cURL / Mailing Lists / curl-library / Single Mail


Re: lib curl nocopy callbacks

From: Philippe Raoult <>
Date: Thu, 16 Jan 2003 14:02:23 +0100

Laurent is an intern in our company but only works one or two days a week.
I will thus reply for him :)

On Thu, 16 Jan 2003 13:30:04 +0100 (MET)
Daniel Stenberg <> wrote:

> Can you please provide us with some more info on how you visualize this API
> to be used, like an example app source code (possibly psuedo code)?

certainly. we used a modified lib500.c test :

  curl_easy_setopt(curl, CURLOPT_BUFF_ALLOC, alloc);
  curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, mywrite);

alloc is just calling malloc at the moment, and mywrite prints the buffer
content and free()s it.
> > the goal is that when the user use the write_callback the return buffer is
> > the same that the one provide by alloc_callback
> >
> > it works well for the body of an html request but the headers have a proper
> > traitment, lib return headers line by line :/ so it make different buffer
> > for this.
> The code reallocates a buffer if it needs to, in order to keep very big
> header lines in a single buffer, yes.
> > a quick working possobility is in case of alloc_callback send headers in
> > raw like body, It's the use who does it's own traitment and it prevent
> > copy.
> I think you're oversimplifying this. There is a reason why libcurl collects
> header lines into one single buffer: it needs to be able to check headers for
> specific contents, why it needs to construct full headers before those checks
> are made.
> But if you have a way to overcome this "problem", then please tell us!

we talked about this with Laurent and i told him to post here because we hadnt been
able to find any clever way to get out of this. I agree that passing raw headers
is bit simplistic. The way i see it, BUFF_ALLOC allows the user to specify a
malloc function to curl. It follows that either :
- the user also has to provide a free function for curl's internal use (and a realloc one too)
- the provided malloc should only be used when data is actually passed to the user.

I'd rather stick with the latter solution because the interface is supposed to be
about not copying data around, and certainly it wouldnt help to use the whole
malloc/realloc/free for curl's internal use.

I thus propose the following logic:
* what is passed to the user from curl via the WRITE callbacks should be allocated
with the provided BUFF_ALLOC function.
* this means that if the user provided a HEADERWRITE function, then the header
buffers *which are passed to it* will be allocated with BUFF_ALLOC
* same goes for data WRITE, but there it is easier

The easy part is already done, and (I || Laurent) will do the rest when we have
reached an agreement on what exactly to do.

Best Regards,

This SF.NET email is sponsored by:
Understand how to protect your customers personal information by implementing
SSL on your Apache Web Server. Click here to get our FREE Thawte Apache
Received on 2003-01-16