cURL / Mailing Lists / curl-library / Single Mail


Re: lib curl nocopy callbacks

From: <>
Date: Fri, 17 Jan 2003 09:33:41 -0500

This is a bit off-topic but I think it is tangential to this discussion.

Has there ever been any thought to having an "advanced" curl mode that does
not rely exclusively on callbacks but could expose the underlying socket
directly from the handle?

There are times when I would like curl to send the request, parse all the
response headers and get to the point where content is ready. Then it
would be nice to have a way to get control of the socket directly and
read/write on it without the overhead of the callback mechanism (and would
obviously eliminate the buffer copy during the content transfer).

I realize this is counter to the entire way the API is currently setup, so
it would probably not be easy to fit it into the current architecture.

| | Philippe Raoult |
| | <> |
| | Sent by: |
| | curl-library-admin_at_lists.sour|
| | |
| | |
| | |
| | 01/17/2003 04:05 AM |
| | Please respond to |
| | curl-library |
| | |
  | |
  | To: |
  | cc: |
  | Subject: Re: lib curl nocopy callbacks |

On Fri, 17 Jan 2003 09:42:50 +0100 (MET)
Daniel Stenberg <> wrote:

> > * what is passed to the user from curl via the WRITE callbacks should
> > allocated with the provided BUFF_ALLOC function.
> And I guess this alloc is performed basicly before each call to read() in
> libcurl? So that the application could for example have a large amount of
> pre-allocated buffers and hand out one at a time when libcurl asks for

this was exactly my idea. It would allow me to do non-blocking IO in the
WRITEDATA callback without doing any copies. This would mean zero [user]
which is the best you can achieve without direct kernel support.

> > * this means that if the user provided a HEADERWRITE function, then the
> > header buffers *which are passed to it* will be allocated with
> Right, and if libcurl needs to realloc that block? Then it will need to
> some kind of realloc magic...

my understanding of the curl code is :
* curl allocates a buffer(1) to receive headers.
* if it's not big enough, realloc it until it fits in
* then parse the content and send it line by line (2) to the user
(if he provided a HEADERWRITE func that is).

if we keep malloc for (1) but use the user-provided callback for (2)
there should be no need to realloc. Correct me if i misunderstood

> Right. Also, we can always bounce some more ideas and fixes back and
> once you guys present the first working patch.

i'll try to do that soon.


This SF.NET email is sponsored by:
Understand how to protect your customers personal information by
SSL on your Apache Web Server. Click here to get our FREE Thawte Apache

This SF.NET email is sponsored by: - A 128-bit supercerts will
allow you to extend the highest allowed 128 bit encryption to all your
clients even if they use browsers that are limited to 40 bit encryption.
Received on 2003-01-17