Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: libcurl read-llike interface.
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 14 Dec 2020 17:35:10 +0100 (CET)
On Mon, 14 Dec 2020, XSLT2.0 via curl-library wrote:
>> we don't get any feedback and nobody writes any pull requests for it.
>
> The code is obviously incomplete, which helps a lot understanding things
> because it is "short enough", but does probably not encourage to write PR?
That would explain it then. But I also don't want to spend time and energy on
something that won't be used. Catch 22!
> It is also pretty un-optimised: the write callback could be called multiple
> times in a multi_perform cycle.
There's this saying about premature optimization...
>> -2) A step towards zero-copy libcurl (a topic much discussed!)
>
>> I don't think it will help with that...
>
> Vocabulary (as often)! It depends on what you call "zero-copy" here.
Sure, zero-copy usually means fewer-copies and that's fine, but I still don't
think the fcurl API is the key to solving that. But I'm also not suggesting
that I have a firm idea of how to do a fewer-copies implementation that A)
makes sense to users and B) makes a performance impact.
In fact, I've never seen a zero-copy suggestion for libcurl that provides A +
B.
> - call libcurl to provide buffer/size where the data is needed
> - call curl_multi_perform (or whatever existing/new function)
> - curl_multi_perform returns when the provided buffer is full.
Feel free to work on implementing that!
> That would trim down the fread code to almost nothing: 2 calls and error
> handling... but possibly pushing down the rest of the fcurl code inside the
> library, which hopefully would take care of limiting the number of
> memcpy/realloc!
If fcurl would ever become popular, we could certainly discuss making it a
part of libcurl proper. That was the plan from the beginning. I just don't see
it happening any time soon...
> But maybe the "caller provided buffers" technique is already possible to
> achieve with "multi-socket"?
No. We deliver and get data to and from transfers the same way, independently
of which API you use, with the read and write callbacks using internal
buffers.
For a zero-copy API, I figure the application would somehow provide the buffer
first and then the callback would be done and tell the application when (a
piece of) the buffer has been populated.
> Could someone point me to a similarly enlightening example as fcurl, but
> using "multi-socket"?
I'm not aware of any such example. To me, it seems that would just complicate
fcurl.
Date: Mon, 14 Dec 2020 17:35:10 +0100 (CET)
On Mon, 14 Dec 2020, XSLT2.0 via curl-library wrote:
>> we don't get any feedback and nobody writes any pull requests for it.
>
> The code is obviously incomplete, which helps a lot understanding things
> because it is "short enough", but does probably not encourage to write PR?
That would explain it then. But I also don't want to spend time and energy on
something that won't be used. Catch 22!
> It is also pretty un-optimised: the write callback could be called multiple
> times in a multi_perform cycle.
There's this saying about premature optimization...
>> -2) A step towards zero-copy libcurl (a topic much discussed!)
>
>> I don't think it will help with that...
>
> Vocabulary (as often)! It depends on what you call "zero-copy" here.
Sure, zero-copy usually means fewer-copies and that's fine, but I still don't
think the fcurl API is the key to solving that. But I'm also not suggesting
that I have a firm idea of how to do a fewer-copies implementation that A)
makes sense to users and B) makes a performance impact.
In fact, I've never seen a zero-copy suggestion for libcurl that provides A +
B.
> - call libcurl to provide buffer/size where the data is needed
> - call curl_multi_perform (or whatever existing/new function)
> - curl_multi_perform returns when the provided buffer is full.
Feel free to work on implementing that!
> That would trim down the fread code to almost nothing: 2 calls and error
> handling... but possibly pushing down the rest of the fcurl code inside the
> library, which hopefully would take care of limiting the number of
> memcpy/realloc!
If fcurl would ever become popular, we could certainly discuss making it a
part of libcurl proper. That was the plan from the beginning. I just don't see
it happening any time soon...
> But maybe the "caller provided buffers" technique is already possible to
> achieve with "multi-socket"?
No. We deliver and get data to and from transfers the same way, independently
of which API you use, with the read and write callbacks using internal
buffers.
For a zero-copy API, I figure the application would somehow provide the buffer
first and then the callback would be done and tell the application when (a
piece of) the buffer has been populated.
> Could someone point me to a similarly enlightening example as fcurl, but
> using "multi-socket"?
I'm not aware of any such example. To me, it seems that would just complicate
fcurl.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ ------------------------------------------------------------------- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2020-12-14