Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: libcurl read-llike interface.
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: XSLT2.0 via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 14 Dec 2020 19:06:40 +0100
> For a zero-copy API, I figure the application would somehow provide
the buffer first and then the callback would be done and tell the
application when (a piece of) the buffer has been populated.
Exactly!
You are right "zero-copy" or rather "caller provided buffers" won't
generally make things "faster" because on most (but not all!) machines,
the bottleneck is usually not the CPU/memory bus.
But instructions you don't do are always faster than the one you do,
aren't they? This means efficiency, so less CPU for the same task.
In some case, I think my Raspberry Pi where the bottleneck is NOT the
network (ethernet + fiber connection) but in fact CPU used for crypto
(eg SSL/TLS), saving cycles makes things somehow quicker. For example, I
doubled "efficiency" (as seen with top/htop) on my fuse driver that uses
libcurl, by changing write buffers from 4K (standard fuse) to 128K (max
fuse write). That saves context switches, which from some benchmark I've
read are equivalent on Linux to the impact of a memcpy() in the range
16k to 32k. Double efficiency does not mean things goes twice as fast...
but that you can do more workload with the same machine in a given
time-slot, which is always beneficial.
And you save electricity, that good for the carbon footprint! :-)
Would the benefit balance the code complexity? It is indeed an excellent
question that you probably have to consider for each new feature.
I don't have the answer, but being "7th open source project" (Kudos!)
means "efficiency" expectations are high, but also features, stability,
maintainability, etc... so indeed choices must be made.
> Feel free to work on implementing that!
I might... when I would consider my own open source projects "done
enough"! I probably have a lot of improvement and optimisations already
in my own code...
But libcurl is a big piece of code and I'll first need to find, in all
those pieces of code, where "internal buffers" are allocated and used,
to assess the work to be done and the complexity that would add, so that
the community decides if it is worth having.
> >Could someone point me to a similarly enlightening example as fcurl,
but using "multi-socket"?
> I'm not aware of any such example. To me, it seems that would just
complicate fcurl.
Sorry, didn't explain myself correctly. I don't mean doing the same as
fcurl.
I mean any "significant" piece of code, not too short like examples in
the official documentation and not too big (the size of fcurl is ideal)
that uses "multi-socket" to get a taste and try to understand the subtle
differences between multi-select and multi-socket!
I found fcurl excellent to understand the taste of multi-select, thanks
again.
Regards
Alain
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-12-14
Date: Mon, 14 Dec 2020 19:06:40 +0100
> For a zero-copy API, I figure the application would somehow provide
the buffer first and then the callback would be done and tell the
application when (a piece of) the buffer has been populated.
Exactly!
You are right "zero-copy" or rather "caller provided buffers" won't
generally make things "faster" because on most (but not all!) machines,
the bottleneck is usually not the CPU/memory bus.
But instructions you don't do are always faster than the one you do,
aren't they? This means efficiency, so less CPU for the same task.
In some case, I think my Raspberry Pi where the bottleneck is NOT the
network (ethernet + fiber connection) but in fact CPU used for crypto
(eg SSL/TLS), saving cycles makes things somehow quicker. For example, I
doubled "efficiency" (as seen with top/htop) on my fuse driver that uses
libcurl, by changing write buffers from 4K (standard fuse) to 128K (max
fuse write). That saves context switches, which from some benchmark I've
read are equivalent on Linux to the impact of a memcpy() in the range
16k to 32k. Double efficiency does not mean things goes twice as fast...
but that you can do more workload with the same machine in a given
time-slot, which is always beneficial.
And you save electricity, that good for the carbon footprint! :-)
Would the benefit balance the code complexity? It is indeed an excellent
question that you probably have to consider for each new feature.
I don't have the answer, but being "7th open source project" (Kudos!)
means "efficiency" expectations are high, but also features, stability,
maintainability, etc... so indeed choices must be made.
> Feel free to work on implementing that!
I might... when I would consider my own open source projects "done
enough"! I probably have a lot of improvement and optimisations already
in my own code...
But libcurl is a big piece of code and I'll first need to find, in all
those pieces of code, where "internal buffers" are allocated and used,
to assess the work to be done and the complexity that would add, so that
the community decides if it is worth having.
> >Could someone point me to a similarly enlightening example as fcurl,
but using "multi-socket"?
> I'm not aware of any such example. To me, it seems that would just
complicate fcurl.
Sorry, didn't explain myself correctly. I don't mean doing the same as
fcurl.
I mean any "significant" piece of code, not too short like examples in
the official documentation and not too big (the size of fcurl is ideal)
that uses "multi-socket" to get a taste and try to understand the subtle
differences between multi-select and multi-socket!
I found fcurl excellent to understand the taste of multi-select, thanks
again.
Regards
Alain
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2020-12-14