Hi Daniel,

On Thu, 18 Sep 2014, Daniel Stenberg wrote:

> On Thu, 18 Sep 2014, Pashupati Kumar wrote:
>
> > I am new to libcurI and still learning my ways around it. I have few
> > questions on it. I discovered that user needs to copy the data in
> > write call-back function and also the way our application behave, it
> > will end up doing copy during read call back function as well. This
> > will be a big draw-back as the application is data intensive.
>
> Will it really? Some people have claimed this over the years but I've never seen
> any real numbers proving this. On the contrary I think this is a common belief
> and suspicion that rarely actually turns out to be an actual problem.
>
> Have you measured how much CPU - in your case - that would be "wasted" on
> this extra copy? Or perhaps if that waste even makes a difference to the total
> performance of your application.
>
I have not measured CPU required to perform copy operation. I understand that impact to application due to additional copy would depend on the nature of application. A telnet kind application wouldn't see impact whereas FTP probably would. To give you further background on the application. It needs to work with Cloud (private/public) object storage. So, here the comparison would be against protocols such as NFS (File) and iSCSI (Block), which is traditionally used for storage traffic.
I came across this paper on memcpy performance. It tells me that a CORE running at 733 MHZ can do 400 MB per second.
http://www.embedded.com/design/configurable-systems/4024961/Optimizing-Memcpy-improves-speed

Please keep in mind that there would be another copy involved in Kernel ( from user data to Socket sk_buf). Are people using CURL library for storage traffic? If not, what is recommended library for doing so.

> > I came across an old mail thread on avoiding copy for
> CURLOPT_WRITEFUNCTION.
> > I don't know what was the final outcome of it.
>
> We discuss this topic every once in a while and it always ends up getting
> burried again when the work to implement something like this ends up being
> more than the gains in having the feature. But maybe it is different this time?
>
> > For read callback function, application data is available in scatter
> > list of buffers. This brings me to my next question. Can libcurl use
> > scatter-gather buffers?
>
> No.
>
Has this been considered before? Since Socket layer provide iovec based interface, will it be a huge change in the CURL library to expose it all the way to user?

> --
>
> / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-09-18