On Thu, Aug 11, 2011 at 10:18 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Thu, 11 Aug 2011, Alex Loukissas wrote:
>
>>> In many situations you won't gain any performance by doing parallell
>>> uploads to a single server, and you'll get a simpler implementation by doing
>>> serial uploads so I'd recommend that. At least for a first shot.
>>>
>>> Just make sure you re-use the same CURL handle _without_ doing
>>> cleanup/init between each individual transfer.
>>
>> Seems like a good plan of attack to use the multi interface as a first
>> shot.
>
> Well, I meant to use the easy interface single-threaded as a first attempt,
> but by all means use the multi interface!
>

I did try reusing the same handle, which (as expected) had a
tremendous positive effect in the number of TIME_WAIT sockets and the
initial tests on performance show some improvement (although not
drastic). Is there a benefit over what I'm doing now (i.e. looping
through the URIs and issuing a curl_easy_perform) versus having a
number of handles in a curl_multi_handle? From what I understand, the
multi interface is doing the same thing (i.e. serially doing a
curl_easy_perform for each handle).

>> Another option would be HTTP pipelining but from what I see it's not in
>> its most mature stage in libcurl. The reason I'm considering pipelining is
>> that the files I'm up/downloading are small size (few to couple hundred KB)
>> and I'm guessing that squeezing many HTTP requests in a single round trip
>> I'll gain some performance. Any thoughts on that?
>
> It is not as mature, no, but perhaps more importantly it is only enabled for
> downloading. If you use the multi interface, you can easily just enable
> pipelining and see how it works/performs.

Great thanks. Reads are more important than writes, so that's OK. I'll
give it a shot.

>
> --
>
>  / daniel.haxx.se
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette:  http://curl.haxx.se/mail/etiquette.html
>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-08-11