On 4/13/2015 10:01 PM, Bryan Christ wrote:
> I've been trying to figure out why http POST uploads are so slow with
> libcurl. Upload speeds continually perform at about 1/10th of the
> expected performance (or less). Many users have reported this behavior
> on our forum. I suspect it has a lot to do with CURL_MAX_WRITE_SIZE
> being set to 16k. Uploads to these same servers through other means
> (JavaScript for example) reach their expected throughput. The code in
> question can be seen here:
>
> https://github.com/MediaFire/mediafire-fuse/blob/master/utils/http.c
> (at approx line 314)
>
> Assuming the issue is the 16K buffer limit, are there any other
> options? Asking users to recompile a custom libcurl with a larger
> buffer size is not very palatable.

If you want help on the list your best bet is a self contained example
that can be used to reproduce and the details at [1]. The buffer issue
is in the TODO [2] but from what I see there and elsewhere the
significance is SFTP related.

Continually or continuously? Is it 100% reproducible? A few ideas:
Maybe your uploads are compressed when they go through the browser, but
they are not compressed when uploaded through libcurl. There is no
compression built in libcurl upload (as far as I know), you would have
to do it manually and attach the header for content encoding gzip.
A different user agent (or lack of one -- the default) when you use
libcurl causes different treatment by the server. This applies to any
header, really.
The IP address returned to the browser is different than the IP address
returned to curl via DNS because the DNS request was made differently.
Or you just got a different IP address because they rotate.
I/O in your program. eg posting a FILE but the I/O is backed up.
Proxy setting is different in the browser than it is in libcurl.

Once you have a way to reproduce try using the curl tool and see if you
get the same result. Also try the latest version.

[1]: http://curl.haxx.se/docs/bugs.html#What_to_report
[2]: http://curl.haxx.se/docs/todo.html#Modified_buffer_size_approach

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-04-14