On Thu, 9 Aug 2012, Florian Pritz wrote:

> When using curl to upload files to a pastebin, I noticed that `curl -v -F
> "file=@/proc/cpuinfo" http://...` will send an http request stating that the
> file is empty, but it reads the contents anyway and the server (apache in my
> case) returns "400 Bad Request" because it didn't expect the content.

Yes. I took a decision many years ago (introduced in libcurl 7.12.1 in August
2004) that I would let libcurl stat() the file first to figure out the file
size (and use that in the request headers) and then read the file contents
later in a streaming manner when providing thr request body. That is necessary
to avoid having to read the entire thing into memory before sending it off -
which libcurl used to do before that.

The downside is that it doesn't handle growing or shrinking files properly.

> Right now I'm using this code[1] to work around the problem. Basically I
> just load the file into memory in case st_size is 0 and pass it as
> CURLFORM_BUFFER{,PTR} instead of CURLFORM_FILE.

Sounds like a decent work-around. Another version could copy the file to a
temporary one and you'd then upload that. It would use less memory but require
temp file management in your app.

> I hope this is enough for someone with more knowledge about libcurl
> internals to fix this properly.

Sorry, but what exactly would a "proper" fix be to this problem you think?

I can think of one fix if we assume that the server is a HTTP/1.1, as then we
could send the whole request using chunked encoding - but libcurl can't
blindly assume 1.1 and some server won't like getting such posted in a chunked
manner.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html