cURL / Mailing Lists / curl-library / Single Mail

curl-library

Uploading large files via form POST

From: Jesse Noller <jnoller_at_archivas.com>
Date: Tue, 20 Apr 2004 17:30:43 -0400

Another strange thing we encountered today. Specifically with curl on
the command line, but I am wondering how to handle this with libcurl in
my app as well.

We have a 3 GB file, we want to upload via a form POST, for instance:

curl -i -F path=/home/foo/tmp/3GB.dat -F
uploadedFile=@/home/foo/tmp/3GB.dat

The process runs for awhile after execution, and then linux throws:
Killed

We're assuming that the URL encoding required for a form POST is
happening as curl reads the file and encodes in memory, the Killed comes
from the system running out of memory (1GB on the system running curl).

The question is this - does curl itself have a method to url encode on
disk, and then pass a pre-encoded file to the server, and secondarily,
how do we avoid this with libcurl?

In python/pycurl's case, it would be a matter of importing httplib and
url encoding the file, but how do you pass libcurl the handle to say
"this file is already encoded?"

-jesse
Received on 2004-04-20