cURL / Mailing Lists / curl-library / Single Mail

curl-library

pyCURL/Libcurl problem with large files

From: Jesse Noller <jnoller_at_archivas.com>
Date: Mon, 22 Mar 2004 11:43:15 -0500

(via http PUT)

I am hoping that someone else working with pycurl catches this - I just
upgraded to 7.11.1 release (for both curl and pycurl) and I am trying to
perform file PUTs against a server with 2GB files.

The problem I am having is that with the normal INFILESIZE setopt, the
size of the file I am feeding libcurl via pyCURL has to be converted to
an INT - files up to 1GB work fine with:

fs = os.path.getsize('file')
c.setopt(c.INFILESIZE, int(fs))

FS is always a long with these bigger files, so I convert it to an INT
before feeding it to libcurl/pycurl.

Now, the problem comes up when doing 2Gb files, the number spit out by
getsize is a long, and can not be converted to an INT (a known python
limitation) - as the number is already a long, I skip conversion, and
try:

if fs > 524288000: #Set the limit to 500 Megs just to be safe.
c.setopt(c.INFILESIZE_LARGE, long(fs))
else:
c.setopt(c.INFILESIZE, int(fs))

This results in an "Error 55, ''" from Libcurl - I grepped the C code,
and found:

hugehelp.c:" 55 Failed sending network data\n"

I'm in the dark here (still learning) - I am assuming it's a problem
with pycurl interfacing with libcurl, but I don't know how to prove
that/fix it.

-jesse
Received on 2004-03-22