cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: pyCURL/Libcurl problem with large files

From: Jesse Noller <jnoller_at_archivas.com>
Date: Mon, 22 Mar 2004 16:59:15 -0500

> (please don't reply to an existing thread when you want to post an entirely
> new thread using a new subject, it messes up threading in mail clients and
> the web archive - post a fresh new mail instead)
>

Apologies - my mistake.

> CURLOPT_INFILESIZE in the libcurl level takes a 'long', which is equal to
> 'int' on 32bit systems.
>

Thanks - unless I convert it from a long() which it is "normally" from
the the os.path.getsize() call I make to an int, this again leads me to
believe this is a python or pycurl problem

> How big is a 'long' and 'int' in python? Don't they match the host's native
> longs and ints?
>

No - not to my knowledge. That's what's bothering me - I tried to find
some docs on what the actual limit of int() conversion is - at what
point I need to use a long() - but google did not shed a lot of light.

An example (file-0 is a 2 gig file)

>>> import os
>>> fs = os.path.getsize('file-0')
>>> print fs
2147532800
>>> fs_int = int(fs)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to int
>>> type(fs)
<type 'long'>
>>>

> CURLOPT_INFILESIZE_LARGE wants the size passed in as a curl_off_t variable,
> not just a long.
>

Let me take a look at the pycurl interface to that. Maybe that's what
I'm doing wrong.

> 55 is CURLE_SEND_ERROR, yes. 'man libcurl-errors' is a nice way to figure out
> libcurl error codes.
>

Well, that *is* easier than grepping c code.

> I think so too, and I don't have any clues either!

Know of a pyCURL-users list? :)
Received on 2004-03-22