cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: FTP large file support patch

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Thu, 11 Dec 2003 19:49:11 -0800

On Thu, Dec 11, 2003 at 05:27:00PM -0800, Dave Meyer wrote:
> Oops. That's right -- plus, some compilers might not even be very happy
> with the %lld conversion for long longs, either. Or, at least, I've heard
> rumor of such beasts. At any rate, I've done as suggested, defining an
> OFF_T_FMT based on the size of off_t (or, rather, the setting of
> _FILE_OFFSET_BITS). The test to pick the correct format might need
> modification, especially if there's a better way to know about the size of
> off_t's based on something that comes out of the configure script, or if
> there's a way to know about the presence of printf/scanf flags, but it
> works for me as it stands right now.

I suggest leaving off the % from OFF_T_FMT so that it's possible to add
flags if desired (e.g. you can't currently do ("%-12" OFF_T_FMT))
It looks to me like _FILE_OFFSET_BITS is probably the right way to do this
check. It's important that it will work even on when being cross compiled,
but I haven't looked to see how configure determines this value.

> A'ighty, point taken. I've added the PROGRESSFUNCTION_BIG key and its
> related stuff. I needed to add an additional callback field to the
> UserDefined section, though, since the two callbacks have different
> signatures. Right now, you're allowed to define both, in which case the
> non-_BIG version gets called in preference to the _BIG version. I don't
> really feel like that's necessarily the right way to do things, though.
>
> Another caveat with this addition is that the data for the progress
> monitoring is still being stored as double values. I was tempted to
> change those to off_t's, but then I realized that might cause trouble.
> If the progress is displayed globally across all files being downloaded,
> for instance, and someone downloads several files whose combined sizes
> exceed the 32-bit range on a platform where off_t is only 32 bits, then
> progress monitoring would fail.

That might be the case, but the kind of systems that will be downloading
a total of > 2 GiB are going to have a 64-bit off_t. For the others, it's
just another form of the 2 GiB size limit.

> It seems like the only way to be certain that one is storing more or less
> the right numbers is to use doubles or long longs. Maybe the function
> should be changed to take long longs instead of off_t's? It seems
> unlikely that anyone would attempt a single curl to download enough data
> to exceed the size of a signed long long (8+ exabytes!!)... But then, I'm
> not sure if all platforms *have* a long long value (in particular, I'd be
> concerned on small systems that didn't have floating point support, which
> was one of the stated reasons for having this).
>
> At any rate, the callback has been defined to take off_t's, and the data
> for the progress counting is being appropriately cast back out of a
> double, so registering such callbacks should work properly up to the size
> of the off_t, although they may suffer some loss of precision.
>
> I've included the new patch. As the list doesn't really like messages
> beyond 40k, I've gzipped it this time around. Should have been doing that
> from the start. :)

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved
-------------------------------------------------------
This SF.net email is sponsored by: IBM Linux Tutorials.
Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
Received on 2003-12-12