cURL / Mailing Lists / curl-library / Single Mail

curl-library

patch for >2GB files

From: Gisle Vanem <gvanem_at_broadpark.no>
Date: Fri, 23 Jan 2004 19:01:20 +0100

Thanks Dave for the url
 ftp://ftp.ncbi.nih.gov/blast/db/FASTA/nt.gz

The file download just fine. But as you noted the REST command
at this server doesn't handle huge files. So I didn't try that yet.

Some minor cosmetic "bugs" remains.

> SIZE nt.gz
< 213 2876516042
> RETR nt.gz
< 150 Opening BINARY mode data connection for nt.gz (2876516042 bytes)

* Connection accepted from server
* Getting file with size: 2876516042
  % Total % Received % Xferd Average Speed Time Curr.
                                 Dload Upload Total Current Left Speed

  0 2743M 0 1024 0 0 431 0 1853:13:27 0:00:02 1853:13:24 431
  0 2743M 0 5404 0 0 2108 0 378:57:49 0:00:02 378:57:46 23297
  0 2743M 0 60528 0 0 17605 0 45:23:06 0:00:03 45:23:03 119k

The huge initial ETA makes the progress line a bit too long. So
successive lines should clear the old trailing text. And >999M replaced by
xx.xG. Here's my suggestion:

--- CVS-latest/lib/progress.c Fri Jan 23 09:02:12 2004
+++ lib/progress.c Fri Jan 23 18:50:43 2004
@@ -57,6 +57,7 @@
 {
 #define ONE_KILOBYTE 1024
 #define ONE_MEGABYTE (1024*1024)
+#define ONE_GIGABYTE (1024*1024*1024)

   if(bytes < 100000) {
     sprintf(max5, "%5Od", (curl_off_t)bytes);
@@ -71,11 +72,16 @@
     sprintf(max5, "%4.1fM", bytes/ONE_MEGABYTE);
     return max5;
   }
- sprintf(max5, "%4OdM", (curl_off_t)bytes/ONE_MEGABYTE);
+ if(bytes < (1000*ONE_MEGABYTE)) {
+ sprintf(max5, "%4OdM", (curl_off_t)bytes/ONE_MEGABYTE);
+ return max5;
+ }
+ /* 1000 MB - 8589934587 GB !! */
+ sprintf(max5, "%4.1fG", bytes/ONE_GIGABYTE);
   return max5;
 }

@@ -192,8 +198,9 @@

 int Curl_pgrsUpdate(struct connectdata *conn)
 {
+ static int longest_len = -1;
   struct timeval now;
- int result;
+ int result, print_len;

   char max5[6][10];
   double dlpercen=0;
@@ -366,7 +373,7 @@
   if(total_expected_transfer)
     total_percen=(double)(total_transfer/total_expected_transfer)*100;

- fprintf(data->set.err,
+ print_len = fprintf(data->set.err,
           "\r%3d %s %3d %s %3d %s %s %s %s %s %s %s",
           (int)total_percen, /* total % */
           max5data(total_expected_transfer, max5[2]), /* total size */
@@ -382,6 +389,11 @@
           time_left, /* time left */
           max5data(data->progress.current_speed, max5[5]) /* current speed */
           );
+
+ if (print_len < longest_len) /* space pad old lines */
+ fprintf(data->set.err, "%*.*s", longest_len - print_len, "");
+ if (print_len > longest_len)
+ longest_len = print_len;

   /* we flush the output stream to make it appear as soon as possible */
   fflush(data->set.err);

-----------

Note there is still problems with MSVC v6, SIZEOF_CURL_OFF_T and
'long long' etc., but MingW has no problems with huge files.

Gisle V.

# rm /bin/laden
/bin/laden: Not found

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
Received on 2004-01-23