cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: curl doesn't download full jar and doesn't fail despite --fail

From: Gabriele Kahlout <gabriele_at_mysimpatico.com>
Date: Tue, 17 May 2011 10:28:45 +0200

curl -D log -O --fail -L
http://downloads.sourceforge.net/project/dp4j/1.2-SNAPSHOT/dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar
     [exec] % Total % Received % Xferd Average Speed Time
Time Time Current
     [exec] Dload Upload Total
Spent Left Speed
     [exec]
     [exec] 0 0 0 0 0 0 0 0 --:--:-- --:--:--
--:--:-- 0 0 0 0 0 0 0 0 0 --:--:--
--:--:-- --:--:-- 0 0 372k 0 1214 0 0 1531 0
0:04:09 --:--:-- 0:04:09 1531 26 372k 26 100k 0 0 56259
0 0:00:06 0:00:01 0:00:05 98407 30 372k 30 112k 0 0
51395 0 0:00:07 0:00:02 0:00:05 78565
curl: (18) transfer closed with 266078 bytes remaining to read

And the log:
HTTP/1.1 302 Found
Access-Control-Allow-Origin: *
X-Powered-By: PHP/5.2.9
Content-Disposition: attachment;
filename="dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar"
Location:
http://master.dl.sourceforge.net/project/dp4j/1.2-SNAPSHOT/dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar
Content-type: text/html
Content-Length: 0
Date: Tue, 17 May 2011 08:26:10 GMT
Server: lighttpd/1.4.26

HTTP/1.1 200 OK
Server: nginx/0.7.63
Date: Tue, 17 May 2011 08:26:10 GMT
Content-Type: application/java-archive
Content-Length: 381684
Last-Modified: Tue, 17 May 2011 08:24:57 GMT
Connection: keep-alive
Accept-Ranges: bytes

Do you blame this on sf.net?

On Sun, May 1, 2011 at 6:37 AM, Gabriele Kahlout
<gabriele_at_mysimpatico.com>wrote:

>
>
> On Sat, Apr 30, 2011 at 11:15 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>
>> On Sat, 30 Apr 2011, Gabriele Kahlout wrote:
>>
>> $ curl -O --fail -L
>>> http://downloads.sourceforge.net/project/dp4j/$v/$jar_file
>>>
>>> However, comparing sizes the downloaded jar is invalid. However --fail
>>> didn't complain.
>>>
>>
>> --fail makes curl fail instead of doing a download if the server returns
>> an error. If it instead returns content, --fail will not have any effect.
>>
>> Your lack of details in what exactly you got and what headers the server
>> responded with, we can't properly guess what happened. In most cases curl
>> detects early disconnects or other failures to download the full file and
>> will tell you about it. There are however edge cases where it can't detect a
>> disconnect from a legitimate end of file.
>>
>> What should I do?
>>>
>>
>> First, show us all HTTP headers that your server responds with in this
>> case.
>
>
> This time it worked:
>
> $ curl -D culr-log -O --fail -L
> http://downloads.sourceforge.net/project/dp4j/1.2-SNAPSHOT/dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar
>
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 372k 100 372k 0 0 138k 0 0:00:02 0:00:02 --:--:--
> 174k
>
> curl-log:
> HTTP/1.1 302 Found
> Access-Control-Allow-Origin: *
> X-Powered-By: PHP/5.2.9
> Content-Disposition: attachment;
> filename="dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar"
> Location:
> http://ignum.dl.sourceforge.net/project/dp4j/1.2-SNAPSHOT/dp4j-1.2-SNAPSHOT-jar-with-dependencies.jar
> Content-type: text/html
> Content-Length: 0
> Date: Sun, 01 May 2011 04:34:41 GMT
> Server: lighttpd/1.4.26
>
> HTTP/1.1 200 OK
> Date: Sun, 01 May 2011 04:34:41 GMT
> Server: Apache
> Last-Modified: Sat, 30 Apr 2011 13:00:22 GMT
> ETag: "881065-5d28d-4a2225f854580"
> Accept-Ranges: bytes
> Content-Length: 381581
> Connection: close
> Content-Type: application/java-archive
> X-Pad: avoid browser bug
>
>
>>
>>
>> Should it be just expected that from time to time curl won't download the
>>> complete jar
>>>
>>
>> A TCP connection isn't failsafe. Sometimes it will break and then curl
>> will have gotten a broken image. But curl is able to inform about when this
>> happens except for some rare cases...
>>
>>
>> I must retry and rely on md5sum
>>>
>>
>> An md5sum could still be a good idea as it would help you to exclude other
>> possible error sources as well.
>>
>> --
>>
>> / daniel.haxx.se
>> -------------------------------------------------------------------
>> List admin: http://cool.haxx.se/list/listinfo/curl-users
>> FAQ: http://curl.haxx.se/docs/faq.html
>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>
>
>
>
> --
> Regards,
> K. Gabriele
>
> --- unchanged since 20/9/10 ---
> P.S. If the subject contains "[LON]" or the addressee acknowledges the
> receipt within 48 hours then I don't resend the email.
> subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧
> time(x) < Now + 48h) ⇒ ¬resend(I, this).
>
> If an email is sent by a sender that is not a trusted contact or the email
> does not contain a valid code then the email is not received. A valid code
> starts with a hyphen and ends with "X".
> ∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
> L(-[a-z]+[0-9]X)).
>
>

-- 
Regards,
K. Gabriele
--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).
If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-05-17