cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Download of files using curl is not reliable

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Wed, 2 Sep 2015 23:38:53 +0200 (CEST)

On Wed, 2 Sep 2015, Freek de Kruijf wrote:

> When downloading rpm files from an openSUSE software repository via a Wi-Fi
> connection I often get the message that checksum does not match. There are
> packages lost on the Wi-Fi connection. A ping of 100 shows a few percent
> loss.

That sounds terrible. A "few" precent is a lot and really hampers your
connections - like you cannot get very fast transfer rates with that. It is
not a normal level for normal home conditions I would say.

Maybe it is an indication that your network is shaky to begin with?

> However using shasum to calculate the check sum, I get an answer that does
> not match the check sum it should have. Repeating the 2 actions often gives
> a different check sum, especially on large files. But also short files are
> not always transferred with a matching check sum.

It is a grave indication that things are wrong. But I expect that you get the
same sort of problems when downloading using other TCP clients as well? curl
doesn't do anything magic with its connections, it is just a plain old TCP
connection where it talks HTTP over.

> I assume TCP is used as the transfer protocol, so I expect this transfer to
> be error free. Is there an explanation for this behavior or is my assumption
> wrong?

First, TCP is not really 100% error free, it is just unlikely to get errors.
You can damage the packets and still get a fine checksum as its only 16bit.
But there's usually more layers of checksums in networks, so actual broken
content to go through is extremely rare.

How exactly do you download these files, using the curl tool directly? If so,
you could try alternative tools to see what results they produce.

I would suspect they give roughly the same problems. If that's true, then I
would suspect something in your surroundings. Network equipment, network card,
network driver, file system, harddrive...

When you get checksum errors or get differences in downloaded content from two
identical files, have you tried to compare them to see what the damaged areas
look like?

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2015-09-02