cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: truncated files

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sun, 7 Apr 2013 19:09:18 +0200 (CEST)

On Sun, 7 Apr 2013, Giuseppe Attardi wrote:

> The settings for the easy handle are:

Which libcurl version on what platform?

> Besides I ask for compressed transfer:
> curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, "gzip");
> I configured curl --without-zlib, because I want to save the file zipped.
> The transfer callback (CURLOPT_WRITEFUNCTION) just copies the received bytes
> to a local file.

You could still have a zlib enabled libcurl if you just add the correct
Accept-Encoding header manually instead.

> Everything works fine, except that occasionally some files are received
> truncated. curl does not report any error on those files.

So what does such a transfer look like, in regards to request headers and
response headers + body?

What makes you suspect this is a libcurl problem to begin with and not a
server-side problem?

> The problem only occurrs when a large number (~ 1000) of files are
> requested, and it happens occasionally for different files.

So if you only add 500 the problem doesn't happen? If so, where's the limit?

> Could you suggest what could cause missing an event?

I'm far from convinced that is actually what's happening.

libcurl just can't "forget" to call the write callback for data it receives,
so if the data was truly received it should've been delivered in a callback as
well.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2013-04-07