cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: first 94208 bytes corrupted! (Ubuntu 14)

From: Michael Wood <esiotrot_at_gmail.com>
Date: Thu, 29 May 2014 06:56:36 +0200

On 29 May 2014 12:56 AM, "Wagner Patriota" <wagner.patriota_at_gmail.com>
wrote:
>
> I can't fsync() a FILE* ... does anybody have any hint on what can be
done here?

Try flush() or fflush()?

> On Wed, May 28, 2014 at 7:36 PM, Wagner Patriota <
wagner.patriota_at_gmail.com> wrote:
>>
>> hey Daniel / Ray, I think I found the solution. I didn't try yet, but
for sure it is related.
>>
http://www.pointsoftware.ch/en/4-ext4-vs-ext3-filesystem-and-why-delayed-allocation-is-bad/
>>
>> My cURL code is fine. The detail is that I write the file as a .tmp file
and then I rename the file.
>> This fast rename is responsible for the losing of data.
>>
>> I need do a fsync() before every rename... have you guys seen this
before?
>>
>> Thanks for attention.
>>
>>
>> On Sat, May 24, 2014 at 1:07 PM, Wagner Patriota <
wagner.patriota_at_gmail.com> wrote:
>>>
>>> On Sat, May 24, 2014 at 12:23 PM, Daniel Stenberg <daniel_at_haxx.se>
wrote:
>>>>
>>>> On Fri, 23 May 2014, Wagner Patriota wrote:
>>>>
>>>>> I noticed I didn't check if the file handle is actually valid. Now I
fixed and I will test a little more... [in bold] But I really don't think
it's the error, once the file it's supposed to write on is a *completely
new path*, almost impossible the file handle not be valid.
>>>>
>>>>
>>>> That wouldn't have been a likely problem anyway. A bad file handle
would not suddently cause zeroes to get written after 94KB.
>>>
>>>
>>> Yes, sure. And I confirm this problem is still happening... randomly...
another detail is that the number of null bytes is 94208 but each time a
new number of null bytes... now I just got 159728.
>>>
>>>>
>>>>
>>>> Your example looks pretty much like what curl the command line tool
does. Does invoking curl on that URL cause the same problem?
>>>
>>>
>>> I tried... via command line it works. My applications downloads
hundreds of pictures by hour. It happens very seldom. But I never saw it
happening on Windows or Mac, only on Ubuntu.
>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> / daniel.haxx.se
>>>> -------------------------------------------------------------------
>>>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>>>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>>
>>>
>>
>
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-05-29