cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: first 94208 bytes corrupted! (Ubuntu 14)

From: Wagner Patriota <wagner.patriota_at_gmail.com>
Date: Wed, 28 May 2014 19:51:27 -0300

I can't fsync() a FILE* ... does anybody have any hint on what can be done
here?

On Wed, May 28, 2014 at 7:36 PM, Wagner Patriota
<wagner.patriota_at_gmail.com>wrote:

> hey Daniel / Ray, I think I found the solution. I didn't try yet, but for
> sure it is related.
>
> http://www.pointsoftware.ch/en/4-ext4-vs-ext3-filesystem-and-why-delayed-allocation-is-bad/
>
> My cURL code is fine. The detail is that I write the file as a .tmp file
> and then I rename the file.
> This fast rename is responsible for the losing of data.
>
> I need do a *fsync()* before every rename... have you guys seen this
> before?
>
> Thanks for attention.
>
>
> On Sat, May 24, 2014 at 1:07 PM, Wagner Patriota <
> wagner.patriota_at_gmail.com> wrote:
>
>> On Sat, May 24, 2014 at 12:23 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
>>
>>> On Fri, 23 May 2014, Wagner Patriota wrote:
>>>
>>> I noticed I didn't check if the file handle is actually valid. Now I
>>>> fixed and I will test a little more... [in bold] But I really don't think
>>>> it's the error, once the file it's supposed to write on is a *completely
>>>> new path*, almost impossible the file handle not be valid.
>>>>
>>>
>>> That wouldn't have been a likely problem anyway. A bad file handle would
>>> not suddently cause zeroes to get written after 94KB.
>>>
>>
>> Yes, sure. And I confirm this problem is still happening... randomly...
>> another detail is that the number of null bytes is 94208 but each time a
>> new number of null bytes... now I just got 159728.
>>
>>
>>>
>>> Your example looks pretty much like what curl the command line tool
>>> does. Does invoking curl on that URL cause the same problem?
>>
>>
>> I tried... via command line it works. My applications downloads hundreds
>> of pictures by hour. It happens very seldom. But I never saw it happening
>> on Windows or Mac, only on Ubuntu.
>>
>>
>>>
>>>
>>> --
>>>
>>> / daniel.haxx.se
>>> -------------------------------------------------------------------
>>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>>
>>
>>
>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-05-29