On Fri, Oct 05, 2012 at 07:03:47PM +0200, Sebastian Rasmussen wrote:
> Alright, how about something along the lines of the attached?
>
> Basically I'm adding a CURLOPT_UNBUFFERED_WRITES which causes libcurl
> to call setbuf() with NULL, and also to call fsync() to make sure
> that the written file data actually makes it to the file system
> (whether it makes it to disk is an entirely different matter).

fsync() is completely orthogonal to unbuffered writes and shouldn't be
included in the patch. On many journaled filesystems, fsync()
essentially forces a complete sync(), affecting throughput of the entire
system. It's a big stick that should be used sparingly. If it's truly
needed, then sync() could always be called in the application's write
callback that occurs immediately after the file write. Admittedly,
that's not as efficient on systems where fsync() doesn't effectively
equal sync(), but it should be needed in so few applications that I'm
not too worried.

> I'm not confident enough about how curl's test harness works to add a
> test that sets this option and tests it. I'm not even sure what such
> a test can do apart from just making sure that the CURLOPT is
> accepted. But as you can see from the patch I'm hoping for the patch
> to eventually be included in 7.27.1. :)

You could do it pretty easily with an external libtest program, using
test 513 as a base. It might be tricky to make completely portable
due to differing stdio buffering implementations, but it should be
possible to make it work most places.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-10-06