"Paul Monteiro" wrote:
> Case A
> 2009-11-27 08:00.00 - Copying file AAAA (1.5GB) from server 1 to server 2
> (the copy will finish at 08:05:00)
> 2009-11-27 08:01.00 - Trying to download AAAA with curl (0.3GB) from
> server 2
> 2009-11-27 08:02.00 - Download complete

So the problem is that cURL downloads a file to which data
is still being appended, and the overall protocol (OS +
server + client) does not allow to distinguish between the
events "read all data available so far" and "read all data
in the file".

  IMHO you should change the triggering even from "the file
is present on the server" to "the file has been fully
uploaded to the server". For example, after AAAA has been
fully uploaded, create a further file AAAA.done (empty or
symlink or whatever does not waste space) and use the event
"AAAA.done is present on the server" as trigger to start
downloading AAAA.

HTH

-- 
Marco Maggi
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html