Hi,
I transfer many files using a batch like so (not all options shown)

 curl -T myfile ftp://10.20.30.40/mydir/fNNN.tmp -Q "-RNFR fNNN.tmp" -Q "-RNTO fNNN.dat"
 if errorlevel 1 goto failure
 copy myfile backup\fNNN.tmp
 rem ...

That works on an XP box. There are a few connections per second from various
clients all using the same batch file. Not an overwhelming volume of data.
(The rename is for remote processes waiting for *.dat)

A couple of days ago, a few dozens of files did not pass from one client.
The backup copy was on the client, certifying that the exit status was 0.
However, the files were not found on the server. We retrieved the logs and
there was no mention of the whole session. (All sessions are exactly similar,
except for the number NNN. The session after NNN-1 was about NNN+k, for some
missing NNNs, intermittently.)

The problem is on the client, because it only happened on that PC. In facts,
the poor machine had a huge amount of opened handles and was working badly.
Our best bet is that curl encountered a critical error and was terminated.
Because the calling batch is called from a service, it was not interactive.
Perhaps, if it were interactive it could have shown a message like

  "The application failed to initialize properly (0xC0000017).
  Click OK to terminate the application."

I'm not sure if a call such as SetErrorMode(SEM_FAILCRITICALERRORS);
http://msdn2.microsoft.com/en-us/library/ms680621.aspx
could have prevented that. In case it would, curl should deserve a switch
to trigger it on... Of course, if the system just didn't have enough
memory to load curl.exe, that call wouldn't have been executed anyway.

As a workaround, we are thinking about replacing those batch lines with

 curl -T myfile ftp://10.20.30.40/mydir/fNNN.tmp
 if errorlevel 1 goto failure
 curl -O ftp://10.20.30.40/mydir/fNNN.tmp
 if errorlevel 1 goto failure
 if not exist fNNN.tmp goto failure
 cmp myfile fNNN.tmp
 if errorlevel 1 goto failure
 curl -X NOOP ftp://10.20.30.40/mydir/ "-RNFR fNNN.tmp" -Q "-RNTO fNNN.dat"
 if errorlevel 1 goto failure
 move fNNN.tmp backup
 del myfile
 rem ...

Besides doubling the ftp traffic, the reliability of the latter is still
questionable. But seeking resource leaks will take time and we must work
in the meanwhile.

Booting more often or switching to a reliable OS would be better solutions,
but they are out of my reach. :-)

Any other ideas?

TIA
Ale Vesely
Received on 2007-06-09