Bugs item #3517441, was opened at 2012-04-13 02:46
Message generated for change (Comment added) made by blackcat101
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3517441&group_id=976
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: client module
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Daniel Theron (blackcat101)
Assigned to: Daniel Stenberg (bagder)
Summary: Incorrect exit codes with multiple file transfer
Initial Comment:
Hi there,
I have noticed this bug since I started using cURL (7.19.6), but only reporting it now.
I am using the latest version 7.25.0 now, and the bug still persists.
In some cases when a failure occurs when sending multiple files with cURL an incorrect exit code "0" is given. Please see my tests below:
$ uname -a
SunOS V240-rd-2 5.10 Generic_147440-12 sun4u sparc SUNW,Sun-Fire-V240
$ curl --version
curl 7.25.0 (sparc-sun-solaris2.10) libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6 libidn/1.24 libssh2/1.4.1
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP
TEST 1: All files transferred successfully - correct exit code:
$ curl --connect-timeout 10 --retry 8 --ftp-port - -# -T {file1,file10,file11,file12,file13,file2,file3,file4,file5,file6,file7,file8,file9} -u athed:Helpme12 ftp://172.21.2.232/
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
$ echo $?
0
TEST 2: Now first half of files transferred successfully - correct exit code:
$ curl --connect-timeout 10 --retry 8 --ftp-port - -# -T {file1,file10,file11,file12,file13,file2,file3,file4,file5,file6,file7,file8,file9} -u athed:Helpme12 ftp://172.21.2.232/
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
####################################### 55.2%
curl: (55) Failed sending data to the peer
0.0%Warning: Transient problem: timeout Will retry in 1 seconds. 1 retries left.
0.0%
curl: (28) connect() timed out!
0.0%
curl: (7) couldn't connect to host
0.0%
curl: (7) couldn't connect to host
0.0%
curl: (7) couldn't connect to host
0.0%
curl: (7) couldn't connect to host
$ echo $?
7
TEST 3: Now a connection drop occurs in the middle of the transfer, 1 file was not successfully transferred, but an incorrect exit code 0 was given:
$ curl --connect-timeout 10 --retry 8 --ftp-port - -# -T {file1,file10,file11,file12,file13,file2,file3,file4,file5,file6,file7,file8,file9} -u athed:Helpme12 ftp://172.21.2.232/
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
0.6%
curl: (55) Send failure: Broken pipe
0.0%
Warning: Transient problem: timeout Will retry in 1 seconds. 8 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 2 seconds. 7 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 4 seconds. 6 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 8 seconds. 5 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 16 seconds. 4 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 32 seconds. 3 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 64 seconds. 2 retries left.
0.0%
Warning: Transient problem: timeout Will retry in 128 seconds. 1 retries left.
0.0%
curl: (28) connect() timed out!
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
$ echo $?
0
TEST 4: Now there is a connection drop in beginning of transfer, and connection has gone up after first few files failed, but has given an incorrect exit code of 0:
$ curl --connect-timeout 1 --retry 1 --ftp-port - -# -T {file1,file10,file11,file12,file13,file2,file3,file4,file5,file6,file7,file8,file9} -u athed:Helpme12 ftp://172.21.2.232/
Warning: Transient problem: timeout Will retry in 1 seconds. 1 retries left.
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
$ echo $?
0
Is it possible to fix this bug? cURL to give correct exit codes for the failures above?
Thanks,
Daniel
----------------------------------------------------------------------
Comment By: Daniel Theron (blackcat101)
Date: 2012-04-19 22:36
Message:
@ yangtse
Are you asking me to make the patch? or are you asking badger?
Im not very familiar with the programming language used for curl, but would
certainly be happy if someone can make this change :)
Thanks,
Daniel
----------------------------------------------------------------------
Comment By: Yang Tse (yangtse)
Date: 2012-04-19 08:54
Message:
[Jumping in]
curl tool --retry* options apply to each 'transfer' of a multi-file or
multi-url command invocation. And as you most likely have verified it does
not abort curl execution upon a definitively-failed transfer, remaining
files or urls are tried unless some sort of very serious error such as an
out of memory condition is encountered.
When using curl with a multi-file or multi-url command invocation the exit
code you get is that of the last transfer.
Given existing implementation it should be quite clean to introduce a new
curl option that aborted a multi-file or multi-url transfer upon first
definitively-failed transfer and return that exit code. Adapting function
is_fatal_error() in tool_operate.c would be the key to this.
Are you up to providing a patch for this in case the change was considered
acceptable?
[Jumping out]
----------------------------------------------------------------------
Comment By: Daniel Theron (blackcat101)
Date: 2012-04-19 06:53
Message:
"I'm just very reluctant to change a behavior such as this without being
very careful and considerate. There may very well be lots of scripts
already relying on existing return codes."
Yes I understand, I would not want to break other people's scripts if they
are reliant on this code.
"What I'm curious about is if this is something that was changed in recent
days or if curl always returned like this. I suspect the latter."
As far as I can remember curl had the same behavior since 7.19.6 (when I
started using curl).
"Related question: what if you work on three URLs and for the first two you
get two different return codes and the third succeeds. Which exit code
should curl return, the first failure or the second?"
Good question, I think the last(second) failure's exit code should be
returned (in our case it would not matter if its returning the exit code of
either the first or second).
But just a thought - shouldn't curl rather abort after the first failure
(when retries are exceeded)? Maybe a new flag to allow curl not to continue
with rest of the files in list if previous one failed? (Then the last
returned code should be for the transfer that failed) - Sorry I do not wish
to over complicate this :)
----------------------------------------------------------------------
Comment By: Daniel Stenberg (bagder)
Date: 2012-04-18 14:28
Message:
I read you.
I'm just very reluctant to change a behavior such as this without being
very careful and considerate. There may very well be lots of scripts
already relying on existing return codes.
What I'm curious about is if this is something that was changed in recent
days or if curl always returned like this. I suspect the latter.
Related question: what if you work on three URLs and for the first two you
get two different return codes and the third succeeds. Which exit code
should curl return, the first failure or the second?
I suspect the optimal way that would allow a user to truly get to know how
things run would be to add something new (perhaps expanding -w) to allow
curl to actually tell its progress about each and every single
request/transfer that it performed.
----------------------------------------------------------------------
Comment By: Daniel Theron (blackcat101)
Date: 2012-04-17 23:00
Message:
My question is, for example - if you download /upload 5 files to the
internet and only 3 of the files was downloaded /uploaded successful. Is it
correct if the status downloaded /uploaded is equal to "Successful" or exit
code "0"?
If we script around curl on unix, the script checks the exit code of curl
and notifies support personnel if a file transfer was not successful. Its
an automated process, its not like we run the command manually and check
for the progress bars to complete :)
I cannot rely on the curl exit code if not all files in the file list was
successfully sent. I dont want to run a curl for each file because
transferring all the files individually to get a correct exit code is much
slower :)
Thanks
Daniel
----------------------------------------------------------------------
Comment By: Daniel Stenberg (bagder)
Date: 2012-04-17 15:32
Message:
But really, when you do multiple requests in a single command line, why
should the failed ones override the successful ones? I mean, if you'd get
your way what would stop someone from coming in and asking for exactly the
reverse?
IMHO there's no clear right or wrong here but more a matter of documenting
exactly how it works...
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3517441&group_id=976
Received on 2012-04-20
Mailing List 
Page updated January 05, 2012.