> From: "Ray Satiro" <raysatiro_at_yahoo.com>
> To: curl-users_at_cool.haxx.se
> Sent: Friday, March 22, 2019 12:45:06 AM
> Subject: Re: curl exit status on curl: (18) server did not report OK,
> got 426

> On 3/21/2019 12:50 PM, Kevin R. Bulgrien wrote:

> > It seems I may have found a case where one cannot trust the exit
> > status
>
> > to report a failed transfer. In this case, the file was truncated.
> > Am
>
> > I missing something essential in my understanding of how to detect
> > file
>
> > transfer errors?
>

> > Given the following shell script fragment:
>

> > curl -v -k --use-ascii -u "${GET_USER}:${GET_PASS}" -o "${GET_DST}"
> > \
>
> > "ftps://${GET_HOST}${GET_PATH:+/}${GET_PATH}/${GET_SRC}"
>
> > RET="${?}"
>
> > echo
>
> > echo "Result = ${RET}"
>

> > Why might I get "Result = 0" when the log reports an error:
>

> > * Trying xx.xx.xx.xx...
>
> > * TCP_NODELAY set
>
> > % Total % Received % Xferd Average Speed Time Time
> > Time
> > Current
>
> > Dload Upload Total Spent Left Speed
>
> > 0 0 0 0 0 0 0 0 --:--:-- --:--:--
> > --:--:-- 0* Connected to regionftp (xx.xx.xx.xx) port 990 (#0)
>
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>
> > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
>
> > } [5 bytes data]
>
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>
> > } [512 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Server hello (2):
>
> > { [81 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Certificate (11):
>
> > { [1717 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Server finished (14):
>
> > { [4 bytes data]
>
> > * TLSv1.0 (OUT), TLS handshake, Client key exchange (16):
>
> > } [262 bytes data]
>
> > * TLSv1.0 (OUT), TLS change cipher, Change cipher spec (1):
>
> > } [1 bytes data]
>
> > * TLSv1.0 (OUT), TLS handshake, Finished (20):
>
> > } [16 bytes data]
>
> > * TLSv1.0 (IN), TLS change cipher, Change cipher spec (1):
>
> > { [1 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Finished (20):
>
> > { [16 bytes data]
>
> > * SSL connection using TLSv1.0 / AES256-SHA
>
> > * Server certificate:
>
> > * subject: OU=Domain Control Validated; CN=*.xx.xx
>
> > * start date: Feb 5 15:50:15 2019 GMT
>
> > * expire date: Mar 8 20:21:38 2021 GMT
>
> > * issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=
> > http://certs.godaddy.com/repository/ ; CN=Go Daddy Secure
> > Certificate Authority - G2
>
> > * SSL certificate verify result: unable to get local issuer
> > certificate (20), continuing anyway.
>
> > 0 0 0 0 0 0 0 0 --:--:-- 0:00:02
> > --:--:-- 0{ [5 bytes data]
>
> > < 220-FileZilla Server version 0.9.41 beta
>
> > < 220-written by Tim Kosse ( Tim.Kosse_at_gmx.de )
>
> > < 220 Please visit http://sourceforge.net/projects/filezilla/ } [5
> > bytes data]
>
> > > USER xxx
> >
>
> > { [5 bytes data]
>
> > < 331 Password required for xxx
>
> > } [5 bytes data]
>
> > > PASS xxx
> >
>
> > { [5 bytes data]
>
> > < 230 Logged on
>
> > } [5 bytes data]
>
> > > PBSZ 0
> >
>
> > { [5 bytes data]
>
> > < 200 PBSZ=0
>
> > } [5 bytes data]
>
> > > PROT P
> >
>
> > { [5 bytes data]
>
> > < 200 Protection level set to P
>
> > } [5 bytes data]
>
> > > PWD
> >
>
> > { [5 bytes data]
>
> > < 257 "/" is current directory.
>
> > * Entry path is '/'
>
> > } [5 bytes data]
>
> > > CWD xxxx
> >
>
> > * ftp_perform ends with SECONDARY: 0
>
> > { [5 bytes data]
>
> > < 250 CWD successful. "/xxxx" is current directory.
>
> > } [5 bytes data]
>
> > > EPSV
> >
>
> > * Connect data stream passively
>
> > { [5 bytes data]
>
> > < 229 Entering Extended Passive Mode (|||5004|)
>
> > * Trying xx.xx.xx.xx...
>
> > * TCP_NODELAY set
>
> > * Connecting to xx.xx.xx.xx (xx.xx.xx.xx) port 5004
>
> > * Connected to regionftp (xx.xx.xx.xx) port 990 (#0)
>
> > } [5 bytes data]
>
> > > TYPE A
> >
>
> > { [5 bytes data]
>
> > < 200 Type set to A
>
> > } [5 bytes data]
>
> > > SIZE xxxxx
> >
>
> > { [5 bytes data]
>
> > < 213 310225
>
> > } [5 bytes data]
>
> > > RETR xxxxx
> >
>
> > { [5 bytes data]
>
> > < 150 Connection accepted
>
> > * Maxdownload = -1
>
> > * Getting file with size: -1
>
> > * Doing the SSL/TLS handshake on the data stream
>
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>
> > * SSL re-using session ID
>
> > } [5 bytes data]
>
> > * TLSv1.0 (OUT), TLS handshake, Client hello (1):
>
> > } [232 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Server hello (2):
>
> > { [81 bytes data]
>
> > * TLSv1.0 (IN), TLS change cipher, Change cipher spec (1):
>
> > { [1 bytes data]
>
> > * TLSv1.0 (IN), TLS handshake, Finished (20):
>
> > { [16 bytes data]
>
> > * TLSv1.0 (OUT), TLS change cipher, Change cipher spec (1):
>
> > } [1 bytes data]
>
> > * TLSv1.0 (OUT), TLS handshake, Finished (20):
>
> > } [16 bytes data]
>
> > * SSL connection using TLSv1.0 / AES256-SHA
>
> > * Server certificate:
>
> > * subject: OU=Domain Control Validated; CN=*.????.net
>
> > * start date: Feb 5 15:50:15 2019 GMT
>
> > * expire date: Mar 8 20:21:38 2021 GMT
>
> > * issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=
> > http://certs.godaddy.com/repository/ ; CN=Go Daddy Secure
> > Certificate Authority - G2
>
> > * SSL certificate verify result: unable to get local issuer
> > certificate (20), continuing anyway.
>
> > 0 302k 0 0 0 0 0 0 --:--:-- 0:00:02
> > --:--:-- 0{ [5 bytes data]
>
> > 65 302k 65 198k 0 0 1641 0 0:03:09 0:02:03
> > 0:01:06 0* Remembering we are in dir "xxxx/"
>
> > } [5 bytes data]
>
> > * TLSv1.0 (OUT), TLS alert, close notify (256):
>
> > } [2 bytes data]
>
> > < 426 Connection timed out, aborting transfer
>
> > * server did not report OK, got 426
>
> > 65 302k 65 198k 0 0 1640 0 0:03:09 0:02:03
> > 0:01:06 0
>
> > * Connection #0 to host regionftp left intact
>
> > curl: (18) server did not report OK, got 426
>

> > Result = 0
>

> > --
>

> > $ curl --version
>
> > curl 7.63.0 (i686-pc-sco3.2v5.0.7) libcurl/7.63.0 OpenSSL/1.0.2q
> > zlib/1.2.11 libssh2/1.8.0
>
> > Release-Date: 2018-12-12
>
> > Protocols: dict file ftp ftps gopher http https imap imaps pop3
> > pop3s
> > rtsp scp sftp smtp smtps telnet tftp
>
> > Features: NTLM NTLM_WB SSL libz TLS-SRP UnixSockets HTTPS-proxy
>
> I went through the latest code and I don't see a way it could happen
> that that command showed curl: (#) message but returned a different
> exit code. If it's reproducible try with the latest curl. Are you
> sure about the code fragment? Is it possible that your curl is
> actually a wrapper script around curl that maybe doesn't record the
> error code? What happens if you do curl -asdf
The actual script has no references to PATH, and is run by cron.

`which` when run by cron, writes: /bin/curl

$ ls -l /bin/curl
lrwxrwxrwx 1 root sys 19 Feb 5 20:02 /bin/curl -> /usr/local/bin/curl

$ file /usr/local/bin/curl
/usr/local/bin/curl: ELF 32-bit LSB executable 80386, dynamically linked, not stripped, debug

I compiled this binary myself. No, it is not a shell and was built from official curl sources. I
did not patch it in any way.

The script the snippet was taken from has no manipulation of PATH in it.

The snippet expanded to include context:

if true
then
# Use curl; required to support ftps.
#
# Load site-specific data.
#
LOC_DATA="/path/to/data"
if [ -s "${LOC_DATA}" ]
then
. ${LOC_DATA}
else
echo
echo "Local data file missing or empty (${LOC_DATA})"
echo
exit
fi

__CWD=`pwd`
cd "${LOC_PATH}"
curl -v -k --use-ascii -u "${GET_USER}:${GET_PASS}" -o "${GET_DST}" \
"ftps://${GET_HOST}${GET_PATH:+/}${GET_PATH}/${GET_SRC}"
RET="${?}"
cd "${__CWD}"
else
cp /path/to/.netrc $HOME/.netrc
chmod 600 $HOME/.netrc
umask 000
ftp -v xx.xx.xx.xx
RET="${?}"
rm -f ${HOME}/.netrc
fi

# For the log, report the exit code returned by the file transfer utility.
(
echo
echo "Result = ${?}"
echo
) >/dev/stderr

Neither GET_PATH nor GET_SRC contains glob characters, but if globbing
was "on" for some reason, the loop in tool_operate.c would not exit
for error 18 and conceptually goes back around a loop again.

Sure, I didn't compile the last 7.64 release yet. A quick, inexperienced glance
at the code changes in https://github.com/curl/curl/compare/curl-7_63_0...master
where it seems the curl: message is output, doesn't seem to show post-7.63.0
changes to warrant thinking newer than 7.63.0 would change the behavior.

I'm not sure its practical to try to reproduce a transient error that was caused
by unknown factors. I have no idea what might have happened to lead to
that particular scenario. I don't have control of the far-end server, or
infrastructure between it an this system. The code has worked fine for a
month or more on several servers. Sadly, it seems faster to distrust curl and
scrape console spew even though its annoying to have to do.

I think `curl -asdf` is a suggestion to continue the aborted download to see
if the rest of the file can be obtained. This fault occurred during an automated
process I was notified after support personnel cleared the error condition. To
try that, I guess I'd have to analyze console output in the script and spawn
a nother command... but it seems simpler just to retry the transmission, so
I'm not sure how that is supposed to help. The file is small enough that
retry is not at all a problem.

-- 
Kevin R. Bulgrien 

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2019-03-23