cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Curl retry without notifying application?

From: Mark Aldred <mark_at_twinstrata.com>
Date: Thu, 27 Aug 2009 10:22:31 -0400

This comment in Curl_retry_request() indicates that curl is doing exactly
what I thought. Sounds like it is retrying a PUT without returning an
error. This behavior is fine for GETs and DELETEs, but not PUTs.

    conn->bits.retry = TRUE; /* mark this as a connection we're about
                                to retry. Marking it this way should
                                prevent i.e HTTP transfers to return
                                error just because nothing has been
                                transfered! */

I'm trying this fix in Curl_http_done(). does this make sense?

  if(!premature && /* this check is pointless when DONE is called before the
                      entire operation is complete */
     !conn->bits.retry &&
     data->req.upload_present != 0 && ///////////////////////// ADDED
THIS LINE
     ((http->readbytecount +
       data->req.headerbytecount -
       data->req.deductheadercount)) <= 0) {
    /* If this connection isn't simply closed to be retried, AND nothing was
       read from the HTTP server (that counts), this can't be right so we
       return an error here */
    failf(data, "Empty reply from server");
    return CURLE_GOT_NOTHING;
  }

On Thu, Aug 27, 2009 at 12:59 AM, Mark Aldred <mark_at_twinstrata.com> wrote:

> I am using libcurl 7.19.6.
>
> My application uses CURLOPT_READFUNCTION and CURLOPT_WRITEFUNCTION copy the
> data into and out of curl. My callback functions us structures to track how
> much data has been copied to maintain sane pointers as the copy progresses.
> Fairly standard stuff. When curl return an error to our application my code
> resets the pointers used in the data copy and retires the curl request (for
> retryable errors). All this works fine.
>
> Today I encountered a situation where it appears I issued a PUT request to
> curl. Curl got the 100-continue from the host, invoked by copy function to
> get all the data then encountered an error. It seems that curl then retried
> the operation without returning an error to my application so that I can
> reset the copy pointers in my structures. The result is that when curl
> invoked the copy function in my application for the retry, my copy function
> returned zero because it thought (correctly) that all the data had been
> copied already. At this point the request hangs.
>
> Does curl retry requests without returning an error to the application so
> that it can prepare for the retry?
>
> Here is an excerpt from my log.
>
> 0x14f7ae0 2009-Aug-26 23:22:11.905625 CURLINFO_HEADER_OUT 23006848 PUT
> /some_url HTTP/1.1
> Accept: */*
> Authorization: AWS 1VVN4RW1K6ACA2246MR2:jONA2hLltDtR7WSR84/L0xsGH8g=
> content-length: 131072
> content-type: application/octet-stream
> date: Thu, 27 Aug 2009 03:22:11 GMT
> host: s3.amazonaws.com
> Expect: 100-continue
>
> 0x14f7ae0 2009-Aug-26 23:22:12.912369 CURLINFO_TEXT 23006848 Done waiting
> for 100-continue
> 0x14f7ae0 2009-Aug-26 23:22:12.912508 _OutboundDataCopy COPY eh:23006848
> copy_size:40960 iovec size:131072 copy size:40960 remaining:131072
> 0x14f7ae0 2009-Aug-26 23:22:12.913001 _OutboundDataCopy COPY eh:23006848
> copy_size:40960 iovec size:131072 copy size:40960 remaining:90112
> 0x14f7ae0 2009-Aug-26 23:22:12.913454 _OutboundDataCopy COPY eh:23006848
> copy_size:40960 iovec size:131072 copy size:40960 remaining:49152
> 0x14f7ae0 2009-Aug-26 23:22:12.913904 _OutboundDataCopy COPY eh:23006848
> copy_size:8192 iovec size:131072 copy size:8192 remaining:8192
> 0x14f7ae0 2009-Aug-26 23:22:12.914067 _OutboundDataCopy DONE bytes
> processed:131072 size:131072 eh:23006848
>
> 0x14f7ae0 2009-Aug-26 23:22:24.665157 CURLINFO_TEXT 23006848 SSL read:
> error:00000000:lib(0):func(0):reason(0), errno 104
> 0x14f7ae0 2009-Aug-26 23:22:24.665243 CURLINFO_TEXT 23006848 Connection
> died, retrying a fresh connect
> 0x14f7ae0 2009-Aug-26 23:22:24.665283 CURLINFO_TEXT 23006848 Closing
> connection #0
> 0x14f7ae0 2009-Aug-26 23:22:24.668640 CURLINFO_TEXT 23006848 Issue another
> request to this URL: 'https://s3.amazonaws.com:443/some_url<https://s3.amazonaws.com/some_url>'
>
> 0x14f7ae0 2009-Aug-26 23:22:24.668719 CURLINFO_TEXT 23006848 Re-using
> existing connection! (#1) with host s3.amazonaws.com
> 0x14f7ae0 2009-Aug-26 23:22:24.668757 CURLINFO_TEXT 23006848 Connected to
> s3.amazonaws.com (72.21.202.97) port 443 (#1)
>
> 0x14f7ae0 2009-Aug-26 23:22:24.741157 CURLINFO_HEADER_IN 23006848 HTTP/1.1
> 100 Continue
> 0x14f7ae0 2009-Aug-26 23:22:24.753208 _OutboundDataCopy DONE bytes
> processed:131072 size:131072 eh:23006848
>
> One last thing. What is this SSL read error?
>
> Thank you for any help you can provide.
>
>
> --
> Mark
>
>

-- 
Mark
Received on 2009-08-27