Thanks for the response! The AWS ELB had been configured with a TCP SSL
listener and the issue manifested itself when the instances behind the ELB
were being replaced - since switching to a HTTPS listener, the problem has
gone away. I didn't get around to installing tcpdump on my embedded device
to monitor the traffic, but I suppose the ELB was doing something poor.

Related question - since moving to the HTTPS listener, things have been
better, but I did run into an issue where a similar persistent GET never
recovered. I strace'd the process and it appeared to be continuously
receiving SIGPIPE's but not exiting curl_easy_perform() (this was after
over 48 hours of running):

poll([{fd=14, events=POLLIN}], 1, 1000) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {323780, 656612984}) = 0
clock_gettime(CLOCK_MONOTONIC, {323780, 657165992}) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, 8) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, NULL, 8) = 0
poll([{fd=14, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0
(Timeout)
clock_gettime(CLOCK_MONOTONIC, {323780, 658354855}) = 0
clock_gettime(CLOCK_MONOTONIC, {323780, 658636234}) = 0
clock_gettime(CLOCK_MONOTONIC, {323780, 658934070}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {323780, 659530997}) = 0
clock_gettime(CLOCK_MONOTONIC, {323780, 659814932}) = 0
poll([{fd=14, events=POLLIN}], 1, 1000) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {323781, 666977604}) = 0
clock_gettime(CLOCK_MONOTONIC, {323781, 667299002}) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, 8) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, NULL, 8) = 0
poll([{fd=14, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0
(Timeout)
clock_gettime(CLOCK_MONOTONIC, {323781, 668484456}) = 0
clock_gettime(CLOCK_MONOTONIC, {323781, 668762235}) = 0
clock_gettime(CLOCK_MONOTONIC, {323781, 669049861}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0xb6d6a0a1}, NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {323781, 669663769}) = 0
clock_gettime(CLOCK_MONOTONIC, {323781, 669813457}) = 0

I have not set NOSIGNAL and I can see in the source that SIGPIPE's will be
ignored, but was curious how normal operation could have gotten into this
state. I'm now running with NOSIGNAL and plan on installing a handler to
retrying the call to curl_easy_perform() (or maybe I need to teardown and
setup again?)

Thanks for your response!

Mark

On Tue, May 2, 2017 at 7:17 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Wed, 26 Apr 2017, Mark Fine via curl-library wrote:
>
> I'm making a persistent PUT to a TCP AWS ELB - it's an endless request
>> that is continuously sending data through a CURLOPT_READFUNCTION callback.
>> During a deploy or some other event I lose connectivity to the AWS ELB and
>> my socket enters CLOSE_WAIT - but my curl request continues to write data
>> to the socket indefinitely.
>>
>
> Wow. How is that even possible? If there's no way to send the data I would
> expect an error sooner or later to get returned from the send() calls
> libcurl uses. It makes me curious where the data goes when it can't send it
> anywhere...

Yes, this was extremely puzzling - it's almost as if the connection was
maintained on the ELB event as the instance was being replaced.

> I have a CURLOPT_XFERINFOFUNCTION callback that also continues to report
>> bytes uploaded!! How can I break out of this situation and end the request?
>>
>
> libcurl is supposed to detect errors and bail out. A closed connection
> should cause such an error.
>
> In my callback should I see if the socket has gone into CLOSE_WAIT?
>>
>
> No, that's not exposed.
>
> Is it possible to do something better with the multi interface - I
>> couldn't tell looking at the source code that it would behave any
>> differently.
>>
>
> No, it's meant to work similarly fine using either interface!
>
> --
>
> / daniel.haxx.se
>

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-05-02