Re: Sending a largish POST request with --limit-rate delays receiving of small server response

From: Fabian Keil via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 26 Mar 2021 11:52:56 +0100

Daniel Stenberg <daniel_at_haxx.se> wrote on 2021-03-26:

> On Fri, 26 Mar 2021, Fabian Keil via curl-library wrote:
>
> > Apparently the rate limit only kicks in after the request has been
> > sent and as a result the (small) response is then received delayed.

> Proposed fix: https://github.com/curl/curl/pull/6797

Thanks for the quick fix.

It works for me when using the test I posted.

After also writing tests for Privoxy using https with Privoxy
https-inspecting the requests I'm still seeing an issue that
seems related.

The <command> used in the test is:
--limit-rate 4k -d "blafasel%repeat[5000 x padding]%" --insecure https://%HOSTIP:%HTTPSPORT/%TESTNUMBER

An excerpt from the trace:
11:25:30.376176 == Info: SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
11:25:30.376197 == Info: ALPN, server did not agree to a protocol
11:25:30.376232 == Info: Server certificate:
11:25:30.376302 == Info: subject: CN=127.0.0.1; O=127.0.0.1; OU=127.0.0.1; C=CZ
11:25:30.376344 == Info: start date: Feb 15 09:25:43 2021 GMT
11:25:30.376373 == Info: expire date: Jun 15 09:25:43 2021 GMT
11:25:30.376425 == Info: issuer: C=DE; ST=Some-State; O=Privoxy Test Company
11:25:30.376457 == Info: SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
11:25:30.376500 == Info: STATE: PROTOCONNECTING => DO handle 0x8030f8808; line 1864 (connection #0)
11:25:30.376908 => Send SSL data, 5 bytes (0x5)
0000: ..._at_.
11:25:30.377005 => Send SSL data, 1 bytes (0x1)
0000: .
11:25:30.377109 => Send header, 157 bytes (0x9d)
0000: POST /6 HTTP/1.1
0012: Host: 127.0.0.1:24984
0029: User-Agent: curl/7.76.0-DEV
0046: Accept: */*
0053: Content-Length: 35008
006a: Content-Type: application/x-www-form-urlencoded
009b:
11:25:30.377269 => Send data, 16227 bytes (0x3f63)
0000: blafaselpaddingpaddingpaddingpaddingpaddingpaddingpaddingpadding
[...]
3f40: paddingpaddingpaddingpaddingpadding
11:25:30.384881 == Info: STATE: DO => DID handle 0x8030f8808; line 1922 (connection #0)
11:25:30.384908 == Info: STATE: DID => PERFORMING handle 0x8030f8808; line 2041 (connection #0)
11:25:30.384927 == Info: STATE: PERFORMING => RATELIMITING handle 0x8030f8808; line 2126 (connection #0)
11:25:34.286387 == Info: STATE: RATELIMITING => PERFORMING handle 0x8030f8808; line 2090 (connection #0)
11:25:34.286504 <= Recv SSL data, 5 bytes (0x5)

So it looks like curl spent more than three seconds doing nothing.

I expected curl to write some data at least once per second
and configured Privoxy with "socket-timeout 3" which results
in the test being flaky. Increasing Privoxy's socket-timeout
works around the issue.

I haven't tried reproducing the issue without using Privoxy as proxy yet.

Fabian


-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-03-26