Sending a largish POST request with --limit-rate delays receiving of small server response

From: Fabian Keil via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 26 Mar 2021 09:17:06 +0100

For a Privoxy regression test I'm trying to send a POST request slowly.

I'm experimenting with <command>s like:
--limit-rate 1k -d "blafasel%repeat[5000 x padding]%" http://%HOSTIP:%HTTPPORT/%TESTNUMBER

Unfortunately the Privoxy log indicates that the whole
request always arrives within a second.

This is reproducible without using Privoxy as proxy.

Apparently the rate limit only kicks in after the request
has been sent and as a result the (small) response is then
received delayed.

The attached patch (gzipped to preserve line endings) adds
a test that reproduces the two related issues.

Excerpt from the trace file:

08:34:20.493951 == Info: STATE: PROTOCONNECT => DO handle 0x7492fe8; line 1848 (connection #0)
08:34:20.516824 => Send header, 159 bytes (0x9f)
0000: POST /494 HTTP/1.1
0014: Host: 127.0.0.1:11820
002b: User-Agent: curl/7.76.0-DEV
0048: Accept: */*
0055: Content-Length: 20000
006c: Content-Type: application/x-www-form-urlencoded
009d:
08:34:20.521942 => Send data, 20000 bytes (0x4e20)
0000: datadatadatadatadatadatadatadatadatadatadatadatadatadatadatadata
0040: datadatadatadatadatadatadatadatadatadatadatadatadatadatadatadata
[...]
4dc0: datadatadatadatadatadatadatadatadatadatadatadatadatadatadatadata
4e00: datadatadatadatadatadatadatadata
08:34:20.653015 == Info: STATE: DO => DID handle 0x7492fe8; line 1922 (connection #0)
08:34:20.654085 == Info: STATE: DID => PERFORMING handle 0x7492fe8; line 2041 (connection #0)
08:34:20.656013 == Info: STATE: PERFORMING => RATELIMITING handle 0x7492fe8; line 2126 (connection #0)
08:34:39.861771 == Info: STATE: RATELIMITING => PERFORMING handle 0x7492fe8; line 2090 (connection #0)
08:34:39.940084 == Info: Mark bundle as not supporting multiuse
08:34:39.944398 == Info: HTTP 1.1 or later with persistent connection
08:34:39.961874 <= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
08:34:39.994767 <= Recv header, 19 bytes (0x13)
0000: Connection: close
08:34:40.004723 <= Recv header, 25 bytes (0x19)
0000: Content-Type: text/html
08:34:40.006156 <= Recv header, 24 bytes (0x18)
0000: X-Connection: swsclose
08:34:40.022740 <= Recv header, 2 bytes (0x2)
0000:
08:34:40.030478 <= Recv data, 21 bytes (0x15)
0000: Received your input..
08:34:40.066967 == Info: nread <= 0, server closed connection, bailing
08:34:40.074474 == Info: STATE: PERFORMING => DONE handle 0x7492fe8; line 2239 (connection #0)
08:34:40.076919 == Info: multi_done
08:34:40.115687 == Info: The cache now contains 0 members
08:34:40.121391 == Info: Closing connection 0

While the test passes the behaviour shown doesn't seem ideal to me.

Fabian


-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-03-26