curl won't reuse connections (using curl_easy)

Hi,

I'm new here, so let me know if the question is inappropriate or anything.
Thanks in advance for your time in reading this.

I'm writing code with the curl_easy interface, and am having a hard time
getting it to reuse connections. Details below.

curl --version
curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11
libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3
pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB
SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

My code runs through a series of unit tests all of which have to connect to
the same host over H2, so I've set up persistent connections. I call
curl_easy_init once, then all the global curl_easy_setopt values, then in a
loop, I make the requests by calling curl_easy_setopt to set post fields,
url, content length, and then curl_easy_perform.

I only call curl_easy_cleanup after all requests have been made.

I have the following options set:

CURLOPT_FOLLOWLOCATION => 1
CURLOPT_SSL_VERIFYPEER => 1
CURLOPT_SSL_VERIFYHOST => 2
CURLOPT_SSLVERSION 7<<16
CURLOPT_HTTP_VERSION = 4
CURLOPT_TCP_FASTOPEN = 1
CURLOPT_TCP_KEEPALIVE = 1
CURLOPT_ACCEPT_ENCODING = ""
CURLOPT_TRANSFER_ENCODING = 1
CURLOPT_POST = 1
CURLOPT_DNS_CACHE_TIMEOUT = 0
CURLOPT_VERBOSE = 1
CURLOPT_SSLCERT
CURLOPT_SSLKEY
CURLOPT_CAINFO

Verbose output looks like this:

* TCP_NODELAY set
* TCP_FASTOPEN_CONNECT set
* Connected to XYZ () port 443 (#0)
* found 3 certificates in /data/certs/ca_chain.pem
* found 401 certificates in /etc/ssl/certs
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* error fetching CN from cert:The requested data were not available.
* common name: (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject:
* start date: Mon, 13 Jul 2020 08:04:30 GMT
* expire date: Mon, 20 Jul 2020 08:05:00 GMT
* issuer: C=US,ST=MA,L=Cambridge,O=XXXXX,CN=YYYYY
* compression: NULL
* ALPN, server accepted to use h2
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=0
* Using Stream ID: 1 (easy handle 0x9036570)
> POST /foo/ HTTP/2

<Request>

* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!

<Response>

* Connection #0 to host XYZ left intact

* Found bundle for host XYZ: 0xe141520 [can multiplex]
* Connection #0 is still name resolving, can't reuse
* Trying IP...
* TCP_NODELAY set
* TCP_FASTOPEN_CONNECT set
* Connected to XYZ () port 443 (#1)
* found 3 certificates in /data/certs/ca_chain.pem
* found 401 certificates in /etc/ssl/certs
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL re-using session ID
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* error fetching CN from cert:The requested data were not available.
* common name: (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject:
* start date: Mon, 13 Jul 2020 08:04:30 GMT
* expire date: Mon, 20 Jul 2020 08:05:00 GMT
* issuer: C=US,ST=MA,L=Cambridge,O=XXXXX,CN=YYYYY
* compression: NULL
* ALPN, server accepted to use h2
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=0
* Using Stream ID: 1 (easy handle 0x9036570)

<Request>

* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< vary: accept-encoding
< content-encoding: gzip
< content-type: text/plain
< date: Mon, 13 Jul 2020 19:30:36 GMT
< server: abcdefg
<Response>

* Connection #1 to host XYZ left intact

* Found bundle for host XYZ: 0xe141520 [can multiplex]
* Connection #0 is still name resolving, can't reuse
* Connection #1 is still name resolving, can't reuse

And so on...

It fairly soon gets to this:

* Connection cache is full, closing the oldest one.
* Closing connection 0
* Connection #5 to host XYZ left intact

This closing of the oldest connection continues from here on in until
either everything is done, or a TLS handshake error puts the connection
into an inconsistent state.

Any idea what I can do to get curl to reuse the H2 connection to the host?

Thanks for your time.

Philip

--
hello world

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-07-13