Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: "getaddrinfo() thread failed to start" under heavy load
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: m brandenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 17 Oct 2023 12:29:55 +0000 ()
On Mon, 16 Oct 2023, Matt Toschlog via curl-library wrote:
> I'm using libcurl in a voice server app running on Ubuntu 20.04 on an
> Amazon AWS EC2 instance. When I get around 500 users on the system I
> start getting the error "getaddrinfo() thread failed to start" in my
> curl_easy_perform() calls.
>
> Memory doesn't seem to be an issue -- I'm not going above 15%
> utilization. Perhaps there's a Linux limitation (number of threads, for
> example) that I'm running up against but I haven't found it.
>
> I'm building with curl 7.68.0, which as far as I can tell is the most
> recent version for the system I'm running on.
>
> Any ideas?
Few ideas but I can confirm. On Debian through Buster and libcurl
7.64, I've seen this on occasion. ~1000 servers with 1000s of
client connections each. I'll get a small, micro-burst of resolver
failures due to thread failure with a hint that resolver
piggy-backing may not be working correctly. Hosts are safe on
memory, process and system fd limits, and process/thread fork
limits. Another resource seems involved but haven't got beyond that.
Problem clears on retry a second or two later.
Date: Tue, 17 Oct 2023 12:29:55 +0000 ()
On Mon, 16 Oct 2023, Matt Toschlog via curl-library wrote:
> I'm using libcurl in a voice server app running on Ubuntu 20.04 on an
> Amazon AWS EC2 instance. When I get around 500 users on the system I
> start getting the error "getaddrinfo() thread failed to start" in my
> curl_easy_perform() calls.
>
> Memory doesn't seem to be an issue -- I'm not going above 15%
> utilization. Perhaps there's a Linux limitation (number of threads, for
> example) that I'm running up against but I haven't found it.
>
> I'm building with curl 7.68.0, which as far as I can tell is the most
> recent version for the system I'm running on.
>
> Any ideas?
Few ideas but I can confirm. On Debian through Buster and libcurl
7.64, I've seen this on occasion. ~1000 servers with 1000s of
client connections each. I'll get a small, micro-burst of resolver
failures due to thread failure with a hint that resolver
piggy-backing may not be working correctly. Hosts are safe on
memory, process and system fd limits, and process/thread fork
limits. Another resource seems involved but haven't got beyond that.
Problem clears on retry a second or two later.
-- Monty Brandenberg, Software Engineer MCB, Inc.
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-10-17