Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Turning off async resolution at runtime
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 2 Feb 2026 23:19:39 +0100 (CET)
On Thu, 29 Jan 2026, Andrew Bell via curl-library wrote:
> I have an application that is making many http requests through curl. Each
> request is handled by a thread. On my OS (OSX) there is a default file
> descriptor limit of 256. When I make a curl request with curl_easy_perform,
> it creates two pipes in addition to creating the socket on which to make the
> request. It appears that these pipes are made for communication internal to
> curl about async DNS resolution. This means that I'm using three file
> descriptors for each request, which means get "out of file descriptor"
> errors when my application has ~80 threads making curl requests.
The socketpair file descriptors are however only used while name resolving.
> It looks like I can disable async resolution at compile-time, which I would
> think would eliminate the pipes and associated file descriptors, but I can't
> control how the library was built for users. I don't see any way I can
> disable this async resolution at runtime, but perhaps I'm missing something.
Nope. Because using sync name resolving is terrible and is normally only used
as a worst-case fallback.
For example: timeouts for sync name resolving is bad, uses signals and is
error-prone. It messes up libcurl's ability to work asynchronously.
> Does anyone have any ideas/recommendations to eliminate the FDs being used
> for DNS resolution or some other ideas on how I can reduce the likelihood of
> running out of FDs when using lots of simultaneous requests?
I would recommend using the multi interface and avoid threads. It probably
also increases performance.
Date: Mon, 2 Feb 2026 23:19:39 +0100 (CET)
On Thu, 29 Jan 2026, Andrew Bell via curl-library wrote:
> I have an application that is making many http requests through curl. Each
> request is handled by a thread. On my OS (OSX) there is a default file
> descriptor limit of 256. When I make a curl request with curl_easy_perform,
> it creates two pipes in addition to creating the socket on which to make the
> request. It appears that these pipes are made for communication internal to
> curl about async DNS resolution. This means that I'm using three file
> descriptors for each request, which means get "out of file descriptor"
> errors when my application has ~80 threads making curl requests.
The socketpair file descriptors are however only used while name resolving.
> It looks like I can disable async resolution at compile-time, which I would
> think would eliminate the pipes and associated file descriptors, but I can't
> control how the library was built for users. I don't see any way I can
> disable this async resolution at runtime, but perhaps I'm missing something.
Nope. Because using sync name resolving is terrible and is normally only used
as a worst-case fallback.
For example: timeouts for sync name resolving is bad, uses signals and is
error-prone. It messes up libcurl's ability to work asynchronously.
> Does anyone have any ideas/recommendations to eliminate the FDs being used
> for DNS resolution or some other ideas on how I can reduce the likelihood of
> running out of FDs when using lots of simultaneous requests?
I would recommend using the multi interface and avoid threads. It probably
also increases performance.
-- / daniel.haxx.se || https://rock-solid.curl.dev -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2026-02-02