On Wed, Aug 8, 2018 at 10:56 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Tue, 7 Aug 2018, James Read wrote:
>
> As long as libcurl is built to use the threaded resolver (default) or the
>>> c-ares backend, it will do name resolving in a non-blocking manner.
>>>
>>
>> I just did a standard install using the latest git clone.
>>
>
> Then it probably built with the threaded resolver. At the end of the
> configure run it says which one it will build with (and the curl -V output
> subtly reveals the info too).
>
>
Output of ./configure says:

resolver: POSIX threaded

which makes me think threaded resolver.

Output of curl -V says:

Features: AsynchDNS

which makes me think c-ares.

How do I compile with c-ares? I'm guessing c-ares gives the best
performance.

> On another note. I've just tested ephiperfifo with a 100k domains.
>> Throughput is below what I was expecting (hoping for). It started off at
>> about 600KB/sec and then dropped to around 40KB/sec.
>>
>
> There are like a million factors that all work together and in this
> statement you put them all together.
>
>
Is there a way of limiting the number of sockets dealt with at any time?

> First, I think you should understand the API properly and make sure that's
> used correctly. Then you should start small and get data from URLs that
> really will give you high speed transfers

I have no way of knowing in advance which URLs would do this. And in the
long term I want to download them all anyway.

> and make sure the application works correctly when using a small amount of
> transfers - perhaps a single one to start with - and then add more when
> you've confirmed that.
>
>
OK

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-08-08