On Sat, 16 Aug 2008, Cory Nelson wrote:

>>> the existing buffer instead of reading or writing, but is otherwise the
>>> same.
>>
>> I don't quite understand this brief description. Can you add some more
>> psuedo code for a client using this suggested API?
>
> The good thing is that curl's public API doesn't need to change one bit to
> use IOCP - curl_multi_perform() is the equivalent of the d.run() above.
> How much the internal code would need to change is another question though!

Ok, I think I start to understand things. In order to get really high
performance on Windows for C10k (10000+ connections), we need this IOCP
approach.

For an ordinary application with < 100 connections, why would an app
particularly insist on such an asynch API for libcurl? Also, given the
portable nature of libcurl, I don't see how we could even dream of providing
an "asynch API" (unless we do threads of course).

>> In my view, asynchronous is mostly just another word for running the stuff
>> in another thread until it has something, and then have a means of telling
>> the first thread when it is done. And you can use libcurl fine already for
>> doing exactly that.
>
> Well, the current way of doing it (select) supports that model too -- it's
> just not anything close to IOCP's efficiency which scales to tens of
> thousands of concurrent operations.

I wouldn't call libcurl's current way to be select-based. libcurl can be
select-based indeed, but libcurl has successfully been used with several tens
of thousands of connections just fine using its multi_socket API that is
portable and functional on a large amount of operating systems. Including
windows.

What possibly is lacking on Windows, as Jamie Lokier possibly suggests, is an
event-based system that is good enough or suitable to do the job in a
convenient manner!

> asio does some magic to make a truly async OpenSSL socket (see
> asio::ssl::stream), but OpenSSL's API, much like almost every other protocol
> library, is not really designed well for this kind of async.

Right, but SSL (with OpenSSL or the other SSL libs) and SSH based protocols
are supported by libcurl's high performance API...

So, the notion that we "need an asynch API" is thus simply based on the fact
that you need to use it on Windows to get high performance when using large
amounts of connections?

-- 
  / daniel.haxx.se