cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl_multi_socket_action performance problem

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sun, 6 May 2012 23:35:39 +0200 (CEST)

On Sun, 6 May 2012, Hershberg, Josh wrote:

> I have things set up so that I receive the timeout callback from the multi
> handle. On receiving the timeout I call curl_multi_socket_action with
> CURL_SOCKET_TIMEOUT. The problem is that if a large number of easy handles
> are added before the timeout fires the call to curl_multi_socket_action
> blocks for a very long time since it iterates through all the sockets and
> connects them, etc.

First, all added handles have a timeout at 1ms set by default so you are
suggesting you're adding very many handles during that millisecond? But sure,
if your app adds many handles at once there will be lots to deal with at
once...

> Needless to say, this can take quite a while and everything else is blocked
> while this is happening.

Well, "quite a while" is not very long - or shouldn't be at least. If it takes
long, I think you're using a synchronous name resolver and that's really hard
to do much about other than suggesting you rebuild to use a more asynchronous
way. With an asynch name resolver I don't see how one call should take a very
long time without it being a bug. Of course I don't know how long "quite a
while" is.

> Is there no way to make this function faster than to simply not accept new
> requests until the socket callback is called?

When a timeout has triggered the curl_multi_socket_action() call, it really
has to perform a certain set of actions and it must do that for all handles
that have timed out. At least I've not figured out any shortcuts for it.

I think I need to understand (A) your problem better, (B) your name resolver
backend and (C) what a long time is - to be able to help out more.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2012-05-06