On Mon, Jul 08, 2013 at 06:06:25PM -0400, Keyur Govande wrote:
> Thanks Dan for the detailed response.
>
> I agree that if feasible, what you suggested would work well. But in
> most large pre-existing code bases, there is no way to keep calling
> cURL function when you get a few class hierarchies deep and passing
> along the cURL object along all the way through to do so seems leaky
> and would involve lots of changes.

There's always global variables <ducks>

> I realize libcurl is not the answer to every problem :-) This
> statement (Enable a "pull" interface. The application that uses
> libcurl decides where and when to ask libcurl to get/send data) in
> Objectives on http://curl.haxx.se/libcurl/c/libcurl-multi.html was
> what led me down the road: ask libcurl to first send, and then the app
> will ask it to receive when it is ready.

That's how it works. It's just that neither your app nor libcurl has control
over *how much* data is able to be sent or received at any moment, e.g. if the
DNS hasn't resolved yet or the socket isn't connected yet, the answer is 0 to
both.

> To take your example, a more conventional RPC would look like:
> 1) DNS lookup + connect in 10ms
> 2) send request in 20ms
> (external service takes 200ms to respond)
> 3) poll() with timeout. If timeout, assume RPC failed and move on.
> 4) If #3 succeeded, receive response in 20ms
> 5) close connection
>
> With the goal being to run other code after step 2 and before step 3
> while the external service is still processing.

True, but the normal case isn't as much fun as the worst case! I was just
trying to show that libcurl can be used efficiently in both the worst and
typical cases.

> To solve my immediate problem where connect() takes long(er) and to
> still use libcurl, the solution seems to be to pass in a connected
> socket file descriptor using CURLOPT_OPENSOCKETFUNCTION and
> CURLOPT_SOCKOPTFUNCTION, and assume when CURLM_OK is returned from
> curl_multi_perform(), the entire request has been flushed over. Then
> call curl_multi_wait() when I'm ready to receive the response. The
> other option is to use a different library or raw sockets to do
> exactly what I need.

I don't think you can necessarily make that assumption. It might be true in
some cases, but not all (think about proxies, TLS, large headers,
etc). There was some talk here a couple of weeks ago about creating a new
curl_easy_getinfo option to return the state of the transfer. If you were sure
that the largest wait was going to be in the turnaround between sending the
request and waiting for the response, then you could use that (future) option
to just loop until that state is reached, do stuff, then continue looping until
the transfer is done.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-07-09