On Mon, Jul 8, 2013 at 6:25 PM, Dan Fandrich <dan_at_coneharvesters.com> wrote:
> On Mon, Jul 08, 2013 at 06:06:25PM -0400, Keyur Govande wrote:
>> Thanks Dan for the detailed response.
>>
>> I agree that if feasible, what you suggested would work well. But in
>> most large pre-existing code bases, there is no way to keep calling
>> cURL function when you get a few class hierarchies deep and passing
>> along the cURL object along all the way through to do so seems leaky
>> and would involve lots of changes.
>
> There's always global variables <ducks>

Nice try! :-D

>> I realize libcurl is not the answer to every problem :-) This
>> statement (Enable a "pull" interface. The application that uses
>> libcurl decides where and when to ask libcurl to get/send data) in
>> Objectives on http://curl.haxx.se/libcurl/c/libcurl-multi.html was
>> what led me down the road: ask libcurl to first send, and then the app
>> will ask it to receive when it is ready.
>
> That's how it works. It's just that neither your app nor libcurl has control
> over *how much* data is able to be sent or received at any moment, e.g. if the
> DNS hasn't resolved yet or the socket isn't connected yet, the answer is 0 to
> both.
>
>> To take your example, a more conventional RPC would look like:
>> 1) DNS lookup + connect in 10ms
>> 2) send request in 20ms
>> (external service takes 200ms to respond)
>> 3) poll() with timeout. If timeout, assume RPC failed and move on.
>> 4) If #3 succeeded, receive response in 20ms
>> 5) close connection
>>
>> With the goal being to run other code after step 2 and before step 3
>> while the external service is still processing.
>
> True, but the normal case isn't as much fun as the worst case! I was just
> trying to show that libcurl can be used efficiently in both the worst and
> typical cases.
>
>> To solve my immediate problem where connect() takes long(er) and to
>> still use libcurl, the solution seems to be to pass in a connected
>> socket file descriptor using CURLOPT_OPENSOCKETFUNCTION and
>> CURLOPT_SOCKOPTFUNCTION, and assume when CURLM_OK is returned from
>> curl_multi_perform(), the entire request has been flushed over. Then
>> call curl_multi_wait() when I'm ready to receive the response. The
>> other option is to use a different library or raw sockets to do
>> exactly what I need.
>
> I don't think you can necessarily make that assumption. It might be true in
> some cases, but not all (think about proxies, TLS, large headers,
> etc). There was some talk here a couple of weeks ago about creating a new
> curl_easy_getinfo option to return the state of the transfer. If you were sure
> that the largest wait was going to be in the turnaround between sending the
> request and waiting for the response, then you could use that (future) option
> to just loop until that state is reached, do stuff, then continue looping until
> the transfer is done.
>

Yeah, in the pathological case, there's tons of stuff that can go
wrong. I too dislike making the implicit assumption and love the the
idea of an 'info' option; I hope it is trunk'ed soon...knowing exactly
the state a curl transfer is in will allow the app to make more
intelligent decisions and also help solve the issue I'm experiencing
as well.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-07-09