On Sun, 17 Mar 2002 theexperts_at_allprodirect.com wrote:

> Esteemed cURLers,
>
> The vast majority of the time required by my application is in waiting for
> the responses from the remote servers. Hence, it really is necessary to
> submit multiple requests and let the remote servers work on them
> simultaneously. Then my script can deal with the results in the order in
> which they are received.

I'm not following the logic here. If there's one single request to one site
that takes a long time, why would issuing multiple requests for smaller
pieces to that same site be faster?

Not to mention that it breaks what RFC2616 says:

   A single-user client SHOULD NOT maintain more than 2 connections with
   any server or proxy.

> I.e.: using a repeating loop keeping track of the current stage of
> progress, submit request to each remote server, check each one for a
> response, read a chunk of the response, if the end of file is reached then
> process the response and output the results.
>
> How would this be implemented with curl, specifically libcurl? After
> searching all over, I have found several references to it, but no
> explanations of how to do it. Perhaps using curlopt_progressfunction or
> curlopt_file/_range/_resume_from? Or does it need to be done with the
> command line interface?
>
> If I have to do it without curl, that's okay, but I would rather not get
> into doing that unless I have to.

Ok, let's take one thing at a time and I'll reply from a libcurl angle,
ignoring for the moment what you may or may not be able to do with the PHP
binding.

Getting ranges: set a byte range to get with CURLOPT_RANGE, and if the remote
server supports it (if we're talking HTTP here), you get the specified range
only.

Multiple simultaneous requests: if you intend to do this using the curl easy
interface, you must pop up one new thread for each concurrent transfer you
intend to do.

Multiple simultaneous requests, using a different approach: the curl multi
interface. While not being announced, it is included in libcurl since a while
back. Using the multi interface, you can do multiple transfers using one
single thread.

For further details, just ask.

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/