Steve Dekorte wrote:
>
> Ben Greear wrote:
> > If you use NON-Blocking IO it doesn't really matter because you will
> > never block on the socket. Of course, you have to buffer non-written
> > bytes in memory so you have to be careful about that...
>
> Right, you won't block, but you can bring your system to a virtual
> standstill via paging if you try to put more data into a socket
> buffer(or program memory) than you have available memory. (assuming your
> disk is much faster than your connection speed to the receiver) Also,
> even if the socket operation isn't blocking, the file operation is. So
> if your curl_handle is pulling data off a drive, it will block for as
> long as that operation takes(which can be a long time for a large file).
>
> Steve

I would think the send algorithm should look something like:

int doSendStuff() {
  getSomeBytesIntoTheBuffer(max_length);
  sendAsManyBytesAsWeCanWithoutBlocking();
  return;
}

The getSomeBytes is the read callback, so user code can either be reading
from disk or doing any other special stuff to get bytes there fast and
efficient. The read callback should never read more than some fixed amount,
and that amount is probably 32k or less. It definately isn't 1MB.

So basically, the curl handle should never block long on a file read because
it should never try to read the whole file at once (NFS of course can screw
things up...but that's a special case.)

---
Ben Greear <greearb_at_candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear