On Fri, 5 Oct 2001, Lucas Adamski wrote:

> I think I noticed a small bug in url.c:

Ah, right! Thanks. That particular code is located in lib/connect.c now, but
the problem was still there.

> First of all, I noticed BUFSIZE is set to 50K by default for each
> curl_easy_init. This eats up quite a bit of ram in a multithreaded
> environment, especially in how zthreads implements c++ mulithreaded (i.e.
> all code is stored in the task class, not in the worker thread itself).
> Is there any huge disadvantage to resetting it to somewhere around 5K?
> A lot of my requests are for the header only, so I don't want to be
> wasting all that ram.

The reason for using such a huge buffer would be purely for speed reasons. We
require less read calls to get more data.

Yes, it would take some profiling to figure out whether or not this actually
makes any difference in real-world. The buffer could very well be set to 5K
without any major (if even noticable) difference.

I'd appriciate if someone would do some benchmarking or could refer to
previously made tests of this kind.

> Secondly, I made the follow hack to allow for a fixed limit on download
> size. Because I've found that some servers don't honor the Range:
> directive, and I don't want to get stuck downloading a 300 meg file by
> accident. Its currently implemented as follows, but it would be much
> nicer to implement this as a curl_easy_setopt I think:

[snip]

> Comments? Thanks,

As said before, I would indeed consider applying such a patch that offers a
CURLOPT_MAXDOWNLOAD option. If you make it work as such, I'll incorporate it
into the main sources.

We could possibly come up with a different approach that would use the given
range data in cases where the remote server doesn't support ranges. But that
would probably be harder.

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/