On Thu, Dec 27, 2012 at 07:36:00PM -0600, Derek Martin wrote:
> 1. It calls realloc() every time through the callback. This is
> terribly inefficient, though it appears to be somewhat required by
> libcurl, since (as I found out in my testing) it does not
> necessarily make the content-length available to you until after
> perform() returns.
> The down side of calling realloc() repeatedly is that it
> contributes to memory fragmentation and can cause your data to be
> copied around multiple times. Neither are especially conducive to
> good performance.

getinmemory.c is designed to be an easy-to-understand demonstration
program, not a production-ready one. But, it's not as inefficient as you
might think given a reasonable realloc() implementation. The one glibc
uses, for example, only actually reallocs at an O(log n) rate.

> Since you SHOULD be able to get the content
> length (from the Content-Length header) before you start reading
> the body, it should be possible to avoid doing a realloc() EVER.
> In practice, libcurl sometimes does, and sometimes does not make
> the content length available prior to reading the body via
> curl_easy_getinfo(). You either get the actual value, or zero.
>
> 2. If you point getinmemory.c at, say, www.google.com (or several
> other sites I tried, including one of my own), you get no results.
> It just prints "0 bytes retrieved" and dies a happy death.
> Clearly, this is the Wrong Thing¢â. Though to be honest, it's not
> clear to me why it does that. It does work with most sites I
> tried... including others of my own.

Have you looked at the headers from the server in these cases? You'll
probably find that there is no Content-Length: header sent. The Web 2.0
world of dynamically-generated pages means that at many sites even the
server doesn't know what the page length is going to be until it's
completely sent.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-12-28