cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: How can I GET a gzipped static resource and have libcurl uncompress it?

From: Mohun Biswas <m_biswas_at_mailinator.com>
Date: Fri, 12 Jun 2009 19:49:27 -0400

Daniel Stenberg wrote:
> On Wed, 10 Jun 2009, Mohun Biswas wrote:
>
>> problem is that as far as I can tell, libcurl won't uncompress it
>> unless it sees the Content-Encoding header, and there's no way to add
>> a custom header to a static resource.
>
> Can't you trick the server into providing one?

I don't think so. For an individual server, it might be possible; I
wouldn't be surprised, for instance, if Apache could be configured to
say "if a file has a .gz extension, serve it with a 'Content-Encoding:
gzip' header" (in fact all gzipped files will have a .gz extension).

However, what I have is a custom client program (using libcurl) plus a
standard web app in a .war package. The whole idea of these .war
packages is that you can drop them into any servlet container and they
will work. Even if a particular container, such as Tomcat, could be
tweaked to do this I'd be giving up container independence and requiring
manual reconfiguration even for the supported containers. So it's a
problem I'd really prefer to solve from the client end where I have
complete control over the code.

>> Am I missing something? All I want is to tell the libcurl library
>> "Here's the URL of a compressed file. Go get it and decompress it".
>
> No, there's no such option. I can see how it could be useful in
> situations such as yours though...

It sounds like you're saying you'd consider a patch but that that's the
only way the functionality would be available. In the meantime, however,
I've done some more profiling (thanks to CURLINFO_CONNECT_TIME etc) and
discovered that the connection costs are only about %10 of the overhead.
So although I may come back to this, because 10% isn't 0%, I think I'll
be looking at the other 90% first.

This assumes that servlet startup overhead would be counted, from the
client perspective, as part of the connection time. I don't know if that
assumption is justified.

MB
Received on 2009-06-13