cURL / Mailing Lists / curl-library / Single Mail


Re: [] Missing Newline after chunk data

From: Tim Heckman <>
Date: Mon, 26 Mar 2012 00:29:08 -0400

My apologies if this ends up at wrong place in the mailing list. For some reason an email wasn't sent to me when a reply was made. As such, I was unable to directly reply to the previous response by Guenter.

I've tracked it down, or so I believe, to LWP+https being the cause here. I've managed to replicate the same issue on Debian 6.0 (Squeeze) as well as the OS X Lion system I run. As best I can tell, when LWP is downloading the file over HTTPS sometimes the request ends early. I was not able to replicate using curl, wget, or Google Chrome.

I tested this my commenting out the '$ua->mirror($url, $txt);' line, and replacing it with the following line:

    print $ua->get($url)->content . "\n"; exit(0);

The first time I ran the script the entire file was downloaded. However, it was a fluke, as the following request attempts varying amounts of the file contents were downloaded. And I was never able to print the full file again. I even tried playing with the "max_size" option in UserAgent. This had no affect, as expected, because the "max_size" option defaults to undef (unlimited). When switching to http, it works perfectly every single time.

I've tried to do some digging and have been unable to find a discussion regarding the change from HTTP to HTTPS. Additionally, I can't say I see any benefit of doing so as the information is publicly accessible. I would be interested in why it should not remain HTTP.


Tim Heckman

List admin:
Received on 2012-03-26