On Sat, Jun 20, 2009 at 4:38 PM, Raymond C. Glassford <
rcglassford_at_copper.net> wrote:

> On the following page:
>
>
> http://download.cnet.com/Macrium-Reflect-Free-Edition/3000-2242_4-10845728.html?part=dl-6288502&subj=dl&tag=button
>
> the page source gave me the following link for the download button:
>
>
> http://dw.com.com/redir?edId=3&siteId=4&oId=3000-2242_4-10845728&ontId=2242_4&spi=abf20204d70bae9a07c1a6598b0a2573&lop=btn&ltype=dl_dlnow&pid=11015803&mfgId=6288502&merId=6288502&pguid=q-Zp-AoPjF0AABWxUcYAAAFt&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-2242_4-10845728.html%3Fspi%3Dabf20204d70bae9a07c1a6598b0a2573%26part%3Ddl-6288502
>
> and, when I did the following:
>
> curl -L -m 60 -o reflect -v -# "
> http://dw.com.com/redir?edId=3&siteId=4&oId=3000-2242_4-10845728&ontId=2242_4&spi=abf20204d70bae9a07c1a6598b0a2573&lop=btn&ltype=dl_dlnow&pid=11015803&mfgId=6288502&merId=6288502&pguid=q-Zp-AoPjF0AABWxUcYAAAFt&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-2242_4-10845728.html%3Fspi%3Dabf20204d70bae9a07c1a6598b0a2573%26part%3Ddl-6288502
> "
>
> I got:
>
> * About to connect() to dw.com.com port 80 (#0)
> * Trying 216.239.122.227 ... connected
> * Connected to dw.com.com (216.239.122.227) port 80 (#0)
>
>> GET
>> /redir?edId=3&siteId=4&oId=3000-2242_4-10845728&ontId=2242_4&spi=abf20204d70bae9a07c1a6598b0a2573&lop=btn&ltype=dl_dlnow&pid=11015803&mfgId=6288502&merId=6288502&pguid=q-Zp-AoPjF0AABWxUcYAAAFt&destUrl=http%3A%2F%
>> 2Fdownload.cnet.com%2F3001-2242_4-10845728.html%3Fspi%3Dabf20204d70bae9a07c1a6598b0a2573%26part%3Ddl-6288502
>> HTTP/1.1
>> User-Agent: curl/7.19.4 (i586-pc-mingw32msvc) libcurl/7.19.4 zlib/1.2.3
>> Host: dw.com.com
>> Accept: */*
>>
>> < HTTP/1.1 302 Found
> < Date: Sat, 20 Jun 2009 13:39:43 GMT
> < Server: Apache/2.0
> < Pragma: no-cache
> < Cache-control: no-cache
> < Location:
> http://download.cnet.com/3001-2242_4-10845728.html?spi=abf20204d70bae9a07c1a6598b0a2573&part=dl-6288502
> < Content-Length: 0
> < P3P: CP="CAO DSP COR CURa ADMa DEVa PSAa PSDa IVAi IVDi CONi OUR OTRi IND
> PHY ONL UNI FIN COM NAV INT DEM STA"
> < Content-Type: text/plain
> <
> * Connection #0 to host dw.com.com left intact
> * Issue another request to this URL: '
> http://download.cnet.com/3001-2242_4-10845728.html?spi=abf20204d70bae9a07c1a6598b0a2573&part=dl-6288502
> '
> * About to connect() to download.cnet.com port 80 (#1)
> * Trying 216.239.122.178 ... connected
> * Connected to download.cnet.com (216.239.122.178) port 80 (#1)
>
>> GET
>> /3001-2242_4-10845728.html?spi=abf20204d70bae9a07c1a6598b0a2573&part=dl-6288502
>> HTTP/1.1
>> User-Agent: curl/7.19.4 (i586-pc-mingw32msvc) libcurl/7.19.4 zlib/1.2.3
>> Host: download.cnet.com
>> Accept: */*
>>
>> < HTTP/1.1 200 OK
> < Date: Sat, 20 Jun 2009 13:39:44 GMT
> < Via: HTTP/1.0 c18-rb-tron-farm1-mav-app2.cnet.com:80(cnwk.proxy.servlet.PathProxyServlet $Revision: 218012 $)
> < Content-Language: en-US
> < Expires: Sat, 20 Jun 2009 13:45:44 GMT
> < Cache-Control: max-age=360
> < Content-Type: text/html; charset=ISO-8859-1
> < Set-Cookie: rbSessionId=Cg+MmEo85qAEn/Jd35I; path=/; expires=Thu,
> 20-Aug-2009 09:39:44 GMT
> < Set-Cookie: rbSessionId=Cg+LiUo85qAEn/JdlrM; path=/; expires=Thu,
> 20-Aug-2009 09:39:44 GMT
> < P3P: CP="CAO DSP COR CURa ADMa DEVa PSAa PSDa IVAi IVDi CONi OUR OTRi IND
> PHY ONL UNI FIN COM NAV INT DEM STA"
> < Transfer-Encoding: chunked
> <
> { [data not shown]
> ########################################################################
> 100.0%* Connection #1 to host download.cnet.com left intact
>
> * Closing connection #0
> * Closing connection #1
>

There's a couple of Set-Cookie lines in there. You might get a bit further
by adding

      -b cookies -c cookies

to the curl command. That'll start up the cookie parser for you.

You should also try doing the same download in a browser like Firefox, with
LiveHTTPHeaders installed. That'll show you exactly what you need to
replicate. I think there's a similar tool for Internet Explorer called
IEhttpheaders, or something like it.

Ralph Mitchell

-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-06-20