I think that "Terraserver.com" is a wierd beast even in terms of IIS4.0
servers. Even getting "valid" URLs (try terraserver.com/Area51.asp) results
in getting a fraction of the URL, _every time_, with 'nc' working consistantly!)

This appears to point to some wierd difference between what 'curl' and what
'nc' or any other browser is sending -- I've tried to make them similar...

'nc' or 'netcat' is a command-line utility to send /receive from a TCP
connection. I use it in the form; 'cat req.txt | nc terraserver.com 80 > nc-test.html'

req.txt contains:
        GET /bad.html HTTP/1.0
        User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)
        Host: terraserver.com
        Pragma: no-cache
        Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
        <newline>

and this all goes out as one packet, using 0x0a (linefeed). I don't think
it's just that, because Windows uses 0x0d 0x0a.

Curl sends:
        GET /bad.html HTTP/1.0
        User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)
        Host: terraserver.com
        Pragma: no-cache
        Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Except there is the two packet thing I mentioned earlier.. (which seems
independ of how many extra headers I include.. ie, -H "Accept-Language","Accept-Encoding", doesn't change the packet count or the split right at the last
0d0a packet.

Other things I've tried: Changing HTTP/1.0 to HTTP/1.1
                          Removing the pragma

The wierdest thing is that using the 'nc' command-line with what appears to
be the same sequence, results in a correct transfer.. Using 'curl' _never_
does (with existing files, I get a 'Connection closed by remote host' error
inside the Transfer routine).

Any ideas? Can you try to pull some HTML off the site in question and see if
you see these same issues?

Try: terraserver.com/Area51.asp
      terraserver.com/image.asp?S=14&T=100&X=39&Y=4&Z=1379&W=1
(or, prepend www. -- doesn't seem to make a difference).

Thanks.
Received on 2000-04-26