cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: cURL hangs when downloading a CSV file from behind a proxy

From: Adam Larsen <aplarsen_at_gmail.com>
Date: Tue, 16 Apr 2013 15:04:54 -0500

Thanks again for your reply.

> Do you have this problem against other sites or when using plain HTTP?

Elsewhere in this batched job, I seem to be able to download straight HTML
files and process their results. It's when I download a file
(Content-Disposition: attachment;) that it hangs. I did try this project
off of the network on a non-proxy connection, and the file downloaded and
moved on just fine.

> Okay, then I don't have any really good theory. I would need more details
such as logs/dumps from the protocol exchange to possibly say something.

Is this the kind of output that would help? You can see at the end that
the downloaded file size is 100% of its expectation, but it stays on that
progress table.

>curl\curl.exe --proxy proxyserver:8080 --proxy-ntlm -
-proxy-user DOMAIN\user:pass --cookie-jar cookies.txt --cookie cookies.t
xt --location-trusted --user-agent 'Mozilla/5.0' --ssl --insecure --output
payp
al-2013-04-15.csv --verbose --data cmd=_history-download-submit --data
history_c
ache= --data type=custom_date_range --data from_a=1 --data from_b=1 --data
from_
c=2013 --data to_a=4 --data to_b=15 --data to_c=2013 --data
custom_file_type=com
ma_completed --data latest_completed_file_type= --data
submit.x=Download+History
 --data form_charset=UTF-8 --data include_shopping_cart_detail=1
https://www.san
dbox.paypal.com/us/cgi-bin/webscr
  % Total % Received % Xferd Average Speed Time Time Time
 Current
                                 Dload Upload Total Spent Left
 Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
  0*
About to connect() to proxy proxy1 port 8080 (#0)
* Trying 10.99.1.1...
* Connected to proxy1 (10.99.1.1) port 8080 (#0)
* Establish HTTP proxy tunnel to www.sandbox.paypal.com:443
* Proxy auth using NTLM with user 'DOMAIN\user'
> CONNECT www.sandbox.paypal.com:443 HTTP/1.1
> Host: www.sandbox.paypal.com:443
> Proxy-Authorization: (removed)
> User-Agent: 'Mozilla/5.0'
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 407 Proxy Authentication Required ( Access is denied. )
< Via: 1.1 ODTMG-02
< Proxy-Authenticate: (removed)
< Connection: Keep-Alive
< Proxy-Connection: Keep-Alive
< Pragma: no-cache
< Cache-Control: no-cache
< Content-Type: text/html
< Content-Length: 0
<
* Rewind stream after send
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to www.sandbox.paypal.com:443
* Proxy auth using NTLM with user 'OCUSD\aduser'
> CONNECT www.sandbox.paypal.com:443 HTTP/1.1
> Host: www.sandbox.paypal.com:443
> Proxy-Authorization: (removed)
> User-Agent: 'Mozilla/5.0'
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< Via: 1.1 ODTMG-02
< Connection: Keep-Alive
< Proxy-Connection: Keep-Alive
<
* Proxy replied OK to CONNECT request
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
  0*
SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using AES256-SHA
* Server certificate:
* subject: 1.3.6.1.4.1.311.60.2.1.3=US;
1.3.6.1.4.1.311.60.2.1.2=Delaware
; businessCategory=Private Organization; serialNumber=3014267; C=US;
postalCode=
95131-2021; ST=California; L=San Jose; street=2211 N 1st St; O=PayPal,
Inc.; OU=
PayPal Production; CN=www.sandbox.paypal.com
* start date: 2011-09-01 00:00:00 GMT
* expire date: 2013-09-30 23:59:59 GMT
* issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network;
OU=Terms of
use at https://www.verisign.com/rpa (c)06; CN=VeriSign Class 3 Extended
Validati
on SSL CA
* SSL certificate verify result: unable to get local issuer
certificate (
20), continuing anyway.
> POST /us/cgi-bin/webscr HTTP/1.1
> User-Agent: 'Mozilla/5.0'
> Host: www.sandbox.paypal.com
> Accept: */*
> Cookie: (cookie content removed from output)
> Content-Length: 258
> Content-Type: application/x-www-form-urlencoded
>
} [data not shown]
100 258 0 0 100 258 0 130 0:00:01 0:00:01 --:--:--
130*
upload completely sent off: 258 out of 258 bytes
100 258 0 0 100 258 0 85 0:00:03 0:00:03 --:--:--
 85<
HTTP/1.1 200 OK
< Date: Tue, 16 Apr 2013 19:53:10 GMT
< Server: Apache
< Content-Disposition: attachment; filename="Download.csv";
filename*=utf-8'en'D
ownload.csv
< X-Cnection: close
< Transfer-Encoding: chunked
< Content-Type: application/x-csv
* Keep sending data to get tossed away!
<
{ [data not shown]
100 18122 0 17864 100 258 769 11 0:00:23 0:00:23 --:--:--
  0

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-04-16