Hi everybody,

I recently run into some disagreement between curl and browsers (chrome,
firefox). Same URL is fetched just fine by browsers.

$ ~/bin/curl -L "
http://www.expedia.ca/pubspec/scripts/eap.asp?PRID=4&GOTO=PROMOS&Page=../../p/promos/winter-vacation-deals?langid=4105&AID=11259876&PID=3211374&affcid=network.cj.3211374.11259876
."
curl: (47) Maximum (50) redirects followed

$ ~/bin/curl -V
curl 7.29.1-DEV (i686-pc-linux-gnu) libcurl/7.29.1-DEV OpenSSL/1.0.1 zlib/
1.2.3.4 libidn/1.23 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3
pop3s rtmp rtsp smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

chrome handles above URL just fine apparently by tweaking path from "
/promos../../p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj"
to
"/p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj"
 during handling a redirect response.

Is there something in RFC that suggests doing this URL transformation? Any
special curl options to use? Any help is appreciated.

Thanks
-Alex

Event log from chrome

t=1361571282933 [st= 0] +REQUEST_ALIVE [dt=838]
t=1361571282934 [st= 1] URL_REQUEST_START_JOB [dt=0]
                            --> load_flags = 68223104
(ENABLE_LOAD_TIMING | MAIN_FRAME | MAYBE_USER_GESTURE |
VERIFY_EV_CERT)
                            --> method = "GET"
                            --> priority = 4
                            --> url =
"http://www.expedia.ca/pubspec/scripts/eap.asp?PRID=4&GOTO=PROMOS&Page=../../p/promos/winter-vacation-deals?langid=4105&AID=11259876&PID=3211374&affcid=network.cj.3211374.11259876."
t=1361571282934 [st= 1] +URL_REQUEST_START_JOB [dt=308]
                            --> load_flags = 68223104
(ENABLE_LOAD_TIMING | MAIN_FRAME | MAYBE_USER_GESTURE |
VERIFY_EV_CERT)
                            --> method = "GET"
                            --> priority = 4
                            --> url =
"http://www.expedia.ca/pubspec/scripts/eap.asp?PRID=4&GOTO=PROMOS&Page=../../p/promos/winter-vacation-deals?langid=4105&AID=11259876&PID=3211374&affcid=network.cj.3211374.11259876."
t=1361571282936 [st= 3] HTTP_CACHE_GET_BACKEND [dt=7]
t=1361571282943 [st= 10] HTTP_CACHE_OPEN_ENTRY [dt=1]
                              --> net_error = -2 (ERR_FAILED)
t=1361571282944 [st= 11] HTTP_CACHE_CREATE_ENTRY [dt=0]
t=1361571282944 [st= 11] HTTP_CACHE_ADD_TO_ENTRY [dt=0]
t=1361571282944 [st= 11] +HTTP_STREAM_REQUEST [dt=1]
t=1361571282945 [st= 12] HTTP_STREAM_REQUEST_BOUND_TO_JOB
                                --> source_dependency = 106697
(HTTP_STREAM_JOB) <chrome://net-internals/#events&s=106697>
t=1361571282945 [st= 12] -HTTP_STREAM_REQUEST
t=1361571282945 [st= 12] +HTTP_TRANSACTION_SEND_REQUEST [dt=0]
t=1361571282945 [st= 12] HTTP_TRANSACTION_SEND_REQUEST_HEADERS
                                --> GET
/pubspec/scripts/eap.asp?PRID=4&GOTO=PROMOS&Page=../../p/promos/winter-vacation-deals?langid=4105&AID=11259876&PID=3211374&affcid=network.cj.3211374.11259876.
HTTP/1.1
                                    Host: www.expedia.ca
                                    Connection: keep-alive
                                    Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
                                    User-Agent: Mozilla/5.0 (X11;
Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56
Safari/537.17
                                    Accept-Encoding: gzip,deflate,sdch
                                    Accept-Language: en-US,en;q=0.8
                                    Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.3
                                    Cookie: [736 bytes were stripped]
t=1361571282945 [st= 12] -HTTP_TRANSACTION_SEND_REQUEST
t=1361571282945 [st= 12] +HTTP_TRANSACTION_READ_HEADERS [dt=293]
t=1361571282945 [st= 12] HTTP_STREAM_PARSER_READ_HEADERS [dt=293]
t=1361571283238 [st=305] HTTP_STREAM_PARSER_READ_HEADERS [dt=0]
t=1361571283238 [st=305] HTTP_TRANSACTION_READ_RESPONSE_HEADERS
                                --> HTTP/1.1 302 Moved Temporarily
                                    Cache-Control: private, max-age=0,
proxy-revalidate, no-store, no-cache, must-revalidate
                                    Content-Type: text/html
                                    Location:
/promos../../p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj3211374
                                    P3P: policyref="/w3c/p3p.xml",
CP="CAO DSP IND COR ADM CONo CUR CUSi DEV PSA PSD DELi OUR COM NAV PHY
ONL PUR UNI"
                                    Pragma: no-cache
                                    Expires: Sat, 07 Jul 2012 21:42:50 GMT
                                    Vary: Accept-Encoding
                                    Content-Encoding: gzip
                                    RTSS: 1
                                    Content-Length: 195
                                    Date: Fri, 22 Feb 2013 22:14:43 GMT
                                    Connection: keep-alive
                                    Set-Cookie: [33 bytes were stripped]
                                    Set-Cookie: [34 bytes were stripped]
                                    Set-Cookie: [79 bytes were stripped]
                                    Set-Cookie: [107 bytes were stripped]
                                    Set-Cookie: [52 bytes were stripped]
                                    Set-Cookie: [41 bytes were stripped]
                                    Set-Cookie: [36 bytes were stripped]
                                    Set-Cookie: [42 bytes were stripped]
                                    Set-Cookie: [43 bytes were stripped]
                                    Set-Cookie: [48 bytes were stripped]
                                    Set-Cookie: [58 bytes were stripped]
                                    Set-Cookie: [72 bytes were stripped]
                                    Set-Cookie: [77 bytes were stripped]
t=1361571283238 [st=305] -HTTP_TRANSACTION_READ_HEADERS
t=1361571283238 [st=305] HTTP_CACHE_WRITE_INFO [dt=0]
t=1361571283241 [st=308] URL_REQUEST_REDIRECTED
                              --> location =
"http://www.expedia.ca/p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj3211374"
t=1361571283242 [st=309] -URL_REQUEST_START_JOB
t=1361571283242 [st=309] URL_REQUEST_START_JOB [dt=0]
                            --> load_flags = 68223104
(ENABLE_LOAD_TIMING | MAIN_FRAME | MAYBE_USER_GESTURE |
VERIFY_EV_CERT)
                            --> method = "GET"
                            --> priority = 4
                            --> url =
"http://www.expedia.ca/p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj3211374"
t=1361571283242 [st=309] +URL_REQUEST_START_JOB [dt=467]
                            --> load_flags = 68223104
(ENABLE_LOAD_TIMING | MAIN_FRAME | MAYBE_USER_GESTURE |
VERIFY_EV_CERT)
                            --> method = "GET"
                            --> priority = 4
                            --> url =
"http://www.expedia.ca/p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj3211374"
t=1361571283243 [st=310] HTTP_CACHE_GET_BACKEND [dt=0]
t=1361571283243 [st=310] HTTP_CACHE_OPEN_ENTRY [dt=0]
                              --> net_error = -2 (ERR_FAILED)
t=1361571283243 [st=310] HTTP_CACHE_CREATE_ENTRY [dt=0]
t=1361571283243 [st=310] HTTP_CACHE_ADD_TO_ENTRY [dt=0]
t=1361571283244 [st=311] +HTTP_STREAM_REQUEST [dt=1]
t=1361571283244 [st=311] HTTP_STREAM_REQUEST_BOUND_TO_JOB
                                --> source_dependency = 106703
(HTTP_STREAM_JOB) <chrome://net-internals/#events&s=106703>
t=1361571283245 [st=312] -HTTP_STREAM_REQUEST
t=1361571283245 [st=312] +HTTP_TRANSACTION_SEND_REQUEST [dt=0]
t=1361571283245 [st=312] HTTP_TRANSACTION_SEND_REQUEST_HEADERS
                                --> GET
/p/promos/winter-vacation-deals?langid=4105&affcid=network.cj.3211374.11259876.&affcid=cj3211374
HTTP/1.1
                                    Host: www.expedia.ca
                                    Connection: keep-alive
                                    Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
                                    User-Agent: Mozilla/5.0 (X11;
Linux i686) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56
Safari/537.17
                                    Accept-Encoding: gzip,deflate,sdch
                                    Accept-Language: en-US,en;q=0.8
                                    Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.3
                                    Cookie: [736 bytes were stripped]
t=1361571283245 [st=312] -HTTP_TRANSACTION_SEND_REQUEST
t=1361571283245 [st=312] +HTTP_TRANSACTION_READ_HEADERS [dt=463]
t=1361571283245 [st=312] HTTP_STREAM_PARSER_READ_HEADERS [dt=463]
t=1361571283708 [st=775] HTTP_TRANSACTION_READ_RESPONSE_HEADERS
                                --> HTTP/1.1 200 OK
                                    Content-Type: text/html;charset=UTF-8
                                    Content-Language: en-CA
                                    Vary: Accept-Encoding
                                    P3P: policyref="/w3c/p3p.xml",
CP="CAO DSP IND COR ADM CONo CUR CUSi DEV PSA PSD DELi OUR COM NAV PHY
ONL PUR UNI"
                                    Pragma: no-cache
                                    Cache-Control: private, max-age=0,
proxy-revalidate, no-store, no-cache, must-revalidate
                                    Expires: Sat, 07 Jul 2012 21:40:25 GMT
                                    Content-Encoding: gzip
                                    RTSS: 1
                                    Content-Length: 18260
                                    Date: Fri, 22 Feb 2013 22:14:43 GMT
                                    Connection: keep-alive
                                    Set-Cookie: [33 bytes were stripped]
                                    Set-Cookie: [34 bytes were stripped]
                                    Set-Cookie: [79 bytes were stripped]
                                    Set-Cookie: [107 bytes were stripped]
                                    Set-Cookie: [118 bytes were stripped]
                                    Set-Cookie: [72 bytes were stripped]
                                    Set-Cookie: [73 bytes were stripped]
                                    Set-Cookie: [76 bytes were stripped]
t=1361571283708 [st=775] -HTTP_TRANSACTION_READ_HEADERS
t=1361571283708 [st=775] HTTP_CACHE_WRITE_INFO [dt=0]
t=1361571283709 [st=776] -URL_REQUEST_START_JOB
t=1361571283726 [st=793] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283726 [st=793] HTTP_TRANSACTION_READ_BODY [dt=1]
t=1361571283727 [st=794] HTTP_TRANSACTION_READ_BODY [dt=25]
t=1361571283752 [st=819] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283753 [st=820] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283753 [st=820] HTTP_TRANSACTION_READ_BODY [dt=15]
t=1361571283769 [st=836] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283769 [st=836] HTTP_TRANSACTION_READ_BODY [dt=1]
t=1361571283770 [st=837] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283771 [st=838] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283771 [st=838] HTTP_TRANSACTION_READ_BODY [dt=0]
t=1361571283771 [st=838] -REQUEST_ALIVE

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-02-22