cURL / Mailing Lists / curl-users / Single Mail

curl-users

RE: Passing cookies during session

From: Lara Bordick <lbordick_at_interconnect.net>
Date: Fri, 18 Jan 2002 15:55:59 -0600

Thank you for the quick response to my question.

Some more information on how the servers I am connecting to work (followed
by the verbose output):

When I connect directly to
https://dataserver.somesite.com/file_I_want_to_download.tar.gz, that server
checks my client certificate AND it looks for either an SSLCRED or SMSESSION
cookie. If it finds a valid SSLCRED and no SMSESSION it sends back a
cleared SSLCRED cookie and a valid SMSESSION and then proceeds with the
download. If it finds a valid SMSESSION it proceeds with the download. If
it finds neither it redirects to https://validationserver.somesite.com which
will check my client certificate, produce a valid SSLCRED and re-direct the
request back to
https://dataserver.somesite.com/file_I_want_to_download.tar.gz where the
process repeats.

****************************************************************************
****************
The highlights of the output of curl -vi -D cookies --cert
my_cert.pem:password -L https://validationserver.somesite.com:
...
>GET /credential_checker?name1=val1&name2=val2&...&nameN=valN
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: validationserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
HTTP/1.1 302 Moved Temporarily
Server: Netscape-Enterprise/4.1
Date: Fri, 18 Jan 2002 16:19:17 GMT
Content-type: magnus-internal/scc
Location: https://validationserver.somesite.com/
Set-Cookie: SSLCRED=k93vtBmLvd....;
path=/; domain=.somesite.com
* Follow to new URL: https://validationserver.somesite.com/
* Connection #0 left intact
* Follows Location: to new URL: 'https://validationserver.somesite.com/'
* Disables POST, goes with GET
* Connection 0 seems to be dead!
* Closing connection #0
* SSL re-using session ID
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-07 19:12:02 GMT
* expire date: 2004-11-07 19:12:02 GMT
* common name: validationserver.somesite.com (matched)
* issuer: blah blah blah
* Connected to validationserver.somesite.com(ip address)
> GET / HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: validationserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Cookie: SSLCRED=k93vtBm.....
HTTP/1.1 200 OK
Server: Netscape-Enterprise/4.1
Date: Fri, 18 Jan 2002 16:19:23 GMT
Set-Cookie: SSLCRED=; path=/; expires=Sun, 22-Jul-2001 16:19:23 GMT;
domain=.somesite.com
Set-Cookie: SMSESSION=xPDVSKopSR9....

....followed by content of home page

....followed by contents of the .tar.gz file

****************************************************************************
****************
highlights of the output of curl -vi -b cookies --cert
my_cert.pem:password -L
https://dataserver.somesite.com/file_I_want_to_download.tar.gz when it is
run WITHOUT any previous commands
...
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-13 15:07:25 GMT
* expire date: 2004-11-13 15:07:25 GMT
* common name: dataserver.somesite.com(matched)
* issuer: blah blah blah
* Connected to dataserver.somesite.com(ip address)
> GET /file_I_want_to_download.tar.gz HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: dataserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  % Total % Received % Xferd Average Speed Time
Curr.
                                 Dload Upload Total Current Left
Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:--
0
* Closing connection #0
* Follows Location: to new URL:
'https://validationserver.somesite.com/credential_checker?name1=val1&name2=v
al2&...&nameN=valN[11:12:04:273]&TARGET=https://dataserver.somesite.com/file
_I_want_to_download.tar.gz '
* Disables POST, goes with GET
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-07 19:12:02 GMT
* expire date: 2004-11-07 19:12:02 GMT
* common name: validationserver.somesite.com (matched)
* issuer: blah blah blah
* Connected to validationserver.somesite.com(ip address)
> GET
credential_checker?name1=val1&name2=val2&...&nameN=valN[11:12:04:273]&TARGET
=https://dataserver.somesite.com/file_I_want_to_download.tar.gz HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: validationserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
* Follow to new URL:
https://dataserver.somesite.com/file_I_want_to_download.tar.gz
  0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:--
0
* Connection #0 left intact
* Follows Location: to new URL:
https://dataserver.somesite.com/file_I_want_to_download.tar.gz '
* Disables POST, goes with GET
* SSL re-using session ID
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-13 15:07:25 GMT
* expire date: 2004-11-13 15:07:25 GMT
* common name: dataserver.somesite.com (matched)
* issuer: blah blah blah
* Connected to dataserver.somesite.com(ip address)
> GET /file_I_want_to_download.tar.gz HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: dataserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*

0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:--
0
* Closing connection #1
* Follows Location: to new URL:
'https://validationserver.somesite.com/credential_checker?name1=val1&name2=v
al2&...&nameN=valN[11:12:07:2642]&TARGET=https://dataserver.somesite.com/fil
e_I_want_to_download.tar.gz'
* Disables POST, goes with GET
* Connection 0 seems to be dead!
* Closing connection #0
* SSL re-using session ID
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-07 19:12:02 GMT
* expire date: 2004-11-07 19:12:02 GMT
* common name: validationserver.somesite.com (matched)
* issuer: blah blah blah
* Connected to validationserver.somesite.com (ip address)
> GET
credential_checker?name1=val1&name2=val2&...&nameN=valN[11:12:07:2642]&TARGE
T=https://dataserver.somesite.com/file_I_want_to_download.tar.gz HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: validationserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Cookie: SSLCRED=ys16/D2c9XR.....
* Follow to new URL:
https://dataserver.somesite.com/file_I_want_to_download.tar.gz
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:--
0
* Connection #0 left intact
* Follows Location: to new URL:
'https://dataserver.somesite.com/file_I_want_to_download.tar.gz'
* Disables POST, goes with GET
* SSL re-using session ID
* SSL connection using RC4-MD5
* Server certificate:
* subject: blah blah blah
* start date: 2001-11-13 15:07:25 GMT
* expire date: 2004-11-13 15:07:25 GMT
* common name: dataserver.somesite.com (matched)
* issuer: blah blah blah
* Connected to dataserver.somesite.com(ip address)
> GET file_I_want_to_download.tar.gz HTTP/1.1
User-Agent: curl/7.9.2 (sparc-sun-solaris2.7) libcurl 7.9.2 (OpenSSL 0.9.6)
Host: dataserver.somesite.com
Pragma: no-cache
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*

0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:--
0
* Closing connection #1
* Follows Location: to new URL:
'https://validationserver.somesite.com/credential_checker?name1=val1&name2=v
al2&...&nameN=valN[11:12:08:4738]&TARGET=https://dataserver.somesite.com/fil
e_I_want_to_download.tar.gz'

...the above repeats until the Max # of redirects is reached

****************************************************************************
****************

Luckily I also have access to the servers that I am downloading from. What
appears to be happening on the server side is in case #1 when I use curl in
2 steps:
- the first connection goes to the validation server and presents the client
certificate
- the validation server generates an SSLCRED cookie and redirects back to
the home page of the validation server
- accessing the home page of the validation server causes the SSLCRED cookie
to be checked
- the SSLCRED cookie is then cleared and an SMSESSION cookie is created
- the transaction ends
- the second call to curl is invoked requesting access to a file on the
dataserver
- the client certificate and the SMSESSION cookie are presented to the
dataserver, which it accepts as valid
-the transfer of the requested file begins

What appears to be happening on the server side is in case #2 when I use
curl in 1 step:
-curl is invoked requesting access to a file on the dataserver
- the client certificate is presented to the dataserver
- no SSLCRED or SMSESSION cookies are presented so the dataserver redirects
to the validation server
- the connection goes to the validation server and presents the client
certificate
- the validation server generates an SSLCRED cookie and redirects back to
the file request of the dataserver
******HERE IS THE IMPORTANT PART WHERE THINGS SEEM TO BE BREAKING DOWN*****
- the logs on the dataserver show that the client does not present a valid
SSLCRED or SMSESSION cookie at this point

I can not tell from the logs if the dataserver is getting the SSLCRED cookie
and determining that it is invalid or if the dataserver is not getting the
SSLCRED cookie at all. Both servers share the same domain and the cookies
are defined to be valid across that whole domain. Is it possible that curl
is not passing the SSLCRED cookie to dataserver.somesite.com because it was
defined by validationserver.somesite.com? Or is it this more likely caused
by an invalid SSLCRED cookie? It is hard for me to tell from the output
what cookies are REALLY being sent and received.

I will be happy to send more logs if necessary but I did not want to
overwhelm you.

-----Original Message-----
From: Daniel Stenberg [mailto:daniel_at_haxx.se]
Sent: Friday, January 18, 2002 3:55 AM
To: Lara Bordick
Cc: Curl Mailinglist
Subject: Re: Passing cookies during session

On Thu, 17 Jan 2002, Lara Bordick wrote:

[my comments are inlined below]

> Using a web browser I can download a particular file that I need at the
> following URL by selecting the appropriate client certificate that is
> installed in my browser:
> https://dataserver.somesite.com/file_I_want_to_download.tar.gz
>
> What actually happens is that when I connect to the above URL, I am
> redirected to a different server, lets call it
> https://validationserver.somesite.com, which handles authentication and
> then sets a cookie which dataserver.somesite.com requires to determine
that
> I am a valid user and then redirects me back to
> https://dataserver.somesite.com/file_I_want_to_download.tar.gz. At this
> point the browser begins the process of downloading this file.
>
> What I really want to do is to download these files using curl because of
> its ability to resume the file transfer where it left off in case of a
> broken connection.
>
> I have been able to make this process work using a two step process.
>
> The first command looks like this:
> curl -D cookies --cert my_cert.pem:password -L
> https://validationserver.somesite.com
>
> The second command looks like this:
> curl -b cookies --cert my_cert.pem:password -L
> https://dataserver.somesite.com/file_I_want_to_download.tar.gz
>
> Is there a way to do this in one step? The documentation led me to
believe
> that I should be able to do this using only the second command.

I would say so now as well.

> In the tutorial it says "if you want to let curl understand cookies from a
> page and follow a location (and thus possibly send back cookies it
> received), you can invoke it like: curl -b nada -L www.cookiesite.com".
> However when I run the second command without doing the first I get stuck
> in a loop between the authentication server and the dataserver because
curl
> does not appear to be passing the cookie received from the authentication
> server to the dataserver.

This sounds like a bug. I realize that this might contain sensitive data,
but
do you think it would be possible for you to use -vi when you perform the
above commands to figure out what goes wrong? Check the 'Set-Cookie: lines
from the server and the 'Cookie:' line sent back by curl.

If you replace the names with suitably other values (as you did with the
site
names) to prevent them to show anything you wouldn't want, we might still
understand why this happens or possibly work out a fix if this truly proves
to be a bug.

> What is the proper syntax for doing this in one step or is it not
possible?

It is meant to work using that syntax!

--
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/

Received on 2002-01-18

This message: [ Message body ]
Next message: Daniel Stenberg: "Re: How do I set"
Previous message: Saul Good: "Re: Curl 7.9.3 pre3 build fails"
In reply to: Daniel Stenberg: "Re: Passing cookies during session"
Next in thread: Daniel Stenberg: "RE: Passing cookies during session"
Reply: Daniel Stenberg: "RE: Passing cookies during session"

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]