Dear all,

I'm definitelly not an expert regarding cURL. Nevertheless I'm a big
fan and I use it quite often for tests purposes on Internet access
platforms at customers premises.

I'm currently facing an issue regarding proxy authentication. The
proxy is a Bluecoat box and the authentication method relies on a LDAP
realm using an origin-cookie-redirect mode. FYI i'm using cURL and
libcurl v7.16.4.

Here's the issue:

when using a Firefox browser and requesting www.cnn.com for instance I
receive a 302 code from the proxy as show below:

GET / HTTP/1.1
Host: www.cnn.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.5)
Gecko/20070713 Firefox/2.0.0.5
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive

HTTP/1.x 302 Found
Location: https://vurl-proxy1.mydomain.dmz/?cfru=aHR0cDovL3d3dy5jbm4uY29tLw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 869

This a normal behavior. Then the browser issues the following request:
GET /?cfru=aHR0cDovL3d3dy5jbm4uY29tLw== HTTP/1.1
Host: vurl-proxy1.mydomain.dmz
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.5)
Gecko/20070713 Firefox/2.0.0.5
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

and received the following answer from the proxy:
HTTP/1.x 401 Unauthorized
Cache-Control: no-cache
Pragma: no-cache
WWW-Authenticate: BASIC realm="LDAP_AD"
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Set-Cookie: BCSI-CSAC104C01=2; Path=/
Connection: close
Content-Length: 1055

As you may noticed the user is now prompted for authentication.
Credentials are now sent:
GET /?cfru=aHR0cDovL3d3dy5jbm4uY29tLw== HTTP/1.1
Host: vurl-proxy1.braibous.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.5)
Gecko/20070713 Firefox/2.0.0.5
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Authorization: Basic amRvZTpCb3VDbzQ3
Cookie: BCSI-CSAC104C01=2; Path=/

Credentials are validated and a credential cookie is set on the
browser for the session. The user acces is granted to the original
URL:
HTTP/1.x 302 Found
Location: http://www.cnn.com/?cfAC104C01=1E640CEF!amRvZTpsZGFwX2FkOsK9rwH7nsDxqpfMcCp3csA=
Cache-Control: no-cache
Pragma: no-cache
P3P: CP="NON ADM OUR STP COM"
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Set-Cookie: CFAC104C01=1E640CEF!amRvZTpsZGFwX2FkOsK9rwH7nsDxqpfMcCp3csA=;
Path=/; Domain=.braibous.com
Connection: close
Content-Length: 898

etc etc ...

Does anyone can teel me if such kind of behavior can be repoduced
using cURL ?From my understanding and my tests the answer is no but
i'm probably wrong.
FYI here is the cURL request I tried:

curl -i -L --proxy-anyauth -U jdoe:password -k -b cookie-store -c
cookie-store --trace-ascii trace.txt www.google.com

Here's the trace file output:
== Info: About to connect() to www.google.com port 8080 (#0)
== Info: Trying 172.16.76.1... == Info: connected
== Info: Connected to www.google.com (172.16.76.1) port 8080 (#0)
=> Send header, 209 bytes (0xd1)
0000: GET http://www.google.com HTTP/1.1
0024: User-Agent: curl/7.16.4 (i586-pc-mingw32msvc) libcurl/7.16.4 Ope
0064: nSSL/0.9.8d zlib/1.2.2
007c: Host: www.google.com
0092: Pragma: no-cache
00a4: Accept: */*
00b1: Proxy-Connection: Keep-Alive
00cf:
<= Recv header, 20 bytes (0x14)
0000: HTTP/1.1 302 Found
<= Recv header, 82 bytes (0x52)
0000: Location: http://vurl-proxy1.mydomain.dmz/?cfru=aHR0cDovL3d3dy5n
0040: b29nbGUuY29tLw==
<= Recv header, 25 bytes (0x19)
0000: Cache-Control: no-cache
<= Recv header, 18 bytes (0x12)
0000: Pragma: no-cache
<= Recv header, 40 bytes (0x28)
0000: Content-Type: text/html; charset=utf-8
== Info: HTTP/1.1 proxy connection set close!
<= Recv header, 25 bytes (0x19)
0000: Proxy-Connection: close
<= Recv header, 19 bytes (0x13)
0000: Connection: close
<= Recv header, 21 bytes (0x15)
0000: Content-Length: 884
<= Recv header, 2 bytes (0x2)
0000:
== Info: Closing connection #0
== Info: Issue another request to this URL:
'http://vurl-proxy1.mydomain.dmz/?cfru=aHR0cDovL3d3dy5nb29nbGUuY29tLw=='
== Info: About to connect() to vurl-proxy1.mydomain.dmz port 80 (#0)
== Info: Trying 172.16.76.101... == Info: connected
== Info: Connected to vurl-proxy1.mydomain.dmz (172.16.76.101) port 80 (#0)
=> Send header, 216 bytes (0xd8)
0000: GET /?cfru=aHR0cDovL3d3dy5nb29nbGUuY29tLw== HTTP/1.1
0036: User-Agent: curl/7.16.4 (i586-pc-mingw32msvc) libcurl/7.16.4 Ope
0076: nSSL/0.9.8d zlib/1.2.2
008e: Host: vurl-proxy1.mydomain.dmz
00ae: Accept: */*
00bb: Cookie: BCSI-CSAC104C01=2
00d6:
<= Recv header, 27 bytes (0x1b)
0000: HTTP/1.1 401 Unauthorized
<= Recv header, 25 bytes (0x19)
0000: Cache-Control: no-cache
<= Recv header, 18 bytes (0x12)
0000: Pragma: no-cache
== Info: Authentication problem. Ignoring this.
<= Recv header, 41 bytes (0x29)
0000: WWW-Authenticate: BASIC realm="LDAP_AD"
<= Recv header, 40 bytes (0x28)
0000: Content-Type: text/html; charset=utf-8
<= Recv header, 25 bytes (0x19)
0000: Proxy-Connection: close
== Info: Replaced cookie BCSI-CSAC104C01="2" for domain
vurl-proxy1.mydomain.dmz, path /, expire 0
<= Recv header, 39 bytes (0x27)
0000: Set-Cookie: BCSI-CSAC104C01=2; Path=/
<= Recv header, 19 bytes (0x13)
0000: Connection: close
<= Recv header, 22 bytes (0x16)
0000: Content-Length: 1070
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 1070 bytes (0x42e)
0000: <HTML><HEAD>.<TITLE>Access Denied</TITLE>.</HEAD>.<BODY>.<FONT f
0040: ace="Helvetica">.<big><strong></strong></big><BR>.</FONT>.<block
0080: quote>.<TABLE border=0 cellPadding=1 width="80%">.<TR><TD>.<FONT
00c0: face="Helvetica">.<big>Access Denied (authentication_failed)</b
0100: ig>.<BR>.<BR>.</FONT>.</TD></TR>.<TR><TD>.<FONT face="Helvetica"
0140: >.Your credentials could not be authenticated: "Credentials requ
0180: ired.". You will not be permitted access until your credentials
01c0: can be verified..</FONT>.</TD></TR>.<TR><TD>.<FONT face="Helveti
0200: ca">.This is typically caused by an incorrect username and/or pa
0240: ssword, but could also be caused by network problems..</FONT>.</
0280: TD></TR>.<TR><TD>.<FONT face="Helvetica" SIZE=2>.<BR>.For assist
02c0: ance, contact your network support team.<br><br>Your request was
0300: categorized by Blue Coat Web Filter as 'Search Engines/Portals'
0340: . <br>If you wish to question or dispute this result, please cli
0380: ck <a href="http://sitereview.bluecoat.com/sitereview.jsp?referr
03c0: er=136&url=http://www.google.com/">here</a>..</FONT>.</TD></TR>.
0400: </TABLE>.</blockquote>.</FONT>.</BODY></HTML>.
== Info: Closing connection #0

cURL stops working right after receiving the 401 message from the
proxy. The WWW-Authenticate is seen as a problem.

Thanks in advance for your answers.
Cheers,
Laurent
Received on 2007-10-03