Bugs item #3085981, was opened at 2010-10-12 08:40
Message generated for change (Comment added) made by dfandrich
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3085981&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: http
Group: bad behaviour
Status: Open
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Carey Klenetsky (cklenetsky)
Assigned to: Daniel Stenberg (bagder)
Summary: NTLM proxy w/authentication + POST problem

Initial Comment:
If you configure curl to use NTLM proxy authentication, but the proxy doesn't require authentication, an HTTP POST will not include the parameters.

The problem seems to come from the fact that curl expects to be in an auth. negotiation state, but the proxy happily forwards on the POST. You can see this with the curl app by sending this command and tracing the network traffic:

curl -x proxy-machine:proxy-port --proxy-ntlm -U user:password -d "abc=123&def=456" server.to.hit

In the case where authentication is on and supported the traffic looks like this (port 8080 is the traffic from curl to the proxy, and 80 from the proxy to the server):

interface: \Device\NPF_{4BD3F68E-3CDB-4B9A-BE85-65F2F79561AC} (172.20.2.0/255.
5.255.0)
filter: (ip or ip6) and ( port 80 or port 8080 )
####
T 172.20.2.147:3097 -> 172.20.2.10:8080 [AP]
  POST http://www.sfbaymagic.com HTTP/1.1..Proxy-Authorization: NTLM TlRMTVNT
  UAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=..User-Agent: curl/7.16.3 (i686-pc-cygw
  in) libcurl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS..Host: www.sf
  baymagic.com..Pragma: no-cache..Accept: */*..Proxy-Connection: Keep-Alive..
  Content-Length: 0..Content-Type: application/x-www-form-urlencoded....
#
T 172.20.2.10:8080 -> 172.20.2.147:3097 [AP]
  HTTP/1.1 407 Proxy Authentication Required ( Access is denied. )..Via: 1.1
   LIMAX32VM..Proxy-Authenticate: NTLM TlRMTVNTUAACAAAABgAGADgAAAAGgokC8OxY+v
  LP1QUAAAAAAAAAAJ4AngA+AAAABQLODgAAAA9DQVJFWTICAAwAQwBBAFIARQBZADIAAQASAEwAS
  QBNAEEAWAAzADIAVgBNAAQAHABjAGEAcgBlAHkAMgAuAGMAbQAuAHQAZQBzAHQAAwAwAGwAaQBt
  AGEAeAAzADIAdgBtAC4AYwBhAHIAZQB5ADIALgBjAG0ALgB0AGUAcwB0AAUAHABjAGEAcgBlAHk
  AMgAuAGMAbQAuAHQAZQBzAHQAAAAAAA==..Connection: Keep-Alive..Proxy-Connection
  : Keep-Alive..Pragma: no-cache..Cache-Control: no-cache..Content-Type: text
  /html..Content-Length: 0 ....
#
T 172.20.2.147:3097 -> 172.20.2.10:8080 [AP]
  POST http://www.sfbaymagic.com HTTP/1.1..Proxy-Authorization: NTLM TlRMTVNT
  UAADAAAAGAAYAEAAAAAYABgAWAAAAAYABgBwAAAACgAKAHYAAAAMAAwAgAAAAAAAAAAAAAAABoK
  JApXaqUAU77gpAAAAAAAAAAAAAAAAAAAAAHCJaYtBbyluhLKj/9c+0cQEP98HcHxUoWNhcmV5Mm
  NrbGVuZXRza3llbmcxMi1kYzc2MDA=..User-Agent: curl/7.16.3 (i686-pc-cygwin) li
  bcurl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS..Host: www.sfbaymag
  ic.com..Pragma: no-cache..Accept: */*..Proxy-Connection: Keep-Alive..Conten
  t-Length: 15..Content-Type: application/x-www-form-urlencoded....abc=123&de
  f=456
#####
T 172.20.2.10:8636 -> 168.75.111.31:80 [AP]
  POST / HTTP/1.1..Via: 1.1 LIMAX32VM..Content-Length: 15..Content-Type: appl
  ication/x-www-form-urlencoded..User-Agent: curl/7.16.3 (i686-pc-cygwin) lib
  curl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS..Host: www.sfbaymagi
  c.com..Pragma: no-cache..Accept: */*..Connection: Keep-Alive....abc=123&def
  =456
##

If the proxy does not require authentication the traffic looks like this instead:

filter: (ip or ip6) and ( port 80 or port 8080 )
####
T 172.20.2.147:3107 -> 172.20.2.10:8080 [AP]
  POST http://www.sfbaymagic.com HTTP/1.1..Proxy-Authorization: NTLM TlRMTVNT
  UAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=..User-Agent: curl/7.16.3 (i686-pc-cygw
  in) libcurl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS..Host: www.sf
  baymagic.com..Pragma: no-cache..Accept: */*..Proxy-Connection: Keep-Alive..
  Content-Length: 0..Content-Type: application/x-www-form-urlencoded....
####
T 172.20.2.10:8643 -> 168.75.111.31:80 [AP]
  POST / HTTP/1.1..Via: 1.1 LIMAX32VM..Content-Length: 0..Content-Type: appli
  cation/x-www-form-urlencoded..User-Agent: curl/7.16.3 (i686-pc-cygwin) libc
  url/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS..Host: www.sfbaymagic
  .com..Pragma: no-cache..Accept: */*..Connection: Keep-Alive....
##

Notice that the POST parameter are not part of the communication.

Even though the version of curl in the trace is 7.16.3 I've tried with 7.21.1 as well and the issue still remains.

----------------------------------------------------------------------

>Comment By: Dan Fandrich (dfandrich)
Date: 2010-10-12 12:57

Message:
You could also try the POST with no authentication, then turn it on if you
get a 401 error code. It means an extra round trip if authentication is
required, but can be done without changing libcurl.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2010-10-12 12:29

Message:
Oh.

Right, that's not quite obviously not supported. It would be easy for you
to just enable one other method as well, as that would then make this work.
Like allowing digest or gssnegotiate as well...

To support method X or none we need to change libcurl.

----------------------------------------------------------------------

Comment By: Carey Klenetsky (cklenetsky)
Date: 2010-10-12 12:23

Message:
The problem I'm facing, though, is that I specifically need it to use NTLM
or none, and not even try basic or digest. I'm not sure if there's a way
to specify that, especially since CURLAUTH_NONE is defined as 0.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2010-10-12 12:12

Message:
This is in fact an expected behavior. If you tell curl explicitly to use
NTLM for a POST, then it will act like this in order to not waste a lot of
bandwidth and doing the entire POST twice - as you told it to use NTLM.

If you instead want curl to "probe" for what auth to use, and possibly not
use any auth at all, then --proxy-anyauth is the option you want.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3085981&group_id=976
Received on 2010-10-12