Hello,

I've been trying to spend some time getting curl to do what I want, and I
just can't get it to work. In a last ditch attempt, I thought I would join
and ask if there IS a way to do what I'm doing...

BACKGROUND

Running a transparent proxy (Websense) using WCCP redirection to redirect
requests to the proxy.

The goal was to write a script using CURL with SSPI support to check the
proxy is working and that it's blocking appropriate web pages.

EXPECTED BEHAVIOUR

curl -L would retrieve the web pages requested, and the output could then
be parsed to see if an 'Access to this web site is blocked' page was
returned.

ACTUAL BEHAVIOUR

Well. The actual behaviour was baffling until I worked it out.
I wrote the script. It worked, up to the limited degree it had been
written. Then I schedule it to run on an hourly schedule to check the
proxy, and it stops working.

I've not quite figured out the details here, but this is what seems to be
happening:

1) I browse the Internet using IE
2) IE authenticates me against the proxy, using Schannel.dll
3) curl works fine for a bit.
4) My NTLM authenticated session that IE set up with the proxy times out.
5) curl stops working, returning "307 Authorization Required" pages from
the Websense proxy.

WHAT DOES WORK

I can explicitly use the proxy, and it all works as expected. I.e., this
works

curl -L --proxy http://192.168.x.x:8080 --proxy-ntlm -U :
http://www.google.com

However, that doesn't allow me to test that the WCCP redirect is working OK.

WHAT DOESN'T WORK

a. Trying to grab the Location: URL in the 307 message, and then going to
it via the proxy itself, in an attempt to initiate authentication (Proxy
comes back with 400 Cycle Detected)
b. Any of the --proxy-ntlm, --ntlm options
--proxy-ntlm -U : is ignored, because it doesn't think it's talking to a
proxy
--ntlm -U : plain doesn't work (I just get the 307 page back.

Now, unless I am missing something ... the easy way to fix this would be to
add a --proxy-transparent option to curl. How would that work you might
wonder?

Well, what would happen is:
- curl options are parsed and curl notes --proxy-transparent option is
present
- curl does an nslookup on www.google.com and opens a connection to it.
- curl's logic then treats the connection as if it was talking to an NTLM
proxy
  - the "--proxy-ntlm -U :" options are treated as if you'd opened an
explicit
    connection to a proxy
  - NTLM authentication occurs
  - the web page is retrieved and returned

Am I correct in thinking extending curl in this way might be the only way
to solve this problem?

Thanks,
Malcolm.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-10-18