Hello,

Is there a way to disable CONNECT proxy tunneling for HTTPS connections?
If not, could it be a sensible evolution request to add such feature?

For security reasons, some proxies offer the possibility to completely
disable the CONNECT request (often used to create opaque tunnels and
bypass proxy restrictions) and require the client to send a plain
https:// request to the proxy, the client effectively offloading all the
SSL stuff onto the proxy and the proxy gaining a better control of the
data exchanged.

For instance, the GNU Squid proxy natively supports this feature (it
needs to be compiled with SSL support enabled which may not be the
default though) and I also faced it with some commercial software.

Curl manpage about the -x, --proxy argument seems to explain that all
operations are made through HTTP and recommends to use -p, --proxytunnel
to enable tunneling. However, I saw that tunneling is automatically
enabled when retrieving an https:// URL.

The beginning of the manpage also indicates the possibility to disable
boolean parameters either by duplicating them or using a syntax such as
--no-option. I tried to play with -p, --no-proxytunnel, --ssl, --no-ssl
with no luck: Curl systematically tries to create a CONNECT tunnel for
https:// URL's.

Investigating Curl source-code, I see that this behavior is indeed
enforced in `lib/url.c':

     /*************************************************************
      * If the protocol is using SSL and HTTP proxy is used, we set
      * the tunnel_proxy bit.
      *************************************************************/
     if((conn->given->flags&PROTOPT_SSL) && conn->bits.httpproxy)
       conn->bits.tunnel_proxy = TRUE;

Here is the associated commit and description:
https://github.com/bagder/curl/commit/2c43d64302cc58532768e7e98fceb0e3032950cc#diff-8fb104c402dc51bdffef05a372f32aa2

     Added a new 'bit' in the connect struct named 'tunnel_proxy' that is
set if
     a connection is tunneled through a proxy. A tunnel is done with
CONNECT,
     either when using HTTPS or FTPS, or if explicitly enabled by the
app.

It seems clear that sending https:// scheme clear requests has not been
taken into account and that Curl, in its current shape, is therefore
unable to deal with such proxies.

I'm not a developer and do not know Curl internals well enough to
quickly provide a reliable patch. Trying to disable the tunneling in the
above mentioned location makes Curl to initiate an SSL-handshake
directly with the proxy, and the flags are read-only so cannot be
altered here.

For my own needs, I modified `lib/http.c' to make the https:// scheme to
be handled the same way as plain http:// scheme:

     --- lib/http.c.orig 2015-12-11 13:57:23.000000000 +0100
     +++ lib/http.c 2015-12-11 13:59:15.000000000 +0100
     @@ -134,17 +134,17 @@
        Curl_http_done, /* done */
        ZERO_NULL, /* do_more */
        Curl_http_connect, /* connect_it */
     - https_connecting, /* connecting */
     + ZERO_NULL, /* connecting */
        ZERO_NULL, /* doing */
     - https_getsock, /* proto_getsock */
     + ZERO_NULL, /* proto_getsock */
        http_getsock_do, /* doing_getsock */
        ZERO_NULL, /* domore_getsock */
        ZERO_NULL, /* perform_getsock */
        ZERO_NULL, /* disconnect */
        ZERO_NULL, /* readwrite */
     - PORT_HTTPS, /* defport */
     - CURLPROTO_HTTPS, /* protocol */
     - PROTOPT_SSL | PROTOPT_CREDSPERREQUEST /* flags */
     + PORT_HTTP, /* defport */
     + CURLPROTO_HTTP, /* protocol */
     + PROTOPT_CREDSPERREQUEST /* flags */
      };
      #endif

This works correctly as Curl now sends plain https:// scheme requests to
my Squid proxy which, in turns, handles the HTTPS request with the
remote host successfully:

     GET https://example.com/somepath HTTP/1.1
     Host: example.com
     User-Agent: curl/7.46.0
     Accept: */*
     Proxy-Connection: Keep-Alive

I understand that Curl by default create a CONNECT tunnel since it is
the most common expected behavior.

However, wouldn't it make sense for Curl to honor a --no-proxytunnel
argument disabling CONNECT tunneling even for https:// URL's so it can
be used natively in such circumstances?

Best regards,
Simon.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-12-11