Re: Help using libcurl with HTTP proxy on Android device

From: David Castillo via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 11 Apr 2023 18:03:20 -0700

> There used to be at least two locations used on Android for
> certificates. Maybe OpenSSL is only using one of them?

Yes, user-installed certificates are stored in the
"/data/misc/user/0/cacerts-added" directory while system certificates are
stored in "/system/etc/security/cacerts" directory. That's why when I
detect there's a proxy, I update the "CURLOPT_CAPATH" option to the
user-installed CA cert directory. But the certificates in the
user-installed directory are installed as DER format and libcurl fails to
open them with this error: `BoringSSL: error:0900006e:PEM
routines:OPENSSL_internal:NO_START_LINE`

> well if the plan is to always use this particular proxy then you can
always bundle the root cert and point libcurl to it as a local file.

Unfortunately the plan is to not always use this particular proxy. The idea
is to allow users to set any proxy they want, for example they can set
Charles, Fiddler, Proxyman or any other proxy.

> AFAIK the ssl callback functions of libcurl could be used to load the
cert in DER format and supply it via memory buffer to the ssl library but
that requires knowledge of the exact ssl library used and not all of them
support the ssl callback functions AFAIK

Do you know where I can look for documentation or anything to see how to do
this?

Although I have the feeling it shouldn't be that complicated to support SSL
proxy on Android with libcurl. I have the feeling that I might be doing
something wrong or missing something. I found that this option exists
CURLOPT_PROXY_CAPATH (https://curl.se/libcurl/c/CURLOPT_PROXY_CAPATH.html)
and I tried to set it, but got again this error: `SSL certificate problem:
self signed certificate in certificate chain`

Then I saw that this proxy CA path option is only used for HTTPS proxies
and I found this CURLOPT_PROXYTYPE option (
https://curl.se/libcurl/c/CURLOPT_PROXYTYPE.html) and saw that HTTP proxy
is set by default, so I tried to set it to CURLPROXY_HTTPS and now I got
this error: `BoringSSL: error:100000f7:SSL
routines:OPENSSL_internal:WRONG_VERSION_NUMBER`

Could it be that my version of libcurl doesn't support HTTPS proxies? This
is the configuration I have when building libcurl

Host setup: aarch64-unknown-linux-android

  Install prefix: /build/curl/arm64-v8a

  Compiler: /
ndk/22.0.7026061/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android23-clang

   CFLAGS: -Qunused-arguments -Wno-pointer-bool-conversion -O2

   CPPFLAGS: -isystem /build/zlib/arm64-v8a/include -isystem
/build/boringssl/arm64-v8a/include

   LDFLAGS: -/build/boringssl/arm64-v8a/lib
/build/zlib/arm64-v8a/lib -L/build/zlib/arm64-v8a/lib
-L/build/boringssl/arm64-v8a/lib

   LIBS: -lssl -lcrypto -lz -lssl -lcrypto -lc++ -lz


  curl version: 7.87.1-DEV

  SSL: enabled (BoringSSL)

  SSH: no (--with-{libssh,libssh2})

  zlib: enabled

  brotli: no (--with-brotli)

  zstd: no (--with-zstd)

  GSS-API: no (--with-gssapi)

  GSASL: no (libgsasl not found)

  TLS-SRP: no (--enable-tls-srp)

  resolver: POSIX threaded

  IPv6: enabled

  Unix sockets: enabled

  IDN: no (--with-{libidn2,winidn})

  Build libcurl: Shared=yes, Static=yes

  Built-in manual: no (--enable-manual)

  --libcurl option: enabled (--disable-libcurl-option)

  Verbose errors: no

  Code coverage: disabled

  SSPI: no (--enable-sspi)

  ca cert bundle: no

  ca cert path: /system/etc/security/cacerts (warning: certs not
found)

  ca fallback: no

  LDAP: no (--enable-ldap / --with-ldap-lib /
--with-lber-lib)

  LDAPS: no (--enable-ldaps)

  RTSP: enabled

  RTMP: no (--with-librtmp)

  PSL: no (libpsl not found)

  Alt-svc: enabled (--disable-alt-svc)

  Headers API: enabled (--disable-headers-api)

  HSTS: enabled (--disable-hsts)

  HTTP1: enabled (internal)

  HTTP2: no (--with-nghttp2, --with-hyper)

  HTTP3: no (--with-ngtcp2, --with-quiche --with-msh3)

  ECH: no (--enable-ech)

  WebSockets: no (--enable-websockets)

  Protocols: DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS
MQTT POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP

  Features: AsynchDNS HSTS HTTPS-proxy IPv6 Largefile NTLM NTLM_WB
SSL UnixSockets alt-svc libz threadsafe

On Tue, Apr 11, 2023 at 2:37 PM Henrik Holst <henrik.holst_at_millistream.com>
wrote:

> well if the plan is to always use this particular proxy then you can
> always bundle the root cert and point libcurl to it as a local file. AFAIK
> the ssl callback functions of libcurl could be used to load the cert in DER
> format and supply it via memory buffer to the ssl library but that requires
> knowledge of the exact ssl library used and not all of them support the ssl
> callback functions AFAIK.
>
> /HH
>
> Den tis 11 apr. 2023 kl 23:09 skrev David Castillo <casvel.d_at_gmail.com>:
>
>> > But did you install it as a new root certificate or as a client
>> certificate on the android device?
>>
>> As far as I understand, I installed it as a new root certificate, but I'm
>> not sure. I went to "Settings -> Passwords & security -> Privacy ->
>> Encryption and Credentials -> Install a certificate -> CA certificate".
>> Then if I go to "Trusted Credentials" on the device, I can see the Charles
>> certificate in the "User" tab
>>
>> > Can you access other https locations? If so then the other root ca:s
>> works on the device for some reason
>>
>> Yeah, I can access HTTPS locations without the proxy. I think the system
>> certificates that are in the "/system/etc/security/cacerts" directory
>> don't have any problem because those are PEM format (I checked that by
>> grabbing one of those certificates and doing "openssl x509 -in
>> <certificate> -inform PEM -text -noout" on my computer).
>>
>> The problem seems to be that when Android installs the new certificate,
>> it converts it to DER format for some reason, which libcurl can't handle. I
>> want to know if there's a way to handle this using libcurl. Maybe there's a
>> way to convert the certificate before libcurl tries to verify it?
>>
>

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-04-12