Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in rustls pkg-config detection #13200

Closed
kpcyrd opened this issue Mar 27, 2024 · 19 comments
Closed

Regression in rustls pkg-config detection #13200

kpcyrd opened this issue Mar 27, 2024 · 19 comments

Comments

@kpcyrd
Copy link
Contributor

kpcyrd commented Mar 27, 2024

I did this

I tried to compile the latest curl release with the rustls TLS backend.

This is the error I get during configure:

[...]
checking for mit-krb5-gssapi options with pkg-config... found
checking for gss.h... no
checking for gssapi/gssapi.h... yes
checking for gssapi/gssapi_generic.h... yes
checking for gssapi/gssapi_krb5.h... yes
checking if GSS-API headers declare GSS_C_NT_HOSTBASED_SERVICE... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for mit-krb5-gssapi options with pkg-config... found
checking if we can link against GSS-API library... yes
checking whether to enable Windows native SSL/TLS... no
checking whether to enable Secure Transport... no
checking whether to enable Amiga native SSL/TLS (AmiSSL v5)... no
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for rustls options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lrustls"
configure: pkg-config: SSL_LDFLAGS: ""
configure: pkg-config: SSL_CPPFLAGS: ""
configure: detected rustls
configure: error: TLS not detected, you will not be able to use HTTPS, FTPS, NTLM and more.
Use --with-openssl, --with-gnutls, --with-wolfssl, --with-mbedtls, --with-schannel, --with-secure-transport, --with-amissl, --with-bearssl or --with-rustls to address this.

pkg-config is correctly setup for librustls:

% pacman -Ql librustls
librustls /usr/
librustls /usr/include/
librustls /usr/include/rustls.h
librustls /usr/lib/
librustls /usr/lib/librustls.so
librustls /usr/lib/librustls.so.0.12.1
librustls /usr/lib/pkgconfig/
librustls /usr/lib/pkgconfig/rustls.pc
librustls /usr/share/
librustls /usr/share/licenses/
librustls /usr/share/licenses/librustls/
librustls /usr/share/licenses/librustls/LICENSE-APACHE
librustls /usr/share/licenses/librustls/LICENSE-ISC
librustls /usr/share/licenses/librustls/LICENSE-MIT

The configure options used are:

--prefix='/usr'
--mandir='/usr/share/man'
--disable-ldap
--disable-ldaps
--disable-manual
--enable-ipv6
--enable-threaded-resolver
--with-gssapi
--with-libssh2
--with-random='/dev/urandom'
--with-ca-bundle='/etc/ssl/certs/ca-certificates.crt'
--with-rustls
--without-openssl
--disable-shared

I noticed this is due to #13179, reverting 647e86a fixes the build.

Full configure output:

checking whether to enable maintainer-specific portions of Makefiles... no
checking whether make supports nested variables... yes
checking whether to enable debug build options... no
checking whether to enable compiler optimizer... (assumed) yes
checking whether to enable strict compiler warnings... no
checking whether to enable compiler warnings as errors... no
checking whether to enable curl debug memory tracking... no
checking whether to enable hiding of library internal symbols... yes
checking whether to enable c-ares for DNS lookups... no
checking whether to disable dependency on -lrt... (assumed no)
checking whether to enable ECH support... no
checking for path separator... :
checking for sed... /usr/bin/sed
checking for grep... /usr/bin/grep
checking that grep -E works... yes
checking for ar... /usr/bin/ar
checking for a BSD-compatible install... /usr/bin/install -c
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking how to run the C preprocessor... gcc -E
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for stdatomic.h... yes
checking if _Atomic is available... yes
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for code coverage support... no
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking curl version... 8.7.1
checking for httpd... no
checking for apache2... no
checking for apachectl... no
checking for apxs... no
configure: httpd/apache2 not in PATH, http tests disabled
configure: apxs not in PATH, http tests disabled
checking for nghttpx... no
checking for caddy... no
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for egrep -e... /usr/bin/grep -E
checking if OS is AIX (to define _ALL_SOURCE)... no
checking if _THREAD_SAFE is already defined... no
checking if _THREAD_SAFE is actually needed... no
checking if _THREAD_SAFE is onwards defined... no
checking if _REENTRANT is already defined... no
checking if _REENTRANT is actually needed... no
checking if _REENTRANT is onwards defined... no
checking for gcc option to enable large file support... none needed
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking whether to build shared libraries with -version-info... yes
checking whether to build shared libraries with -no-undefined... no
checking whether to build shared libraries with -mimpure-text... no
checking whether to build shared libraries with PIC... yes
checking whether to build static libraries with PIC... yes
checking whether to build shared libraries only... no
checking whether to build static libraries only... yes
checking for windres... no
checking for inline... inline
checking if cpp -P is needed... yes
checking if cpp -P works... yes
checking if compiler is DEC/Compaq/HP C... no
checking if compiler is HP-UX C... no
checking if compiler is IBM C... no
checking if compiler is Intel C... no
checking if compiler is clang... no
checking if compiler is GNU C... yes
checking compiler version... gcc '1302' (raw: '13.2.1')
checking if compiler is SGI MIPSpro C... no
checking if compiler is SGI MIPS C... no
checking if compiler is SunPro C... no
checking if compiler is Tiny C... no
checking whether build target is a native Windows one... no
checking if compiler accepts some basic options... yes
configure: compiler options added: -Werror-implicit-function-declaration 
checking if compiler optimizer assumed setting might be used... no
checking if compiler accepts strict warning options... yes
configure: compiler options added: -Wno-system-headers 
checking if compiler halts on compilation errors... yes
checking if compiler halts on negative sized arrays... yes
checking if compiler halts on function prototype mismatch... yes
checking if compiler supports hiding library internal symbols... yes
checking whether build target supports WIN32 file API... no
checking whether build target supports WIN32 crypto API... no
checking for good-to-use Darwin CFLAGS... no
checking whether to link macOS CoreFoundation, CoreServices, and SystemConfiguration frameworks... no
checking to see if the compiler supports __builtin_available()... no
checking whether to support http... yes
checking whether to support ftp... yes
checking whether to support file... yes
checking whether to support ldap... no
checking whether to support ldaps... no
checking whether to support rtsp... yes
checking whether to support proxies... yes
checking whether to support dict... yes
checking whether to support telnet... yes
checking whether to support tftp... yes
checking whether to support pop3... yes
checking whether to support imap... yes
checking whether to support smb... yes
checking whether to support smtp... yes
checking whether to support gopher... yes
checking whether to support mqtt... no
checking whether to build documentation... yes
checking whether to provide built-in manual... no
checking whether to enable generation of C code... yes
checking whether to use libgcc... no
checking if X/Open network library is required... no
checking for gethostbyname... yes
checking whether build target is a native Windows one... (cached) no
checking for proto/bsdsocket.h... no
checking for connect in libraries... yes
checking for sys/types.h... (cached) yes
checking for sys/time.h... yes
checking for monotonic clock_gettime... yes
checking for clock_gettime in libraries... no additional lib required
checking if monotonic clock_gettime works... yes
checking for sys/types.h... (cached) yes
checking for sys/time.h... (cached) yes
checking for raw monotonic clock_gettime... yes
checking for pkg-config... /usr/bin/pkg-config
checking for zlib options with pkg-config... found
checking for zlib.h... yes
configure: found both libz and libz.h header
checking for BrotliDecoderDecompress in -lbrotlidec... yes
checking for brotli/decode.h... yes
checking for ZSTD_createDStream in -lzstd... yes
checking for zstd.h... yes
checking whether to enable IPv6... yes
checking if struct sockaddr_in6 has sin6_scope_id member... yes
checking if argv can be written to... yes
checking if GSS-API support is requested... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for mit-krb5-gssapi options with pkg-config... found
checking for gss.h... no
checking for gssapi/gssapi.h... yes
checking for gssapi/gssapi_generic.h... yes
checking for gssapi/gssapi_krb5.h... yes
checking if GSS-API headers declare GSS_C_NT_HOSTBASED_SERVICE... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for mit-krb5-gssapi options with pkg-config... found
checking if we can link against GSS-API library... yes
checking whether to enable Windows native SSL/TLS... no
checking whether to enable Secure Transport... no
checking whether to enable Amiga native SSL/TLS (AmiSSL v5)... no
checking for pkg-config... (cached) /usr/bin/pkg-config
checking for rustls options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lrustls"
configure: pkg-config: SSL_LDFLAGS: ""
configure: pkg-config: SSL_CPPFLAGS: ""
configure: detected rustls
configure: error: TLS not detected, you will not be able to use HTTPS, FTPS, NTLM and more.
Use --with-openssl, --with-gnutls, --with-wolfssl, --with-mbedtls, --with-schannel, --with-secure-transport, --with-amissl, --with-bearssl or --with-rustls to address this.

I expected the following

The build passes without reverting any commits.

curl/libcurl version

curl 8.7.1

operating system

Arch Linux

@bagder
Copy link
Member

bagder commented Mar 27, 2024

/cc @Kangie

Kangie added a commit to Kangie/curl that referenced this issue Mar 27, 2024
The previous pkg-config code would successfully detect rustls
but did not set all appropriate variables and call the right macros to
properly configure cURL.

Closes: curl#13200
@Kangie
Copy link
Contributor

Kangie commented Mar 27, 2024

Sorry! How does it behave with that patch applied?

I was able to start building, but came up with this:

vtls/rustls.c: In function 'cr_init_backend':
vtls/rustls.c:425:21: error: implicit declaration of function 'rustls_root_cert_store_builder_new'; did you mean 'rustls_root_cert_store_add_pem'? [-Werror=implicit-function-declaration]
  425 |     roots_builder = rustls_root_cert_store_builder_new();
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                     rustls_root_cert_store_add_pem
vtls/rustls.c:425:19: warning: assignment to 'struct rustls_root_cert_store_builder *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
  425 |     roots_builder = rustls_root_cert_store_builder_new();
      |                   ^
vtls/rustls.c:429:16: error: implicit declaration of function 'rustls_root_cert_store_builder_add_pem'; did you mean 'rustls_root_cert_store_add_pem'? [-Werror=implicit-function-declaration]
  429 |       result = rustls_root_cert_store_builder_add_pem(roots_builder,
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                rustls_root_cert_store_add_pem
vtls/rustls.c:435:9: error: implicit declaration of function 'rustls_root_cert_store_builder_free'; did you mean 'rustls_root_cert_store_free'? [-Werror=implicit-function-declaration]
  435 |         rustls_root_cert_store_builder_free(roots_builder);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         rustls_root_cert_store_free
vtls/rustls.c:443:16: error: implicit declaration of function 'rustls_root_cert_store_builder_load_roots_from_file'; did you mean 'rustls_client_config_builder_load_roots_from_file'? [-Werror=implicit-function-declaration]
  443 |       result = rustls_root_cert_store_builder_load_roots_from_file(
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                rustls_client_config_builder_load_roots_from_file
vtls/rustls.c:454:14: error: implicit declaration of function 'rustls_root_cert_store_builder_build'; did you mean 'rustls_root_cert_store_add_pem'? [-Werror=implicit-function-declaration]
  454 |     result = rustls_root_cert_store_builder_build(roots_builder, &roots);
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |              rustls_root_cert_store_add_pem
vtls/rustls.c:463:24: error: implicit declaration of function 'rustls_web_pki_server_cert_verifier_builder_new' [-Werror=implicit-function-declaration]
  463 |     verifier_builder = rustls_web_pki_server_cert_verifier_builder_new(roots);
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vtls/rustls.c:463:22: warning: assignment to 'struct rustls_web_pki_server_cert_verifier_builder *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
  463 |     verifier_builder = rustls_web_pki_server_cert_verifier_builder_new(roots);
      |                      ^
vtls/rustls.c:465:14: error: implicit declaration of function 'rustls_web_pki_server_cert_verifier_builder_build' [-Werror=implicit-function-declaration]
  465 |     result = rustls_web_pki_server_cert_verifier_builder_build(
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vtls/rustls.c:467:5: error: implicit declaration of function 'rustls_web_pki_server_cert_verifier_builder_free' [-Werror=implicit-function-declaration]
  467 |     rustls_web_pki_server_cert_verifier_builder_free(verifier_builder);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vtls/rustls.c:470:7: error: implicit declaration of function 'rustls_server_cert_verifier_free'; did you mean 'rustls_client_cert_verifier_free'? [-Werror=implicit-function-declaration]
  470 |       rustls_server_cert_verifier_free(server_cert_verifier);
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |       rustls_client_cert_verifier_free
vtls/rustls.c:476:5: error: implicit declaration of function 'rustls_client_config_builder_set_server_verifier'; did you mean 'rustls_client_config_builder_set_certified_key'? [-Werror=implicit-function-declaration]
  476 |     rustls_client_config_builder_set_server_verifier(config_builder,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |     rustls_client_config_builder_set_certified_key
cc1: some warnings being treated as errors
make[2]: *** [Makefile:3197: vtls/libcurl_la-rustls.lo] Error 1

I suspect that's because rustls for gentoo is a bit old - I'm trying to package an updated version now.

Kangie added a commit to Kangie/curl that referenced this issue Mar 27, 2024
The previous pkg-config code would successfully detect rustls
but did not set all appropriate variables and call the right macros to
properly configure cURL.

Closes: curl#13200
Kangie added a commit to Kangie/curl that referenced this issue Mar 27, 2024
The previous pkg-config code would successfully detect rustls
but did not set all appropriate variables and call the right macros to
properly configure cURL.

Closes: curl#13200
@kpcyrd
Copy link
Contributor Author

kpcyrd commented Mar 27, 2024

Ah yes, gentoo seems to be on librustls 0.10.0 but curl 8.7.0 was specifically ported to librustls >= 0.12.0 in #12989.

@xnox
Copy link

xnox commented Mar 27, 2024

Sure but https://github.com/rustls/rustls-ffi doesn't have any pkg-config files does it? where/what do you expect to provide rustls.pc file?

@xnox
Copy link

xnox commented Mar 27, 2024

let me try using cargo-ci to build rustls-ffi to gain pgkconfig.

@kpcyrd
Copy link
Contributor Author

kpcyrd commented Mar 27, 2024

The recent 0.12.1 release has added cargo-c support that was contributed in rustls/rustls-ffi#274. The pkgconfig files are generated by cargo-c as part of the build. The issue also documents how to build it at the end. :)

@Kangie
Copy link
Contributor

Kangie commented Mar 28, 2024

@kpcyrd does the fixed pkg-config detection work resolve the issue for you? (#13202).

It all looks good for my end but we're still dealing with some rustls packaging issues for 0.12.1 with Gentoo - should be able to validate sometime this afternoon.

And with rustls-ffi fixed for Gentoo.... Yes, this now builds for me with rustls as the only tls backend using 0.12.1.

@bagder
Copy link
Member

bagder commented Mar 28, 2024

@kpcyrd do we get a thumbs up from you as well on this PR?

@kpcyrd
Copy link
Contributor Author

kpcyrd commented Mar 28, 2024

Sorry for the late reply, it does indeed fix the build but curl is now linking to openssl for some reason (despite --without-openssl):

% diff before.txt after.txt 
3a4,5
>  0x0000000000000001 (NEEDED)             Shared library: [libssl.so.3]
>  0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.3]
11c13
<  0x000000006ffffffe (VERNEED)            0x3b18
---
>  0x000000006ffffffe (VERNEED)            0x3b20
  • before.txt is curl 8.7.1 with 647e86a reverted
  • after.txt is curl 8.7.1 with d7675bf backported

It links to libssl in addition to librustls, in curl-rustls --version output it prints:

curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 rustls-ffi/0.12.1/rustls/0.22 zlib/1.3.1 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.60.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz PSL SPNEGO SSL threadsafe UnixSockets zstd

I'm really bad at reading m4 unfortunately.

@bagder
Copy link
Member

bagder commented Mar 28, 2024

curl is now linking to openssl

Isn't that simply because libssh2 uses OpenSSL?

@kpcyrd
Copy link
Contributor Author

kpcyrd commented Mar 28, 2024

libssh2 does indeed pull in libssl according it ldd, but the output above is from readelf -d /usr/bin/curl-rustls | rg NEED, meaning the curl binary itself has a direct dependency in libssl.

@Kangie
Copy link
Contributor

Kangie commented Mar 28, 2024

I can't see anything in the updated M4 that would cause that.

Comparing an 8.6.0 build without any of the M4 changes to 8.7.1, both built with the following configure options:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --datarootdir=/usr/share --disable-dependency-tracking --disable-silent-rules --disable-static --docdir=/usr/share/doc/curl-8.6.0-r1 --htmldir=/usr/share/doc/curl-8.6.0-r1/html --with-sysroot=/ --libdir=/usr/lib64 --without-ca-fallback --with-ca-bundle=/etc/ssl/certs/ca-certificates.crt --without-gnutls --without-mbedtls --without-rustls --with-rustls --with-default-ssl-backend=rustls --enable-alt-svc --enable-basic-auth --enable-bearer-auth --enable-digest-auth --enable-kerberos-auth --enable-negotiate-auth --enable-aws --enable-dict --disable-ech --enable-file --enable-ftp --disable-gopher --enable-hsts --enable-http --enable-imap --disable-ldap --disable-ldaps --enable-ntlm --disable-ntlm-wb --enable-pop3 --enable-rt --enable-rtsp --disable-smb --without-libssh2 --enable-smtp --disable-telnet --enable-tftp --enable-tls-srp --enable-ares --enable-cookies --enable-dateparse --enable-dnsshuffle --enable-doh --enable-symbol-hiding --enable-http-auth --enable-ipv6 --enable-largefile --enable-manual --enable-mime --enable-netrc --enable-progress-meter --enable-proxy --enable-socketpair --disable-sspi --disable-static --enable-pthreads --enable-threaded-resolver --disable-versioned-symbols --without-amissl --without-bearssl --without-brotli --with-fish-functions-dir=/usr/share/fish/vendor_completions.d --with-nghttp2 --without-hyper --without-libidn2 --without-gssapi --without-libgsasl --with-libpsl --without-msh3 --without-nghttp3 --without-ngtcp2 --without-quiche --without-librtmp --without-schannel --without-secure-transport --without-test-caddy --without-test-httpd --without-test-nghttpx --disable-websockets --without-winidn --without-wolfssl --with-zlib --with-zstd --with-zsh-functions-dir=/usr/share/zsh/site-functions --with-test-nghttpx=/usr/bin/nghttpx

I see:

curl 8.6.0 (x86_64-pc-linux-gnu) libcurl/8.6.0 rustls-ffi/0.10.0/rustls/0.21.0 zlib/1.3.1 zstd/1.5.5 c-ares/1.27.0 libpsl/0.21.5 nghttp2/1.60.0
Release-Date: 2024-01-31
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps tftp
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz PSL SSL threadsafe UnixSockets zstd
curl => /usr/bin/curl (interpreter => /lib64/ld-linux-x86-64.so.2)
    libcurl.so.4 => /usr/lib64/libcurl.so.4
        libcares.so.2 => /usr/lib64/libcares.so.2
        libnghttp2.so.14 => /usr/lib64/libnghttp2.so.14
        libpsl.so.5 => /usr/lib64/libpsl.so.5
            libidn2.so.0 => /usr/lib64/libidn2.so.0
            libunistring.so.5 => /usr/lib64/libunistring.so.5
        librustls.so.0.10 => /usr/lib64/librustls.so.0.10
            libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1
            ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
        libm.so.6 => /usr/lib64/libm.so.6
        libzstd.so.1 => /usr/lib64/libzstd.so.1
    libz.so.1 => /usr/lib64/libz.so.1
    libc.so.6 => /usr/lib64/libc.so.6

curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 rustls-ffi/0.12.1/rustls/0.22 zlib/1.3.1 zstd/1.5.5 c-ares/1.27.0 libpsl/0.21.5 nghttp2/1.60.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps tftp
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz PSL SSL threadsafe UnixSockets zstd
curl => /usr/bin/curl (interpreter => /lib64/ld-linux-x86-64.so.2)
    libcurl.so.4 => /usr/lib64/libcurl.so.4
    libcares.so.2 => /usr/lib64/libcares.so.2
    libnghttp2.so.14 => /usr/lib64/libnghttp2.so.14
    libpsl.so.5 => /usr/lib64/libpsl.so.5
        libidn2.so.0 => /usr/lib64/libidn2.so.0
        libunistring.so.5 => /usr/lib64/libunistring.so.5
    librustls.so.0.12.1 => /usr/lib64/librustls.so.0.12.1
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1
        ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
    libzstd.so.1 => /usr/lib64/libzstd.so.1
    libz.so.1 => /usr/lib64/libz.so.1
    libc.so.6 => /usr/lib64/libc.so.6

It is interesting that neither shows 'rustls' in the version info, but I don't think this (or linking against OpenSSL) is a regression, just another subtly-different configure bug. Or something.

Edit: I can be wrong - system libcurl. Oops! Updated results above... I don't see any libssl.

It's family time for most of this weekend - I might take a deeper look next week if this is still unresolved.

I had a bug to close so I backported this patch to 8.6.0 and 8.5.0 and tested in a musl chroot:

curl 8.6.0 (x86_64-gentoo-linux-musl) libcurl/8.6.0 rustls-ffi/0.10.0/rustls/0.21.0 zlib/1.3 c-ares/1.25.0 libpsl/0.21.5
Release-Date: 2024-01-31
Protocols: dict file ftp ftps http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smtp smtps tftp
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IPv6 Largefile libz PSL SSL threadsafe UnixSockets
curl => /usr/bin/curl (interpreter => /lib/ld-musl-x86_64.so.1)
    libcurl.so.4 => /usr/lib/libcurl.so.4
    libcares.so.2 => /usr/lib/libcares.so.2
    libpsl.so.5 => /usr/lib/libpsl.so.5
        libidn2.so.0 => /usr/lib/libidn2.so.0
        libunistring.so.5 => /usr/lib/libunistring.so.5
    librustls.so.0.10 => /usr/lib/librustls.so.0.10
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-gentoo-linux-musl/13/libgcc_s.so.1
    libz.so.1 => /usr/lib/libz.so.1
    libc.so => /usr/lib/libc.so

curl 8.5.0 (x86_64-gentoo-linux-musl) libcurl/8.5.0 rustls-ffi/0.10.0/rustls/0.21.0 zlib/1.3 c-ares/1.25.0 libpsl/0.21.5 (+libidn2/2.3.7)
Release-Date: 2023-12-06
Protocols: dict file ftp ftps http https imap imaps mqtt pop3 pop3s rtsp smtp smtps tftp
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IPv6 Largefile libz PSL SSL threadsafe UnixSockets
curl => /usr/bin/curl (interpreter => /lib/ld-musl-x86_64.so.1)
    libcurl.so.4 => /usr/lib/libcurl.so.4
        libcares.so.2 => /usr/lib/libcares.so.2
        libpsl.so.5 => /usr/lib/libpsl.so.5
            libidn2.so.0 => /usr/lib/libidn2.so.0
            libunistring.so.5 => /usr/lib/libunistring.so.5
        librustls.so.0.10 => /usr/lib/librustls.so.0.10
            libgcc_s.so.1 => /usr/lib/gcc/x86_64-gentoo-linux-musl/13/libgcc_s.so.1
    libz.so.1 => /usr/lib/libz.so.1
    libc.so => /usr/lib/libc.so
    
# and for fun with libtree on 8.5.0
/usr/bin/curl
├── libcurl.so.4 [ld.so.conf]
│   ├── libcares.so.2 [ld.so.conf]
│   ├── libz.so.1 [ld.so.conf]
│   ├── librustls.so.0.10 [ld.so.conf]
│   └── libpsl.so.5 [ld.so.conf]
│       ├── libidn2.so.0 [ld.so.conf]
│       │   └── libunistring.so.5 [ld.so.conf]
│       └── libunistring.so.5 [ld.so.conf]
└── libz.so.1 [ld.so.conf]

  

Kangie added a commit to Kangie/curl that referenced this issue Mar 29, 2024
The previous pkg-config code would successfully detect rustls
but did not set all appropriate variables and call the right macros to
properly configure cURL.

Closes: curl#13200
@Kangie
Copy link
Contributor

Kangie commented Mar 29, 2024

fixed some logic in the 'with path and pkgconf' section, where some variables could get set early but not related to this :)

@kpcyrd
Copy link
Contributor Author

kpcyrd commented Mar 31, 2024

It seems there are further issues: rustls/rustls-ffi#407

Maybe we should consider reverting 647e86a and the followup changes.

(There's way too much autotools in my life right now)

@cpu
Copy link
Contributor

cpu commented Mar 31, 2024

I can confirm that reverting both 9c42098 and 647e86a resolves the issues I see locally, and the issue reported in #13248 when using curl and a dynamically linked librustls.

@ecnelises
Copy link

ecnelises commented Apr 9, 2024

Is there any plan for 8.7.2 after fixing this?

@Kangie
Copy link
Contributor

Kangie commented Apr 9, 2024

Is there any plan for 8.7.2 after fixing this?

I can't speak for the project but curl typically only release patch revisions for severe issues, distributions and other downstream users typically backport patches (like this) if they feel like it's something they need before the next official release.

@bagder
Copy link
Member

bagder commented Apr 9, 2024

We have not decided to do an 8.7.2 patch release.

  • rustls support is still experimental
  • this issue "only" affects the build
  • no one has yelled loud enough during the first two weeks since the previous release
  • we are only six weeks away from the next release

Our rough guidance documentation for doing "early releases" is here: https://github.com/curl/curl/blob/master/docs/EARLY-RELEASE.md

@ecnelises
Copy link

Thanks! That makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

6 participants