Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error building cURL 7.61.0 on SLES 11 SP3 #2890

Closed
mathomp4 opened this issue Aug 16, 2018 · 10 comments
Closed

Error building cURL 7.61.0 on SLES 11 SP3 #2890

mathomp4 opened this issue Aug 16, 2018 · 10 comments
Labels
build TLS

Comments

@mathomp4
Copy link
Contributor

I did this

I recently tried building 7.61.0 but it threw an error. cURL 7.60.0 works just fine on this machine, but the OS is a bit old, so I'm wondering if I've finally hit a "final" version of cURL for it?

Using:

gcc (GCC) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

as my compiler on SLES 11 SP3, I configured cURL with:

  $ ./configure \
    --prefix=/discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux \
    --includedir=/discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux/include/ \
    --libdir=/discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux/lib \
    --with-zlib=/discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux \
    --disable-ldap --enable-manual --disable-shared --enable-static \
    --without-libidn CCFLAGS=-fPIC  CC=gcc CXX=g++ FC=ifort

Configure succeeds, but on make I get:

configure: Configured to build curl/libcurl:

  curl version:     7.61.0
  Host setup:       x86_64-pc-linux-gnu
  Install prefix:   /discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux
  Compiler:         gcc
  SSL support:      enabled (OpenSSL)
  SSH support:      no      (--with-libssh2)
  zlib support:     enabled
  brotli support:   no      (--with-brotli)
  GSS-API support:  no      (--with-gssapi)
  TLS-SRP support:  no      (--enable-tls-srp)
  resolver:         POSIX threaded
  IPv6 support:     enabled
  Unix sockets support: enabled
  IDN support:      no      (--with-{libidn2,winidn})
  Build libcurl:    Shared=no, Static=yes
  Built-in manual:  enabled
  --libcurl option: enabled (--disable-libcurl-option)
  Verbose errors:   enabled (--disable-verbose)
  SSPI support:     no      (--enable-sspi)
  ca cert bundle:   no
  ca cert path:     /etc/ssl/certs/
  ca fallback:      no
  LDAP support:     no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
  LDAPS support:    no      (--enable-ldaps)
  RTSP support:     enabled
  RTMP support:     no      (--with-librtmp)
  metalink support: no      (--with-libmetalink)
  PSL support:      no      (libpsl not found)
  HTTP2 support:    disabled (--with-nghttp2)
  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP

make[1]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src'
make[1]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src'
make[2]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src/curl'
Making install in lib
make[3]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src/curl/lib'
  CC       vauth/libcurl_la-vauth.lo
  CC       vauth/libcurl_la-cleartext.lo
  CC       vauth/libcurl_la-cram.lo
  CC       vauth/libcurl_la-digest.lo
  CC       vauth/libcurl_la-digest_sspi.lo
  CC       vauth/libcurl_la-krb5_gssapi.lo
  CC       vauth/libcurl_la-krb5_sspi.lo
  CC       vauth/libcurl_la-ntlm.lo
  CC       vauth/libcurl_la-ntlm_sspi.lo
  CC       vauth/libcurl_la-oauth2.lo
  CC       vauth/libcurl_la-spnego_gssapi.lo
  CC       vauth/libcurl_la-spnego_sspi.lo
  CC       vtls/libcurl_la-openssl.lo
  CC       vtls/libcurl_la-gtls.lo
  CC       vtls/libcurl_la-vtls.lo
  CC       vtls/libcurl_la-nss.lo
  CC       vtls/libcurl_la-polarssl.lo
  CC       vtls/libcurl_la-polarssl_threadlock.lo
  CC       vtls/libcurl_la-axtls.lo
  CC       vtls/libcurl_la-cyassl.lo
  CC       vtls/libcurl_la-schannel.lo
vtls/openssl.c: In function ‘Curl_ossl_cleanup’:
vtls/openssl.c:1019:3: error: implicit declaration of function ‘ENGINE_cleanup’ [-Werror=implicit-function-declaration]
   ENGINE_cleanup();
   ^~~~~~~~~~~~~~
  CC       vtls/libcurl_la-darwinssl.lo
  CC       vtls/libcurl_la-schannel_verify.lo
  CC       vtls/libcurl_la-gskit.lo
  CC       vtls/libcurl_la-mbedtls.lo
  CC       libcurl_la-file.lo
  CC       libcurl_la-timeval.lo
  CC       libcurl_la-base64.lo
  CC       libcurl_la-hostip.lo
  CC       libcurl_la-progress.lo
  CC       libcurl_la-formdata.lo
  CC       libcurl_la-cookie.lo
  CC       libcurl_la-http.lo
  CC       libcurl_la-sendf.lo
  CC       libcurl_la-ftp.lo
cc1: some warnings being treated as errors
make[3]: *** [vtls/libcurl_la-openssl.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src/curl/lib'
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src/curl'
make[1]: *** [curl.install] Error 2
make[1]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src'
make: *** [install] Error 2

NOTE: I mainly build cURL so that I can add OpenDAP support to netCDF in a set of base libraries for a climate model. I don't really need cURL to do much fancy, so I'm willing to --disable anything you might like me to try.

I expected the following

Well, the make to succeed.

curl/libcurl version

Trying to build 7.61.0

operating system

SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 3
@mathomp4
Copy link
Contributor Author

Further information. cURL 7.61.0 built fine on a SLES12 SP3 machine. In doing the build I noticed that SLES 11 said:
TLS-SRP support: no (--enable-tls-srp)
while the SLES 12 build:
TLS-SRP support: enabled

And on SLES 12, it seemed to "skip" the vauth/vtls build:

  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP

make[1]: Leaving directory '/nobackupp2/gmao_SIteam/Baselibs/ESMA-Baselibs-5.1.4/src'
make[1]: Entering directory '/nobackupp2/gmao_SIteam/Baselibs/ESMA-Baselibs-5.1.4/src'
make[2]: Entering directory '/nobackupp2/gmao_SIteam/Baselibs/ESMA-Baselibs-5.1.4/src/curl'
Making install in lib
make[3]: Entering directory '/nobackupp2/gmao_SIteam/Baselibs/ESMA-Baselibs-5.1.4/src/curl/lib'
  CC       libcurl_la-timeval.lo
  CC       libcurl_la-file.lo
  CC       libcurl_la-base64.lo
  CC       libcurl_la-hostip.lo
  CC       libcurl_la-progress.lo
  CC       libcurl_la-formdata.lo
  CC       libcurl_la-cookie.lo
  CC       libcurl_la-http.lo
  CC       libcurl_la-sendf.lo
  CC       libcurl_la-ftp.lo

With 7.60.0 on SLES11, I get:

configure: Configured to build curl/libcurl:

  curl version:     7.60.0
  Host setup:       x86_64-pc-linux-gnu
  Install prefix:   /discover/swdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.3/x86_64-unknown-linux-gnu/ifort_18.0.3.222-intelmpi_18.0.3.222/Linux
  Compiler:         gcc
  SSL support:      enabled (OpenSSL)
  SSH support:      no      (--with-libssh2)
  zlib support:     enabled
  brotli support:   no      (--with-brotli)
  GSS-API support:  no      (--with-gssapi)
  TLS-SRP support:  no      (--enable-tls-srp)
  resolver:         POSIX threaded
  IPv6 support:     enabled
  Unix sockets support: enabled
  IDN support:      no      (--with-{libidn2,winidn})
  Build libcurl:    Shared=no, Static=yes
  Built-in manual:  enabled
  --libcurl option: enabled (--disable-libcurl-option)
  Verbose errors:   enabled (--disable-verbose)
  SSPI support:     no      (--enable-sspi)
  ca cert bundle:   no
  ca cert path:     /etc/ssl/certs/
  ca fallback:      no
  LDAP support:     no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
  LDAPS support:    no      (--enable-ldaps)
  RTSP support:     enabled
  RTMP support:     no      (--with-librtmp)
  metalink support: no      (--with-libmetalink)
  PSL support:      no      (libpsl not found)
  HTTP2 support:    disabled (--with-nghttp2)
  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP

make[1]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.3/src'
make[1]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.3/src'
make[2]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.3/src/curl'
Making install in lib
make[3]: Entering directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.3/src/curl/lib'
  CC       libcurl_la-hostip.lo
  CC       libcurl_la-file.lo
  CC       libcurl_la-timeval.lo
  CC       libcurl_la-base64.lo
  CC       libcurl_la-progress.lo
  CC       libcurl_la-formdata.lo
  CC       libcurl_la-cookie.lo

So it has the same TLS-SRP support indicator, but it also doesn't build into vauth/vtls.

Looking at the systems, the SLES 11 has gnutls 2.4.1 and SLES 12 has gnutls 3.3.27. Would that matter?

@bagder
Copy link
Member

bagder commented Aug 16, 2018

vtls/openssl.c:1019:3: error: implicit declaration of function ‘ENGINE_cleanup’ [-Werror=implicit-function-declaration]
ENGINE_cleanup();

So this looks like a problem with your OpenSSL version. We changed how we use the engine features because it is provided by default in openssl since a while back. Did you make a special build of OpenSSL without it? What openssl version is this?

@mathomp4
Copy link
Contributor Author

mathomp4 commented Aug 17, 2018 •
edited

Looks to be: OpenSSL 0.9.8j-fips 07 Jan 2009

The machines I can build on:
OpenSSL 1.0.2k-fips 26 Jan 2017
OpenSSL 1.0.2j 26 Sep 2016

As I said, old OS.

I'm guessing I might have to degrade cURL on the SLES 11 machine? I'm not sure the admins will want to upgrade OpenSSL if possible. (It's a supercomputing cluster that will eventually move to SLES 12).

Or is there a flag I could pass to avoid this?

@bagder
Copy link
Member

bagder commented Aug 17, 2018

ENGINE_cleanup should only be used by libcurl if configure detected that it exists:

curl/lib/vtls/openssl.c

Lines 1050 to 1053 in ba58ce6

#ifdef HAVE_ENGINE_CLEANUP
/* Free engine list */
ENGINE_cleanup();
#endif

so I'm a little puzzled why it tries to use this with OpenSSL 0.9.8j which presumably doesn't have it. Maybe config.log contains some clue if you check for the ENGINE_cleanup check in there. Also, lib/curl_config.h should have a line like:

/* Define to 1 if you have the `ENGINE_cleanup' function. */
/* #undef HAVE_ENGINE_CLEANUP */

@bagder
Copy link
Member

bagder commented Aug 17, 2018

old OS

curl doesn't really care about the age of your OS. It builds on way older installs than yours too. It should build on anything close to posix with a c89 compiler. What is sometimes problematic on the old ones is the 3rd party libs and the versions of them, as curl has a lowest-version supported for each of them.

A big challenge with the oldies is that they're less tested and used so build errors tend to sneak in more often just because it was so long time since someone verified the build on that system.

@mathomp4
Copy link
Contributor Author

Hmm. Well, I do see a lot of OpenSSL changes here: 38203f1, which is why I assume my 7.60.0 build has these lines:

checking for ENGINE_init... yes
checking openssl/engine.h usability... yes
checking openssl/engine.h presence... yes
checking for openssl/engine.h... yes
checking for ENGINE_load_builtin_engines... yes

in the configure output, but 7.61.0 doesn't?

As you are the expert, I'm attaching my config.log and make logs from builds of 7.60.0 and 7.61.0 on the same machine with the same compilers. Hopefully you can see something pertinent.
curl-7.61.0.make.log
curl-7.60.0.make.log
curl-7.61.0.config.log
curl-7.60.0.config.log

@mathomp4
Copy link
Contributor Author

Also, I don't know if this could cause it, but on this system there does seem to be a couple libopenssl:

libopenssl1_0_0-1.0.1g-0.57.1
libopenssl0_9_8-32bit-0.9.8j-0.105.1
libopenssl0_9_8-0.9.8j-0.105.1
libopenssl-devel-0.9.8j-0.105.1

But only one has the devel package, which is why I assume configure is seeing the 0.9.8?

I can also report that if I try to build with: --without-ssl --with-gnutls, I get a different crash:

  CC       vtls/libcurl_la-polarssl_threadlock.lo
  CC       vtls/libcurl_la-axtls.lo
vtls/gtls.c:35:29: fatal error: gnutls/abstract.h: No such file or directory
 #include <gnutls/abstract.h>
                             ^
compilation terminated.
make[3]: *** [vtls/libcurl_la-gtls.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
  CC       vtls/libcurl_la-cyassl.lo
make[3]: Leaving directory `/gpfsm/dswdev/mathomp4/Baselibs/ESMA-Baselibs-5.1.4/src/curl/lib'

Which is true. The libgnutls-devel-2.4.1-24.39.51.1 package does not seem to have gnutls/abstract.h. Perhaps that GnuTLS is too old?

@bagder
Copy link
Member

bagder commented Aug 17, 2018 •
edited

I do see a lot of OpenSSL changes here: 38203f1

Yes, and that change made the engine support require OpenSSL 1.0.1 or later. So for your 0.9.8, it would not use it. But still it does...

on this system there does seem to be a couple libopenssl

That is likely to be the problem. It could be that configure finds one lib to setup things for, and then when you build curl it compiles and links with another and thus you get compiler errors.

with: --without-ssl --with-gnutls, I get a different crash

That's not a crash, that's a compiler error. But it looks like we broke compatibility with really old gnutls versions a while ago with e644866 (curl 7.39.0) gnutls/abstract.h seems to have been introduced in gnutls 2.11.3 so I presume nothing older will work without patching the code.

bagder added a commit that referenced this issue Aug 21, 2018
@bagder
INTERNALS: require GnuTLS >= 2.11.3
362e9cc 
Since the public pinning support was brought in e644866. GnuTLS
2.11.3 was released in October 2010.

Figured out in #2890
@bagder
Copy link
Member

bagder commented Aug 24, 2018

Summed up, the problems shown in here so far has been too old dependencies. curl itself will build on these systems either without these dependencies or with updated versions of them. Closing this issue.

@bagder bagder closed this as completed Aug 24, 2018
xquery pushed a commit to xquery/curl that referenced this issue Sep 3, 2018
@bagder @xquery
INTERNALS: require GnuTLS >= 2.11.3
c0676a1 
Since the public pinning support was brought in e644866. GnuTLS
2.11.3 was released in October 2010.

Figured out in curl#2890
falconindy pushed a commit to falconindy/curl that referenced this issue Sep 10, 2018
@bagder @falconindy
INTERNALS: require GnuTLS >= 2.11.3
d385f36 
Since the public pinning support was brought in e644866. GnuTLS
2.11.3 was released in October 2010.

Figured out in curl#2890
@mipon
Copy link

mipon commented Nov 5, 2018

Got the same error with 7.61.1 and tried 7.62.0 and it went through. Thanks for fixing it.

@lock lock bot locked as resolved and limited conversation to collaborators Feb 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
build TLS
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bagder @mathomp4 @mipon