Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot build curl 7.81.0 with --with-wolfssl #8292

Closed
harrysarson opened this issue Jan 17, 2022 · 12 comments
Closed

Cannot build curl 7.81.0 with --with-wolfssl #8292

harrysarson opened this issue Jan 17, 2022 · 12 comments
Labels
build TLS

Comments

@harrysarson
Copy link
Contributor

I did this

CURL_VERSION=curl-7.81.0

curl https://curl.se/download/$CURL_VERSION.tar.gz | tar -xz
curl -L https://github.com/wolfSSL/wolfssl/archive/refs/tags/v4.8.1-stable.tar.gz | tar -xz

mkdir wolfssl-install
mkdir wolfssl-dir

ls

pushd wolfssl-4.8.1-stable
    
autoreconf -fi
./configure --prefix=$(realpath ../wolfssl-install)  --enable-opensslextra
make -j$(nproc) 
make test 
make install

popd
pushd $CURL_VERSION

autoreconf -fi
./configure --with-wolfssl=$(realpath ../wolfssl-install) 
make -j$(nproc)

popd

I expected the following

Curl can build with wolfssl (all the commands run successfully), as it does with CURL_VERSION=curl-7.79.1. Instead it fails to build with error:

md5.c:85:10: fatal error: openssl/md5.h: No such file or directory
   85 | #include <openssl/md5.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
In file included from http_aws_sigv4.c:33:
curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:2474: libcurl_la-md5.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from sha256.c:29:
curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [Makefile:2404: libcurl_la-http_aws_sigv4.lo] Error 1
make[2]: *** [Makefile:2621: libcurl_la-sha256.lo] Error 1
In file included from vauth/digest.c:38:
../lib/curl_sha256.h:35:10: fatal error: openssl/sha.h: No such file or directory
   35 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.

(Note CURL_VERSION=curl-7.80.0 also fails).

curl/libcurl version

7.81.0

operating system

WSL: Linux D-00076 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Notes

I suspect #7806 is at fault here. It expects that the wolfssl openssl compatability layer uses openssl/*.h include paths. However, (atleast as my reading of https://www.wolfssl.com/docs/wolfssl-manual/ch13/) the wolfssl openssl compatability layer actually puts the header files at wolfssl/openssl/*.h:

image

I have a patch that fixes this for me locally that I can submit if the above conclusion is correct.
@jay
Copy link
Member

jay commented Jan 17, 2022

I get a bunch of errors with master and less with 7.79.1. The errors I get in 7.79.1 are because of # include <openssl/ssl.h>. If the compatibility layer is enabled is it supposed to include the parent wolfssl include directory? I'm using VS project files which include only the wolfssl source directory and then one can write #include <wolfssl/... to include from include directory wolfssl/wolfssl but not #include <openssl/... to include wolfssl/wolfssl/openssl.

@bagder
Copy link
Member

bagder commented Jan 17, 2022

--enable-opensslextra

You probably need more openssl capability enabled. Try with --enable-all first and see if that doesn't get you going.

It expects that the wolfssl openssl compatability layer uses openssl/.h include paths. However, (atleast as my reading of https://www.wolfssl.com/docs/wolfssl-manual/ch13/) the wolfssl openssl compatability layer actually puts the header files at wolfssl/openssl/.h:

The configure script adapts to that. We successfully build with wolfSSL in our CI builds for every commit and for every PR after all...

@bagder
Copy link
Member

bagder commented Jan 17, 2022

If the compatibility layer is enabled is it supposed to include the parent wolfssl include directory

The compatibility layer needs to be enabled for curl to build with wolfSSL.

@bagder
Copy link
Member

bagder commented Jan 17, 2022

Try with --enable-all first

I should mention that wolfSSL will soon ship with an --enable-curl option for their configure which will make this process a little easier...

jay added a commit to jay/curl that referenced this issue Jan 19, 2022
@jay
projects: Fix Visual Studio wolfSSL configurations
6f4e8d6 
- Change build-wolfssl.bat to disable SSLv3, enable TLSv1.3, enable
  wolfSSL_DES_ecb_encrypt (needed by NTLM) and enable alt cert chains.

- Disable warning C4214 'bit field types other than int'.

- Add include directory wolfssl\wolfssl.

wolfSSL offers OpenSSL API compatibility that libcurl uses, and some
recent change in libcurl included an include file for wolfSSL like
openssl/foo.h, which has a path like wolfssl\wolfssl\openssl\foo.h.

The include directory issue was reported in curl#8292 but it's currently
unclear whether this type of change is needed for other build systems.

Bug: curl#8292
Reported-by: Harry Sarson

Closes #xxxx
@jay jay mentioned this issue Jan 19, 2022
Closed
@jay
Copy link
Member

jay commented Jan 19, 2022 •
edited

Ref: https://github.com/curl/curl/blob/curl-7_81_0/m4/curl-wolfssl.m4

What does pkg-config --cflags-only-I wolfssl show? It looks like we don't add wolfssl/openssl directly so the only way it would be added is if it came from pkg-config.

You probably need more openssl capability enabled. Try with --enable-all first and see if that doesn't get you going

I don't think it is practical to enable-all. There may be some flag that does it, but I can't find it. (edit: Fair enough, as a diagnostic --enable-all may be helpful here.)

@bagder
Copy link
Member

bagder commented Jan 19, 2022

It looks like we don't add wolfssl/openssl directly

We add the wolfssl part only, as the openssl part is what we use in the include names:

curl/m4/curl-wolfssl.m4

Lines 137 to 152 in fde0925

AC_DEFINE(HAVE_WOLFSSL_DES_ECB_ENCRYPT, 1,
[if you have wolfSSL_DES_ecb_encrypt])
if test -n "$addcflags"; then
dnl use a for loop to strip off whitespace
for f in $addcflags; do
CPPFLAGS="$f/wolfssl $CPPFLAGS"
AC_MSG_NOTICE([Add $f/wolfssl to CPPFLAGS])
break
done
else
dnl user didn't give a path, so guess/hope they installed wolfssl
dnl headers to system default location
CPPFLAGS="-I/usr/include/wolfssl $CPPFLAGS"
AC_MSG_NOTICE([Add /usr/include/wolfssl to CPPFLAGS])
fi
WOLFSSL_NTLM=1

@jay
Copy link
Member

jay commented Jan 19, 2022

We add the wolfssl part only, as the openssl part is what we use in the include names:

Ok I missed that was added when wolfSSL_DES_ecb_encrypt.

@bagder
Copy link
Member

bagder commented Jan 20, 2022

When I added that logic, curl would still build without that function and just have less functionality. Since then we broke that flexibility. We should probably make configure check for the functions we need instead of causing link errors much later.

@harrysarson
Copy link
Contributor Author

hey, I confirmed that the build issue I was facing seems resolved by configuring wolfssl with --enable-all, thanks!

Would there be interest in keeping support for building curl against wolfssl configured with just --enable-opensslextra? I don't think the changes needed to restore support would be too great.

@bagder
Copy link
Member

bagder commented Jan 20, 2022

I think it would be cool to make curl adapt to whatever features wolfSSL was built with, instead of us insisting on a particular feature set: yes.

@jay
Copy link
Member

jay commented Jan 21, 2022

Couldn't there also be a real openssl include directory (like say in /usr/local/include/openssl) that would end up taking precedence before wolfssl/wolfssl/openssl?

@bagder
Copy link
Member

bagder commented Jan 21, 2022

It probably could, but that would then be a different problem.

jay added a commit that referenced this issue Jan 21, 2022
@jay
projects: Fix Visual Studio wolfSSL configurations
f74b6d8 
- Change build-wolfssl.bat to disable SSLv3, enable TLSv1.3, enable
  wolfSSL_DES_ecb_encrypt (needed by NTLM) and enable alt cert chains.

- Disable warning C4214 'bit field types other than int'.

- Add include directory wolfssl\wolfssl.

wolfSSL offers OpenSSL API compatibility that libcurl uses, and some
recent change in libcurl included an include file for wolfSSL like
openssl/foo.h, which has a path like wolfssl\wolfssl\openssl\foo.h.

The include directory issue was reported in #8292 but it's currently
unclear whether this type of change is needed for other build systems.

Bug: #8292
Reported-by: Harry Sarson

Closes #8298
@harrysarson harrysarson mentioned this issue Jan 21, 2022
Closed
@bagder bagder closed this as completed in 392fdb0 Jan 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build TLS
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix #8292: allow building curl when wolfssl configured with --enable-opensslextra harrysarson/curl
3 participants
@bagder @jay @harrysarson