Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake FetchContent with ZLIB fails (>=8.0.0) #11285

Closed
gvne opened this issue Jun 9, 2023 · 7 comments
Closed

CMake FetchContent with ZLIB fails (>=8.0.0) #11285

gvne opened this issue Jun 9, 2023 · 7 comments
Labels

Comments

@gvne
Copy link

gvne commented Jun 9, 2023

I did this

Include curl with zlib support in my CMake project using FetchContent.
Minimal CMakeLists.txt:

cmake_minimum_required(VERSION 3.25)

project(test)

include(FetchContent)
FetchContent_Declare(zlib
  GIT_REPOSITORY https://github.com/madler/zlib.git
  GIT_TAG v1.2.13
)
FetchContent_MakeAvailable(zlib)
set(ZLIB_INCLUDE_DIRS ${zlib_SOURCE_DIR})
set(ZLIB_LIBRARY zlibstatic CACHE STRING "")
set(ZLIB_LIBRARIES ${ZLIB_LIBRARY})
set(ZLIB_FOUND 1)
add_library(ZLIB::ZLIB ALIAS zlibstatic)  # required for CMake > 3.4

FetchContent_Declare(curl
  URL https://github.com/curl/curl/releases/download/curl-8_1_2/curl-8.1.2.tar.xz
)
FetchContent_MakeAvailable(curl)

I expected the following

The CMake configuration to run smoothly

curl/libcurl version

8.1.2 (Issue started with 8.0.0)

operating system

Linux pop-os 6.2.6-76060206-generic #202303130630168375320722.04~77c1465 SMP PREEMPT_DYNAMIC Wed M x86_64 x86_64 x86_64 GNU/Linux

What I get

When configuring, I get the following error:

[cmake] -- Looking for idn2_lookup_ul in idn2;OpenSSL::SSL;OpenSSL::Crypto;ZLIB::ZLIB
[cmake] CMake Error at build/CMakeFiles/CMakeScratch/TryCompile-2M2kVT/CMakeLists.txt:25 (target_link_libraries):
[cmake]   Target "cmTC_70ab4" links to:
[cmake] 
[cmake]     ZLIB::ZLIB
[cmake] 
[cmake]   but the target was not found.  Possible reasons include:
[cmake] 
[cmake]     * There is a typo in the target name.
[cmake]     * A find_package call is missing for an IMPORTED target.
[cmake]     * An ALIAS target is missing.

When digging a little, I found something I don't understand in the CMake macro check_library_exists_concat

Do we need to concatenate the searched library with the current state of the CURL_LIBS variable when running the check_library_exists ?
Removing it fixes my issue.

Thanks !

@jzakrzewski
Copy link
Contributor

I don't like this behaviour either (the same for the headers checks), but my understanding is that this is basically cloning the autotools behaviour, thus probably making the logic easier to maintain.

@bagder bagder added the cmake label Jun 9, 2023
@bagder
Copy link
Member

bagder commented Jun 9, 2023

If the code breaks the build, then what's the proposed fix? The configure build seems to work so if so this was an incomplete "mimic"...

I doubt that trying to "clone behavior" is a very good idea long term, as the two systems are very different, using different paradigms and they are also often maintained and updated by a different set of people.

@jzakrzewski
Copy link
Contributor

Well, it's like that since I can remember.
In fact, curl's CMake build even reads some Makefile.am files in to extract some information from there.

Maybe, just maybe, re-writing it from scratch would make sense (there are benefits and downsides of doing so).

@bagder
Copy link
Member

bagder commented Jun 9, 2023

It is not really "Makefile.am files" though. They are files we share between the build systems to avoid having to maintain two separate lists of the same things.

@jzakrzewski
Copy link
Contributor

May very well be - I know nothing about autotools - and it makes sense to have single source of truth like that.

@gvne
Copy link
Author

gvne commented Jun 12, 2023

I understand that my first suggestion of removing the CURL_LIBS has an impact I don't understand entirely.

The real issue after all is that this list does not only contains libraries but also CMake targets. Would removing targets from the list before calling check_library_exists be a valid option to you ?

@nmoinvaz
Copy link
Contributor

We have encountered this same issue. We don't use FetchContent but we use the same ZLIB::ZLIB CMake alias.

If you guys are looking to continue to emulate autotools, here is a possible solution that removes targets from the list before performing check_library_exists.

macro(check_library_exists_concat LIBRARY SYMBOL VARIABLE)
  # Enumerate CURL_LIBS and add each library to CMAKE_REQUIRED_LIBRARIES if it
  # is not a CMake target. System libraries should not depend on CMake targets.
  set(CMAKE_REQUIRED_LIBRARIES)
  foreach(LIB ${CURL_LIBS})
    if (NOT TARGET ${LIB})
      set(CMAKE_REQUIRED_LIBRARIES ${LIB} ${CMAKE_REQUIRED_LIBRARIES})
    endif()
  endforeach()
  # Using CMake targets in the call to check_library_exists() may cause it to fail
  # unless the target is IMPORTED.  
  check_library_exists("${LIBRARY}" ${SYMBOL} "${CMAKE_LIBRARY_PATH}"
    ${VARIABLE})
  set(CMAKE_REQUIRED_LIBRARIES)
  if(${VARIABLE})
    set(CURL_LIBS ${LIBRARY} ${CURL_LIBS})
  endif()
endmacro()

The comment for the check_library_exists_concat macro mentions:

# Some libraries depend on others to link correctly.

I don't doubt the claim, but I'm not aware of a specific instance where this is the case. However, if you don't want to continue to emulate autotools then this might be a simpler solution:

macro(check_library_exists_concat LIBRARY SYMBOL VARIABLE)
  check_library_exists("${LIBRARY}" ${SYMBOL} "${CMAKE_LIBRARY_PATH}"
    ${VARIABLE})
  if(${VARIABLE})
    set(CURL_LIBS ${LIBRARY} ${CURL_LIBS})
  endif()
endmacro()

@bagder I can submit a PR if you have a specific way you want to go.

vszakats added a commit to vszakats/curl that referenced this issue Oct 9, 2023
The idea of `check_library_exists_concat()` is that it detects an
optional component and adds it to the list of libs that we also use in
subsequent component checks. This caused problems when detecting
components, with unnecessary dependencies that were not yet built.

CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set the libs
used for component checks, which we already use in most of the cases.
That left 4 uses of `check_library_exists_concat()`. Only one of these
actually needed the 'concat' feature (ldap/lber).

Delete this function and replace it with standard
`check_library_exists()` and manual management of our `CURL_LIBS`
list we use when linking build targets. And special logic to handle the
ldap/lber case.

(We have a similar function for headers: `check_include_file_concat`.
It works, but problematic for performance reasons and because it hides
the actual headers necessary in `check_symbol_exists()` calls.)

Ref: curl#11537
Fixes curl#11285
Fixes curl#11648
Closes #xxxxx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.

4 participants