Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AIX XL C/C++ V16.1 comes with clang/clang++ but is not fully clang compatible #3474

Closed
kdekker opened this issue Jan 15, 2019 · 9 comments
Closed

AIX XL C/C++ V16.1 comes with clang/clang++ but is not fully clang compatible #3474

kdekker opened this issue Jan 15, 2019 · 9 comments
Labels
build

Comments

@kdekker
Copy link
Contributor

kdekker commented Jan 15, 2019 •
edited by bagder

I did this

IBM quite recently released Xl C/C++ V16.1 for AIX (see announcement). This compiler comprises the 'good old' known xlc (and variants) C-compiler, but this product now also contains xlclang and xlclang++. Apparently, support for newer standards like C11/C++11/C++14 is not added to XL C/C++ but to xlclang (C) or xlclang++ (C++). The xlclang compiler supports some/most of the clang compiler but not all. When calling configure with the following flags:

LDFLAGS="-L/vobs/obj.AIX71/thirdparty/OpenSSL/64bit/openssl-1.1.1/build/lib -L/vobs/obj.AIX71/thirdparty/zlib/64bit/zlib-1.2.11/build/lib" LIBS="-lm -ldl /vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/lib/libz.a" CC="xlclang" CFLAGS="-O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 " AR="/usr/bin/ar -X 32_64" MAKE=gmake MAKEFLAGS= MFLAGS= ./configure --prefix=/vobs/obj.AIX71/thirdparty/cURL/64bit/curl-7.61.1/build --with-ssl=/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit --with-zlib=/vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit --disable-silent-rules --enable-ipv6 --disable-shared --disable-ldap --disable-manual --without-libidn2 --without-libssh2 || rm -f "/vobs/obj.AIX71/thirdparty/cURL/64bit/curl-7.61.1/config.status"

(with xlclang in the PATH, /opt/IBM/xlC/16.1.0/bin/xlclang)

I then get a configure result that adds -Qunused flag. See (part of) configure output:

checking if cpp -P is needed... no
checking if compiler is DEC/Compaq/HP C... no
checking if compiler is HP-UX C... no
checking if compiler is IBM C... no
checking if compiler is Intel C... no
checking if compiler is clang... yes
checking if compiler is GNU C... no
checking if compiler is LCC... no
checking if compiler is SGI MIPSpro C... no
checking if compiler is SGI MIPS C... no
checking if compiler is SunPro C... no
checking if compiler is Tiny C... no
checking if compiler is Watcom C... no
checking if compiler accepts some basic options... yes
configure: compiler options added: -Qunused-arguments

Unfortunately, the -Qunused flag is not supported by IBM's xlclang, see:

gmake[2]: Entering directory '/vobs/obj.AIX71/thirdparty/cURL/64bit/curl-7.61.1/lib' /bin/sh ../libtool --tag=CC --mode=compile xlclang -DHAVE_CONFIG_H -I../include -I../lib -I../lib -DBUILDING_LIBCURL -DCURL_STATICLIB -DCURL_HIDDEN_SYMBOLS -I/vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include/openssl -fvisibility=hidden -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments -MT libcurl_la-file.lo -MD -MP -MF .deps/libcurl_la-file.Tpo -c -o libcurl_la-file.lotest -f 'file.c' || echo './'file.c libtool: compile: xlclang -DHAVE_CONFIG_H -I../include -I../lib -I../lib -DBUILDING_LIBCURL -DCURL_STATICLIB -DCURL_HIDDEN_SYMBOLS -I/vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include/openssl -fvisibility=hidden -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments -MT libcurl_la-file.lo -MD -MP -MF .deps/libcurl_la-file.Tpo -c file.c -o libcurl_la-file.o **/opt/IBM/xlC/16.1.0/bin/.orig/xlclang: warning: 1501-216 command option -Qunused-arguments is not recognized - passed to ld** mv -f .deps/libcurl_la-file.Tpo .deps/libcurl_la-file.Plo

See also the compiler reference manual at compiler ref

I expected the following

I'm not sure how to fix this. It is probably easy to adjust configure in such way that -Qunused is not added if $CC maches xlclang. In that case, this option can be omitted. However, I don't know the procedure 'how to add support for a new compiler'.

Because I'm building a static libcurl, I don't really suffer from this issue. That may probably change for people who like to build a shared library. In that case, the linker is invoked with incorrect options.

I would be nice if configure fully recognizes AIX's xlclang and passes correct compiler flags to it.

A similar/related thing is another compiler option that does not work correctly (but does not raise further errors):

checking whether the xlclang linker (/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... /opt/IBM/xlC/16.1.0/bin/.orig/xlclang: error: 1501-294 No input file specified. Please use -qhelp for more information.
AIX lib.a(lib.so.V)

I think that _xl_clang is probably more XL than clang. Please refer to IBM's compiler reference documentation for more information about what options are supported.

In case if greater details are wanted, test or something else, feel free to ask. If wanted, I can upload configure related files.

curl/libcurl version

[curl -V output]
curl 7.61.1

operating system

AIX 7.1 TL4
@bagder bagder added the build label Jan 15, 2019
bagder added a commit that referenced this issue Jan 15, 2019
@bagder
curl-compilers.m4: detect xlclang
3d2646d 
Since it isn't totally clang compatible, we detect this IBM clang
front-end and if detected, avoids some clang specific magic.

Reported-by: Kees Dekker
Fixes #3474
@bagder bagder mentioned this issue Jan 15, 2019
Closed
@bagder
Copy link
Member

bagder commented Jan 15, 2019

Thanks!

Please try out my patch in #3476. It should at least fix the use of the non-supported option.

If the checking dynamic linker characteristics problem remains (as I suspect), I will be interested in getting a full log. That output is done by some of the macros that is shipped by autoconf and isn't shown by a test we've written.

@kdekker
Copy link
Contributor Author

kdekker commented Jan 15, 2019

Many thanks for very quick reply. I've updated the m4/curl-compilers.m4 with the modified file and reran configure (clean/from scratch).

I still get (part from config.log):

configure:17209: checking if compiler accepts some basic options
configure:17233: xlclang -c -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments  conftest.c >&5
/opt/IBM/xlC/16.1.0/bin/.orig/xlclang: warning: 1501-216 command option -Qunused-arguments is not recognized - passed to ld
configure:17233: $? = 0
configure:17263: xlclang -o conftest -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments  -L/vobs/obj.AIX71/thirdparty/OpenSSL/64bit/openssl-1.1.1/build/lib -L/vobs/obj.AIX71/thirdparty/zlib/64bit/zlib-1.2.11/build/lib conftest.c -lm -ldl /vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/lib/libz.a >&5
/opt/IBM/xlC/16.1.0/bin/.orig/xlclang: warning: 1501-216 command option -Qunused-arguments is not recognized - passed to ld
ld: 0706-027 The -Q flag is ignored.
ld: 0711-301 WARNING: Symbol specified with the -u flag not defined: nused-arguments
configure:17263: $? = 0

When I start the build, I get:

gmake[1]: Entering directory '/vobs/obj.dbg.AIX71/thirdparty/cURL/64bit/curl-7.61.1/lib'
gmake  all-am
gmake[2]: Entering directory '/vobs/obj.dbg.AIX71/thirdparty/cURL/64bit/curl-7.61.1/lib'
/bin/sh ../libtool  --tag=CC   --mode=compile xlclang -DHAVE_CONFIG_H   -I../include -I../lib -I../lib  -DBUILDING_LIBCURL -DCURL_STATICLIB -DCURL_HIDDEN_SYMBOLS -I/vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include/openssl   -fvisibility=hidden -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments  -MT libcurl_la-file.lo -MD -MP -MF .deps/libcurl_la-file.Tpo -c -o libcurl_la-file.lo `test -f 'file.c' || echo './'`file.c
libtool: compile:  xlclang -DHAVE_CONFIG_H -I../include -I../lib -I../lib -DBUILDING_LIBCURL -DCURL_STATICLIB -DCURL_HIDDEN_SYMBOLS -I/vobs/tt/../common/thirdparty/zlib/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include -I/vobs/tt/../common/thirdparty/OpenSSL/build/IBM_RS6000/64bit/include/openssl -fvisibility=hidden -O2 -qinlglue -qtune=balanced -qmaxmem=-1 -qstrict -brtl -qro -qarch=pwr4 -qcheck=divzero -qalias=noansi -qfuncsect=noimplicitstaticref -qcheck=nullptr -qchars=unsigned -std=c11 -q64 -Qunused-arguments -MT libcurl_la-file.lo -MD -MP -MF .deps/libcurl_la-file.Tpo -c file.c -o libcurl_la-file.o
/opt/IBM/xlC/16.1.0/bin/.orig/xlclang: warning: 1501-216 command option -Qunused-arguments is not recognized - passed to ld
mv -f .deps/libcurl_la-file.Tpo .deps/libcurl_la-file.Plo

In the output of configure, the following line still exists:

configure: compiler options added: -Qunused-arguments

Attached: config.log, config.status and the output of the configure command (all in onze tar.gz file).
configure.logs.tar.gz

@bagder
Copy link
Member

bagder commented Jan 15, 2019

You also need to run ./buildconf to generate a new configure with the updated script before you invoke ./configure again - which will require you to have the autotools installed!

@kdekker
Copy link
Contributor Author

kdekker commented Jan 15, 2019

Sorry, please forgive my innocence. Unfortunately, autoconf was not installed. I've obtained it from the AIX (Linux) Toolbox (including automake). But finally I stuck down on libtoolize. It is not in the AIX Toolbox (repositories). I will trust you on this fix and will await until it will be released in a next cURL release. For now, it is just a warning that does apparently not harm now.

@bagder
Copy link
Member

bagder commented Jan 15, 2019

We can actually do it even better if you're up for it. I'll land this (#3476) patch today, as I believe it works, and tomorrow you can get a daily snapshot which will have the configure script generated and you can use that to verify if the issue is truly fixed or not.

@bagder bagder closed this as completed in 2fa0d57 Jan 15, 2019
@kdekker
Copy link
Contributor Author

kdekker commented Jan 16, 2019

I've downloaded the snapshot from https://curl.haxx.se/snapshots/ (taken 16. January download).

Configure still says:

checking if compiler is Watcom C... no
checking if compiler accepts some basic options... yes
configure: compiler options added: -Qunused-arguments 
checking if compiler accepts debug disabling options... yes
configure: compiler options added:

As a result, still messages about the -Qunused option passed to ld are being displayed.

Also some confusing (but it is no error, just an observation) output my still be produced by configure:

checking dynamic linker characteristics... /opt/IBM/xlC/16.1.0/bin/.orig/xlclang: error: 1501-294 No input file specified. Please use -qhelp for more information.
AIX lib.a(lib.so.V)

I..e. I did not expect that error message are part of the output. The job of configure is finding out the compiler and OS capabilities, but if something does not exist, it should not be reported as error. I did not figure out in detail whether a wrong call to the compiler was done or whether some stderr output was not suppressed. IBM AIX works slightly different in making a shared library (for xlc). For xlclang, they both support the -qmkshrobj as well as the -shared option.

I double cheched m4/curl-compilers.m4 and configure; both contain a line about if xlclang. See attachment for more details.
config-aix.tar.gz

@bagder bagder reopened this Jan 16, 2019
bagder added a commit that referenced this issue Jan 16, 2019
@bagder
curl-compilers.m4: check for __ibmxl__ to detect xlclang
1bcbd4f 
Follow-up to 2fa0d57. The __xlc__ symbol is only defined there if a
particular flag is used for legacy macros.

Fixes #3474
@bagder bagder mentioned this issue Jan 16, 2019
Closed
@bagder
Copy link
Member

bagder commented Jan 16, 2019

Here's why:

configure:16334: checking if compiler is xlclang
configure:16358: xlclang -E   conftest.c
configure:16358: $? = 0
configure:16385: result: no

and reading the documentation again I see that __xlc__ is the wrong macro to check for. It seems __ibmxl__ is the proper one. See #3479. You can try this by a manual search-and-replace in the configure script if you want to.

@kdekker
Copy link
Contributor Author

kdekker commented Jan 16, 2019

Do you work 24h a day?? Thanks for very quick reply.

I guess you've found the ref 😄. Hoewever, I'm not sure how you then can distinguish real xlc from xlclang. Even in XL C/C++ V16.1 product, both xlc as well as xlclang (and xlC/xlclang++) exist.

Output of configure is now (copied some parts):

checking if xlclang supports -c -o file.o... (cached) yes
checking whether the xlclang linker (/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... /opt/IBM/xlC/16.1.0/bin/.orig/xlclang: error: 1501-294 No input file specified. Please use -qhelp for more information.
AIX lib.a(lib.so.V)
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... no

checking if cpp -P is needed... no
checking if compiler is DEC/Compaq/HP C... no
checking if compiler is HP-UX C... no
checking if compiler is IBM C... no
checking if compiler is Intel C... no
checking if compiler is clang... yes
checking if compiler is xlclang... yes
checking if compiler is GNU C... no
checking if compiler is LCC... no
checking if compiler is SGI MIPSpro C... no
checking if compiler is SGI MIPS C... no
checking if compiler is SunPro C... no
checking if compiler is Tiny C... no
checking if compiler is Watcom C... no
checking if compiler accepts debug disabling options... yes
configure: compiler options added: 
checking if compiler optimizer assumed setting might be used... no

The second part is ok. The first part is up to you to decide to do something with.
No warnings are raised now when starting the build, because no -Qunused flag is added. Thanks, this issue has been fixed now.

@bagder
Copy link
Member

bagder commented Jan 16, 2019

I'm not sure how you then can distinguish real xlc from xlclang.

I think this is doing the right thing. It only checks for xlclang if it first found the clang variable, which I hope won't happen in the regular xlc case.

Thanks for verifying the update. I'll merge that into master.

@bagder bagder closed this as completed in 26d7f00 Jan 16, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Apr 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
build
Milestone
No milestone
2 participants
@bagder @kdekker