Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

autobuilds: test 1560 [URL API] failing #10193

Closed
MarcelRaad opened this issue Jan 1, 2023 · 6 comments
Closed

autobuilds: test 1560 [URL API] failing #10193

MarcelRaad opened this issue Jan 1, 2023 · 6 comments
Labels
tests URL

Comments

@MarcelRaad
Copy link
Member

I did this

Look at the autobuilds at https://curl.se/dev/builds.html. Since commit 901392c, @mkauf 's NetBSD builds and Kirill Sokolovsky's Linux builds are failing for the newly extended test 1560 with:

=== Start of file stderr1560
URL: -
/curl/tests/libtest/lib1560.c:1058 returned 7 (A memory function failed). URL: 'https://r[C3][A4]ksm[C3][B6]rg[C3][A5]s.se/path?q#frag'
=== End of file stderr1560

I cannot reproduce the error with a curl configured as close as possible to the failing Linux configuration with the Windows Subsystem for Linux (tried with Ubuntu, Debian, Kali, openSUSE Tumbleweed, and Alpine).

I expected the following

Test 1560 succeeding.

curl/libcurl version

NetBSD:

Host setup: armv7eb-unknown-netbsdelf9.3-eabihf
Install prefix: /usr/local
Compiler: gcc
CFLAGS: -Werror-implicit-function-declaration -g -O0 -std=gnu89 -pedantic -Wall -W -Wpointer-arith -Wwrite-strings -Wunused -Wshadow -Winline -Wnested-externs -Wmissing-declarations -Wmissing-prototypes -Wno-long-long -Wbad-function-cast -Wfloat-equal -Wno-multichar -Wsign-compare -Wundef -Wno-format-nonliteral -Wendif-labels -Wstrict-prototypes -Wdeclaration-after-statement -Wold-style-definition -Wstrict-aliasing=3 -Wcast-align -Wtype-limits -Wold-style-declaration -Wmissing-parameter-type -Wempty-body -Wclobbered -Wignored-qualifiers -Wconversion -Wno-sign-conversion -Wvla -ftree-vrp -Wdouble-promotion -Wformat=2 -Warray-bounds=2 -Wshift-negative-value -Wshift-overflow=2 -Wnull-dereference -fdelete-null-pointer-checks -Wduplicated-cond -Wunused-const-variable -Wduplicated-branches -Wrestrict -Walloc-zero -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=4 -Wno-system-headers -pthread
CPPFLAGS: -isystem /usr/pkg/include -isystem /usr/pkg/include
LDFLAGS: -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/pkg/lib -L/usr/pkg/lib
LIBS: -lnghttp2 -lidn2 -lssl -lcrypto -lssl -lcrypto -lldap -lz

curl version: 7.87.1-DEV
SSL: enabled (OpenSSL)
SSH: no (--with-{libssh,libssh2})
zlib: enabled
brotli: no (--with-brotli)
zstd: no (--with-zstd)
GSS-API: no (--with-gssapi)
GSASL: no (libgsasl not found)
TLS-SRP: enabled
resolver: POSIX threaded
IPv6: enabled
Unix sockets: enabled
IDN: enabled (libidn2)
Build libcurl: Shared=yes, Static=yes
Built-in manual: enabled
--libcurl option: enabled (--disable-libcurl-option)
Verbose errors: enabled (--disable-verbose)
Code coverage: disabled
SSPI: no (--enable-sspi)
ca cert bundle: no
ca cert path: /etc/openssl/certs
ca fallback: no
LDAP: enabled (OpenLDAP)
LDAPS: enabled
RTSP: enabled
RTMP: no (--with-librtmp)
PSL: no (libpsl not found)
Alt-svc: enabled (--disable-alt-svc)
Headers API: enabled (--disable-headers-api)
HSTS: enabled (--disable-hsts)
HTTP1: enabled (internal)
HTTP2: enabled (nghttp2)
HTTP3: no (--with-ngtcp2, --with-quiche --with-msh3)
ECH: no (--enable-ech)
WebSockets: no (--enable-websockets)
Protocols: DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS LDAP LDAPS MQTT POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP
Features: AsynchDNS HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile NTLM NTLM_WB SSL TLS-SRP UnixSockets alt-svc libz threadsafe

Linux:

Host setup: x86_64-pc-linux-gnu
Install prefix: /usr/local
Compiler: gcc
CFLAGS: -Werror-implicit-function-declaration -g -O0 -std=gnu89 -pedantic -Wall -W -Wpointer-arith -Wwrite-strings -Wunused -Wshadow -Winline -Wnested-externs -Wmissing-declarations -Wmissing-prototypes -Wno-long-long -Wbad-function-cast -Wfloat-equal -Wno-multichar -Wsign-compare -Wundef -Wno-format-nonliteral -Wendif-labels -Wstrict-prototypes -Wdeclaration-after-statement -Wold-style-definition -Wstrict-aliasing=3 -Wcast-align -Wtype-limits -Wold-style-declaration -Wmissing-parameter-type -Wempty-body -Wclobbered -Wignored-qualifiers -Wconversion -Wno-sign-conversion -Wvla -ftree-vrp -Wdouble-promotion -Wformat=2 -Warray-bounds=2 -Wshift-negative-value -Wshift-overflow=2 -Wnull-dereference -fdelete-null-pointer-checks -Wduplicated-cond -Wunused-const-variable -Wduplicated-branches -Wrestrict -Walloc-zero -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=4 -Warith-conversion -Wenum-conversion -Wno-system-headers -pthread
CPPFLAGS:
LDFLAGS:
LIBS: -lnghttp2 -lidn2 -lpsl -lssl -lcrypto -lssl -lcrypto -lldap -llber -lzstd -lz

curl version: 7.87.1-DEV
SSL: enabled (OpenSSL)
SSH: no (--with-{libssh,libssh2})
zlib: enabled
brotli: no (--with-brotli)
zstd: enabled (libzstd)
GSS-API: no (--with-gssapi)
GSASL: no (libgsasl not found)
TLS-SRP: enabled
resolver: POSIX threaded
IPv6: enabled
Unix sockets: enabled
IDN: enabled (libidn2)
Build libcurl: Shared=yes, Static=yes
Built-in manual: enabled
--libcurl option: enabled (--disable-libcurl-option)
Verbose errors: enabled (--disable-verbose)
Code coverage: disabled
SSPI: no (--enable-sspi)
ca cert bundle: /etc/ssl/certs/ca-certificates.crt
ca cert path: no
ca fallback: no
LDAP: enabled (OpenLDAP)
LDAPS: enabled
RTSP: enabled
RTMP: no (--with-librtmp)
PSL: enabled
Alt-svc: enabled (--disable-alt-svc)
Headers API: enabled (--disable-headers-api)
HSTS: enabled (--disable-hsts)
HTTP1: enabled (internal)
HTTP2: enabled (nghttp2)
HTTP3: no (--with-ngtcp2, --with-quiche --with-msh3)
ECH: no (--enable-ech)
WebSockets: no (--enable-websockets)
Protocols: DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS LDAP LDAPS MQTT POP3 POP3S RTSP SMB SMBS SMTP SMTPS TELNET TFTP
Features: AsynchDNS HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile NTLM NTLM_WB PSL SSL TLS-SRP UnixSockets alt-svc libz threadsafe zstd

operating system

NetBSD:

********* System characteristics ********
* curl 7.87.1-DEV (armv7eb-unknown-netbsdelf9.3-eabihf)
* libcurl/7.87.1-DEV OpenSSL/1.1.1k zlib/1.2.10 libidn2/2.3.3 nghttp2/1.49.0
* Features: alt-svc AsynchDNS Debug HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB SSL threadsafe TLS-SRP TrackMemory UnixSockets
* Disabled:
* Host: cubieboard
* System: NetBSD cubieboard 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug 4 15:30:37 UTC 2022 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC evbarm
* OS: netbsd
*
*** DISABLES memory tracking when using threaded resolver
*
* Servers: SSL HTTP-IPv6 HTTP-unix FTP-IPv6
* Env:
* Seed: 251056
*****************************************

Linux:

********* System characteristics ********
* curl 7.87.1-DEV (x86_64-pc-linux-gnu)
* libcurl/7.87.1-DEV OpenSSL/1.0.2u zlib/1.2.11 zstd/1.5.0 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.0) nghttp2/1.45.1
* Features: alt-svc AsynchDNS Debug HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL threadsafe TLS-SRP TrackMemory UnixSockets zstd
* Disabled:
* Host: scan
* System: Linux scan 4.1.15-gentoo-r1 #2 SMP Thu Feb 18 14:16:46 UTC 2016 x86_64 Intel(R) Xeon(R) CPU X5675 @ 3.07GHz GenuineIntel GNU/Linux
* OS: linux
*
*** DISABLES memory tracking when using threaded resolver
*
* Servers: HTTP-IPv6 HTTP-unix FTP-IPv6
* Env:
* Seed: 231329
*****************************************
@bagder
Copy link
Member

bagder commented Jan 1, 2023

We probably need to make sure they use UTF-8 locales for test 1560.

bagder added a commit that referenced this issue Jan 1, 2023
@bagder
test1560: use a UTF8-using locale when run
9c62871 
There are odd cases that don't use UTF8 and then the IDN handling goes
wrong.

Reported-by: Marcel Raad
Fixes #10193
@bagder bagder mentioned this issue Jan 1, 2023
Closed
@bagder bagder closed this as completed in f27262b Jan 2, 2023
@bagder
Copy link
Member

bagder commented Jan 2, 2023

There's still a risk that this fix does not work, since there is no guarantee that systems have UTF8 locales around, but then I'm not really sure how we should go about and detect that and skip the tests on those systems...

@dfandrich
Copy link
Contributor

dfandrich commented Jan 2, 2023 •
edited

There are a number of tests that check that already in <precheck>, such as test955.

@bagder
Copy link
Member

bagder commented Jan 2, 2023

Ah right. Thanks!

@bagder
Copy link
Member

bagder commented Jan 2, 2023

except that test 1560 needs a condition like "using winidn OR !using-IDN OR has a UTF-8 locale" ...

@sharewax
Copy link

sharewax commented May 23, 2023 •
edited

On my build Oracle Linux 9 I have an issue with this 1560 test.

test 1560...[URL API]
../libtool --mode=execute /usr/bin/valgrind --tool=memcheck --quiet --leak-check=yes --suppressions=../../tests/valgrind.supp --num-callers=16 --log-file=log/valgrind1560 ./libtest/lib1560 - > log/stdout1560 2> log/stderr1560
CMD (768): ../libtool --mode=execute /usr/bin/valgrind --tool=memcheck --quiet --leak-check=yes --suppressions=../../tests/valgrind.supp --num-callers=16 --log-file=log/valgrind1560 ./libtest/lib1560 - > log/stdout1560 2> log/stderr1560
 1560: stdout FAILED:
--- log/check-expected  2023-05-23 08:31:11.619504665 +0000
+++ log/check-generated 2023-05-23 08:31:11.619504665 +0000
@@ -1 +0,0 @@
-success[LF]
== Contents of files in the log/ dir after test 1560
=== Start of file check-expected
 success[LF]
=== End of file check-expected
=== Start of file commands.log
 ../libtool --mode=execute /usr/bin/valgrind --tool=memcheck --quiet --leak-check=yes --suppressions=../../tests/valgrind.supp --num-callers=16 --log-file=log/valgrind1560 ./libtest/lib1560 - > log/stdout1560 2> log/stderr1560
=== End of file commands.log
=== Start of file ftpserver.cmd
 Testnum 1560
=== End of file ftpserver.cmd
=== Start of file stderr1560
 ../libtool: line 4460: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory
 ../libtool: line 4460: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory
 ../libtool: line 4460: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory
 URL: -
 ../../../tests/libtest/lib1560.c:1112 returned 7 (A memory function failed). URL: 'https://räksmörgås.se/path?q#frag'
 Get URL
 in: https://räksmörgås.se/path?q#frag
 returned 7 (expected 0)
=== End of file stderr1560

Am I right, that I need some additional locale package requirements for build?
It seems that I need add this one. Thank you

# needed for test1560 to succeed
BuildRequires: glibc-langpack-en

bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
@bagder @bch
test1560: use a UTF8-using locale when run
90cf39f 
There are odd cases that don't use UTF8 and then the IDN handling goes
wrong.

Reported-by: Marcel Raad
Fixes curl#10193
Closes curl#10196
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests URL
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test1560: use a UTF8-using locale when run curl/curl
4 participants
@bagder @dfandrich @MarcelRaad @sharewax