Skip to content

remove unneeded guards around PUNY2IDN #17364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

jacobmealey
Copy link
Contributor

This PR removes some guards around IDN which were causing punycode encoded domains starting with www (or anything other than xn-- to fail the conversion. Removing the if statement allows libcurl to pass the urls directly to libidn2 and propagate errors up, instead of trying to detect a bad punycode early.

This came out of digging in to an issue @dfandrich opened in trurl: curl/trurl#392

I've done tests both with standalone C programs as well as in trurl, but I'm happy to try anything folks suspect will break.

Thanks!

@github-actions github-actions bot added the tests label May 16, 2025
@jacobmealey jacobmealey changed the title remove uneeded guards around PUNY2IDN remove unneeded guards around PUNY2IDN May 16, 2025
@jacobmealey
Copy link
Contributor Author

that's a lot of red!

@testclutch
Copy link

Analysis of PR #17364 at c1ac8a8b:

Test 1560 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 216 different CI jobs (the link just goes to one of them).

Generated by Testclutch

@jacobmealey
Copy link
Contributor Author

I was testing it by statically linking libcurl and it was working well, but when I dynamically link it the same curl_url_get returns the punycoded url. I've made sure the curl-config in my path is the one built from curl. I've configured curl as follows:

./configure --with-openssl --with-idn2 --with-ldap --enable-test-bundles --enable-debug 

@bagder
Copy link
Member

bagder commented May 16, 2025

The same fix is also needed here:

diff --git a/lib/urlapi.c b/lib/urlapi.c
index 071ee057eb..0ec3de32ec 100644
--- a/lib/urlapi.c
+++ b/lib/urlapi.c
@@ -1588,11 +1588,11 @@ CURLUcode curl_url_get(const CURLU *u, CURLUPart what,
         *part = allochost;
 #endif
       }
     }
     else if(depunyfy) {
-      if(Curl_is_ASCII_name(u->host)  && !strncmp("xn--", u->host, 4)) {
+      if(Curl_is_ASCII_name(u->host)) {
 #ifndef USE_IDN
         return CURLUE_LACKS_IDN;
 #else
         char *allochost;
         CURLcode result = Curl_idn_encode(*part, &allochost);

@jacobmealey
Copy link
Contributor Author

Thanks for the hint. I was chasing fake configuration errors. I'm not sure why those two tests are failing, it looks like one is flagged as flaky in testclutch, but the OmniOS one is a bit confusing. I have to get ready for work but I can look at this more this weekend.

Verified

This commit was signed with the committer’s verified signature.
jacobmealey Jacob Mealey
add more IDN/punycode test
@bagder
Copy link
Member

bagder commented May 16, 2025

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

3 participants