Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libidn2 bug in Windows Unicode build #7228

Closed
dEajL3kA opened this issue Jun 10, 2021 · 0 comments
Closed

libidn2 bug in Windows Unicode build #7228

dEajL3kA opened this issue Jun 10, 2021 · 0 comments
Labels
unicode Unicode, code page, character encoding Windows Windows-specific

Comments

@dEajL3kA
Copy link

dEajL3kA commented Jun 10, 2021

Hello,

I think in Windows Unicode build of curl we should be calling the idn2_lookup_u8() function of libidn2, instead of idn2_lookup_ul() function. That is because in Windows Unicode build all internal char* strings in curl are always encoded in UTF-8 (apprently curl converts given CLI arguments from UTF-16 to UTF-8 internally), but the idn2_lookup_ul() function assumes the system "default" encoding - which on most Windows systems is something like Windows-1252 codepage, not UTF-8. Hence libidn2 fails to decode the UTF-8 string. Replacing call to idn2_lookup_ul() with idn2_lookup_u8() in url.c fixes this for me.

Thank you and best regards.

@danielgustafsson danielgustafsson added the Windows Windows-specific label Jun 10, 2021
vszakats added a commit to vszakats/curl that referenced this issue Jun 11, 2021
Reported-by: dEajL3kA on github
Fixes curl#7228
vszakats added a commit to vszakats/curl that referenced this issue Jun 12, 2021
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to use the UTF-8 flavour of the libidn2 API.

Reported-by: dEajL3kA on github
Fixes curl#7228
vszakats added a commit to vszakats/curl that referenced this issue Jun 12, 2021
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to call the UTF-8 flavour of the libidn2 API.

Reported-by: dEajL3kA on github
Fixes curl#7228
jay added a commit to jay/curl that referenced this issue Jun 14, 2021
- For Windows Unicode builds of libcurl that use libidn, attempt to
  convert the hostname from UTF-8 and if that fails use local encoding
  as a fallback.

Prior to this change Windows Unicode builds that use libidn attempted to
convert the hostname from the local encoding only. The curl tool passes
the command-line URL (and therefore the hostname) in Unicode's UTF-8
encoding, so the conversion would fail. Since other applications may
pass locally encoded URLs to libcurl we'll keep local encoding as a
fallback.

Reported-by: dEajL3kA@users.noreply.github.com
Assisted-by: Viktor Szakats

Fixes curl#7228
Closes curl#7246
Closes #xxxx
vszakats added a commit to vszakats/curl that referenced this issue Jun 15, 2021
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to call the UTF-8 flavour of the libidn2 API.

Reported-by: dEajL3kA on github
Assisted-by: Jay Satiro
Fixes curl#7228
vszakats added a commit to vszakats/curl that referenced this issue Jun 15, 2021
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to call the UTF-8 flavour of the libidn2 API.

Reported-by: dEajL3kA on github
Assisted-by: Jay Satiro
Fixes curl#7228
vszakats added a commit to vszakats/curl that referenced this issue Jun 15, 2021
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to call the UTF-8 flavour of the libidn2 API.

Reported-by: dEajL3kA on github
Assisted-by: Jay Satiro
Reviewed-by: Marcel Raad
Fixes curl#7228
@vszakats vszakats added the unicode Unicode, code page, character encoding label Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
unicode Unicode, code page, character encoding Windows Windows-specific
3 participants