Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

./configure fails with Unicode build #7229

Closed
dEajL3kA opened this issue Jun 10, 2021 · 13 comments
Closed

./configure fails with Unicode build #7229

dEajL3kA opened this issue Jun 10, 2021 · 13 comments
Labels
build help wanted KNOWN_BUGS material unicode Unicode, code page, character encoding Windows Windows-specific

Comments

@dEajL3kA
Copy link

dEajL3kA commented Jun 10, 2021

I did this

I am trying to build curl with Unicode support on Windows (using MinGW/MSYS2).

In order to enable Unicode, I added options -munciode and -mconsole to the CFLAGS explicitely, because apparently ./configure doesn't have a dedicated --enable-unicode switch. Or did I miss something?

Unforntunately, this caused ./configure to fail with "No working C compiler" error. My analysis of the config.log showed that ./configure uses a lot of "test" programs that have main() hardcoded, but should be using wmain() when Unicode is used! In the actual curl source codes we already do have the required #ifdefs to use either main() or wmain() as needed. Only the ./configure has problem! My workaround was to replace all occurences of "main" in ./configure with "wmain", using a simple sed command. This does work. After patching configure, the Unicode version of curl compiled without any problem!

sed -i -E 's/\bmain[[:space:]]*\(([^\(\)]*)\)/wmain(\1)/g' configure

But this workaround is rather cumbersome and other people may run into the same issue. So I think it would be much preferable to have an --enable-unicode switch that not only addes -munciode and -mconsole to the CFLAGS automatically but also uses the "correct" main function in the "test" programs. At least make ./configure work with -munciode -mconsole options.

I expected the following

CFLAGS="-municode -mconsole" ./configure && make should "just work" without the need to patch configure.

Even better, ./configure --enable-unicode && make should be available.

curl/libcurl version

7.77.0

operating system

Windows 10, using latest MinGW/MSYS2 compiler + build environment

MINGW32_NT-10.0-19042 3.2.0-340.x86_64 2021-05-24 19:32 UTC x86_64 Msys
gcc version 10.3.0 (Rev2, Built by MSYS2 project)
@dEajL3kA
Copy link
Author

dEajL3kA commented Jun 10, 2021

EDIT: Better solution exists. Please see here!


Possible workaround, if we don't want to patch the configure script, would be using a tiny "shim" library that implements the wmain() function, in order to fix the linker error about missing wmain (or wWinMain) entry point, and that does nothing but invoking the "real" main() function. Libary needs to be added to LIBS environment variable when invoking ./configure.

libwmain.c:

extern int main(int argc, char *argv[]);
int wmain(void) {
	char *args[] = { "a.exe", (void*)0 };
	return main(1, (char**)args);
}

Makefile:

libwmain.a: libwmain.o
	$(AR) rcs libwmain.a libwmain.o
libwmain.o: libwmain.c
	$(CC) -march=i386 -Os -o libwmain.o -c libwmain.c

To build curl:
CFLAGS="-munciode -mconsole" LIBS="-lwmain" ./configure

Note: In my test, this did not collide with curl's own wmain() method, but did fix the conftests.

@jay
Copy link
Member

jay commented Jun 11, 2021

Can you use -DUNICODE -D_UNICODE instead

@dEajL3kA
Copy link
Author

dEajL3kA commented Jun 11, 2021

Can you use -DUNICODE -D_UNICODE instead

Sorry, but nope. Or I am missing something? 😕

  • According to my tests, -municode -mconsole already implies -DUNICODE -D_UNICODE, so explicitely adding -DUNICODE -D_UNICODE to CFLAGS in addition to -municode -mconsole does not change anything for me.

  • Setting only -DUNICODE -D_UNICODE in CFLAGS but not -municode -mconsole causes MinGW/GCC to still link against main() rather than wmain() – this I have just verified! As a result, ./configure does succeed without additional hacks, yes. But, in the end, the linking of curl.exe fails with an undefined reference error – which is expected, because with -D_UNICODE we are telling tool_main.c to generate wmain() instead of main(), but at the same time MinGW/GCC is expecting a main() function due to the lack of -municode -mconsole switches. Hence, setting -DUNICODE -D_UNICODE alone is not sufficient 😞

This was tested with latest MinGW/GCC:
gcc version 10.3.0 (Rev2, Built by MSYS2 project)

Best regards.

@jay
Copy link
Member

jay commented Jun 13, 2021

It seems this is the way curl-for-win does it:

  CURL_CFLAG_EXTRAS="${CURL_CFLAG_EXTRAS} -DUNICODE -D_UNICODE"
  CURL_LDFLAG_EXTRAS_EXE="${CURL_LDFLAG_EXTRAS_EXE} -municode"

So we'd have to apply -municode only for the curl tool?

/cc @vszakats

@dEajL3kA
Copy link
Author

dEajL3kA commented Jun 14, 2021

So we'd have to apply -municode only for the curl tool?

Yes, I think so. But it seems that CURL_LDFLAG_EXTRAS_EXE does nothing for me. I did a full-text search on both, configure and the resulting Makefile, but could not find the identifier CURL_LDFLAG_EXTRAS_EXE at all. Does it even exist in "vanilla" curl?

However, what does work for me is:

CFLAGS="-DUNICODE -D_UNICODE" ./configure
make curl_LDFLAGS="-all-static -municode -mconsole"

So that probably is the way to go. Still, figuring this out was wearisome. Maybe add a ./configure option for Unicode build?

Thank you and best regards.

(BTW: What is the difference between CURL_CFLAG_EXTRAS and CFLAGS variables?)

@jay
Copy link
Member

jay commented Jun 14, 2021

Maybe add a ./configure option for Unicode build?

Seems like a good idea.

(BTW: What is the difference between CURL_CFLAG_EXTRAS and CFLAGS variables?)

I think it was added to pass cflags without overriding the default CFLAGS. It's not documented and I may be wrong...

@vszakats
Copy link
Member

vszakats commented Jun 14, 2021

CURL_LDFLAG_EXTRAS_EXE is Makefile.m32 specific, so it indeed doesn't do anything with ./configure builds.

CURL_CFLAG_EXTRAS is also something used only in Makefile.m32. This build method doesn't support a CFLAGS from the environment, so this "extra" envvar is used to inject extra C flags into it.

@bagder
Copy link
Member

bagder commented Oct 2, 2021

This issue seems to have stalled. Is there anything we can/should do about this report? Any proposed next steps?

@vszakats
Copy link
Member

vszakats commented Oct 3, 2021

I think it'd be best to complete Unicode support in libcurl and then fix the different build systems to enable it correctly. Though these two can be done in any order or in parallel. @jay had a nice patch for the former, though I continue to think that fall-back methods are bad and should not be used in libcurl. (Yes, OpenSSL uses one such method, but OpenSSL also keeps at least one Windows vulnerability open since years without intention to fix it (along some other past woes on Windows), so I think it should not be regarded as the yardstick for Windows-development specifically.)

@dEajL3kA
Copy link
Author

dEajL3kA commented Oct 5, 2021

I think it'd be best to complete Unicode support in libcurl and then fix the different build systems to enable it correctly. Though these two can be done in any order or in parallel. @jay had a nice patch for the former, though I continue to think that fall-back methods are bad and should not be used in libcurl.

👍 ⚡

@curl curl temporarily blocked Thanayaby Nov 20, 2021
@curl curl deleted a comment from Thanayaby Nov 20, 2021
@bagder
Copy link
Member

bagder commented Sep 24, 2022

Anyone care to write this issue up in a brief summary for adding to the KNOWN_BUGS document?

@jay
Copy link
Member

jay commented Sep 25, 2022

IMO this isn't a bug, it's either an enhancement or a feature request. The user wants an easier way to tell autotools to build curl with Windows Unicode support, like ./configure --enable-windows-unicode

@vszakats vszakats added the unicode Unicode, code page, character encoding label Feb 16, 2023
@vszakats
Copy link
Member

vszakats commented Mar 11, 2023

I tried to fix this without the sed trick, and tested: -Dmain=wmain.
Oddly enough, it worked!

CPPFLAGS="${CPPFLAGS} -Dmain=wmain"
CPPFLAGS="${CPPFLAGS} -DUNICODE -D_UNICODE"
LDFLAGS="${LDFLAGS} -municode"

It'd be better to restrict -Dmain=wmain internal to the ./configure run, if possible.

If this is a feasible solution, adding an --enable-windows-unicode option is just the icing on the cake.


This new warning came up, unrelated to either of the above hacks:

../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.

vszakats added a commit to vszakats/curl that referenced this issue Mar 11, 2023
llvm/clang and gcc doesn't recognize the wmain() function in Unicode
Windows builds:

```
../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.
```

Before this patch, we already silenced it with CMake. This patch moves
the silencing to the source, so that it applies to all build tools.

Bug: curl#7229 (comment)

Closes #xxxxx
vszakats added a commit that referenced this issue Mar 11, 2023
llvm/clang and gcc doesn't recognize the wmain() function in Unicode
Windows builds:

llvm/clang:
```
../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.
```

gcc:
```
../../src/tool_main.c:239:5: warning: no previous prototype for 'wmain' [-Wmissing-prototypes]
  239 | int wmain(int argc, wchar_t *argv[])
      |     ^~~~~
```

Before this patch, we already silenced it with CMake. This patch moves
the silencing to the source, so that it applies to all build tools.

Bug: #7229 (comment)

Reviewed-by: Marcel Raad
Closes #10744
vszakats added a commit to curl/curl-for-win that referenced this issue Mar 11, 2023
@bagder bagder closed this as completed in 7128ae8 May 22, 2023
bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
llvm/clang and gcc doesn't recognize the wmain() function in Unicode
Windows builds:

llvm/clang:
```
../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.
```

gcc:
```
../../src/tool_main.c:239:5: warning: no previous prototype for 'wmain' [-Wmissing-prototypes]
  239 | int wmain(int argc, wchar_t *argv[])
      |     ^~~~~
```

Before this patch, we already silenced it with CMake. This patch moves
the silencing to the source, so that it applies to all build tools.

Bug: curl#7229 (comment)

Reviewed-by: Marcel Raad
Closes curl#10744
bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build help wanted KNOWN_BUGS material unicode Unicode, code page, character encoding Windows Windows-specific
Development

No branches or pull requests

4 participants