Right now the test seems to take between 15 and 20 minutes
on our Windows CI environments.

The test should not even take 1 minute, why does it take 15?

If a 100 threads take 20 minutes, won't 42 threads then still take a rather excessive 8.4 minutes?

Maybe the test should rather be changed to have a max time for which it can execute?

@jay not sure, on my machine it takes less than a minute. I think the CI VMs have trouble with great parallelism due to limited number of cores.

@bagder I wasn't sure if the issue scaled in a linear fashion. I thought let's try 42 and see what happens. As stated above, I think the main problem is the thread pressure on our CI hosts. Locally it works fast and fine as expected.

As I guessed it does not seem to be a linear issue.

Before / without this PR: https://ci.appveyor.com/project/curlorg/curl/builds/44661247/job/fq5hqnkj4bjr90p5#L4987

test 3026...[curl_global_init thread-safety]
-------e--- OK (1510 out of 1510, remaining: 00:00, took 683.379s, duration: 31:14)

After / with this PR: https://ci.appveyor.com/project/curlorg/curl/builds/44653968/job/hk9etjq6tuwo0ca7#L4988

test 3026...[curl_global_init thread-safety]
-------e--- OK (1510 out of 1510, remaining: 00:00, took 87.706s, duration: 19:43

Overall this seems to cut quite some time from the AppVeyor CI runs which run tests.

Before vs. After cmake builds (top 4 MSVC, bottom 4 MSYS/MinGW):

Before vs. After autotools builds (3 MSYS/MinGW and 1 CygWin):

Seems like this is primarily affecting the 4 cmake-based MSYS/MinGW in the middle.

To sum up, only builds using the following compilers on AppVeyor are affected by the slowness after all:

• C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin
• C:\mingw-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin
• C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\bin
• C:\MinGW\bin

So I guess it is their threading model which is causing the trouble. @MarcelRaad any suggestions?

Edit: confirmed also by Azure CI running classic MinGW and similar variants:

• https://dev.azure.com/daniel0244/curl/_build/results?buildId=11061&view=logs&j=94094f6a-a458-5f8b-6d31-7c5891e4b222&t=d09beab2-e8e0-5504-136f-3960c96c0e10&l=2883
• https://dev.azure.com/daniel0244/curl/_build/results?buildId=11061&view=logs&j=631be2fa-3c4d-5580-deda-f1b6e63c7b2e&t=29c088dd-9cac-5b20-1649-eeba1a1a7c48&l=2857
• https://dev.azure.com/daniel0244/curl/_build/results?buildId=11061&view=logs&j=e551d610-6f0a-5051-bc97-b36613a028cf&t=14d88f67-387f-52ca-f069-ab0dc711f319&l=2857

This is still set a draft but wants a review. Isn't that contradictory?

I will need to reproduce this locally and figure out a better way to shorten the runtime for legacy MinGW builds.

Update: I was able to reproduce this locally with classic/legacy MinGW. Next up is identifying the root cause / finding workaround.

Root cause: the slow part is loading of iphlpapi.dll (or any other library) as part of Curl_win32_init during curl_global_init.

Should we just close this?

Root cause: the slow part is loading of iphlpapi.dll (or any other library) as part of Curl_win32_init during curl_global_init.

I pushed a change to your branch that calls loadlibrary on secur32 and iphlpapi in the main thread. This way each time curl_global_init/cleanup is called from the worker threads and it calls loadlibrary/freelibrary on those two libraries it should just increase/decrease their refcount. This may or may not work depending on if initialization of the library is taking a long time, and not the actual loading.

@jay thanks, it is definitely the loading part according to my tests. But even reversing the binaries and running them outside of msys-shells (v1 or v2) did not help me identify the difference between the various fast (mingw-w64 from msys2) and slow (any mingw on msys1) build variants.

@jay to resolve the merge conflict which is blocking Azure Pipelines and AppVeyor you may delete my remaining commit.

job windows msys v1_mingw32 now shows:

2022-11-16T07:40:24.7492812Z test 3026...[curl_global_init thread-safety]
2022-11-16T07:40:25.0123541Z -------e--- OK (1526 out of 1530, remaining: 00:02, took 0.268s, duration: 17:07)

Does that mean this is fixed or should I be checking a different job?

Thanks a lot @jay. Yeah, all of the msys v1 jobs experienced the slowness issue, so if one is fixed, all of them should be.

Will you take it completely from here?

Will you take it completely from here?

Yup np. I'm going to wait on the CI for this last fixup to complete then I will land it.

Sounds good, thanks again.

CI looks good to me. I think this is ready for merge.