Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sometimes transfers timeout earlier than CURLOPT_CONNECTTIMEOUT #3602

Closed
accountantM opened this issue Feb 22, 2019 · 36 comments
Closed

sometimes transfers timeout earlier than CURLOPT_CONNECTTIMEOUT #3602

accountantM opened this issue Feb 22, 2019 · 36 comments

Comments

@accountantM
Copy link

accountantM commented Feb 22, 2019

I did this

used curl_multi interface in php to test about 400 proxies at the same time with a CURLOPT_CONNECTTIMEOUT set 30, however sometimes the later requests (about 150 from 400) all outputs "Connection time-out" and the total_time is very small (about 0.002 s)!
this is a question on stackoverflow.com
and this is a php class that tests this behaviour CurlMultiTimeoutBugTester

This is a sample output from the tester, please notice the later requests

I expected the following

every request takes his chance and tries to connect for 30 seconds then outputs "Connection timed out after 30005 milliseconds" and total_time is 30 seconds or succeed

curl/libcurl version

7.59.0 and 7.64.0

operating system

Linux CentOS and Linux fedora

@bagder bagder changed the title sometimes later requests in curl_multi timeout very small than CURLOPT_CONNECTTIMEOUT sometimes transfers timeout earlier than CURLOPT_CONNECTTIMEOUT Feb 22, 2019
@bagder
Copy link
Member

bagder commented Feb 22, 2019

I'm afraid the amount of PHP code plus PHP binding layering here is a bit too thick for me to tell exactly what's going on or where exactly the issue lies. We need this example converted to plain C so that we can check exactly what's going on and why. It seems odd that this issue should suddenly start to appear after a certain amount of parallel transfers. Is it out of sockets somehow? If you limit this program to only test for 100 proxies, does then always work correctly?

@accountantM
Copy link
Author

Thank you very much Daniel for your replay. I did more tests
test2_100proxies_bugAppear

test3_100proxies_bugAppear

test4_407proxies_NoBug

/test5_407proxies_BugAppears

test6_30proxies_NoBug

I did about 100 tests and the bug appeared in about 20 tests of them (I did the tests on both a production server Linux centos, and laptop Linux fedora)

regarding the sockets question, I don't know if they out of sockets or not, I just use the php curl_* functions, and I don't know what are they doing behind the scenes.

I also noted something. The reports that don't have the issue are having a normal total_time from the very FIRST request (e.g 1.999564 seconds while the CURLOPT_CONNECTTIMEOUT is 2 seconds ), and the reports with the issue are having late total_time from the very FIRST request (e.g 3.020935 seconds while the CURLOPT_CONNECTTIMEOUT is 2 seconds)

I don't know C but I will download a C compiler and an IDE and will try my best to write this example in plain C.
Thank you very much Daniel for your effort and time in cURL which is almost the only reliable solution for PHP developers for making parallel http requests

@bagder
Copy link
Member

bagder commented Feb 23, 2019

I wrote a C adaptation of what I believe is the same spirit as the original PHP code. See issue-3602.c.

It does however not seem to reproduce the problem here when I use it with curl from current git master.

@accountantM
Copy link
Author

Thank you for making this C code, I will run it on the same 2 machines I run the php code on, and will tell you the results, just give me some time

@accountantM
Copy link
Author

It appeared again ! :(
I run your code one time then I added these 3 lines to your code

double time;
curl_easy_getinfo(e, CURLINFO_TOTAL_TIME, &time);
fprintf(stderr, "returned: %d - %s time:%f\n", msg->data.result, err, time);

and run another time

I tested on my laptop a fedora with curl 7.59.0 and the same issue happened in the 2 tests, I will try to run the same code on my cantos production server curl 7.64.0, it has much more resources(CPU and internet connection ) than my laptop and tell you the results, if you want me to record my screen while doing the tests, just tell me. This is the output in the terminal

The Second Test result

Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
returned: 7 - Failed to connect to 83.239.97.26 port 50223: Connection refused time:0.231454
returned: 7 - Failed to connect to 85.52.217.114 port 42544: Connection refused time:0.231213
returned: 7 - Failed to connect to 85.193.216.85 port 59205: Connection refused time:0.231104
returned: 7 - Failed to connect to 85.193.229.6 port 57712: Connection refused time:0.231026
returned: 7 - Failed to connect to 78.128.124.9 port 47654: No route to host time:0.229756
returned: 7 - Failed to connect to 78.187.50.200 port 30852: Connection refused time:0.229622
returned: 7 - Failed to connect to 80.249.229.64 port 42186: Connection refused time:0.229047
returned: 7 - Failed to connect to 81.16.10.141 port 59352: Connection refused time:0.228993
returned: 7 - Failed to connect to 81.163.62.221 port 31299: Connection refused time:0.228936
returned: 56 - Received HTTP code 503 from proxy after CONNECT time:1.781811
returned: 7 - Failed to connect to 83.13.63.38 port 60860: Connection refused time:0.228710
returned: 56 - Proxy CONNECT aborted time:1.781721
returned: 7 - Failed to connect to 46.151.108.6 port 33727: Connection refused time:0.228485
returned: 7 - Failed to connect to 46.164.149.40 port 36551: Connection refused time:0.228429
returned: 7 - Failed to connect to 51.254.182.52 port 61133: Connection refused time:0.227951
returned: 56 - Proxy CONNECT aborted time:2.474736
returned: 7 - Failed to connect to 65.173.110.238 port 37252: Connection refused time:0.227615
returned: 7 - Failed to connect to 41.78.172.20 port 61169: Connection refused time:0.227199
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.227029
returned: 28 - Connection time-out time:0.226973
returned: 28 - Connection time-out time:0.226918
returned: 28 - Connection time-out time:0.226861
returned: 28 - Connection time-out time:0.226806
returned: 28 - Connection time-out time:0.226747
returned: 28 - Connection time-out time:0.226692
returned: 28 - Connection time-out time:0.226637
returned: 28 - Connection time-out time:0.226581
returned: 28 - Connection time-out time:0.226522
returned: 28 - Connection time-out time:0.226459
returned: 28 - Connection time-out time:0.226383
returned: 28 - Connection time-out time:0.226324
returned: 28 - Connection time-out time:0.226260
returned: 28 - Connection time-out time:0.226197
returned: 28 - Connection time-out time:0.226143
returned: 28 - Connection time-out time:0.226086
returned: 28 - Connection time-out time:0.226029
returned: 28 - Connection time-out time:0.225971
returned: 28 - Connection time-out time:0.225915
returned: 28 - Connection time-out time:0.225858
returned: 28 - Connection time-out time:0.225722
returned: 28 - Connection time-out time:0.225665
returned: 28 - Connection time-out time:0.225607
returned: 28 - Connection time-out time:0.225552
returned: 28 - Connection time-out time:0.225497
returned: 28 - Connection time-out time:0.225422
returned: 28 - Connection time-out time:0.225354
returned: 28 - Connection time-out time:0.225296
returned: 28 - Connection time-out time:0.225238
returned: 28 - Connection time-out time:0.225182
returned: 28 - Connection time-out time:0.225126
returned: 28 - Connection time-out time:0.225067
returned: 28 - Connection time-out time:0.225013
returned: 28 - Connection time-out time:0.224956
returned: 28 - Connection time-out time:0.224899
returned: 28 - Connection time-out time:0.224843
returned: 28 - Connection time-out time:0.224784
returned: 28 - Connection time-out time:0.224728
returned: 28 - Connection time-out time:0.224672
returned: 28 - Connection time-out time:0.224607
returned: 28 - Connection time-out time:0.224545
returned: 28 - Connection time-out time:0.224481
returned: 28 - Connection time-out time:0.224420
returned: 28 - Connection time-out time:0.224364
returned: 7 - Failed to connect to 95.79.54.143 port 80: Connection refused time:0.224306
returned: 7 - Failed to connect to 95.87.127.133 port 50594: Connection refused time:0.224246
returned: 7 - Failed to connect to 95.140.27.135 port 60275: Connection refused time:0.224191
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998953
returned: 7 - Failed to connect to 96.9.69.230 port 53281: No route to host time:0.224073
returned: 7 - Failed to connect to 96.9.73.79 port 61057: Connection refused time:0.224011
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998958
returned: 7 - Failed to connect to 101.109.255.243 port 50098: Connection refused time:0.223900
returned: 7 - Failed to connect to 101.255.124.202 port 42608: Connection refused time:0.223845
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.223730
returned: 28 - Connection time-out time:0.223673
returned: 28 - Connection time-out time:0.223617
returned: 28 - Connection time-out time:0.223561
returned: 28 - Connection time-out time:0.223467
returned: 28 - Connection time-out time:0.223402
returned: 28 - Connection time-out time:0.223342
returned: 28 - Connection time-out time:0.223274
returned: 28 - Connection time-out time:0.223215
returned: 28 - Connection time-out time:0.223153
returned: 28 - Connection time-out time:0.223096
returned: 28 - Connection time-out time:0.223039
returned: 28 - Connection time-out time:0.222971
returned: 28 - Connection time-out time:0.222914
returned: 28 - Connection time-out time:0.222854
returned: 28 - Connection time-out time:0.222800
returned: 28 - Connection time-out time:0.222738
returned: 28 - Connection time-out time:0.222683
returned: 28 - Connection time-out time:0.222624
returned: 28 - Connection time-out time:0.222549
returned: 28 - Connection time-out time:0.222484
returned: 28 - Connection time-out time:0.222428
returned: 28 - Connection time-out time:0.222371
returned: 28 - Connection time-out time:0.222312
returned: 28 - Connection time-out time:0.222258
returned: 28 - Connection time-out time:0.222202
returned: 28 - Connection time-out time:0.222146
returned: 7 - Failed to connect to 103.115.100.246 port 47788: Connection refused time:0.222089
returned: 7 - Failed to connect to 103.192.169.186 port 35865: Connection refused time:0.222033
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.221974
returned: 28 - Connection time-out time:0.221916
returned: 28 - Connection time-out time:0.221858
returned: 28 - Connection time-out time:0.221707
returned: 28 - Connection time-out time:0.221633
returned: 28 - Connection time-out time:0.221559
returned: 28 - Connection time-out time:0.221500
returned: 28 - Connection time-out time:0.221437
returned: 28 - Connection time-out time:0.221378
returned: 28 - Connection time-out time:0.221321
returned: 28 - Connection time-out time:0.221254
returned: 28 - Connection time-out time:0.221193
returned: 28 - Connection time-out time:0.221133
returned: 28 - Connection time-out time:0.221078
returned: 28 - Connection time-out time:0.221020
returned: 28 - Connection time-out time:0.220964
returned: 28 - Connection time-out time:0.220899
returned: 28 - Connection time-out time:0.220812
returned: 28 - Connection time-out time:0.220732
returned: 28 - Connection time-out time:0.220632
returned: 28 - Connection time-out time:0.220550
returned: 28 - Connection time-out time:0.220467
returned: 28 - Connection time-out time:0.220407
returned: 28 - Connection time-out time:0.220352
returned: 28 - Connection time-out time:0.220294
returned: 7 - Failed to connect to 1.20.97.166 port 32588: Connection refused time:0.220236
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998930
returned: 7 - Failed to connect to 1.20.99.185 port 44450: Connection refused time:0.220113
returned: 7 - Failed to connect to 1.20.100.42 port 47962: Connection refused time:0.220053
returned: 7 - Failed to connect to 12.189.124.100 port 44791: Connection refused time:0.219459
returned: 7 - Failed to connect to 18.228.217.208 port 80: Connection refused time:0.219224
returned: 7 - Failed to connect to 24.172.34.114 port 52143: Connection refused time:0.219168
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.219107
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.218934
returned: 28 - Connection time-out time:0.218875
returned: 28 - Connection time-out time:0.218815
returned: 28 - Connection time-out time:0.218734
returned: 28 - Connection time-out time:0.218676
returned: 28 - Connection time-out time:0.218619
returned: 28 - Connection time-out time:0.218561
returned: 28 - Connection time-out time:0.218401
returned: 28 - Connection time-out time:0.218343
returned: 7 - Failed to connect to 111.92.243.154 port 37539: Connection refused time:0.218285
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998973
returned: 7 - Failed to connect to 113.53.83.157 port 35969: Connection refused time:0.218171
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.218111
returned: 28 - Connection time-out time:0.218056
returned: 28 - Connection time-out time:0.217999
returned: 28 - Connection time-out time:0.217943
returned: 28 - Connection time-out time:0.217879
returned: 28 - Connection time-out time:0.217802
returned: 28 - Connection time-out time:0.217721
returned: 28 - Connection time-out time:0.217659
returned: 28 - Connection time-out time:0.217600
returned: 28 - Connection time-out time:0.217541
returned: 28 - Connection time-out time:0.217487
returned: 28 - Connection time-out time:0.217428
returned: 28 - Connection time-out time:0.217371
returned: 28 - Connection time-out time:0.217312
returned: 28 - Connection time-out time:0.217254
returned: 28 - Connection time-out time:0.217197
returned: 28 - Connection time-out time:0.217142
returned: 28 - Connection time-out time:0.217083
returned: 28 - Connection time-out time:0.217017
returned: 28 - Connection time-out time:0.216952
returned: 28 - Connection time-out time:0.216895
returned: 28 - Connection time-out time:0.216825
returned: 28 - Connection time-out time:0.216765
returned: 28 - Connection time-out time:0.216696
returned: 28 - Connection time-out time:0.216639
returned: 28 - Connection time-out time:0.216581
returned: 28 - Connection time-out time:0.216525
returned: 7 - Failed to connect to 125.209.108.26 port 53126: Connection refused time:0.216464
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998957
returned: 7 - Failed to connect to 131.108.63.213 port 57604: No route to host time:0.216169
returned: 7 - Failed to connect to 134.0.16.6 port 47631: Connection refused time:0.216043
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.215986
returned: 28 - Connection time-out time:0.215926
returned: 28 - Connection time-out time:0.215856
returned: 28 - Connection time-out time:0.215796
returned: 28 - Connection time-out time:0.215740
returned: 28 - Connection time-out time:0.215683
returned: 28 - Connection time-out time:0.215616
returned: 28 - Connection time-out time:0.215560
returned: 28 - Connection time-out time:0.215502
returned: 28 - Connection time-out time:0.215440
returned: 28 - Connection time-out time:0.215384
returned: 28 - Connection time-out time:0.215321
returned: 28 - Connection time-out time:0.215257
returned: 28 - Connection time-out time:0.215199
returned: 28 - Connection time-out time:0.215136
returned: 28 - Connection time-out time:0.215076
returned: 28 - Connection time-out time:0.215021
returned: 28 - Connection time-out time:0.214952
returned: 28 - Connection time-out time:0.214886
returned: 28 - Connection time-out time:0.214826
returned: 28 - Connection time-out time:0.214764
returned: 28 - Connection time-out time:0.214706
returned: 28 - Connection time-out time:0.214649
returned: 28 - Connection time-out time:0.214592
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.214467
returned: 28 - Connection time-out time:0.214420
returned: 28 - Connection time-out time:0.214366
returned: 28 - Connection time-out time:0.214313
returned: 28 - Connection time-out time:0.214248
returned: 28 - Connection time-out time:0.214194
returned: 28 - Connection time-out time:0.214140
returned: 28 - Connection time-out time:0.214085
returned: 28 - Connection time-out time:0.214013
returned: 28 - Connection time-out time:0.213940
returned: 28 - Connection time-out time:0.213881
returned: 28 - Connection time-out time:0.213826
returned: 28 - Connection time-out time:0.213772
returned: 28 - Connection time-out time:0.213714
returned: 28 - Connection time-out time:0.213662
returned: 28 - Connection time-out time:0.213595
returned: 28 - Connection time-out time:0.213541
returned: 28 - Connection time-out time:0.213485
returned: 28 - Connection time-out time:0.213430
returned: 28 - Connection time-out time:0.213371
returned: 28 - Connection time-out time:0.213316
returned: 28 - Connection time-out time:0.213255
returned: 28 - Connection time-out time:0.213197
returned: 28 - Connection time-out time:0.213144
returned: 28 - Connection time-out time:0.213090
returned: 28 - Connection time-out time:0.213024
returned: 28 - Connection time-out time:0.212968
returned: 28 - Connection time-out time:0.212894
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998988
returned: 28 - Connection time-out time:0.212786
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998977
returned: 56 - Proxy CONNECT aborted time:2.998915
returned: 7 - Failed to connect to 180.247.199.128 port 8080: Connection refused time:0.212449
returned: 7 - Failed to connect to 182.16.163.138 port 8080: Connection refused time:0.211957
returned: 56 - Proxy CONNECT aborted time:2.998403
returned: 7 - Failed to connect to 182.53.206.40 port 52214: Connection refused time:0.211751
returned: 7 - Failed to connect to 182.75.21.30 port 46527: Connection refused time:0.211696
returned: 7 - Failed to connect to 182.253.201.10 port 36776: Connection refused time:0.211638
returned: 7 - Failed to connect to 185.136.151.20 port 58371: No route to host time:0.211217
returned: 7 - Failed to connect to 185.194.24.187 port 37854: No route to host time:0.211080
returned: 56 - Proxy CONNECT aborted time:2.997658
returned: 7 - Failed to connect to 187.111.192.142 port 30485: Connection refused time:0.210794
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.210737
returned: 28 - Connection time-out time:0.210679
returned: 28 - Connection time-out time:0.210621
returned: 28 - Connection time-out time:0.210565
returned: 28 - Connection time-out time:0.210500
returned: 28 - Connection time-out time:0.210441
returned: 28 - Connection time-out time:0.210383
returned: 28 - Connection time-out time:0.210324
returned: 28 - Connection time-out time:0.210267
returned: 28 - Connection time-out time:0.210183
returned: 28 - Connection time-out time:0.210119
returned: 28 - Connection time-out time:0.209971
returned: 28 - Connection time-out time:0.209910
returned: 28 - Connection time-out time:0.209854
returned: 28 - Connection time-out time:0.209791
returned: 28 - Connection time-out time:0.209736
returned: 28 - Connection time-out time:0.209678
returned: 28 - Proxy CONNECT aborted due to timeout time:2.998994
returned: 28 - Connection time-out time:0.209565
returned: 7 - Failed to connect to 195.39.71.252 port 56935: No route to host time:0.209494
returned: 56 - Proxy CONNECT aborted time:2.998923
returned: 7 - Failed to connect to 195.239.178.110 port 40448: Connection refused time:0.209318
returned: 7 - Failed to connect to 196.32.109.73 port 56930: Connection refused time:0.208875
returned: 7 - Failed to connect to 196.38.8.91 port 8080: Connection refused time:0.208817
returned: 7 - Failed to connect to 197.98.180.162 port 61047: Connection refused time:0.208695
returned: 7 - Failed to connect to 197.232.55.224 port 39385: Connection refused time:0.208632
returned: 56 - Proxy CONNECT aborted time:2.998258
returned: 7 - Failed to connect to 197.248.224.242 port 80: Connection refused time:0.208516
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.208458
returned: 28 - Connection time-out time:0.208397
returned: 28 - Connection time-out time:0.208340
returned: 28 - Connection time-out time:0.208275
returned: 28 - Connection time-out time:0.208217
returned: 28 - Connection time-out time:0.208158
returned: 28 - Connection time-out time:0.208103
returned: 28 - Connection time-out time:0.208047
returned: 7 - Failed to connect to 202.3.72.6 port 38725: Connection refused time:0.207987
returned: 7 - Failed to connect to 202.49.183.168 port 30309: Connection refused time:0.207932
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.207857
returned: 28 - Connection time-out time:0.207803
returned: 28 - Connection time-out time:0.207745
returned: 28 - Connection time-out time:0.207690
returned: 28 - Connection time-out time:0.207633
returned: 28 - Connection time-out time:0.207573
returned: 28 - Connection time-out time:0.207512
returned: 28 - Connection time-out time:0.207454
returned: 28 - Connection time-out time:0.207395
returned: 28 - Connection time-out time:0.207318
returned: 28 - Connection time-out time:0.207252
returned: 28 - Connection time-out time:0.207194
returned: 28 - Connection time-out time:0.207135
returned: 28 - Connection time-out time:0.207080
returned: 28 - Connection time-out time:0.207018
returned: 28 - Connection time-out time:0.206960
returned: 28 - Connection time-out time:0.206902
returned: 28 - Connection time-out time:0.206846
returned: 28 - Connection time-out time:0.206783
returned: 28 - Connection time-out time:0.206726
returned: 28 - Connection time-out time:0.206669
returned: 28 - Connection time-out time:0.206614
returned: 28 - Connection time-out time:0.206561
returned: 28 - Connection time-out time:0.206503
Perform 102 parallel transfers
returned: 28 - Connection timed out after 3031 milliseconds time:3.031532
returned: 28 - Connection timed out after 3031 milliseconds time:3.031229
returned: 28 - Connection timed out after 3030 milliseconds time:3.031067
returned: 28 - Operation timed out after 3030 milliseconds with 0 out of 0 bytes received time:3.030905
returned: 28 - Connection timed out after 3030 milliseconds time:3.030608
returned: 28 - Connection timed out after 3030 milliseconds time:3.030560
returned: 28 - Connection timed out after 3030 milliseconds time:3.030488
returned: 28 - Connection timed out after 3030 milliseconds time:3.030433
returned: 28 - Operation timed out after 3030 milliseconds with 0 out of 0 bytes received time:3.030386
returned: 28 - Operation timed out after 3029 milliseconds with 0 out of 0 bytes received time:3.030431
returned: 28 - Connection timed out after 3029 milliseconds time:3.030391
returned: 28 - Connection timed out after 3029 milliseconds time:3.030231
returned: 28 - Connection timed out after 3029 milliseconds time:3.030134
returned: 28 - Connection timed out after 3029 milliseconds time:3.030063
returned: 28 - Operation timed out after 3029 milliseconds with 0 out of 0 bytes received time:3.030000
returned: 28 - Operation timed out after 3029 milliseconds with 0 out of 0 bytes received time:3.030929
returned: 28 - Connection timed out after 3029 milliseconds time:3.030920
returned: 28 - Connection timed out after 3029 milliseconds time:3.030890
returned: 28 - Operation timed out after 3029 milliseconds with 0 out of 0 bytes received time:3.030795
returned: 28 - Operation timed out after 3028 milliseconds with 0 out of 0 bytes received time:3.030691
returned: 28 - Connection timed out after 3028 milliseconds time:3.030651
returned: 28 - Connection timed out after 3028 milliseconds time:3.030599
returned: 28 - Connection timed out after 3028 milliseconds time:3.030562
returned: 28 - Operation timed out after 3028 milliseconds with 0 out of 0 bytes received time:3.030523
returned: 28 - Operation timed out after 3028 milliseconds with 0 out of 0 bytes received time:3.031417
returned: 28 - Connection timed out after 3028 milliseconds time:3.032299
returned: 28 - Connection timed out after 3028 milliseconds time:3.032282
returned: 28 - Connection timed out after 3028 milliseconds time:3.032008
returned: 28 - Operation timed out after 3028 milliseconds with 0 out of 0 bytes received time:3.031968
returned: 28 - Connection timed out after 3028 milliseconds time:3.031888
returned: 28 - Connection timed out after 3027 milliseconds time:3.031791
returned: 28 - Operation timed out after 3027 milliseconds with 0 out of 0 bytes received time:3.031621
returned: 28 - Operation timed out after 3027 milliseconds with 0 out of 0 bytes received time:3.031474
returned: 28 - Operation timed out after 3027 milliseconds with 0 out of 0 bytes received time:3.032432
returned: 28 - Operation timed out after 3027 milliseconds with 0 out of 0 bytes received time:3.033476
returned: 28 - Operation timed out after 3027 milliseconds with 0 out of 0 bytes received time:3.034504
returned: 28 - Connection timed out after 3027 milliseconds time:3.034445
returned: 28 - Connection timed out after 3027 milliseconds time:3.034376
returned: 28 - Connection timed out after 3026 milliseconds time:3.034275
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.034176
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.034162
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.034147
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.034120
returned: 28 - Connection timed out after 3026 milliseconds time:3.037546
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.037588
returned: 28 - Operation timed out after 3026 milliseconds with 0 out of 0 bytes received time:3.038710
returned: 28 - Connection timed out after 3026 milliseconds time:3.038711
returned: 28 - Connection timed out after 3022 milliseconds time:3.035311
returned: 28 - Connection timed out after 3018 milliseconds time:3.031357
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031299
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031277
returned: 28 - Connection timed out after 3018 milliseconds time:3.031250
returned: 28 - Connection timed out after 3018 milliseconds time:3.031215
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031135
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031103
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031187
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031099
returned: 28 - Connection timed out after 3018 milliseconds time:3.031075
returned: 28 - Operation timed out after 3018 milliseconds with 0 out of 0 bytes received time:3.031035
returned: 28 - Operation timed out after 3017 milliseconds with 0 out of 0 bytes received time:3.030822
returned: 28 - Connection timed out after 3017 milliseconds time:3.030797
returned: 28 - Connection timed out after 3015 milliseconds time:3.028053
returned: 28 - Connection timed out after 3015 milliseconds time:3.028012
returned: 28 - Connection timed out after 3014 milliseconds time:3.027971
returned: 28 - Operation timed out after 3014 milliseconds with 0 out of 0 bytes received time:3.027855
returned: 28 - Connection timed out after 3013 milliseconds time:3.026234
returned: 28 - Operation timed out after 3011 milliseconds with 0 out of 0 bytes received time:3.024291
returned: 28 - Connection timed out after 3011 milliseconds time:3.024270
returned: 28 - Operation timed out after 3011 milliseconds with 0 out of 0 bytes received time:3.024179
returned: 28 - Connection timed out after 3010 milliseconds time:3.024104
returned: 28 - Connection timed out after 3010 milliseconds time:3.024078
returned: 28 - Connection timed out after 3010 milliseconds time:3.024041
returned: 28 - Connection timed out after 3010 milliseconds time:3.024005
returned: 28 - Connection timed out after 3010 milliseconds time:3.023964
returned: 28 - Connection timed out after 3010 milliseconds time:3.023912
returned: 28 - Operation timed out after 3010 milliseconds with 0 out of 0 bytes received time:3.023856
returned: 28 - Connection timed out after 3010 milliseconds time:3.023658
returned: 28 - Connection timed out after 3010 milliseconds time:3.023473
returned: 28 - Connection timed out after 3010 milliseconds time:3.023427
returned: 28 - Connection timed out after 3009 milliseconds time:3.023387
returned: 28 - Connection timed out after 3009 milliseconds time:3.023348
returned: 28 - Operation timed out after 3009 milliseconds with 0 out of 0 bytes received time:3.023306
returned: 28 - Operation timed out after 3009 milliseconds with 0 out of 0 bytes received time:3.023269
returned: 28 - Connection timed out after 3009 milliseconds time:3.023173
returned: 28 - Connection timed out after 3009 milliseconds time:3.023065
returned: 28 - Connection timed out after 3009 milliseconds time:3.023022
returned: 28 - Connection timed out after 3009 milliseconds time:3.022925
returned: 28 - Connection timed out after 3007 milliseconds time:3.021473
returned: 28 - Connection timed out after 3007 milliseconds time:3.021295
returned: 28 - Connection timed out after 3007 milliseconds time:3.021234
returned: 28 - Connection timed out after 3007 milliseconds time:3.021192
returned: 28 - Connection timed out after 3007 milliseconds time:3.021163
returned: 28 - Connection timed out after 3007 milliseconds time:3.021131
returned: 28 - Connection timed out after 3007 milliseconds time:3.021088
returned: 28 - Connection timed out after 3007 milliseconds time:3.020924
Perform 7 parallel transfers
Perform 7 parallel transfers
Perform 7 parallel transfers
Perform 7 parallel transfers
returned: 7 - Failed to connect to 217.113.17.226 port 50948: Connection refused time:0.200511
Perform 6 parallel transfers
returned: 7 - Failed to connect to 213.6.199.94 port 41320: Connection refused time:0.211043
returned: 7 - Failed to connect to 213.87.42.53 port 40367: No route to host time:0.210922
Perform 4 parallel transfers
returned: 7 - Failed to connect to 221.120.163.242 port 56969: Connection refused time:0.450393
Perform 3 parallel transfers
Perform 3 parallel transfers
Perform 3 parallel transfers
Perform 3 parallel transfers
Perform 3 parallel transfers
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
        
    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 50px;
        background-color: #fff;
        border-radius: 1em;
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        body {
            background-color: #fff;
        }
        div {
            width: auto;
            margin: 0 auto;
            border-radius: 0;
            padding: 1em;
        }
    }
    </style>    
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
Perform 2 parallel transfers
Perform 2 parallel transfers
returned: 28 - Connection timed out after 3001 milliseconds time:3.001430
returned: 28 - Connection timed out after 3000 milliseconds time:3.000816

The First Test Result


Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
returned: 7 - Failed to connect to 85.52.217.114 port 42544: Connection refused
returned: 7 - Failed to connect to 85.193.216.85 port 59205: Connection refused
Perform 400 parallel transfers
returned: 7 - Failed to connect to 83.239.97.26 port 50223: Connection refused
returned: 7 - Failed to connect to 85.193.229.6 port 57712: Connection refused
returned: 7 - Failed to connect to 80.249.229.64 port 42186: Connection refused
returned: 7 - Failed to connect to 81.16.10.141 port 59352: Connection refused
returned: 7 - Failed to connect to 83.13.63.38 port 60860: Connection refused
returned: 7 - Failed to connect to 51.254.182.52 port 61133: Connection refused
Perform 399 parallel transfers
returned: 7 - Failed to connect to 83.219.1.80 port 56004: Connection refused
Perform 398 parallel transfers
Perform 398 parallel transfers
returned: 7 - Failed to connect to 89.189.189.115 port 59249: Connection refused
returned: 7 - Failed to connect to 78.187.50.200 port 30852: Connection refused
returned: 7 - Failed to connect to 81.163.62.221 port 31299: Connection refused
returned: 7 - Failed to connect to 46.151.108.6 port 33727: Connection refused
returned: 7 - Failed to connect to 46.164.149.40 port 36551: Connection refused
returned: 7 - Failed to connect to 91.214.240.19 port 50481: Connection refused
returned: 7 - Failed to connect to 91.230.199.174 port 60926: Connection refused
returned: 7 - Failed to connect to 92.246.220.206 port 40396: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 93.87.66.66 port 61831: Connection refused
returned: 7 - Failed to connect to 93.116.57.4 port 55674: Connection refused
returned: 7 - Failed to connect to 93.177.150.112 port 45186: Connection refused
returned: 7 - Failed to connect to 94.102.124.207 port 35836: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 94.180.245.1 port 61251: Connection refused
returned: 7 - Failed to connect to 94.228.21.66 port 55295: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Proxy CONNECT aborted due to timeout
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 12.189.124.100 port 44791: Connection refused
returned: 7 - Failed to connect to 18.228.217.208 port 80: Connection refused
returned: 7 - Failed to connect to 24.172.34.114 port 52143: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 7 - Failed to connect to 118.173.232.43 port 48204: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 7 - Failed to connect to 118.174.234.129 port 54431: Connection refused
returned: 7 - Failed to connect to 118.175.93.28 port 61285: Connection refused
returned: 7 - Failed to connect to 118.175.93.164 port 37694: Connection refused
returned: 7 - Failed to connect to 118.179.69.233 port 44505: Connection refused
returned: 7 - Failed to connect to 118.179.84.106 port 34236: Connection refused
returned: 7 - Failed to connect to 119.82.253.155 port 37436: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 122.102.41.82 port 56067: Connection refused
returned: 7 - Failed to connect to 122.176.65.143 port 42175: Connection refused
returned: 7 - Failed to connect to 124.41.211.213 port 54986: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 125.27.251.208 port 30753: Connection refused
returned: 7 - Failed to connect to 125.209.108.26 port 53126: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Proxy CONNECT aborted due to timeout
returned: 28 - Proxy CONNECT aborted due to timeout
returned: 7 - Failed to connect to 163.53.150.230 port 30195: Connection refused
returned: 7 - Failed to connect to 163.53.182.148 port 39486: Connection refused
returned: 7 - Failed to connect to 165.90.211.69 port 61692: No route to host
returned: 7 - Failed to connect to 166.249.54.244 port 55867: Connection refused
returned: 7 - Failed to connect to 168.181.121.195 port 51514: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 169.255.67.244 port 37804: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 176.98.75.120 port 45259: Connection refused
returned: 7 - Failed to connect to 176.98.76.210 port 45280: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 176.111.10.136 port 46560: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 56 - Proxy CONNECT aborted
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 177.75.143.198 port 43223: Connection refused
returned: 7 - Failed to connect to 177.220.171.54 port 60729: Connection refused
returned: 7 - Failed to connect to 177.223.52.242 port 42363: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 178.75.58.58 port 41514: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 180.247.199.128 port 8080: Connection refused
returned: 7 - Failed to connect to 182.16.163.138 port 8080: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to connect to 182.53.206.40 port 52214: Connection refused
returned: 7 - Failed to connect to 182.75.21.30 port 46527: Connection refused
returned: 7 - Failed to connect to 182.253.201.10 port 36776: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 7 - Failed to connect to 187.111.192.142 port 30485: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 7 - Failed to connect to 188.130.240.17 port 52343: Connection refused
returned: 56 - Proxy CONNECT aborted
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 7 - Failed to connect to 196.32.109.73 port 56930: Connection refused
returned: 7 - Failed to connect to 196.38.8.91 port 8080: Connection refused
returned: 7 - Failed to connect to 197.98.180.162 port 61047: Connection refused
returned: 7 - Failed to connect to 197.232.55.224 port 39385: Connection refused
returned: 7 - Failed to connect to 197.248.224.242 port 80: Connection refused
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 28 - Connection time-out
returned: 7 - Failed to connect to 206.189.137.227 port 3128: Connection refused
returned: 7 - Failed to connect to 212.28.237.131 port 59588: Connection refused
returned: 7 - Failed to connect to 213.6.199.94 port 41320: Connection refused
returned: 7 - Failed to connect to 213.87.42.53 port 40367: No route to host
returned: 7 - Failed to receive SOCKS4 connect request ack.
returned: 28 - Connection time-out
returned: 28 - Connection time-out
Perform 139 parallel transfers
returned: 28 - Connection timed out after 3252 milliseconds
returned: 28 - Connection timed out after 3252 milliseconds
returned: 28 - Operation timed out after 3252 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3251 milliseconds
returned: 28 - Connection timed out after 3251 milliseconds
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3251 milliseconds
returned: 28 - Connection timed out after 3251 milliseconds
returned: 28 - Operation timed out after 3251 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3250 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Operation timed out after 3250 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Operation timed out after 3250 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3250 milliseconds
returned: 28 - Connection timed out after 3249 milliseconds
returned: 28 - Connection timed out after 3249 milliseconds
returned: 28 - Connection timed out after 3249 milliseconds
returned: 28 - Connection timed out after 3249 milliseconds
returned: 28 - Operation timed out after 3249 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3249 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3248 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Operation timed out after 3248 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Operation timed out after 3248 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3248 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3247 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3246 milliseconds
returned: 28 - Connection timed out after 3245 milliseconds
returned: 28 - Connection timed out after 3245 milliseconds
returned: 28 - Operation timed out after 3245 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3245 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3245 milliseconds
returned: 28 - Connection timed out after 3245 milliseconds
returned: 28 - Operation timed out after 3245 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3245 milliseconds
returned: 28 - Operation timed out after 3245 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3244 milliseconds
returned: 28 - Operation timed out after 3244 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3244 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3244 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3244 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3239 milliseconds
returned: 28 - Connection timed out after 3239 milliseconds
returned: 28 - Operation timed out after 3239 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3239 milliseconds
returned: 28 - Operation timed out after 3238 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3238 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3238 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3238 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3238 milliseconds
returned: 28 - Operation timed out after 3238 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3235 milliseconds
returned: 28 - Operation timed out after 3235 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3235 milliseconds
returned: 28 - Operation timed out after 3235 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3234 milliseconds
returned: 28 - Operation timed out after 3234 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3232 milliseconds
returned: 28 - Connection timed out after 3232 milliseconds
returned: 28 - Connection timed out after 3232 milliseconds
returned: 28 - Connection timed out after 3231 milliseconds
returned: 28 - Connection timed out after 3230 milliseconds
returned: 28 - Connection timed out after 3229 milliseconds
returned: 28 - Connection timed out after 3229 milliseconds
returned: 28 - Connection timed out after 3229 milliseconds
returned: 28 - Connection timed out after 3228 milliseconds
returned: 28 - Operation timed out after 3228 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3228 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3228 milliseconds
returned: 28 - Connection timed out after 3228 milliseconds
returned: 28 - Operation timed out after 3228 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3228 milliseconds
returned: 28 - Operation timed out after 3228 milliseconds with 0 out of 0 bytes received
returned: 28 - Operation timed out after 3228 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3227 milliseconds
returned: 28 - Connection timed out after 3226 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3225 milliseconds
returned: 28 - Connection timed out after 3223 milliseconds
returned: 28 - Operation timed out after 3223 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3223 milliseconds
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Operation timed out after 3221 milliseconds with 0 out of 0 bytes received
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Connection timed out after 3221 milliseconds
returned: 28 - Connection timed out after 3008 milliseconds
returned: 28 - Connection timed out after 3006 milliseconds

@accountantM
Copy link
Author

I tried more 10 times and it appeared in 3 times of them (on Linux Fedora curl 7.59)

@accountantM
Copy link
Author

I suppressed the output using the code in this example and allowed up to 30 seconds for connect and total 40 seconds for time out

curl_easy_setopt(e, CURLOPT_CONNECTTIMEOUT_MS, 30000L);
curl_easy_setopt(e, CURLOPT_TIMEOUT_MS, 40000L);

and it appeared also several times (notice the requests with "Connection time-out" and small total time)

Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
Perform 400 parallel transfers
returned: 7 - Failed to connect to 83.239.97.26 port 50223: Connection refused time:0.229924
returned: 7 - Failed to connect to 85.52.217.114 port 42544: Connection refused time:0.229732
returned: 7 - Failed to connect to 85.193.229.6 port 57712: Connection refused time:0.229597
Perform 400 parallel transfers
returned: 7 - Failed to connect to 83.219.1.80 port 56004: Connection refused time:0.232214
returned: 7 - Failed to connect to 81.16.10.141 port 59352: Connection refused time:0.230316
returned: 7 - Failed to connect to 83.13.63.38 port 60860: Connection refused time:0.229997
returned: 7 - Failed to connect to 51.254.182.52 port 61133: Connection refused time:0.229354
Perform 400 parallel transfers
Perform 400 parallel transfers
returned: 7 - Failed to connect to 89.189.189.115 port 59249: Connection refused time:0.236263
returned: 7 - Failed to connect to 78.187.50.200 port 30852: Connection refused time:0.235708
returned: 7 - Failed to connect to 80.249.229.64 port 42186: Connection refused time:0.235154
returned: 7 - Failed to connect to 81.163.62.221 port 31299: Connection refused time:0.235023
returned: 7 - Failed to connect to 46.151.108.6 port 33727: Connection refused time:0.234517
returned: 7 - Failed to connect to 46.164.149.40 port 36551: Connection refused time:0.234454
returned: 7 - Failed to connect to 91.214.240.19 port 50481: Connection refused time:0.231967
returned: 7 - Failed to connect to 91.230.199.174 port 60926: Connection refused time:0.231779
returned: 7 - Failed to connect to 92.246.220.206 port 40396: Connection refused time:0.231244
returned: 7 - Failed to connect to 93.87.66.66 port 61831: Connection refused time:0.231102
returned: 7 - Failed to connect to 93.116.57.4 port 55674: Connection refused time:0.231033
returned: 7 - Failed to connect to 93.177.150.112 port 45186: Connection refused time:0.230920
returned: 7 - Failed to connect to 94.102.124.207 port 35836: Connection refused time:0.230864
returned: 56 - Proxy CONNECT aborted time:1.948208
returned: 7 - Failed to connect to 94.180.245.1 port 61251: Connection refused time:0.230755
returned: 7 - Failed to connect to 94.228.21.66 port 55295: Connection refused time:0.230644
returned: 7 - Failed to connect to 95.66.140.121 port 59078: Connection refused time:0.230359
returned: 7 - Failed to connect to 95.79.54.143 port 80: Connection refused time:0.230302
returned: 7 - Failed to connect to 95.87.127.133 port 50594: Connection refused time:0.230226
returned: 7 - Failed to connect to 95.140.27.135 port 60275: Connection refused time:0.230171
returned: 7 - Failed to connect to 96.9.69.230 port 53281: No route to host time:0.230059
returned: 7 - Failed to connect to 96.9.73.79 port 61057: Connection refused time:0.229997
returned: 56 - Proxy CONNECT aborted time:2.305897
returned: 7 - Failed to connect to 101.109.255.243 port 50098: Connection refused time:0.229888
returned: 7 - Failed to connect to 101.255.124.202 port 42608: Connection refused time:0.229833
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.229720
returned: 7 - Failed to connect to 103.28.243.11 port 37120: Connection refused time:0.229551
returned: 7 - Failed to connect to 103.75.32.132 port 53130: Connection refused time:0.229351
returned: 7 - Failed to connect to 103.75.34.121 port 46210: Connection refused time:0.229252
returned: 7 - Failed to connect to 103.76.170.50 port 60240: Connection refused time:0.229178
returned: 7 - Failed to connect to 103.76.188.85 port 44827: Connection refused time:0.229117
returned: 7 - Failed to connect to 103.76.188.209 port 45190: Connection refused time:0.229060
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.228946
returned: 7 - Failed to connect to 103.94.169.114 port 30372: Connection refused time:0.228600
returned: 7 - Failed to connect to 103.106.148.200 port 53712: Connection refused time:0.228431
returned: 7 - Failed to connect to 103.108.62.14 port 49271: No route to host time:0.228249
returned: 7 - Failed to connect to 103.115.100.246 port 47788: Connection refused time:0.228070
returned: 7 - Failed to connect to 103.192.169.186 port 35865: Connection refused time:0.228016
returned: 7 - Failed to connect to 103.205.27.46 port 52311: Connection refused time:0.227904
returned: 7 - Failed to connect to 103.206.201.179 port 37938: Connection refused time:0.227848
returned: 7 - Failed to connect to 103.221.254.2 port 36778: Connection refused time:0.227726
returned: 7 - Failed to connect to 103.226.227.218 port 43010: Connection refused time:0.227613
returned: 7 - Failed to connect to 103.229.177.51 port 45275: Connection refused time:0.227556
returned: 7 - Failed to connect to 103.237.174.82 port 55326: Connection refused time:0.227497
returned: 56 - Proxy CONNECT aborted time:7.222469
returned: 56 - Proxy CONNECT aborted time:7.222414
returned: 56 - Proxy CONNECT aborted time:7.222391
returned: 56 - Proxy CONNECT aborted time:7.222365
returned: 56 - Proxy CONNECT aborted time:7.222343
returned: 7 - Failed to connect to 110.74.196.9 port 54848: Connection refused time:0.226904
returned: 7 - Failed to connect to 110.74.219.10 port 30906: Connection refused time:0.226848
returned: 7 - Failed to connect to 1.10.186.48 port 38104: Connection refused time:0.226793
returned: 7 - Failed to connect to 1.10.186.157 port 51976: Connection refused time:0.226734
returned: 56 - Proxy CONNECT aborted time:7.222284
returned: 7 - Failed to connect to 1.10.188.12 port 35573: Connection refused time:0.226620
returned: 7 - Failed to connect to 1.10.188.205 port 43580: Connection refused time:0.226563
returned: 7 - Failed to connect to 1.20.97.166 port 32588: Connection refused time:0.226507
returned: 7 - Failed to connect to 1.20.99.185 port 44450: Connection refused time:0.226383
returned: 7 - Failed to connect to 1.20.100.42 port 47962: Connection refused time:0.226327
returned: 7 - Failed to connect to 12.189.124.100 port 44791: Connection refused time:0.225727
returned: 7 - Failed to connect to 18.228.217.208 port 80: Connection refused time:0.225462
returned: 7 - Failed to connect to 24.172.34.114 port 52143: Connection refused time:0.225405
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.225347
returned: 56 - Proxy CONNECT aborted time:7.221507
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.225174
returned: 56 - Proxy CONNECT aborted time:7.221422
returned: 56 - Proxy CONNECT aborted time:7.221395
returned: 35 - error:1408F10B:SSL routines:ssl3_get_record:wrong version number time:7.221262
returned: 7 - Failed to connect to 110.172.135.234 port 53822: Connection refused time:0.224762
returned: 56 - Proxy CONNECT aborted time:7.229473
returned: 7 - Failed to connect to 111.92.243.154 port 37539: Connection refused time:0.224645
returned: 7 - Failed to connect to 113.53.83.157 port 35969: Connection refused time:0.224510
returned: 56 - Proxy CONNECT aborted time:8.432910
returned: 7 - Failed to connect to 115.178.27.125 port 48497: Connection refused time:0.224276
returned: 7 - Failed to connect to 117.196.236.162 port 43989: Connection refused time:0.224151
returned: 7 - Failed to connect to 118.172.201.109 port 54347: Connection refused time:0.224094
returned: 7 - Failed to connect to 118.173.232.5 port 55124: Connection refused time:0.224038
returned: 7 - Failed to connect to 118.173.232.43 port 48204: Connection refused time:0.223979
returned: 7 - Failed to connect to 118.174.234.129 port 54431: Connection refused time:0.223866
returned: 7 - Failed to connect to 118.175.93.28 port 61285: Connection refused time:0.223804
returned: 7 - Failed to connect to 118.175.93.164 port 37694: Connection refused time:0.223742
returned: 7 - Failed to connect to 118.179.69.233 port 44505: Connection refused time:0.223686
returned: 7 - Failed to connect to 118.179.84.106 port 34236: Connection refused time:0.223623
returned: 7 - Failed to connect to 119.82.253.155 port 37436: Connection refused time:0.223498
returned: 56 - Proxy CONNECT aborted time:9.082033
returned: 56 - Proxy CONNECT aborted time:9.082023
returned: 7 - Failed to connect to 122.102.41.82 port 56067: Connection refused time:0.223266
returned: 7 - Failed to connect to 122.176.65.143 port 42175: Connection refused time:0.223210
returned: 7 - Failed to connect to 124.41.211.213 port 54986: Connection refused time:0.223152
returned: 7 - Failed to connect to 124.41.211.231 port 49949: Connection refused time:0.223085
returned: 56 - Proxy CONNECT aborted time:9.081812
returned: 7 - Failed to connect to 125.27.251.208 port 30753: Connection refused time:0.222901
returned: 7 - Failed to connect to 125.209.108.26 port 53126: Connection refused time:0.222834
returned: 56 - Proxy CONNECT aborted time:9.081675
returned: 7 - Failed to connect to 131.108.63.213 port 57604: No route to host time:0.222527
returned: 7 - Failed to connect to 134.0.16.6 port 47631: Connection refused time:0.222416
returned: 56 - Proxy CONNECT aborted time:9.966099
returned: 56 - Proxy CONNECT aborted time:9.966078
returned: 7 - Failed to connect to 139.255.64.10 port 41309: Connection refused time:0.222063
returned: 7 - Failed to connect to 141.101.236.49 port 56000: Connection refused time:0.221989
returned: 7 - Failed to connect to 154.0.15.166 port 49713: Connection refused time:0.221785
returned: 56 - Proxy CONNECT aborted time:10.558608
returned: 56 - Proxy CONNECT aborted time:10.558525
returned: 7 - Failed to connect to 163.53.150.230 port 30195: Connection refused time:0.221473
returned: 7 - Failed to connect to 163.53.182.148 port 39486: Connection refused time:0.221414
returned: 7 - Failed to connect to 165.90.211.69 port 61692: No route to host time:0.221358
returned: 7 - Failed to connect to 166.249.54.244 port 55867: Connection refused time:0.221299
returned: 7 - Failed to connect to 168.181.121.195 port 51514: Connection refused time:0.221175
returned: 56 - Proxy CONNECT aborted time:11.015879
returned: 7 - Failed to connect to 169.255.67.244 port 37804: Connection refused time:0.220992
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.220811
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.220675
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.220618
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.220392
returned: 7 - Failed to connect to 170.210.236.1 port 34792: Connection refused time:0.220336
returned: 7 - Failed to connect to 170.245.173.87 port 4145: Connection refused time:0.220218
returned: 7 - Failed to connect to 175.100.5.52 port 45940: Connection refused time:0.220099
returned: 56 - Proxy CONNECT aborted time:12.537595
returned: 7 - Failed to connect to 176.98.75.120 port 45259: Connection refused time:0.219983
returned: 7 - Failed to connect to 176.98.76.210 port 45280: Connection refused time:0.219925
returned: 7 - Failed to connect to 176.111.10.136 port 46560: Connection refused time:0.219812
returned: 7 - Failed to connect to 176.119.18.92 port 30996: Connection refused time:0.219744
returned: 56 - Proxy CONNECT aborted time:12.537487
returned: 56 - Proxy CONNECT aborted time:12.537472
returned: 7 - Failed to connect to 177.75.143.198 port 43223: Connection refused time:0.219352
returned: 7 - Failed to connect to 177.220.171.54 port 60729: Connection refused time:0.219302
returned: 7 - Failed to connect to 177.223.52.242 port 42363: Connection refused time:0.219252
returned: 56 - Proxy CONNECT aborted time:12.537244
returned: 7 - Failed to connect to 178.75.58.58 port 41514: Connection refused time:0.219146
returned: 56 - Proxy CONNECT aborted time:12.537207
returned: 7 - Failed to connect to 180.247.199.128 port 8080: Connection refused time:0.218729
returned: 56 - Proxy CONNECT aborted time:12.536955
returned: 56 - Proxy CONNECT aborted time:12.536947
returned: 56 - Proxy CONNECT aborted time:12.536771
returned: 7 - Failed to connect to 182.16.163.138 port 8080: Connection refused time:0.218295
returned: 7 - Failed to connect to 182.53.206.40 port 52214: Connection refused time:0.218133
returned: 7 - Failed to connect to 182.75.21.30 port 46527: Connection refused time:0.218071
returned: 7 - Failed to connect to 182.253.201.10 port 36776: Connection refused time:0.218012
returned: 7 - Failed to connect to 185.136.151.20 port 58371: No route to host time:0.217623
returned: 56 - Proxy CONNECT aborted time:12.536069
returned: 56 - Proxy CONNECT aborted time:12.536051
returned: 7 - Failed to connect to 187.111.192.142 port 30485: Connection refused time:0.217268
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.217216
returned: 7 - Failed to connect to 188.130.240.17 port 52343: Connection refused time:0.217168
returned: 56 - Proxy CONNECT aborted time:12.892296
returned: 56 - Proxy CONNECT aborted time:12.892214
returned: 56 - Proxy CONNECT aborted time:13.536803
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.216659
returned: 56 - Proxy CONNECT aborted time:13.913305
returned: 7 - Failed to connect to 190.248.136.18 port 53963: Connection refused time:0.216547
returned: 56 - Proxy CONNECT aborted time:13.913289
returned: 56 - Proxy CONNECT aborted time:13.913266
returned: 56 - Proxy CONNECT aborted time:14.169642
returned: 56 - Proxy CONNECT aborted time:14.169630
returned: 56 - Proxy CONNECT aborted time:14.169613
returned: 7 - Failed to connect to 195.39.71.252 port 56935: No route to host time:0.216184
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.216130
returned: 56 - Proxy CONNECT aborted time:14.169664
returned: 7 - Failed to connect to 195.239.178.110 port 40448: Connection refused time:0.216027
returned: 7 - Failed to connect to 196.32.109.73 port 56930: Connection refused time:0.215625
returned: 7 - Failed to connect to 196.38.8.91 port 8080: Connection refused time:0.215575
returned: 7 - Failed to connect to 197.98.180.162 port 61047: Connection refused time:0.215470
returned: 7 - Failed to connect to 197.232.55.224 port 39385: Connection refused time:0.215422
returned: 7 - Failed to connect to 197.248.224.242 port 80: Connection refused time:0.215319
returned: 7 - Failed to connect to 200.7.205.194 port 51910: No route to host time:0.215141
returned: 7 - Failed to connect to 200.35.49.73 port 45186: Connection refused time:0.215087
returned: 7 - Failed to connect to 200.71.123.185 port 34343: Connection refused time:0.215017
returned: 7 - Failed to connect to 200.195.188.2 port 32954: Connection refused time:0.214956
returned: 7 - Failed to connect to 201.234.253.24 port 59238: Connection refused time:0.214854
returned: 7 - Failed to connect to 202.3.72.6 port 38725: Connection refused time:0.214802
returned: 7 - Failed to connect to 202.49.183.168 port 30309: Connection refused time:0.214751
returned: 7 - Failed to connect to 202.57.55.242 port 52634: Connection refused time:0.214595
returned: 56 - Proxy CONNECT aborted time:28.221193
returned: 7 - Failed to connect to 202.84.79.11 port 41247: Connection refused time:0.214483
returned: 7 - Failed to connect to 202.178.125.99 port 33853: Connection refused time:0.214374
returned: 7 - Failed to connect to 203.17.150.95 port 38498: Connection refused time:0.214319
returned: 7 - Failed to connect to 203.76.147.26 port 45684: Connection refused time:0.214264
returned: 7 - Failed to connect to 203.81.176.254 port 49901: Connection refused time:0.214209
returned: 7 - Failed to connect to 203.142.69.242 port 47619: Connection refused time:0.214093
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.214032
returned: 7 - Failed to connect to 203.174.15.234 port 40604: Connection refused time:0.213981
returned: 7 - Failed to connect to 203.177.133.148 port 44632: Connection refused time:0.213911
returned: 7 - Failed to connect to 206.189.137.227 port 3128: Connection refused time:0.213840
returned: 7 - Failed to connect to 212.28.237.131 port 59588: Connection refused time:0.213544
returned: 56 - Proxy CONNECT aborted time:29.303009
returned: 56 - Proxy CONNECT aborted time:29.303008
returned: 56 - Proxy CONNECT aborted time:29.091600
returned: 7 - Failed to connect to 213.6.199.94 port 41320: Connection refused time:0.001740
returned: 7 - Failed to connect to 213.87.42.53 port 40367: No route to host time:0.000161
returned: 7 - Failed to connect to 217.113.17.226 port 50948: Connection refused time:0.000079
returned: 7 - Failed to connect to 221.120.163.242 port 56969: Connection refused time:0.000084
Perform 209 parallel transfers
returned: 7 - Failed to connect to 83.167.203.174 port 44848: Connection refused time:0.239455
returned: 56 - Proxy CONNECT aborted time:29.591965
returned: 7 - Failed to connect to 86.38.39.21 port 54703: Connection refused time:0.238637
returned: 56 - Proxy CONNECT aborted time:29.591667
returned: 56 - Proxy CONNECT aborted time:29.591681
returned: 56 - Proxy CONNECT aborted time:29.591659
returned: 56 - Proxy CONNECT aborted time:29.591481
returned: 56 - Proxy CONNECT aborted time:29.903486
returned: 56 - Proxy CONNECT aborted time:29.903468
returned: 7 - Failed to connect to 80.55.32.238 port 59513: Connection refused time:0.237515
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.237451
returned: 28 - Connection time-out time:0.237390
returned: 28 - Connection time-out time:0.237329
returned: 28 - Connection time-out time:0.237269
returned: 28 - Connection time-out time:0.237066
returned: 28 - Connection time-out time:0.237003
returned: 28 - Proxy CONNECT aborted due to timeout time:0.236941
returned: 28 - Connection time-out time:0.236817
returned: 28 - Connection time-out time:0.236757
returned: 28 - Proxy CONNECT aborted due to timeout time:0.236693
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.236484
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.236419
returned: 28 - Connection time-out time:0.236360
returned: 28 - Proxy CONNECT aborted due to timeout time:0.236297
returned: 28 - Connection time-out time:0.236159
returned: 28 - Connection time-out time:0.236097
returned: 28 - Proxy CONNECT aborted due to timeout time:0.236039
returned: 28 - Connection time-out time:0.235977
returned: 28 - Connection time-out time:0.235917
returned: 28 - Connection time-out time:0.235843
returned: 28 - Connection time-out time:0.235784
returned: 28 - Connection time-out time:0.235722
returned: 28 - Connection time-out time:0.235661
returned: 28 - Connection time-out time:0.235599
returned: 28 - Connection time-out time:0.235532
returned: 28 - Connection time-out time:0.235471
returned: 28 - Connection time-out time:0.235412
returned: 28 - Connection time-out time:0.235347
returned: 28 - Connection time-out time:0.235275
returned: 28 - Connection time-out time:0.235215
returned: 28 - Connection time-out time:0.235153
returned: 28 - Connection time-out time:0.235090
returned: 28 - Connection time-out time:0.235029
returned: 28 - Connection time-out time:0.234955
returned: 28 - Connection time-out time:0.234889
returned: 28 - Connection time-out time:0.234827
returned: 28 - Connection time-out time:0.234765
returned: 28 - Connection time-out time:0.234697
returned: 28 - Connection time-out time:0.234637
returned: 28 - Connection time-out time:0.234575
returned: 28 - Connection time-out time:0.234513
returned: 28 - Proxy CONNECT aborted due to timeout time:29.999019
returned: 28 - Connection time-out time:0.234391
returned: 7 - Failed to receive SOCKS4 connect request ack. time:0.234322
returned: 28 - Connection time-out time:0.234256
returned: 28 - SSL connection timeout time:30.000666
returned: 28 - Connection time-out time:1.159972
returned: 28 - Connection time-out time:1.159979
returned: 28 - Connection time-out time:1.159916
returned: 28 - Connection time-out time:1.159851
returned: 28 - SSL connection timeout time:30.001952
returned: 28 - Proxy CONNECT aborted due to timeout time:1.698839
returned: 28 - SSL connection timeout time:30.003141
returned: 28 - Proxy CONNECT aborted due to timeout time:1.948405
returned: 28 - Proxy CONNECT aborted due to timeout time:1.948429
returned: 28 - Connection time-out time:1.948289
returned: 28 - Proxy CONNECT aborted due to timeout time:1.948158
returned: 28 - Connection time-out time:1.948076
returned: 28 - Proxy CONNECT aborted due to timeout time:1.948051
returned: 28 - SSL connection timeout time:30.003838
returned: 28 - Connection time-out time:2.306061
returned: 28 - Connection time-out time:2.305970
returned: 28 - Connection time-out time:2.305780
returned: 28 - Connection time-out time:2.305725
returned: 28 - Proxy CONNECT aborted due to timeout time:2.305701
returned: 28 - Connection time-out time:2.305616
returned: 28 - Proxy CONNECT aborted due to timeout time:2.305584
returned: 28 - Connection time-out time:2.305276
returned: 28 - Connection time-out time:2.305219
returned: 28 - SSL connection timeout time:30.003848
returned: 28 - SSL connection timeout time:30.005159
returned: 28 - SSL connection timeout time:30.006580
returned: 28 - SSL connection timeout time:30.007869
returned: 28 - Connection time-out time:4.495685
returned: 28 - Connection time-out time:4.495632
returned: 28 - Proxy CONNECT aborted due to timeout time:4.495638
returned: 28 - Connection time-out time:4.495579
returned: 28 - SSL connection timeout time:30.008879
returned: 28 - Connection time-out time:5.147788
returned: 28 - SSL connection timeout time:30.009989
returned: 28 - SSL connection timeout time:30.011031
returned: 28 - SSL connection timeout time:30.012132
returned: 28 - Connection time-out time:6.684989
returned: 28 - SSL connection timeout time:30.013130
returned: 28 - Proxy CONNECT aborted due to timeout time:7.222439
returned: 28 - Proxy CONNECT aborted due to timeout time:7.222315
returned: 28 - Proxy CONNECT aborted due to timeout time:7.222155
returned: 28 - Connection time-out time:7.222039
returned: 28 - Proxy CONNECT aborted due to timeout time:7.222012
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221986
returned: 28 - Connection time-out time:7.221932
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221909
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221880
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221822
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221776
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221684
returned: 28 - Connection time-out time:7.221619
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221597
returned: 28 - Connection time-out time:7.221452
returned: 28 - Connection time-out time:7.221338
returned: 28 - Proxy CONNECT aborted due to timeout time:7.221312
returned: 28 - Connection time-out time:7.221257
returned: 28 - Connection time-out time:7.229398
returned: 28 - SSL connection timeout time:30.011509
returned: 28 - SSL connection timeout time:30.012622
returned: 28 - Connection time-out time:8.432901
returned: 28 - SSL connection timeout time:30.013196
returned: 28 - Proxy CONNECT aborted due to timeout time:9.082047
returned: 28 - Proxy CONNECT aborted due to timeout time:9.082003
returned: 28 - Proxy CONNECT aborted due to timeout time:9.081835
returned: 28 - Proxy CONNECT aborted due to timeout time:9.081700
returned: 28 - SSL connection timeout time:30.012905
returned: 28 - Proxy CONNECT aborted due to timeout time:9.545297
returned: 28 - Proxy CONNECT aborted due to timeout time:9.545337
returned: 28 - SSL connection timeout time:30.013569
returned: 28 - Connection time-out time:9.966022
returned: 28 - SSL connection timeout time:30.014432
returned: 28 - Connection time-out time:10.216905
returned: 28 - SSL connection timeout time:30.015214
returned: 28 - Proxy CONNECT aborted due to timeout time:10.558574
returned: 28 - Proxy CONNECT aborted due to timeout time:10.558539
returned: 28 - Connection time-out time:10.558326
returned: 28 - SSL connection timeout time:30.015570
returned: 28 - SSL connection timeout time:30.016443
returned: 28 - SSL connection timeout time:30.017401
returned: 28 - SSL connection timeout time:30.018377
returned: 28 - Connection time-out time:12.095881
returned: 28 - SSL connection timeout time:30.018953
returned: 28 - Connection time-out time:12.537638
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537517
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537509
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537442
returned: 28 - Connection time-out time:12.537385
returned: 28 - Connection time-out time:12.537331
returned: 28 - Connection time-out time:12.537160
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537146
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537152
returned: 28 - Connection time-out time:12.537106
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537093
returned: 28 - Proxy CONNECT aborted due to timeout time:12.537065
returned: 28 - Connection time-out time:12.536970
returned: 28 - Connection time-out time:12.536899
returned: 28 - Connection time-out time:12.536843
returned: 28 - Connection time-out time:12.536789
returned: 28 - Connection time-out time:12.536697
returned: 28 - Connection time-out time:12.536641
returned: 28 - Connection time-out time:12.536479
returned: 28 - Connection time-out time:12.536427
returned: 28 - Connection time-out time:12.536360
returned: 28 - Connection time-out time:12.536302
returned: 28 - Proxy CONNECT aborted due to timeout time:12.536283
returned: 28 - Proxy CONNECT aborted due to timeout time:12.536258
returned: 28 - Connection time-out time:12.536184
returned: 28 - Connection time-out time:12.536132
returned: 28 - Connection time-out time:12.536082
returned: 28 - Connection time-out time:12.536003
returned: 28 - SSL connection timeout time:30.016987
returned: 28 - Connection time-out time:12.892240
returned: 28 - Connection time-out time:12.892142
returned: 28 - Connection time-out time:12.892088
returned: 28 - SSL connection timeout time:30.017749
returned: 28 - SSL connection timeout time:30.018299
returned: 28 - Connection time-out time:14.169583
returned: 28 - Connection time-out time:14.169513
returned: 28 - Connection time-out time:14.169459
returned: 28 - Connection time-out time:14.169408
returned: 28 - Connection time-out time:14.169349
returned: 28 - Connection time-out time:14.169295
returned: 28 - Connection time-out time:14.169187
returned: 28 - Connection time-out time:14.169068
returned: 28 - Connection time-out time:14.168988
returned: 28 - SSL connection timeout time:30.018212
returned: 28 - Proxy CONNECT aborted due to timeout time:14.659355
returned: 28 - SSL connection timeout time:30.018616
returned: 28 - Connection time-out time:28.221143
returned: 28 - SSL connection timeout time:30.019418
returned: 28 - Proxy CONNECT aborted due to timeout time:29.303328
returned: 28 - Connection time-out time:29.303150
returned: 28 - Connection time-out time:29.303095
returned: 28 - Proxy CONNECT aborted due to timeout time:29.303077
returned: 28 - Connection time-out time:29.303017
returned: 28 - Proxy CONNECT aborted due to timeout time:29.302996
Perform 17 parallel transfers
returned: 28 - Connection timed out after 30070 milliseconds time:30.070682
returned: 28 - Connection timed out after 30070 milliseconds time:30.070515
returned: 28 - Connection timed out after 30070 milliseconds time:30.070414
returned: 28 - Operation timed out after 30070 milliseconds with 0 out of 0 bytes received time:30.070117
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.070104
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.070083
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.069924
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.071586
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.071599
returned: 28 - Connection timed out after 30069 milliseconds time:30.071570
returned: 28 - Connection timed out after 30069 milliseconds time:30.071539
returned: 28 - Operation timed out after 30069 milliseconds with 0 out of 0 bytes received time:30.071400
returned: 28 - Operation timed out after 30068 milliseconds with 0 out of 0 bytes received time:30.071174
returned: 28 - Operation timed out after 30067 milliseconds with 0 out of 0 bytes received time:30.071614
returned: 28 - Operation timed out after 30065 milliseconds with 0 out of 0 bytes received time:30.069246
Perform 2 parallel transfers
returned: 28 - Connection timed out after 30000 milliseconds time:30.000108
Perform 1 parallel transfers
returned: 28 - Operation timed out after 30000 milliseconds with 0 out of 0 bytes received time:29.999563
[root@localhost Debug]# 

@bagder
Copy link
Member

bagder commented Feb 24, 2019

Thanks, I'll dig deeper.

@bagder
Copy link
Member

bagder commented Feb 24, 2019

I updated my example code. It now measures the elapsed time by itself as well and then outputs both the time libcurl tells for TOTAL_TIME as well as the private timer. It shows that the time reported by libcurl seems to be grossly wrong in these cases and the actual time that it actually took is pretty much what we set the timeout limit to. Example output from my tests would look like this:

returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39406 us (real: 3036371)
returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39380 us (real: 3036374)
returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39345 us (real: 3036376)
returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39311 us (real: 3036377)
returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39284 us (real: 3036380)
returned: 28 - Connection time-out: -16 Curl_is_connected time:0.039 / 39257 us (real: 3036383)

Here libcurl says it took 39000 microseconds for each of these transfers to fail, but the application itself measured them to 3036000 microseconds. 39 vs 3036 milliseconds.

I haven't yet figured out why this happens...

@bagder
Copy link
Member

bagder commented Feb 24, 2019

Ok, I figured it out. The upside is that this is only CURLINFO_TOTAL_TIME reporting the wrong time, it isn't actually returning prematurely. PR coming right up.

bagder added a commit that referenced this issue Feb 24, 2019
Failing to do so would make the CURLINFO_TOTAL_TIME timeout to not get
updated correctly and could end up getting reported to the users to be
completely wrong (way too small).

Reported-by: accountantM on github
Fixes #3602
@bagder bagder closed this as completed in aa7b813 Feb 24, 2019
@accountantM
Copy link
Author

accountantM commented Feb 26, 2019

I'm sorry I was too late to respond to you, as I was doing some tests and making some reports for you.

First of all thank you very much Daniel for fixing this issue in this remarkable short time.

The Problem here - and it is my fault not to describe it better with a focus on the main issue - is that it is not the small total_time problem that made me report this to you, it is why these requests are timing out? they shouldn't!

I don't know much about cURL or C language but in the past 14 days I have used cURL multi interface for testing public proxies hundreds of thousands of times as part of my current project (a PHP application for gathering and testing the free/public proxies on the internet), and I'm telling you there is something unusual happening here in the multi feature and timing out that I don't understand.

these 400 proxies in the php and C code should have about 40-80 working proxies(and by working I mean I can connect to them in less than 30 seconds, they don't refuse or timeout) you can prove this by testing each one of them alone without the multi interface. but if I tested them with curl multi interface, SOMETIMES , I get toooooooooo many timeout errors 28 by curl, almost 3-7 proxies out of the 407 connects without errors.

HOW TO REPRODUCE:

1- get a list of free public proxies (like the ones in the C code or from websites like free-proxy-list.net) around 400 proxies

2- test each of them alone(without the multi interface) so you can estimate how many proxies of them is actually working (cURL errno 0) and how many timesout (errno 28) (I expect about 40 out of 400 will connect with no errors) (CONNECT_TIMEOUT = 30)

3- Now test them using the multi_interface and check how many timeout error (errno 28) happens and how many works (errno 0) (CONNECT_TIMEOUT = 30)

4-repeat number 3 until you suddenly find almost all of them timesout!! (here is the problem, why they are timing out sometimes ?)

I run an automatic test by a cron job every 2 minutes for the past ~24 hours by the a C code and PHP code from my laptop and a production server and saved the results in this file, you can see that some tests reproduces the problem (too many 28 errors almost all the proxies, while the normal results should be about 80 proxies are working without errors)

I'm sorry I couldn't focus on the main issue here as the total_time problem got all the attention

This is my first time I use github and I don't know if this should be reported as new issue or not and if you will get notified about this post or not as it is closed

@bagder
Copy link
Member

bagder commented Feb 26, 2019

You're just randomly trying remote sites if they work they way you want them to, and you do that at different times and you draw conclusions and seem to think that their behavior should be predictable and remain stable. I don't think you can do that since you have no idea about their logic, load, DOS-preventions or other decisions they make or do that changes connection times and success rate over time.

I've found nothing in curl that indicates that there's a problem in its connection logic or the timeout handling of those connect attempts. I believe you're seeing and experience genuine connection timeouts because your local TCP stack's connection attempt is not completed within the given time restrictions.

I'm not suggesting that there can't be any errors in curl in this area, but the method you're using is next to useless in proving that there is. If you can reproduce such timeouts against a server we know for sure is up and accepting and for which we can inspect the traffic and logs, then I'm certainly willing and very keen in digging further on this topic. But as long as you're connecting to random sites on the Internet that you have no idea about their inner workings, this is a very weak case.

@bagder
Copy link
Member

bagder commented Feb 26, 2019

Of course, you can also monitor your TCP connections with a tool like Wireshark to see what's actually happening on the TCP level. Do they actually ever connect without curl noticing within the timeout period? If so, that's a curl bug. If not, then curl is right. Doing that on a massive amount of parallel connections will of course be tricky and hard.

@accountantM
Copy link
Author

accountantM commented Feb 26, 2019

If you can reproduce such timeouts against a server we know for sure is up and accepting and for which we can inspect the traffic and logs

I don't have such amount of proxies that is under my control, but we know there are about 70~90 are working out of the same 400 proxies because if I tried each of them alone without the curl_multi I get the same rate, and curl_multi itself most of the times make the same rate, however SOMETIMES curl_multi timeout ALMOST ALL OF THEM (they are different servers from different networks in different countries, it can't be something wrong, like protection or load suddenly happens to all of them)

Just take a deeper look on column 5(number of connections without errors) and column 7(number of connections with timeout error 28) out of the 407 proxies, and you will notice that there are about 80 proxies are working but sometimes curl_multi says almost all of them timeout!. For example here is the tests from 93 to 100


c(1)	c(2)			c(3)	c(4)	c(5)	c(6)	c(7)	c(8)	c(9)	c(10)				c(11)

93	2019-02-24 21:50:02	0	30.86	99	154	87	2	65	PHP 7.2.14(Linux Fedora)	7.59.0
94	2019-02-24 21:52:02	0	31.68	97	156	90	2	62	PHP 7.2.14(Linux Fedora)	7.59.0
95	2019-02-24 21:54:07	0	30.19	4	156	229	0	18	PHP 7.2.14(Linux Fedora)	7.59.0
96	2019-02-24 21:56:03	0	30.05	5	157	229	0	16	PHP 7.2.14(Linux Fedora)	7.59.0
97	2019-02-24 21:58:01	0	31.49	84	147	133	2	41	PHP 7.0.33 (Linux CenoOS)	7.64.0
98	2019-02-24 21:58:02	0	31.17	97	152	90	2	66	PHP 7.2.14(Linux Fedora)	7.59.0
99	2019-02-24 22:00:02	0	30.76	86	146	137	2	36	PHP 7.0.33 (Linux CenoOS)	7.64.0
100	2019-02-24 22:00:21	0	31.14	99	155	89	2	62	PHP 7.2.14(Linux Fedora)	7.59.0

Is this normal ? curl_multi most of the times report 97/90 (noeror/error28) rate(the same rate I always get if I tried each of them alone/synchronously without curl_multi) and sometimes curl_multi report(4/229)!

I understand this is very hard to debug since we don't control the proxies servers, I'm just reporting to you what I found. I'm also going to write a multi threading code that uses curl for 1 request per thread without the curl_multi interface and do the same tests to be sure about the works/timeouts rate if I didn't use curl_multi

@accountantM
Copy link
Author

accountantM commented Feb 26, 2019

If you can reproduce such timeouts against a server we know for sure is up and accepting and for which we can inspect the traffic and logs

the timeouts are happening to the connections to the proxies (CURLOPT_CONNECTTIMEOUT), not the entire operation timeout (CURLOPT_TIMEOUT) including the proxies connections to the final destination server (www.example.com in this case)

@accountantM
Copy link
Author

If you can reproduce such timeouts against a server we know for sure is up and accepting and for which we can inspect the traffic and logs

I can do this easily on my production server if you want me to, I can prepare it to be the final destination server as a replacement to www.example.com that all the proxies will connect to, and log every request came from these proxies the way you want (Actually this is the heart of my public proxies testing application)

For example I can replace example.com with mydomain/proxiesConnectHere and prepare it in less than one hour to log every request came to it from the proxies, if you want this tell me

@accountantM
Copy link
Author

... want them to, and you do that at different times ....

I done the tests from 3 different sources (the C code on my laptop, the PHP code on my laptop, and a PHP code on my server, my laptop is from a network in Egypt and the server is from a network in the US).

Notice the time of the tests in the file and you will see that at the same time the C code reproduced the problem, the PHP code works fine, or the other way, or the laptop reproduce and in the same time the server works fine, this is an indicator that the problem is in my program not the proxies.

@bagder
Copy link
Member

bagder commented Feb 26, 2019

Again, I can't rule out that there is a problem somewhere in curl. But without reliable, reproducible behavior I don't know how to determine when a timeout is "real" or not by debugging curl.

Do the TCP handshakes complete within the timeout period or not. Is it curl that doesn't notice the connection or is the connection not happening (fast enough). Those are the questions we want answered. That would probably require that you analyze the network traffic.

Continuing to post times or aggregated data won't help us to get those questions answered. They're just observations that might show a curl issue, but they might also just be problems elsewhere.

@bagder
Copy link
Member

bagder commented Feb 26, 2019

Another approach, which could be better but probably also hard to pull off, would be to somehow setup proxies with fixed and controlled behavior that perhaps would have various degrees of slowness in the TCP handshakes, and then we could run curl against such a setup and see how it behaves when doing this using many parallel connections down to only one.

@accountantM
Copy link
Author

Do the TCP handshakes complete within the timeout period or not. Is it curl that doesn't notice the connection or is the connection not happening (fast enough). Those are the questions we want answered. That would probably require that you analyze the network traffic.

I'm sorry Daniel, I'm not such a deep networks developer, I'm a PHP/MySQL developer with only 4 years of experience in developing ERP systems (my area of experience is in sales, purchases, inventory, accounting and stuff like that) I merely can open wireshark and watch the traffic at such low level for fun(I don't understand 95% of the things I see in wireshark) I even took about 8 hours just to remember how to compile a simple C file.
What I have provided here is a try to provide something that hopefully could be helpful to shed some light on what I have noticed while using the curl_multi with these proxies.

Anyway, I guess this is the end of this issue for me, thank you very much for your interest and for fixing the total_time issue.

@accountantM
Copy link
Author

accountantM commented Mar 29, 2019

As you know I can't do more regarding this issue, but I wrote a C program that uses curl with threading and tested the same proxies, and the threads program never shows the timeout issue. I asked for more help from the community and I hope this issue gets your attention 🙏.

@bagder
Copy link
Member

bagder commented Mar 29, 2019

As you know I can't do more regarding this issue

Apparently you could...

@accountantM
Copy link
Author

accountantM commented Mar 29, 2019

Apparently you could...

I'm sorry I don't clearly understand what you mean, do you mean that this C threads program can help in the issue ?

@JL2210
Copy link

JL2210 commented Mar 30, 2019

@accountantM It may not be my place to make a comment here, but I think that maybe you shouldn't hard press this issue. One aspect of it has been fixed, and the other is only reproducible very rarely. Possibly try changing your network connection and try again, but I certainly doubt this is work the 100 bounty you've given it on Stack Overflow. Possibly just close the Stack Overflow question and wait every so often before you go knocking on the doors of ~400 proxy sites?
One more thing: Check your bandwidth. I know a few issues that I've had with curl simply go away at times when there is low bandwidth in my area.

Hope you can figure out your issue 👍.

@accountantM
Copy link
Author

accountantM commented Mar 30, 2019

and the other is only reproducible very rarely.

WOW, this is very enough for me to set 500 bounty on the problem not only 100, I can't rely on curl_multi since it has a problem that shows up rarely!, for me I will just continue my project using curl on threads, and leave this problem for the time, may be one day Daniel give it another deeper look.

but I think that maybe you shouldn't hard press this issue

No, I should, curl is great and that is why I spent too much time and effort on this problem, because I want to see curl perfect and I'm very sure there is something wrong in curl_multi and proxies and this area.

Hope you can figure out your issue +1.

thanks :)

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Doesn't Curl have a threaded resolver? Is that what you mean by "curl on threads"?
Why do you need to test these proxies, anyway? Just find one that works when the problem doesn't show itself, and use that. Read the SO question more carefully. Why not test every one individually if that works?
If Curl's multi-threaded resolver works for you, then just use that.
I'll try and reproduce and get back to you. What network connection are you on?

@accountantM
Copy link
Author

Doesn't Curl have a threaded resolver?

Ummm, I don't know!

Is that what you mean by "curl on threads"?

No, what I meant by "curl on threads" is the C program that I wrote that uses curl on threads for testing the same set of proxies 407.

the curl_multi program sometimes shows the issue
the curl on threads program never shows the issue

The 2 programs are here, and the tests results are here

Why do you need to test these proxies, anyway? Just find one that works when the problem doesn't show itself, and use that.

I'm working on a project which I want to gather the public/free proxies from the internet, save them in my database, test them , rank them (you can find plenty of them if you googled free-proxy-list )

Why not test every one individually if that works?

because testing and using thousands of public proxies synchronously will take ages, as you know they are very slow, most of them will take 60 seconds tell it responds or timeout, and also because after this project I'm going to make a crawler that will use these proxies, and for the same reason a crawler MUST do it's requests in parallel to save time.

If Curl's multi-threaded resolver works for you, then just use that.

Yes, I will definitely do that since curl_multi interface has this issue

I'll try and reproduce and get back to you.

Thank you very much for that ❤️

What network connection are you on?

I tested from my production server from a network in the U.S and from my laptop from a network in Egypt. The production server offcurse has more resources (CPU/network) than my development laptop, however curl_multi still shows the problem sometimes on both machines

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Enable the threaded resolver by passing --enable-threaded-resolver to Curl's configure script.
I have three instances of that program running in the background with all their output redirected to logs, I'll get back to you on how it goes.

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Wow! All three consistently behave the same way! Here's the three logs:
log3.txt
log2.txt
log1.txt
I'm on a mobile hotspot in Indiana, if that helps.

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Strace output about where all the 28 errors are:

rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
close(405)                              = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
close(406)                              = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
write(2, "returned: 28 - Connection timed "..., 61returned: 28 - Connection timed out after 31088 milliseconds
) = 61
rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
write(2, "returned: 28 - Connection timed "..., 61returned: 28 - Connection timed out after 31088 milliseconds
) = 61
rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7e6770eab000}, NULL, 8) = 0
write(2, "returned: 28 - Operation timed o"..., 91returned: 28 - Operation timed out after 31084 milliseconds with 0 out of 0 bytes received
) = 91

@bagder I'm getting this behavior consistently, and I can provide more debug info. If you don't mind, could you reopen this?

I'm building Wireshark and latest Curl right now, I'll give you the output as soon as possible.

@accountantM
Copy link
Author

I see you are getting 0 successful connections and about ~270 timeouts, If you tried the threads program you will see the normal behaviour "about 50 proxies will work and about only 200 timeouts". Thank you veeeeeeeery much for reproducing this

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Curl version 7.64.1 configured with

./configure --prefix=/usr --host=x86_64-cros-linux-musl --enable-shared --disable-static --with-libidn2 --with-ca-bindle=/etc/ssl/certs/ca-certificates.crt --with-ca-path=/etc/ssl/certs --with-zlib --enable-cookies --enable-unix-sockets --with-ssl=/etc/ssl --enable-crypto-auth --enable-pthreads --enable-threaded-resolver --enable-openssl-auto-load-config --enable-ipv6 --enable-libcurl-option --enable-{http,ftp,file,proxy,telnet,manual}

Trying to build right now.

@JL2210
Copy link

JL2210 commented Mar 30, 2019

I got hung up on building Wireshark. Is there any alternative that doesn't require Qt?

@JL2210
Copy link

JL2210 commented Mar 30, 2019

Latest Curl binary exhibits the same problem.

@accountantM
Copy link
Author

I downloaded Wireshark and used it to capture the traffic while each of the 2 C programs was running, I also filtered the traffic to the proxies list used by the 2 C programs, and saved the files on GitHub.

the curl-threads program (the expected behavior)

63 successful connections and 158 connections timeout error 28 out of 407 proxies.

the curl_multi program (the unexpected behavior)

0 successful connections and 272 connections timeout error 28 out of 407 proxies.

(You can open the .pcapng files using wireshark and see the recorded traffic on my computer while both expected/unexpected behavior (I filtered the traffic to the 407 proxies IPs and left Wireshark open for a little while after the 30 seconds of curl limit because I saw some packets still showing up, I don't know Wireshark and this level of networking, but i thought this could be useful))

@bagder I noticed in Wireshark that some proxy servers do reply with some packets, however curl_muti reported 0 successful connections, I don't know if this is normal replying package or not.

Hey Daniel, I'm sorry if I looked rude, It is not what I meant !

@accountantM
Copy link
Author

I don't know if this info is important or not, check the conversation with the proxy 92.244.36.66 in the normal behaviour (63 successful connections) , and the up normal behaviour (0 successful connections ) , I copied them from the wireshark files I attached.

(Please note the second column, which is the time in seconds since the beginning of capture)

This is a normal conversation(I think) done by the threads program


     84 0.034029043    192.168.1.4           92.244.36.66          TCP      74     54586 → 60789 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1257494912 TSecr=0 WS=128
    428 0.270022357    92.244.36.66          192.168.1.4           TCP      74     60789 → 54586 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1412 SACK_PERM=1 TSval=1267582832 TSecr=1257494912 WS=128
    429 0.270058784    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=1257495149 TSecr=1267582832
    622 0.675855971    192.168.1.4           92.244.36.66          TLSv1.2  75     Ignored Unknown Record
    702 0.809616671    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [ACK] Seq=1 Ack=10 Win=14592 Len=0 TSval=1267582886 TSecr=1257495554
    749 0.929540399    92.244.36.66          192.168.1.4           TLSv1.2  74     Ignored Unknown Record
    750 0.929597614    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=10 Ack=9 Win=64256 Len=0 TSval=1257495808 TSecr=1267582898
    756 0.941049080    192.168.1.4           92.244.36.66          TLSv1.2  583    Client Hello
    892 1.106986090    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [ACK] Seq=9 Ack=527 Win=15616 Len=0 TSval=1267582912 TSecr=1257495820
   1043 1.209269928    92.244.36.66          192.168.1.4           TLSv1.2  1466   Server Hello
   1044 1.209324607    92.244.36.66          192.168.1.4           TCP      1466   60789 → 54586 [ACK] Seq=1409 Ack=527 Win=15616 Len=1400 TSval=1267582925 TSecr=1257495820 [TCP segment of a reassembled PDU]
   1045 1.209348995    92.244.36.66          192.168.1.4           TCP      1362   60789 → 54586 [PSH, ACK] Seq=2809 Ack=527 Win=15616 Len=1296 TSval=1267582925 TSecr=1257495820 [TCP segment of a reassembled PDU]
   1046 1.209462350    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=527 Ack=4105 Win=60160 Len=0 TSval=1257496088 TSecr=1267582925
   1179 1.539569877    92.244.36.66          192.168.1.4           TLSv1.2  438    Certificate, Server Key Exchange, Server Hello Done
   1197 1.580210575    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=527 Ack=4477 Win=64128 Len=0 TSval=1257496459 TSecr=1267582958
   1219 1.635410978    192.168.1.4           92.244.36.66          TLSv1.2  192    Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
   1294 1.829758463    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [ACK] Seq=4477 Ack=653 Win=15616 Len=0 TSval=1267582987 TSecr=1257496514
   1329 1.949428398    92.244.36.66          192.168.1.4           TLSv1.2  117    Change Cipher Spec, Encrypted Handshake Message
   1330 1.949451169    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=653 Ack=4528 Win=64128 Len=0 TSval=1257496828 TSecr=1267582999
   1335 1.949840335    192.168.1.4           92.244.36.66          TLSv1.2  149    Application Data
   1403 2.099557713    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [ACK] Seq=4528 Ack=736 Win=15616 Len=0 TSval=1267583014 TSecr=1257496828
   1462 2.228937082    92.244.36.66          192.168.1.4           TLSv1.2  1466   Application Data
   1463 2.228966359    92.244.36.66          192.168.1.4           TLSv1.2  332    Application Data
   1466 2.229014228    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=736 Ack=6194 Win=62464 Len=0 TSval=1257497107 TSecr=1267583026
   1470 2.229285691    192.168.1.4           92.244.36.66          TLSv1.2  97     Encrypted Alert
   1473 2.230883546    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [FIN, ACK] Seq=767 Ack=6194 Win=64128 Len=0 TSval=1257497109 TSecr=1267583026
   1544 2.369578765    92.244.36.66          192.168.1.4           TCP      78     [TCP Dup ACK 1403#1] 60789 → 54586 [ACK] Seq=6194 Ack=736 Win=15616 Len=0 TSval=1267583041 TSecr=1257497107 SLE=767 SRE=768
   1545 2.369636929    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [ACK] Seq=6194 Ack=768 Win=15616 Len=0 TSval=1267583042 TSecr=1257497108
   1548 2.379611340    92.244.36.66          192.168.1.4           TCP      66     60789 → 54586 [FIN, ACK] Seq=6194 Ack=768 Win=15616 Len=0 TSval=1267583042 TSecr=1257497108
   1549 2.379692716    192.168.1.4           92.244.36.66          TCP      66     54586 → 60789 [ACK] Seq=768 Ack=6195 Win=64128 Len=0 TSval=1257497258 TSecr=1267583042

And this is a not normal conversation(I think) done by the curl_multi program

     93 0.006345191    192.168.1.4           92.244.36.66          TCP      74     53722 → 60789 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1256460177 TSecr=0 WS=128
    451 0.327029244    92.244.36.66          192.168.1.4           TCP      74     60789 → 53722 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1412 SACK_PERM=1 TSval=1267479365 TSecr=1256460177 WS=128
    452 0.327059091    192.168.1.4           92.244.36.66          TCP      66     53722 → 60789 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=1256460498 TSecr=1267479365
   1292 4.091443880    192.168.1.4           92.244.36.66          TLSv1.2  75     Ignored Unknown Record
   1299 4.258931691    92.244.36.66          192.168.1.4           TCP      66     60789 → 53722 [ACK] Seq=1 Ack=10 Win=14592 Len=0 TSval=1267479760 TSecr=1256464262
   1305 4.405811238    92.244.36.66          192.168.1.4           TLSv1.2  74     Ignored Unknown Record
   1306 4.405871188    192.168.1.4           92.244.36.66          TCP      66     53722 → 60789 [ACK] Seq=10 Ack=9 Win=64256 Len=0 TSval=1256464576 TSecr=1267479774
   1307 4.419284482    192.168.1.4           92.244.36.66          TLSv1.2  583    Client Hello
   1315 4.555773723    92.244.36.66          192.168.1.4           TCP      66     60789 → 53722 [ACK] Seq=9 Ack=527 Win=15616 Len=0 TSval=1267479789 TSecr=1256464590
   1318 4.716822574    92.244.36.66          192.168.1.4           TLSv1.2  1466   Server Hello
   1319 4.716871684    92.244.36.66          192.168.1.4           TCP      1466   60789 → 53722 [ACK] Seq=1409 Ack=527 Win=15616 Len=1400 TSval=1267479805 TSecr=1256464590 [TCP segment of a reassembled PDU]
   1321 4.726711231    92.244.36.66          192.168.1.4           TCP      1362   60789 → 53722 [PSH, ACK] Seq=2809 Ack=527 Win=15616 Len=1296 TSval=1267479805 TSecr=1256464590 [TCP segment of a reassembled PDU]
   1322 4.757013120    192.168.1.4           92.244.36.66          TCP      66     53722 → 60789 [ACK] Seq=527 Ack=4105 Win=60160 Len=0 TSval=1256464928 TSecr=1267479805
   1338 4.905027969    92.244.36.66          192.168.1.4           TLSv1.2  438    Certificate, Server Key Exchange, Server Hello Done
   1339 4.905076421    192.168.1.4           92.244.36.66          TCP      66     53722 → 60789 [ACK] Seq=527 Ack=4477 Win=59904 Len=0 TSval=1256465076 TSecr=1267479823
   2111 30.055897223   192.168.1.4           92.244.36.66          TCP      66     53722 → 60789 [RST, ACK] Seq=527 Ack=4477 Win=64128 Len=0 TSval=1256490227 TSecr=1267479823


Did you see that? the time of the last packet sent from my computer (30.055897223) while the time of the packet before it is (4.905076421)! Why my computer took all these seconds to send the packet in the curl_multi program while it didn't in the curl threads program ? I don't know if this is normal or not, it might be useful or not.

@lock lock bot locked as resolved and limited conversation to collaborators Jun 29, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Development

Successfully merging a pull request may close this issue.

3 participants