cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Curl HTTP Get Requests Slow Under MS-Windows?

From: Kev <s7g2vp2_at_yahoo.co.uk>
Date: Mon, 27 Jan 2014 19:32:58 +0000 (GMT)

Hi Marc. Thanks for your email. Unfortunately, it isn't related to the progress bar. I've inserted a return 0; at the top of Curl_pgrsUpdate() so that it does not update the progress bar at all and the download time does not change. I have also ruled out Curl_client_write() having any impact by adding a return CURLE_OK; at the start. I have confirmed that one of our applications which uses libCurl suffers from this problem so it is not related to the command line tool itself. If it wasn't for the fact that IE and wfetch on my Windows 7 VM and one of our Windows Server 2008 VMs downloads the files several times faster than than Curl I would have said it was an issue with our VMs but unfortunately, everything points to an issue with libCurl. So far i've tried the following: 1. Setting SO_RCVBUF to different values 2. Setting SO_RCVTIMEO / SO_SNDTIMEO 3. Setting SIO_SET_COMPATIBILITY_MODE 4. Setting SO_SNDBUF 5. Changing CURL_MAX_WRITE_SIZE 6. Removing various Sleep commands from the libCurl source code 7. Analysing traffic via Wireshark (Wireshark cannot keep up with the data coming in from wfetch so could not compare) 8. Downloading large files from other servers on our internal network to see if it is server related I did notice something strange when I used a tool called ApiMonitor to trace the WinSock API calls. When I downloaded the file in question using wfetch I could see a constant stream of select and recv calls until the download was completed. However, when I downloaded the same file via Curl I noticed there were two select calls for every recv call and that sometimes the API calls paused but the curl progress continued to increment. This could mean that Curl isn't keeping up with the data being received on the wire but could just mean that ApiMonitor can not handle the throughput. Regards, Kevin. On Monday, 27 January 2014, 16:05, "Doughty, Marc" <marc_doughty@brown.edu> wrote: I was wondering if maybe the 'progress bar' changes made recently caused a regression in Windows Performance. I filed a problem that showed that the progress bar was causing the Windows console to burn a CPU a few months ago, and it was patched by lowering the progress bar update frequency. Earlier last week I saw activity relating to more changes to the progress bar code. Could this be related? Does this symptom only appear when the progress bar option is used? Does cURL or the Windows Console Host application spike in CPU usage when you see the slowdowns? - Marc Doughty "If you aren't sure who is the give-way vessel, you are the give-way vessel." On Sat, Jan 25, 2014 at 5:44 PM, Daniel Stenberg <daniel@haxx.se> wrote: On Fri, 24 Jan 2014, Kev wrote: > > >If it hadn't been for the fact that IE and wfetch retrieve the file much faster I would have put the blame on my VM. >> >>Is there anything else you can suggest? >> > Sure, for example the simple thing to increase CURL_MAX_WRITE_SIZE by a factor of 4 or 8 or something. > >Other than so, I'm a bit puzzled and I would perhaps try to monitor (if possible) what the other faster networking tools do to make this work. > >I don't have windows and I don't know windows, I'm not a suitable person to debug this myself, but I certainly would like what weird things that are going on here. One would imagine a modern operating system would just do fast transfers by default... > > >-- > > / daniel.haxx.se >------------------------------------------------------------------- >List admin: http://cool.haxx.se/list/listinfo/curl-users >FAQ:        http://curl.haxx.se/docs/faq.html >Etiquette:  http://curl.haxx.se/mail/etiquette.html > ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-users FAQ:        http://curl.haxx.se/docs/faq.html Etiquette:  http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-01-27