Okay, this above patch definitely does help.

In a new test case I'm running:
Before: 50225479 calls over 8 seconds.
After: 1389 calls over 8 seconds.

So a definite improvement, and CPU is no longer being taxed like mad. However, the amount of calls is still way way above "about once per second".

I notice now also when I don't limit upload/download speed, that it is also being called several hundred times per second. So it seems this patch fixes the issue where rate limiting itself causes an insane amount of callback, which is excellent. But we still need to fix the issue that too much calling is occuring per second as well.

---
** [bugs:#1238] progressfunction called too much**
**Status:** open-confirmed
**Labels:** progresscallback rate-limiting 
**Created:** Mon Jun 03, 2013 08:55 PM UTC by Daniel Stenberg
**Last Updated:** Thu Jun 13, 2013 06:31 PM UTC
**Owner:** Daniel Stenberg
(bug reported by Miguel Angel in http://curl.haxx.se/mail/lib-2013-05/0254.html)
I am using pycurl in a small script to download coursera videos, it's working 
fine but the callback i installed to display a progress bar in console it's 
called 22188558 times in 70 seconds of execution, it hangs 1 core of the machine 
all the time. 
given the documentation states should be called "roughly once per second or 
sooner" this seems too much for me, i am using this code to use it, it's python 
but i think it's pretty readable.
    ::: python 
    curl = pycurl.Curl() 
    curl.setopt(curl.URL, url) 
    if rate_limit is not None: 
        curl.setopt(curl.MAX_RECV_SPEED_LARGE, rate_limit) 
    file_store = open(filename, "wb") 
    curl.setopt(curl.WRITEDATA, file_store) 
    curl.setopt(curl.NOPROGRESS, 0) 
    curl.setopt(curl.PROGRESSFUNCTION, curl_progress) 
    curl.setopt(curl.FOLLOWLOCATION, 1) 
    #cookies 
    curl.setopt(curl.COOKIEJAR, cookies_filename) 
    curl.setopt(curl.COOKIEFILE, cookies_filename) 
    try: 
        curl.perform() 
    except: 
        import traceback 
        my_logger.error(u"Error downloading file: %s" % traceback.format_exc()) 
    #cleaning 
    curl.close() 
    file_store.close() 
no matter that i define curl_progress as: 
def curl_progress(dl_total, dl_now, ul_total, ul_now): 
    return 0 
it's called so much times that hangs 1 cpu core anyway. 
---
Sent from sourceforge.net because you indicated interest in <https://sourceforge.net/p/curl/bugs/1238/>
To unsubscribe from further messages, please visit <https://sourceforge.net/auth/subscriptions/>