Bugs item #1603712, was opened at 2006-11-27 13:28
Message generated for change (Comment added) made by sebone
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1603712&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: compile or build problem
Group: bad behaviour
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Sebastien WILLEMIJNS (sebone)
Assigned to: Daniel Stenberg (bagder)
Summary: curl 7.16-0 now use "limit rate" badly

Initial Comment:
http://80.247.230.136/bug/curl7160-bug-in-limit-rate.bmp

When i use "--limit-rate" in 7-16.0 this function crash/stops software when speed is equal or is more value i choose

========> choose 5K as value and software stops, choose 5000M and software do not stop !!! <========

bug are not been detected in 7.15.5 because the package is not present in curl archive packs ;)

my config XP SP2 100% patched French version
versions are both nossl

----------------------------------------------------------------------

>Comment By: Sebastien WILLEMIJNS (sebone)
Date: 2007-01-03 10:00

Message:
Logged In: YES
user_id=433875
Originator: YES

done...

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-03 09:45

Message:
Logged In: YES
user_id=1680849
Originator: NO

Thank you, 20070103 snapshot works for me.

Sebastien, could you look at it so that the ticked could be closed?

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-01-02 22:36

Message:
Logged In: YES
user_id=1110
Originator: NO

Fix committed just now. I noticed we already had the work-around I
mentioned in src/main.c and I re-used the same approach here.

Let me know if you find any problems with this.

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-02 16:16

Message:
Logged In: YES
user_id=1680849
Originator: NO

Hmm, indeed, I see even stronger claim:

"... select() function _shall_ block for the time specified."

So your solution seems the most reasonable.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-01-02 15:51

Message:
Logged In: YES
user_id=1110
Originator: NO

According to POSIX, select() _should_ wait that timeout if the arguments
are NULL so I think I'll leave the select() for non-windows and have the
windows do Sleep().

usleep() is not universerally available

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-02 13:53

Message:
Logged In: YES
user_id=1680849
Originator: NO

The problem is, select doesn't wait a timeout if it sees that arguments
are NULLS. It returns -1 immediately.
And I just noticed that (unsurprisingly) curl consumes 100% of CPU time if
Sleep is replaced to select.

We may have to use Sleep() on win32, and usleep() on POSIX-systems. Should
autoconf allow us to determine if usleep is available?

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-02 13:40

Message:
Logged In: YES
user_id=1680849
Originator: NO

If I replace code (Sleep with select) as you said, curl appears to still
work.
But just because it returns 0 in this case.

If make the code affected by patch to look as follows:

  timeout.tv_sec = timeout_ms / 1000;
  timeout.tv_usec = (timeout_ms % 1000) * 1000;

#ifdef WIN32
/* On win32, it seems select() returns ENOENT if no descriptor at all is
   given so in that case we just do a Sleep() for the given number of
   milliseconds */
  if((readfd == CURL_SOCKET_BAD) && (writefd == CURL_SOCKET_BAD)) {
    ret = select(0, NULL, NULL, NULL, &timeout);
    //Sleep(timeout_ms);
    return ret;
  }
#endif

... then curl again fetches just the first chunk of the file.

----------------------------------------------------------------------

Comment By: Sebastien WILLEMIJNS (sebone)
Date: 2007-01-02 13:04

Message:
Logged In: YES
user_id=433875
Originator: YES

please ask me when this win32 patch will be added 4 test at home even in
nighty build and his link ;)

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-01-02 12:42

Message:
Logged In: YES
user_id=1110
Originator: NO

Yes it would have to always return 0 after the select(). The benefit with
using select() instead of Sleep() is that I can use it unconditionally for
all systems since it should work like that on most systems.

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-02 12:29

Message:
Logged In: YES
user_id=1680849
Originator: NO

If I replace code (Sleep with select) as you said, curl appears to still
work.
But just because it returns 0 in this case.

If make the code affected by patch to look as follows:

  timeout.tv_sec = timeout_ms / 1000;
  timeout.tv_usec = (timeout_ms % 1000) * 1000;

#ifdef WIN32
/* On win32, it seems select() returns ENOENT if no descriptor at all is
   given so in that case we just do a Sleep() for the given number of
   milliseconds */
  if((readfd == CURL_SOCKET_BAD) && (writefd == CURL_SOCKET_BAD)) {
    ret = select(0, NULL, NULL, NULL, &timeout);
    //Sleep(timeout_ms);
    return ret;
  }
#endif

... then curl again fetches just the first chunk of the file.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-01-02 11:56

Message:
Logged In: YES
user_id=1110
Originator: NO

FreeBSD you say.

Hm, so out of curiosity does windows "do right" if you instead of the
Sleep() call select() with NULL arguments like:

select(0, NULL, NULL, NULL, &timeout);

Move the timeout calculation to before the conditional too. This select
thing should work on all POSIX systems afaik. We use select() on (at least)
Mac OS X so we should make sure it does as good as possible...

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-02 09:42

Message:
Logged In: YES
user_id=1680849
Originator: NO

Yes, it fixes the problem for win32.

The comment is a bit misleading, though: I have tested select() on
FreeBSD, it behaves similarly there (except that errno == EINVAL).
Curl_select works just because it uses poll, not select (yes, poll
correctly timeouts with zero descriptors supplied). I don't currently have
any linux boxes at hand to check select there.

So, if there are other platforms with select but not poll available, curl
remains broken on them. However, I'm not aware of any.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-01-01 22:33

Message:
Logged In: YES
user_id=1110
Originator: NO

Thanks for your info and help.

Does the attached patch below solve the problem then? (It makes
Curl_select() simply Sleep() for the given time if both the read and write
sockets are unset)
File Added: curl-select-win32.patch

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-01 17:36

Message:
Logged In: YES
user_id=1680849
Originator: NO

Some additional notes:
 - I am using Jan 1 2007 snapshot, i.e. with patch applied, and compiling
it with Visual C 6.
 - In my (default) configuration, Curl_select() uses plain select(), which
doesn't want to work (returns error) when all fd sets are FD_ZERO'ed.

----------------------------------------------------------------------

Comment By: vvs (victor_snezhko)
Date: 2007-01-01 16:01

Message:
Logged In: YES
user_id=1680849
Originator: NO

I also experience this problem.
After some debugging, I see that there is a loop in Transfer(), the first
pass of which works fine. On the second pass Curl_select() is called with
fd_read == fd_write == -1.
It returns ENOENT (errno == 2), and the whole transfer stops.

BTW, in this case Transfer() returns CURLE_OK, and error doesn't propagate
to caller...

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2006-12-19 09:01

Message:
Logged In: YES
user_id=1110
Originator: NO

Follow-up from the curl-users mailing list discussion: this clearly breaks
the connection on Windows when the rate limiting logic is used (i.e if you
set the limit to way over your bandwidth there's no problem seen). This
does however seem to work fine on Linux.

Someone with a debugger (or similar) on windows would need to set a few
breakpoints and single-step over a few functions to see what happens and
why. I don't have any such and I'll therefore mark this bug "later", close
it and add it to the KNOWN_BUGS document - since nobody has stepped forward
offering to do what needs to be done.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2006-12-10 12:47

Message:
Logged In: YES
user_id=1110
Originator: NO

re-opened since Sebastien still experience connection closes

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2006-12-06 16:47

Message:
Logged In: NO

please reopen this bug :-(

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2006-12-05 14:42

Message:
Logged In: YES
user_id=1110
Originator: NO

I fixed the limit-rate problem that made it not limiting the rate
properly, but I have not seen nor addressed any crashs/stops.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2006-11-28 08:05

Message:
Logged In: YES
user_id=1110
Originator: NO

(for the sake of keeping info in this report as well) --limit-rate is
indeed broken in 7.16.0, but I have not found any reasons for the aborted
transfers:

http://curl.haxx.se/mail/archive-2006-11/0135.html

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1603712&group_id=976
Received on 2007-01-03