Feature idea: wakeup option for curl_multi_wait/poll
Date: Tue, 12 Nov 2019 08:50:29 +0000
We have a use-case where we use a single curl multi with lots of easy connections which are added/removed in a relatively adhoc manner, anytime.
We previously used the multi_socket API with libuv for this, but we'd like to migrate to the curl_multi_poll API.
The problem is that when we for example want to add a new easy connection, we have to wait until curl_multi_poll wakes up.
We could play with setting the timeout low enough so this waiting time is not too long, but not too low to keep CPU usage low.
We found a solution to this, where we create a socketpair (or emulate one on Windows just like Curl_socketpair does), and add one of the ends as an extra fd.
This way we can wake the thread up by writing to the other end.
This works pretty well, but I think this could be a feature of the multi API inside curl, probably useful for others as well.
As the Curl_socketpair function is not public, doing this manually includes copying code as well.
Idea about how this could be added to the API:
1. Two fds should be added to the Curl_multi struct left in an invalid state at first (so there won't be a perf penalty for those not wanting to use this feature)
2. There should be a new CURLMOPT_ENABLE_WAKEUP option that initializes a socketpair and stores the fds when set to 1L. (if some platform doesn't support socketpair at all, it could return an errorcode (for example CURLM_BAD_SOCKET, CURLM_UNKNOWN_OPTION or some new error code just added for this), which could be handled by applications for example by lowering the timeout to make the unfortunately necessary waits shorter)
3. There should be a new function curl_multi_wakeup which writes one byte to the writing end of the socketpair.
4. If the option was set successfully, Curl_multi_wait should add the reading end of the socketpair to its fd list, and after the actual Curl_poll call, if this fd is marked POLLIN, then it should try to read out all data from it and discard it. (This is not so trivial, because it should avoid reading in a long loop if there are lots of wakeup calls meanwhile).
Would you be interested in supporting this feature?
Received on 2019-11-12