Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: curl websockets
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 2 Jul 2021 23:53:24 +0200 (CEST)
On Fri, 2 Jul 2021, Felipe Gasper wrote:
>> Or perhaps with the type and flags just made as a single bitmask.
>
> This will make it impossible to specify a continuation frame rather than a
> data frame. (Context: when a message is fragmented, the messageĒs first
> frame is text/binary, and the latter ones are continuation.)
"CURL_WS_MORE" is my currently suggested flag to say to libcurl that there's
more data coming that belongs to the same data packet. Doesn't that work?
And if doesn't work, can't we just invent flags to make a way that works?
> Also, if WS were to add a 3rd data type -- e.g., via some extension -- then
> it would be possible to specify CURL_WS_TEXT | CURL_WS_FANCY_NEW_EXT_TYPE,
> which means curl would need to detect that.
We would have some saved bits for potential future use for sure, just as we
would have saved types, so a future curl version could get support for it. I
don't think we can imagine a way that can add support for such new features
without an update.
> Alternatively, have separate curl_ws_send_text and curl_ws_send_binary functions?
That could certainly work and would save users sending text from having to set
a "this is text" bit. My personal preference is to rather keep it to one
function with the bits for type, but I'm here to listen in what peeps want and
I will not push my own preferences if I'm in a minority.
>> curl_ws_recv() could just be made to not return until the entire packet can
>> be returned. Or with an option to return a partial one.
>
> I assume you mean something like EAGAIN rather than actually blocking?
Yes, sorry I didn't even think about that. I'm positive users will
immediately come up with use cases where they want 22 parallel websocket
connections going on so they cannot block. Or perhaps we can offer some kind
of blocking abilitites, but non-blocking is a must-support feature.
Another thing that struck me is that we probably want to offer a decent way to
do non-blocking ws-connects as well for the 22-connections user who won't
accept that the application will block once for each setup...
> Having the caller call curl_ws_recv() directly would mean that that same
> caller would need to know when the socket has data to read. I donĒt know if
> thatĒs always a safe assumption.
Yeah, that's one of my outstanding questions. How do we do that?
For CONNECT_ONLY, libcurl allows the application to extract the socket for the
transfer so that it can select() on it, but I have a feeling that's a little
too raw here. We might need to provide a function for this purpose.
curl_ws_poll() maybe. It needs some further thinking.
> Also, WS includes ping/pong, so even if the socket has data there might not
> be any application-level data incoming. (Ping/pong isnĒt usually exposed to
> the application.)
Yes, that certainly needs to be taken into account and be handled correctly.
>> An area for consideration is how to deal with the buffer (size). I mean, if
>> it wants to wait for a full packet to arrive I suppose it can be large and
>> it might not fit in the user-provided buffer, so the application would need
>> to call it multiple times do drain the data.
>
> Would this argue in favour of a callback, then? Curl could just allocate as
> it knows it needs. The caller, then, could either memcpy or assume ownership
> of the buffer (by, e.g., returning CURL_WS_NOFREE from the callback).
We wouldn't need a callback for that, the recv-function could just as well
return a pointer to an allocated buffer of a given size that has the received
packet if we would like that kind.
That said, providing allocated memory blocks to the application is a rather
quirky (and unusual) way to deliver data and it doesn't sit very well with me.
We could also just return a bit (CURL_WS_FRAG) in the 'recvflags' which means
it was only a fragment and that it needs to call the recv function again to
get the next piece of the data.
>> https://github.com/curl/curl/wiki/WebSockets#api
>
> I feel funny asking this, but is there a comment feature in this wiki
> system? I donĒt see such, but it would seem useful.
Yeah, it would be super useful but there is no such thing. I suppose you can
edit the wiki and insert comments with some markup (color?) that stands out.
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-07-02
Date: Fri, 2 Jul 2021 23:53:24 +0200 (CEST)
On Fri, 2 Jul 2021, Felipe Gasper wrote:
>> Or perhaps with the type and flags just made as a single bitmask.
>
> This will make it impossible to specify a continuation frame rather than a
> data frame. (Context: when a message is fragmented, the messageĒs first
> frame is text/binary, and the latter ones are continuation.)
"CURL_WS_MORE" is my currently suggested flag to say to libcurl that there's
more data coming that belongs to the same data packet. Doesn't that work?
And if doesn't work, can't we just invent flags to make a way that works?
> Also, if WS were to add a 3rd data type -- e.g., via some extension -- then
> it would be possible to specify CURL_WS_TEXT | CURL_WS_FANCY_NEW_EXT_TYPE,
> which means curl would need to detect that.
We would have some saved bits for potential future use for sure, just as we
would have saved types, so a future curl version could get support for it. I
don't think we can imagine a way that can add support for such new features
without an update.
> Alternatively, have separate curl_ws_send_text and curl_ws_send_binary functions?
That could certainly work and would save users sending text from having to set
a "this is text" bit. My personal preference is to rather keep it to one
function with the bits for type, but I'm here to listen in what peeps want and
I will not push my own preferences if I'm in a minority.
>> curl_ws_recv() could just be made to not return until the entire packet can
>> be returned. Or with an option to return a partial one.
>
> I assume you mean something like EAGAIN rather than actually blocking?
Yes, sorry I didn't even think about that. I'm positive users will
immediately come up with use cases where they want 22 parallel websocket
connections going on so they cannot block. Or perhaps we can offer some kind
of blocking abilitites, but non-blocking is a must-support feature.
Another thing that struck me is that we probably want to offer a decent way to
do non-blocking ws-connects as well for the 22-connections user who won't
accept that the application will block once for each setup...
> Having the caller call curl_ws_recv() directly would mean that that same
> caller would need to know when the socket has data to read. I donĒt know if
> thatĒs always a safe assumption.
Yeah, that's one of my outstanding questions. How do we do that?
For CONNECT_ONLY, libcurl allows the application to extract the socket for the
transfer so that it can select() on it, but I have a feeling that's a little
too raw here. We might need to provide a function for this purpose.
curl_ws_poll() maybe. It needs some further thinking.
> Also, WS includes ping/pong, so even if the socket has data there might not
> be any application-level data incoming. (Ping/pong isnĒt usually exposed to
> the application.)
Yes, that certainly needs to be taken into account and be handled correctly.
>> An area for consideration is how to deal with the buffer (size). I mean, if
>> it wants to wait for a full packet to arrive I suppose it can be large and
>> it might not fit in the user-provided buffer, so the application would need
>> to call it multiple times do drain the data.
>
> Would this argue in favour of a callback, then? Curl could just allocate as
> it knows it needs. The caller, then, could either memcpy or assume ownership
> of the buffer (by, e.g., returning CURL_WS_NOFREE from the callback).
We wouldn't need a callback for that, the recv-function could just as well
return a pointer to an allocated buffer of a given size that has the received
packet if we would like that kind.
That said, providing allocated memory blocks to the application is a rather
quirky (and unusual) way to deliver data and it doesn't sit very well with me.
We could also just return a bit (CURL_WS_FRAG) in the 'recvflags' which means
it was only a fragment and that it needs to call the recv function again to
get the next piece of the data.
>> https://github.com/curl/curl/wiki/WebSockets#api
>
> I feel funny asking this, but is there a comment feature in this wiki
> system? I donĒt see such, but it would seem useful.
Yeah, it would be super useful but there is no such thing. I suppose you can
edit the wiki and insert comments with some markup (color?) that stands out.
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-07-02