Re: curl websockets

From: bch via curl-library <curl-library_at_cool.haxx.se>
Date: Fri, 2 Jul 2021 15:14:17 -0700

On Fri, Jul 2, 2021 at 14:55 Daniel Stenberg via curl-library <
curl-library_at_cool.haxx.se> wrote:

> On Fri, 2 Jul 2021, Felipe Gasper wrote:
>
> >> Or perhaps with the type and flags just made as a single bitmask.
> >
> > This will make it impossible to specify a continuation frame rather than
> a
> > data frame. (Context: when a message is fragmented, the message’s first
> > frame is text/binary, and the latter ones are continuation.)
>
> "CURL_WS_MORE" is my currently suggested flag to say to libcurl that
> there's
> more data coming that belongs to the same data packet. Doesn't that work?
>
> And if doesn't work, can't we just invent flags to make a way that works?
>
> > Also, if WS were to add a 3rd data type -- e.g., via some extension --
> then
> > it would be possible to specify CURL_WS_TEXT |
> CURL_WS_FANCY_NEW_EXT_TYPE,
> > which means curl would need to detect that.
>
> We would have some saved bits for potential future use for sure, just as
> we
> would have saved types, so a future curl version could get support for it.
> I
> don't think we can imagine a way that can add support for such new
> features
> without an update.
>
> > Alternatively, have separate curl_ws_send_text and curl_ws_send_binary
> functions?
>
> That could certainly work and would save users sending text from having to
> set
> a "this is text" bit. My personal preference is to rather keep it to one
> function with the bits for type, but I'm here to listen in what peeps want
> and
> I will not push my own preferences if I'm in a minority.
>
> >> curl_ws_recv() could just be made to not return until the entire packet
> can
> >> be returned. Or with an option to return a partial one.
> >
> > I assume you mean something like EAGAIN rather than actually blocking?
>
> Yes, sorry I didn't even think about that. I'm positive users will
> immediately come up with use cases where they want 22 parallel websocket
> connections going on so they cannot block. Or perhaps we can offer some
> kind
> of blocking abilitites, but non-blocking is a must-support feature.
>
> Another thing that struck me is that we probably want to offer a decent
> way to
> do non-blocking ws-connects as well for the 22-connections user who won't
> accept that the application will block once for each setup...
>
> > Having the caller call curl_ws_recv() directly would mean that that same
> > caller would need to know when the socket has data to read. I don’t know
> if
> > that’s always a safe assumption.
>
> Yeah, that's one of my outstanding questions. How do we do that?
>
> For CONNECT_ONLY, libcurl allows the application to extract the socket for
> the
> transfer so that it can select() on it, but I have a feeling that's a
> little
> too raw here. We might need to provide a function for this purpose.
> curl_ws_poll() maybe. It needs some further thinking.


Why is that too raw (honest question)?
When I’ve used libcurl connect-only (to do my TLS negotiation) I treat it
like a socket thereafter. What about websockets specifically makes using
the same existing approach distasteful?

Or alternatively, is the (hypothetical) curl_ws_poll() going to be useful
to my existing CONNECT _ONLY non-ws sockets?

Cheers,

-bch



>
> > Also, WS includes ping/pong, so even if the socket has data there might
> not
> > be any application-level data incoming. (Ping/pong isn’t usually exposed
> to
> > the application.)
>
> Yes, that certainly needs to be taken into account and be handled
> correctly.
>
> >> An area for consideration is how to deal with the buffer (size). I
> mean, if
> >> it wants to wait for a full packet to arrive I suppose it can be large
> and
> >> it might not fit in the user-provided buffer, so the application would
> need
> >> to call it multiple times do drain the data.
> >
> > Would this argue in favour of a callback, then? Curl could just allocate
> as
> > it knows it needs. The caller, then, could either memcpy or assume
> ownership
> > of the buffer (by, e.g., returning CURL_WS_NOFREE from the callback).
>
> We wouldn't need a callback for that, the recv-function could just as well
> return a pointer to an allocated buffer of a given size that has the
> received
> packet if we would like that kind.
>
> That said, providing allocated memory blocks to the application is a
> rather
> quirky (and unusual) way to deliver data and it doesn't sit very well with
> me.
>
> We could also just return a bit (CURL_WS_FRAG) in the 'recvflags' which
> means
> it was only a fragment and that it needs to call the recv function again
> to
> get the next piece of the data.
>
> >> https://github.com/curl/curl/wiki/WebSockets#api
> >
> > I feel funny asking this, but is there a comment feature in this wiki
> > system? I don’t see such, but it would seem useful.
>
> Yeah, it would be super useful but there is no such thing. I suppose you
> can
> edit the wiki and insert comments with some markup (color?) that stands
> out.
>
> --
>
> / daniel.haxx.se
> | Commercial curl support up to 24x7 is available!
> | Private help, bug fixes, support, ports, new features
> | https://www.wolfssl.com/contact/
> -------------------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html


-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Received on 2021-07-03