Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
WebSocket feature request: is it possible to call write function when full frame is loaded only?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Vitalii B. Avramenko via curl-library <curl-library_at_lists.haxx.se>
Date: Fri, 3 Feb 2023 09:58:41 +0200
Hello,
I used `ixwebsocket` before and now switched to new `libcurl` websocket feature. `libcurl` is feature rich library that contains many things that `ixwebsocket` lacks, but its new websocket interface has an issue.
I use the library to connect to trading website that sends trades as a stream of JSON objects via websocket connection. I use `CURLOPT_WRITEFUNCTION` to read trades indefinitely while the executable is running. When I did the same for `ixwebsocket`, I always received full JSON objects, one per websocket message, so I think it is how the stream works. I used the same callback for `CURLOPT_WRITEFUNCTION` and found that sometimes (rarely) I receive partial JSON objects, for example, the 1st call of my write function gives `{"Hello":`, and the 2nd one gives `"World"}`. I think the reason is that websocket frame is divided between different TCP packets.
Such partial data may be OK for HTTP protocol when we know for sure that we have "request/response" pattern and we can detect the end of data by HTTP protocol itself, for example, with `Content-Length` header. But with websocket generally speaking we don't have any way to know where is end of frame with `CURLOPT_WRITEFUNCTION`. Specifically for JSON we can try to use stream JSON parser to detect the end of object, but even that is not 100% reliable. What if server is buggy and sends something infinite, for example, it sends something like `{"object":` always? In such a way our stream JSON parser will get something like `{"object":{"object":{"object"`, etc... and how to know who is buggy, the parser or the server? So, we need any way to know where is websocket frame border, and because `CURLOPT_WRITEFUNCTION` doesn't give us that information, we need a guarantee that `CURLOPT_WRITEFUNCTION` will call our callback when full frame is downloaded only, or at least we need the option that will allow us to reques
t such behavior (something like `CURLOPT_WEBSOCKET_FULL_FRAMES_ONLY`).
For now we can also ask to receive raw data from websocket, but it will require to implement websocket messages parser in each program, in my opinion such generic functionality is better for library that for application.
Date: Fri, 3 Feb 2023 09:58:41 +0200
Hello,
I used `ixwebsocket` before and now switched to new `libcurl` websocket feature. `libcurl` is feature rich library that contains many things that `ixwebsocket` lacks, but its new websocket interface has an issue.
I use the library to connect to trading website that sends trades as a stream of JSON objects via websocket connection. I use `CURLOPT_WRITEFUNCTION` to read trades indefinitely while the executable is running. When I did the same for `ixwebsocket`, I always received full JSON objects, one per websocket message, so I think it is how the stream works. I used the same callback for `CURLOPT_WRITEFUNCTION` and found that sometimes (rarely) I receive partial JSON objects, for example, the 1st call of my write function gives `{"Hello":`, and the 2nd one gives `"World"}`. I think the reason is that websocket frame is divided between different TCP packets.
Such partial data may be OK for HTTP protocol when we know for sure that we have "request/response" pattern and we can detect the end of data by HTTP protocol itself, for example, with `Content-Length` header. But with websocket generally speaking we don't have any way to know where is end of frame with `CURLOPT_WRITEFUNCTION`. Specifically for JSON we can try to use stream JSON parser to detect the end of object, but even that is not 100% reliable. What if server is buggy and sends something infinite, for example, it sends something like `{"object":` always? In such a way our stream JSON parser will get something like `{"object":{"object":{"object"`, etc... and how to know who is buggy, the parser or the server? So, we need any way to know where is websocket frame border, and because `CURLOPT_WRITEFUNCTION` doesn't give us that information, we need a guarantee that `CURLOPT_WRITEFUNCTION` will call our callback when full frame is downloaded only, or at least we need the option that will allow us to reques
t such behavior (something like `CURLOPT_WEBSOCKET_FULL_FRAMES_ONLY`).
For now we can also ask to receive raw data from websocket, but it will require to implement websocket messages parser in each program, in my opinion such generic functionality is better for library that for application.
-- Best regards, Vitalii B. Avramenko. -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-02-03