Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Too many calls to the write callback
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Henrik Holst via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 8 Feb 2022 18:42:42 +0100
Hi,
I'm trying to figure out why I get so many calls to my write callback.
This is against a HTTP/1.1 server and with strace I can see that curl
receives the whole reply in one single recv (which is translated to a
recvfrom by libc) but for some reason my write callback is called 9 times
at what is suspiciously close to on each newline but then comes a larger
chunk with many newlines at once.
Looking at the code in libcurl it looks like curl should simply chunk the
data into CURL_MAX_WRITE_SIZE sized chunks so I'm a bit confused on what is
happening here. FYI this is plain HTTP and not HTTPS.
recvfrom(8, "HTTP/1.1 200 OK\r\nServer: Restlet"..., 16384, 0, NULL, NULL)
= 562
write_callback= '<?xml version="1.0" encoding="utf-8"?>' = 38
write_callback= '<content>' = 9
write_callback= '\n<user>millistream</user>' = 25
write_callback= '\n<most-recent-change/>' = 22
write_callback= '\n<channels>' = 11
write_callback= '\n<channel>' = 10
write_callback= '\n<name>product-ALLAGSPRO</name>' = 31
write_callback= '\n<items/>\n</channel>\n</channels>\n</content>' = 43
write_callback= '\n' = 1
The expected behaviour would be to get called once with realsize=190.
As you can see above recvfrom returns with 562 bytes and then I get 9 calls
to write_callback from that, so the chunking is not due to the network
transfer but happens inside curl. Due to the varying sizes this is not due
to hitting some buffer limit (or we would not trigger on 9 after 38) either.
The output here is " write_callback='%.*s' = %lu\n", (int)realsize, ptr,
realsize);"
I have not changed any of the buffer sizes and this is with libcurl 7.68
/HH
Date: Tue, 8 Feb 2022 18:42:42 +0100
Hi,
I'm trying to figure out why I get so many calls to my write callback.
This is against a HTTP/1.1 server and with strace I can see that curl
receives the whole reply in one single recv (which is translated to a
recvfrom by libc) but for some reason my write callback is called 9 times
at what is suspiciously close to on each newline but then comes a larger
chunk with many newlines at once.
Looking at the code in libcurl it looks like curl should simply chunk the
data into CURL_MAX_WRITE_SIZE sized chunks so I'm a bit confused on what is
happening here. FYI this is plain HTTP and not HTTPS.
recvfrom(8, "HTTP/1.1 200 OK\r\nServer: Restlet"..., 16384, 0, NULL, NULL)
= 562
write_callback= '<?xml version="1.0" encoding="utf-8"?>' = 38
write_callback= '<content>' = 9
write_callback= '\n<user>millistream</user>' = 25
write_callback= '\n<most-recent-change/>' = 22
write_callback= '\n<channels>' = 11
write_callback= '\n<channel>' = 10
write_callback= '\n<name>product-ALLAGSPRO</name>' = 31
write_callback= '\n<items/>\n</channel>\n</channels>\n</content>' = 43
write_callback= '\n' = 1
The expected behaviour would be to get called once with realsize=190.
As you can see above recvfrom returns with 562 bytes and then I get 9 calls
to write_callback from that, so the chunking is not due to the network
transfer but happens inside curl. Due to the varying sizes this is not due
to hitting some buffer limit (or we would not trigger on 9 after 38) either.
The output here is " write_callback='%.*s' = %lu\n", (int)realsize, ptr,
realsize);"
I have not changed any of the buffer sizes and this is with libcurl 7.68
/HH
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.htmlReceived on 2022-02-08