Re: Multiple READFUNCTION callbacks not happening
Date: Fri, 20 Jan 2017 11:41:07 +0530
Thanks for looking into this. My responses to some of the points you made:
*> Is the read callback function returning the correct number of bytes it's*
*> supplying? Is it returning too many or too few in total compared to what
the> program sets with CURLOPT_INFILESIZE? It sounds like you've checked
that> already, though*
If for a payload of 23K, we set the CURLOPT_INFILESIZE to 23K, we get only
one callback with size*nmemb = 16K.
There are no subsequent callbacks and hence the server keeps waiting
before timing out.
*> Without seeing code it's hard to say. *
The URLSession code is here:
The easy handle is configured here:
The READFUNCTION is configured here:
Do let me know if you need more clarity on the code.
*> You can look at the various example **programs to see how they do it,
but sounds like you have the basics already.*
Yes, I did look at the post_callback.c example and it works well with large
payloads, with the version of libcurl I'm using.
*> There may be something specific to URLSession involved here. You could
try> installing a debug callback function and seeing if there are any clues
We do have a debug function installed. But I couldn't find anything
substantial in the output it dumps.
Here is the debug output:
I am unsure if a combination of curl options (which I don't know about)
is causing this problem.
Is there a way to dump all the options set on an easy handle? Though the
programmer must know what option he/she's setting, I am wondering if
there's a way to have it as a debug output.
I'll continue to investigate. Any help/guidance from you will be highly
On Thu, Jan 12, 2017 at 2:48 AM, Dan Fandrich <dan_at_coneharvesters.com>
> On Thu, Jan 12, 2017 at 01:34:04AM +0530, Pushkar Kulkarni wrote:
> > I am working with the URLSession implementation in open source
> Swift,which is
> > based on libcurl. I have a question to ask. But a code snippet is not
> > feasible, so kindly bear with me.
> I'm not familiar with URLSession so my answers will be from the
> perspective of
> the native libcurl C binding. URLSession could do something completely
> > An HTTP POST request is made using the CURLOPT_READFUNCTION callback,
> > which the HTTP request body is passed. In my tests, I have seen that the
> > READFUNCTION callback allows writing a maximum of 16K
> > bytes of data to the buffer. So, for request body sizes of more than
> 16K, I’d
> > expect this callback to happen multiple times until the READFUNCTION
> > a 0. Is that right?
> Correct. However, a full POST consists of sending data and receiving the
> response so both CURLOPT_READFUNCTION and CURLOPT_WRITEFUNCTION would be
> to handle both directions. But libcurl will keep calling the read callback
> function until it returns end-of-file or an error occurs.
> > However, for body sizes more than 16K bytes in size I see the callback
> > happening only once. Since the subsequent callbacks don’t happen, the
> > eventually times out. Hence, large requests or file uploads with sizes >
> > simply fail.
> Is the read callback function returning the correct number of bytes it's
> supplying? Is it returning too many or too few in total compared to what
> program sets with CURLOPT_INFILESIZE? It sounds like you've checked that
> already, though.
> > I can confirm that the first (and only) time the callback happens, the
> > READFUNCTION does NOT return 0. I can also confirm that we do set the
> > CURLOPT_INFILESIZE value to the expected size of the body. Though I am
> not sure
> > if the CURLOPT_POSTFIELDSIZE option would make a difference here, I did
> > setting it to the size of the request body but that didn’t help. I have
> > fiddled with a few HTTP headers as well, but I can’t get the callback to
> > for more than once.
> > Am I missing some config option here? Can someone help please?
> Without seeing code it's hard to say. You can look at the various example
> programs to see how they do it, but sounds like you have the basics
> There may be something specific to URLSession involved here. You could try
> installing a debug callback function and seeing if there are any clues in
> >>> Dan
> List admin: https://cool.haxx.se/list/listinfo/curl-library
> Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-01-20