On Fri, 31 Aug 2018, Syberichs Stefan (ETAS/ERS-CON) wrote:

> - We publish a boundary signature in the header before the payload transmission

You make your own multipart? Sure, that's not a problem.

> - the transmission should end when the boundary signature is found in the
> next body (after the payload, which is binary data)

In libcurl's view, the upload ends when POSTFIELDSIZE number of bytes have
been sent to the server.

> curl_easy_setopt(etaso_DMHttp_Prv_session_ast[idx_u16].curl_pst,
> CURLOPT_POSTFIELDSIZE_LARGE, cfgUser_pst->txContentLen_u32);

That suffix '_u32' made me concerned. CURLOPT_POSTFIELDSIZE_LARGE should be a
curl_off_t which rarely is 32 bit. Of course 'u32' might mean something else.

> CURLOPT_COPYPOSTFIELDS, cfgUser_pst->userData_pchr);

Wait, what's this? You set *both* postfields and a callback? I suppose libcurl
will just pick one of those but since you set both, what do you expect it to
do with them? They're pretty much mutually exclusive. You set a callback *OR*
you pass the data to send.

>>> Do you set Content-Length yourself ?

> Yes, in my understanding of the mechanism: we add the length of the boundary
> signature / header information to the payload -> gives the content-length

Avoid setting Content-Length yourself. libcurl will set it for you to the
value you tell it the size is, which avoids them ending up different. As such
a difference will cause issues.

> 1) with libcurl 7.61-0. I tend to think that this is the way it is supposed
> to work: The bare file size is , with the boundary information and header
> stuff it adds up to 29108 bytes. After the last callback call, where the
> boundary is finally transmitted (34 bytes), it says "We are completely
> uploaded and fine"

This looks like its working fine?

> 2) exactly the same client code, but we link against libcurl 7.35.0

> This time, after the 200/OK packet, there is yet another READFUNCTION
> callback invocation, where there is no data to transmit, so we return 0

> - but: with libcurl 7.35.0 there is an extra invokation of the READFUNCTION
> callback, in which we return 0

Ok, so current libcurl doesn't call that extra callback. I don't think we've
ever promised that extra zero-size callback will happen...

> But our design currently forces us to treat the two cases differently (there
> is a state machine involved, which does or does not (7.61) expect the extra
> READFUNCTION invocation). If we link against 7.61.0 and keep waiting for the
> final 0-byte READFUNCTION, we would wait forever, so we check libcurl
> versions currently as a workaround - but that is certainly not nice...

That sounds like your state machine assumes things libcurl doesn't promise.
Why not simply make "libcurl's transfer completed OK" to be the signal to go
that last final state instead of assuming you'll get the zero size callback?

> - what is the reason for the different behaviour ?

We reserve the right to change internals that isn't part of the ABI or ABI. I
can't recall this decision exactly but presumably we fixed a bug, optimized
something or improved the code otherwise and deemed that callback unnecessary
or just in the way.

> - from when did it possibly change ?

I don't know. I spent some time trying to find out, but I only found much
older changes than 7.35.0 (like commit a07a865d5dff7, which was included in
7.21.5)... If it's important to you, I suggest you bisect.

> - how can we make our client so generic that it works with all libcurl
> versions say between 7.35 and 7.61 ?

Don't assume a final zero-size read callback to happen.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html