Hi Rich,

> 1. Libcurl is returning message data line-by-line, with two
> callbacks per line - one for the line data and the other for the
> CRLF. This seems like strange behavior. I'd coded as if I were
> getting the data off a TCP connection - might get one byte,
> might get the whole message in one shot, might get anything
> between. So I'm curious as to what the intent is here.

Yes - that's by design.

You could quite genuinely argue that it is strange behaviour and wouldn't it
be better to pass data on block by block as it comes off the socket.

However, there were problems with the existing CRLF.CRLF checking code that
needed fixing and it was generally simpler for Daniel and I to implement
this checking code this way. With this fix, I did question whether I should
buffer the lines back up before passing them onto whatever application was
using libcurl but I quite quickly came to the assumption that most
applications, will probably buffer it up themselves... thus it seemed quite
inefficient for libcurl to buffer up lines of data whilst checking for the
dot and then for an application programmer to do roughly the same once
received from libcurl.

The reason for the two callbacks is that the checking code will send you all
the data up to a CRLF, which it then has to buffer (if my memory serves me
correctly) in case the CRLF.CRLF runs over the end of a packet into the
next. The second callback is when sufficient data has been received by
libcurl and it has realised that the line wasn't a CRLF.CRLF so it then
passes the CRLF onto you.

It should be possible to get libcurl to send the block of data it receives
off the connection if it doesn't contain any part of the CRLF.CRLF less any
part of that if it is received at the end of the packet, but both Daniel and
I produced a few patches in an attempt to fix this, all of which failed the
test harnesses - However, we are open to patches that are able to pass the
data on in less calls it if you fancy the challenge ;-)

> 2. Libcurl is dropping the final CRLF from the data.

It's probably a combination of 1) me being slightly lazy and 2) me
misinterpreted the spec :(

I should have picked that up, but the existing code that had problems,
stripped the final CRLF off so I maintained compatibility with that rather
than questioning what the existing code was doing and fixing it. A fix
should be fairly simple to do so I will have a go at that later ;-)

Regards

Steve

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-02-14