The last PR in the "client writer" series. This clarifies the handling of server responses by folding the code for the complicated protocols into their protocol handlers. This concerns mainly HTTP and its bastard sibling RTSP.

Nomenclature

The terms "read" and "write" are often used without clear context if they refer to the connect or the client/application side of a transfer. This PR uses "read/write" for operations on the client side and "send/receive" for the connection, e.g. server side. If this is considered useful, we can revisit renaming of further methods in another PR.

Protocol Handler Interface

Curl's protocol handler readwrite() method been changed:

-  CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
-                        const char *buf, size_t blen,
-                        size_t *pconsumed, bool *readmore);
+  CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+                         bool is_eos, bool *done);

The name was changed to clarify that this writes reponse data to the client side. The parameter changes are:

• conn removed as it always operates on data->conn
• pconsumed removed as the method needs to handle all data on success
• readmore removed as no longer necessary
• is_eos as indicator that this is the last call for the transfer response (end-of-stream).
• done TRUE on return iff the transfer response is to be treated as finished

This change affects many files only because of updated comments in handlers that provide no implementation. The real change is that the HTTP protocol handlers now provide an implementation.

HTTP/RTSP write_resp()

The HTTP protocol handlers write_resp() implementation will get passed all raw data of a server response for the transfer. The HTTP/1.x formatted status and headers, as well as the undecoded response body. Curl_http_write_resp_hds() is used internally to parse the response headers and pass them on. This method is public as the RTSP protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general content encoding writer stack, just like other encodings. A new flag CLIENTWRITE_EOS was added for the last client write. This allows writers to verify that they are in a valid end state. The chunked decoder will check if it indeed has seen the last chunk.

General transfer.c handling

The general response handling in transfer.c:466 happens in function readwrite_data(). This mainly operates now like:

static CURLcode readwrite_data(data, ...)
{
  do {
    Curl_xfer_recv_resp(data, buf)
    ...
    Curl_xfer_write_resp(data, buf)
    ...
  } while(interested);
  ...
}

All the response data handling is implemented in Curl_xfer_write_resp(). It calls the protocol handler's
write_resp() implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also means that anyone in possession of such data may call Curl_xfer_write_resp(). This was implemented for HTTP/2 in #12468 to demonstrate the effect this has on transfer handling.