curl / Mailing Lists / curl-users / Single Mail

curl-users

Re: How to intercept curl to extract the raw requests and the raw responses?

From: Ray Satiro <raysatiro_at_yahoo.com>
Date: Fri, 9 Feb 2018 17:01:34 -0500

On 2/9/2018 10:30 AM, Peng Yu wrote:
> In the following example, `curl` should send an HTTP GET request to httpbin.org.
>
> $ curl -g -sS http://httpbin.org/get
> {
> "args": {},
> "headers": {
> "Accept": "*/*",
> "Connection": "close",
> "Host": "httpbin.org",
> "User-Agent": "curl/7.57.0"
> },
> "origin": "165.91.87.88",
> "url": "http://httpbin.org/get"
> }
>
> The request probably starts with the following lines and I'd like to
> see them in the raw request. Is there a way to intercept curl so that
> I can see the raw requests as well as the raw responses?
>
> GET /get HTTP/1.1
> Host:httpbin.org

It depends, it's subject to some breakage. If it's just for your eyes
and you're not feeding it to another program you could parse it out of
verbose or trace-ascii (verbose is better because it doesn't split
headers longer than 64 bytes like trace-ascii does). Sent headers start
with > and received headers start with <

curl -v -fsS google.com 2>&1 1>/dev/null | grep -E "^(<|>|curl: )"

(in windows use NUL instead of /dev/null)

or as a function in bash

headers(){ curl -v -fsS "$1" 2>&1 1>/dev/null | grep -E "^(<|>|curl: )"; }
headers google.com

(the curl: is to show errors, remove that part if you don't want to see
them)

Any library used by curl could output to stderr its own > and < at the
same time which would get caught up, or curl could change it. If you
want parseable defined sent header lines you would have to use libcurl's
CURLOPT_DEBUGFUNCTION [1][2] and handle CURLINFO_HEADER_OUT.

[1]: https://curl.haxx.se/libcurl/c/CURLOPT_DEBUGFUNCTION.html
[2]:
https://github.com/curl/curl/blob/curl-7_58_0/docs/examples/debug.c#L100

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-02-09