curl / Mailing Lists / curl-users / Single Mail

curl-users

Re: How to intercept curl to extract the raw requests and the raw responses?

From: Ray Satiro <raysatiro_at_yahoo.com>
Date: Fri, 9 Feb 2018 17:05:05 -0500

On 2/9/2018 5:01 PM, Ray Satiro wrote:
> On 2/9/2018 10:30 AM, Peng Yu wrote:
>> In the following example, `curl` should send an HTTP GET request to httpbin.org.
>>
>> $ curl -g -sS http://httpbin.org/get
>> {
>> "args": {},
>> "headers": {
>> "Accept": "*/*",
>> "Connection": "close",
>> "Host": "httpbin.org",
>> "User-Agent": "curl/7.57.0"
>> },
>> "origin": "165.91.87.88",
>> "url": "http://httpbin.org/get"
>> }
>>
>> The request probably starts with the following lines and I'd like to
>> see them in the raw request. Is there a way to intercept curl so that
>> I can see the raw requests as well as the raw responses?
>>
>> GET /get HTTP/1.1
>> Host:httpbin.org
>
> It depends, it's subject to some breakage. If it's just for your eyes
> and you're not feeding it to another program you could parse it out of
> verbose or trace-ascii (verbose is better because it doesn't split
> headers longer than 64 bytes like trace-ascii does). Sent headers
> start with > and received headers start with <
>
> curl -v -fsS google.com 2>&1 1>/dev/null | grep -E "^(<|>|curl: )"
>
> (in windows use NUL instead of /dev/null)
>
> or as a function in bash
>
> headers(){ curl -v -fsS "$1" 2>&1 1>/dev/null | grep -E "^(<|>|curl: )"; }
> headers google.com
>
> (the curl: is to show errors, remove that part if you don't want to
> see them)
>
> Any library used by curl could output to stderr its own > and < at the
> same time which would get caught up, or curl could change it. If you
> want parseable defined sent header lines you would have to use
> libcurl's CURLOPT_DEBUGFUNCTION [1][2] and handle CURLINFO_HEADER_OUT.
>
>
> [1]: https://curl.haxx.se/libcurl/c/CURLOPT_DEBUGFUNCTION.html
> [2]:
> https://github.com/curl/curl/blob/curl-7_58_0/docs/examples/debug.c#L100

 -f pre-empts 4xx response headers with an error. So remove -f if you
want to see 4xx response headers

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2018-02-09