Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: How to dry-run a http request
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Dan Fandrich via curl-library <curl-library_at_lists.haxx.se>
Date: Sat, 15 Feb 2025 08:16:30 -0800
On Sat, Feb 15, 2025 at 04:21:04PM +0100, Jeroen Ooms via curl-library wrote:
> For the R bindings, we have a popular debugging mode to 'dry-run' a
> http request. By this we mean that the user builds and performs a
> http-request in the regular way, but instead of actually connecting to
> the requested host, we trick libcurl to make the http request to a
> local server, which records the full payload that libcurl has
> uploaded, and we return this blob to the user.
>
> Currently the way this is implemented is a bit cumbersome. We first
> start a http-server in the background. Then we take the handle
> provided by the user, and replace CURLOPT_URL to substitute the host
> with "http://localhost". Then we modify CURLOPT_HTTPHEADER to set the
> 'Host' request header to the host:port that was specified by the user
> in the original url, and let libcurl perform the request. The fake
> server always returns 200.
>
> Especially the part where we need to run our own http server is a bit
> annoying. I was wondering if there would perhaps be a simpler method
> trick libcurl to send a http request to a local file or buffer, from
> where we can read it. Preferably something that we can do entirely in
> C (things like nc are difficult to port).
When curl is talking HTTP, it expects an HTTP server on the other end. Avoiding
that is not going to be generally possible. But, if you're just interested in
the client side of the transaction and stick with HTTP (not HTTPS) and HTTP
1.1 and avoid a negotiated authentication protocol, you might be able to get
away with providing a CURLOPT_OPENSOCKETFUNCTION that points to a file and
setting instantaneous CURLOPT_EXPECT_100_TIMEOUT_MS and
CURLOPT_SERVER_RESPONSE_TIMEOUT timeouts. It's quite possible that libcurl
won't like a file descriptor in place of a socket, but you might get lucky.
Dan
Date: Sat, 15 Feb 2025 08:16:30 -0800
On Sat, Feb 15, 2025 at 04:21:04PM +0100, Jeroen Ooms via curl-library wrote:
> For the R bindings, we have a popular debugging mode to 'dry-run' a
> http request. By this we mean that the user builds and performs a
> http-request in the regular way, but instead of actually connecting to
> the requested host, we trick libcurl to make the http request to a
> local server, which records the full payload that libcurl has
> uploaded, and we return this blob to the user.
>
> Currently the way this is implemented is a bit cumbersome. We first
> start a http-server in the background. Then we take the handle
> provided by the user, and replace CURLOPT_URL to substitute the host
> with "http://localhost". Then we modify CURLOPT_HTTPHEADER to set the
> 'Host' request header to the host:port that was specified by the user
> in the original url, and let libcurl perform the request. The fake
> server always returns 200.
>
> Especially the part where we need to run our own http server is a bit
> annoying. I was wondering if there would perhaps be a simpler method
> trick libcurl to send a http request to a local file or buffer, from
> where we can read it. Preferably something that we can do entirely in
> C (things like nc are difficult to port).
When curl is talking HTTP, it expects an HTTP server on the other end. Avoiding
that is not going to be generally possible. But, if you're just interested in
the client side of the transaction and stick with HTTP (not HTTPS) and HTTP
1.1 and avoid a negotiated authentication protocol, you might be able to get
away with providing a CURLOPT_OPENSOCKETFUNCTION that points to a file and
setting instantaneous CURLOPT_EXPECT_100_TIMEOUT_MS and
CURLOPT_SERVER_RESPONSE_TIMEOUT timeouts. It's quite possible that libcurl
won't like a file descriptor in place of a socket, but you might get lucky.
Dan
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-02-15