cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Scripting curl from the inside out (Re: curl the next few years)

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Thu, 19 Jun 2014 20:58:05 +0200

On Thu, Jun 19, 2014 at 01:15:07PM -0400, Rich Gray wrote:
> Daniel Stenberg wrote on the libcurl list:
> >1. Embed a language interpreter (lua?). For that middle ground where curl
> > isn’t enough and a libcurl binding feels “too much”.
>
> I have been toying with the idea of an option which would allow
> control of curl via scripting *between* multiple requests, allowing a
> single curl instance to perform multiple transfers to one or more
> hosts. Because curl would not die between commands, connections can
> stay open, DNS caches can be maintained, etc., making the repeated
> accesses more efficient.
>
> Imagine a "--exec <program>" option which would invoke a program
> (script) at the completion of a command. Curl would pass the just
> completed command string, the result and other useful info to the
> program as arguments and/or environment variables. The program would
> check the result of the curl run, optionally operate on any data and
> then communicate back to curl what it wants to do next (if anything)
> by sending commands back to curl through a pipe.

Leveraging the exiting --write-out option as the protocol from curl
to the exec program, and the --config option as the protocol from
the exec program to curl seems to me like a straightforward and
unsurprising protocol for this. But I wonder how many applications
would benefit enough from the efficiency gains to implement something like
this. Admittedly, there have been times when I've wished I could take advantage
of connection reuse while scripting something with curl, but the real gains
would have been pretty minimal for me which is why I never bothered using a
scripting language and libcurl binding to accomplish it.

If curl gains a lua/Python/Guile embedded interpreter, and enough example
recipes to make using it for reasonable simple things easy for beginners, this
will probably become redundant.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-06-19