curl / Mailing Lists / curl-users / Single Mail

curl-users

Re: URL parsing for the command line tool?

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Thu, 16 Aug 2018 15:45:46 +0200 (CEST)

On Wed, 15 Aug 2018, David Niklas wrote:

> Please try something more real world (even contrived). Why would you want to
> change hosts? Your example makes no sense to me.

I was just trying to provoke thoughts and ideas, I didn't have any particular
use case in mind.

Features we can consider:

1. host-specific sections in config files. So .curlrc can specify for example
a specific user-agent to use only when connecting to example.com and a
different user-agent for example.org.

2. command-line variables based on the most recently used URL. If you want to
save the output from a download in a directory named as the host name with the
file name part also from the URL:
  "curl https://example.com/file -o "%{url_host}/%{url_file}".

> export in1="../download.html"
>
> for i in DragonFlyBSD FreeBSD NetBSD; do
> curl --base-url $base --output-url - $in1 "#" $i | ./download_curl.sh
> done

Or just a way to apply a relative URL on the absolute one before it is used:

   curl http://example.org/foo --rel-url "../here/it/is" -O

That could be fun for those who download a HTML page and want to download
something that is pointed to with a relative URL within that.

  curl $url > raw.html
  extract_hrefs;
  for i in $all_hrefs; do
    curl $url --rel-url "$i" -O
  done

> As an award for your work on curl I award you, the extra large virtual happy
> face.

Thank you!

-- 
  / daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2018-08-16