Re: Implementing IPFS support in cURL

From: Mark Gaiser via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 24 Mar 2022 01:24:27 +0100

On Thu, Mar 24, 2022 at 12:41 AM Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Wed, 23 Mar 2022, Mark Gaiser via curl-library wrote:
>
> > - ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > becomes this format:
> > - <gateway>/ipfs/QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > with an actual example being:
>
> So there's never any path or query in an ipfs URL? How about fragment?
>
> What should happen if a path or query is specified?
>

That should be forwarded to the gateway.
For example, i actually had this bug in Brave where "ipfs://<cid>?a=b" was
passed as:
<gateway>/ipfs/<cid>
as opposed to:
<gateway>/ipfs/<cid>?a=b

I think it's the simplest approach to just bluntly forward everything after
"ipfs://" to <gateway>/ipfs/<dump here> which would be a perfectly valid
URL.


>
> > - seturl (urlapi.c line 805) doesn't understand "ipfs://<cid>" and
> > considers <cid> to be the "host".
>
> Now you're talking about the url API and that's libcurl code.
>
> Of course libcurl has no magic special knowledge of ipfs:// URLs, and from
> previous discussions we learned that there also is no ipfs URL syntax
> standard. Is there one now?
>

No there is not. Sorry if I confused you about that.

>
> URLs are made to be generic. Why does the parser need to know about ipfs?
> Ig
> the <cid> is the host from the URL, does that matter?
>

I might have been wrong here. My thinking was that "ipfs://<cid>" should be
handled as:
protocol: ipfs
path: cid

But this is a non issue if it's purely handled on the tool side.


>
> > An approach that could potentially work is:
> > 1. Let parseurl fail with "ipfs://" (actually seturl, but called from
> > parseurl)
>
> Then you're talking about libcurl code. I thought we agreed that this
> change
> would be commandline tool only? The tool code is all in src/.
>

I wasn't aware that to be a decision, I viewed it as a mere hint of a
possible way to implement it but to investigate further to see if that's
the best approach.

So just to get the opinions aligned. Am i correct in assuming that all of
the IPFS logic needed for cURL, those being:
- Transform ipfs://<cid> to <gateway>/ipfs/<cid>
- Detecting which gateway to use
should be implemented purely in the tool code?

I'm merely searching for a clean way to implement this that is acceptable
to both :)
It's fine by me to implement it all on the tooling side.

If it does go in the tooling side then you will get at least one "out of
place" looking if where it checks for an ipfs and ipns protocol to do
something custom in that case. As far as i can tell there is no tooling
code currently that handles any other protocol in such a custom way. This
is why I'm asking for feedback on the approach here.


>
> --
>
> / daniel.haxx.se
> | Commercial curl support up to 24x7 is available!
> | Private help, bug fixes, support, ports, new features
> | https://curl.se/support.html
>

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html

Received on 2022-03-24