Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Thoughts about upstreaming wcurl
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Samuel Henrique via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 4 Jul 2024 23:23:43 +0100
Hello Dan,
On Thu, 4 Jul 2024 at 19:51, Dan Fandrich via curl-library <curl-library_at_lists.haxx.se> wrote:
>
> As it is, I don't think wcurl adds substantial enough value to carry along
> upstream. Right now, it doesn't do much beyond setting a few standard options
> on curl itself, which people can do themselves already in a number of different
> ways to suit them.
I agree it's quite simple, yes, and the plan is to keep it as simple as
possible (more on that below).
> You mention a few possible improvements, but there seems to be a lack of
> clarity of what wcurl is supposed to be. Is it a more simple interface to curl
> for certain use cases? Is it a low-resource way to access curl functionality
> with a wget interface? Is it a wget replacement rewritten from the ground up
> using libcurl to perform transfers?
That's a good point, I didn't do a proper job at strictly documenting the goal
of the tool and why, so here's an opportunity.
The purpose of wcurl is partly defined by the fact that it's not upstream, so:
* To be a command that lets users download files without having to pass any
parameters, using curl;
* To be as simple as it can, trying to do as little processing as possible.
The goal is to keep it as simple as possible to both allow others to easily
review the code and to have a low maintenance burden, while also not
interfering with curl upstream by trying not to change the user's expectation
too much (in the sense that one can easily manually do everything wcurl does
with plain curl).
Since it has to be simple, there's no trying to be similar to wget, so the
strict answer is: wcurl is a curl wrapper that lets you download files with a
parameter-less command, without aiming for wget compatibility, as close to
plain curl as possible.
Now, I say the purpose/goal changes because the moment something like this gets
upstream, you have much more freedom to change it and do things like mimicking
wget parameters and increase the code complexity and features. This is also
because curl upstream should be the ones deciding how a tool named so similar
to "curl" behaves (at least if the tool is not almost plain curl).
> When I saw the announcement, I thought it was the second case, namely a way for
> people used to wget to run curl. It wouldn't add functionality that curl didn't
> already have, but it would give access to what it DID have in a familiar way.
> I talked about this a bit on IRC, since this is an idea I've had for a long
> time that I've never acted on. My idea was that a small script would translate
> options for features that curl already supports and just call curl to perform
> the transfer in the end. So --bind-address would be translated to --interface,
> --no-clobber would translate to --skip-existing, --https-only would translate
> to "--proto -all,https", etc. And the options (like -r) that curl doesn't
> support would cause an error.
That sounds ideal, sorry I can't join curl's IRC as Libera is not bridged to
matrix, but I should find a solution for that.
> As it is, the current wcurl doesn't do that and can't do that because its only
> current option (-o) is already defined by both wget and curl in completely
> different and incompatible ways. So, given that, I don't really see what the
> path forward is for further wcurl improvements, unless it just becomes a third
> new utility for performing network transfers.
At this stage it would still be perfectly fine to change that parameter to
something that doesn't clash with curl or wget, I'm wary of "burning" the wcurl
name by cementing this -o/--opts parameter and not allowing for this in the
future (if it's in your plans).
After we solve the current getopt compatibility issue, we could also see how
feasible it would be to drop -o/--opts and instead reword some options as you
suggest, but I think the options and what they do are better to be defined by
you (curl upstream).
> I personally think that second case hits a sweet spot in that it adds
> significant functionality (a new user interface for curl) that many people are
> familiar with in only a few KB of space without significant portability
> concerns (although it sounds like the getopt issue needs to be solved). That
> would be a Busybox-style approach to wget functionality. Actually, Busybox
> already supports a "wget" applet, but reimplemented the Busybox way and only
> supporting 14 options. A curl version could support many, many more wget
> options and would probably take even less space, if we assume that curl is
> already available.
Do I understand correctly that you're saying wcurl is close to what you have in
mind, only missing the nice parameters you gave as an example? It sounds like
what you had in mind is also a simple portable script (but with its own
parameters instead of doing the -o wrapping).
> The third case (wget reimplemented) would be better as a separate project since
> its added functionality strays away from what curl is trying to be.
I agree, that's something I wouldn't do while using the "wcurl" name and not
being directly upstream.
Thank you for the thoughtful reply, Dan!
Cheers,
Date: Thu, 4 Jul 2024 23:23:43 +0100
Hello Dan,
On Thu, 4 Jul 2024 at 19:51, Dan Fandrich via curl-library <curl-library_at_lists.haxx.se> wrote:
>
> As it is, I don't think wcurl adds substantial enough value to carry along
> upstream. Right now, it doesn't do much beyond setting a few standard options
> on curl itself, which people can do themselves already in a number of different
> ways to suit them.
I agree it's quite simple, yes, and the plan is to keep it as simple as
possible (more on that below).
> You mention a few possible improvements, but there seems to be a lack of
> clarity of what wcurl is supposed to be. Is it a more simple interface to curl
> for certain use cases? Is it a low-resource way to access curl functionality
> with a wget interface? Is it a wget replacement rewritten from the ground up
> using libcurl to perform transfers?
That's a good point, I didn't do a proper job at strictly documenting the goal
of the tool and why, so here's an opportunity.
The purpose of wcurl is partly defined by the fact that it's not upstream, so:
* To be a command that lets users download files without having to pass any
parameters, using curl;
* To be as simple as it can, trying to do as little processing as possible.
The goal is to keep it as simple as possible to both allow others to easily
review the code and to have a low maintenance burden, while also not
interfering with curl upstream by trying not to change the user's expectation
too much (in the sense that one can easily manually do everything wcurl does
with plain curl).
Since it has to be simple, there's no trying to be similar to wget, so the
strict answer is: wcurl is a curl wrapper that lets you download files with a
parameter-less command, without aiming for wget compatibility, as close to
plain curl as possible.
Now, I say the purpose/goal changes because the moment something like this gets
upstream, you have much more freedom to change it and do things like mimicking
wget parameters and increase the code complexity and features. This is also
because curl upstream should be the ones deciding how a tool named so similar
to "curl" behaves (at least if the tool is not almost plain curl).
> When I saw the announcement, I thought it was the second case, namely a way for
> people used to wget to run curl. It wouldn't add functionality that curl didn't
> already have, but it would give access to what it DID have in a familiar way.
> I talked about this a bit on IRC, since this is an idea I've had for a long
> time that I've never acted on. My idea was that a small script would translate
> options for features that curl already supports and just call curl to perform
> the transfer in the end. So --bind-address would be translated to --interface,
> --no-clobber would translate to --skip-existing, --https-only would translate
> to "--proto -all,https", etc. And the options (like -r) that curl doesn't
> support would cause an error.
That sounds ideal, sorry I can't join curl's IRC as Libera is not bridged to
matrix, but I should find a solution for that.
> As it is, the current wcurl doesn't do that and can't do that because its only
> current option (-o) is already defined by both wget and curl in completely
> different and incompatible ways. So, given that, I don't really see what the
> path forward is for further wcurl improvements, unless it just becomes a third
> new utility for performing network transfers.
At this stage it would still be perfectly fine to change that parameter to
something that doesn't clash with curl or wget, I'm wary of "burning" the wcurl
name by cementing this -o/--opts parameter and not allowing for this in the
future (if it's in your plans).
After we solve the current getopt compatibility issue, we could also see how
feasible it would be to drop -o/--opts and instead reword some options as you
suggest, but I think the options and what they do are better to be defined by
you (curl upstream).
> I personally think that second case hits a sweet spot in that it adds
> significant functionality (a new user interface for curl) that many people are
> familiar with in only a few KB of space without significant portability
> concerns (although it sounds like the getopt issue needs to be solved). That
> would be a Busybox-style approach to wget functionality. Actually, Busybox
> already supports a "wget" applet, but reimplemented the Busybox way and only
> supporting 14 options. A curl version could support many, many more wget
> options and would probably take even less space, if we assume that curl is
> already available.
Do I understand correctly that you're saying wcurl is close to what you have in
mind, only missing the nice parameters you gave as an example? It sounds like
what you had in mind is also a simple portable script (but with its own
parameters instead of doing the -o wrapping).
> The third case (wget reimplemented) would be better as a separate project since
> its added functionality strays away from what curl is trying to be.
I agree, that's something I wouldn't do while using the "wcurl" name and not
being directly upstream.
Thank you for the thoughtful reply, Dan!
Cheers,
-- Samuel Henrique <samueloph> -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2024-07-05