Hi Anthony,
Am 20.06.2012 06:29, schrieb Anthony Bryan:
> (moved over from libcurl, since this list is probably more appropriate)
if it would be a pure usage issue then probably yes - but development of
both libcurl *and* curl is discussed on the curl-library list ...

> Guenter, don't know if you got my previous thanks for
> metalink/libmetalink patches, so thank you :)
oh, a big thanks to you and Tatsuhiro too for writing libmetalink so
that the metalink XML processing can be hidden from apps like curl ;-)

> we are already having a similar discussion...if you are using a
> metalink download, by default it will write to the filename taken from
> inside the metalink (except for some sanitizing done within
> libmetalink).
yes, thats clear to me, but this is the filename of the final target the
metalink points to, but I was talking about the storage of the metalink
data itself ...
usually you might find a metalink URL like this:
http://download.documentfoundation.org/libreoffice/stable/3.5.4/win/x86/LibO_3.5.4_Win_x86_install_multi.msi.meta4
this is fine, and works without issue, and the metalink download happens
in 3 stages:
1) download the file LibO_3.5.4_Win_x86_install_multi.msi.meta4 and
store it locally (this works on any platform since the metalink file
LibO_3.5.4_Win_x86_install_multi.msi.meta4 has no illegal chars in its
filename)
2) parse this file with help of libmetalink
3) download the file LibO_3.5.4_Win_x86_install_multi.msi
now the problem I see is in those cases when the metalink data is
generated with a CGI script like what we have on the libcurl download page:
http://curl.haxx.se/metalink.cgi?curl=win64-ssl-sspi
now it happens at stage 1 that curl wants to store the metalink data to
a local file 'metalink.cgi?curl=win64-ssl-sspi' which is an illegal
filename on some platforms due to the '?' ...
maybe its even half an issue of our CGI script which might lack to send
a filename in the headers which curl could use instead? (I dont know ...)

> any OS specific sanitizing for curl stuff should probably be in curl,
> I'm guessing, as the '?' illegal characters are not related to
> metalink.
sure and agreed - and exactly the reason why I brought it to the
curl-library list so that others can make some suggestions how we might
code around this issue; sure I can go and just commit something which
fixes the issue, but I like to hear what others think is the best
solution before I commit fixes.

> metalink downloads are off by default
no, I disagree here!

> (and support is not enabled by default at compile time).
yes, but lets not mix now between enabling metalink support at compile
time (which is done via configure option --enable-libmetalink) and usage
of curl's metalink feature if compiled-in; and be sure my curl *has* it
compiled-in:
~ # curl -V
curl 7.27.0-20120617 (x86_64-unknown-linux-gnu) libcurl/7.27.0-20120617
OpenSSL/1.0.0e zlib/1.2.7 libidn/1.22 libssh2/1.2.9
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps
pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz Metalink

> perhaps some text could be shown like "Metalink detected but not used.
> Metalink use is enabled with --metalink"
just to make clear how the current curl metalink implementation acts
(and why I disagreed above about 'metalink downloads are off by default'):
~ # curl --metalink
http://download.documentfoundation.org/libreoffice/stable/3.5.4/win/x86/LibO_3.5.4_Win_x86_install_multi.msi.meta4
Warning: Could not parse Metalink file:
Warning:
http://download.documentfoundation.org/libreoffice/stable/3.5.4/win/x8
Warning: 6/LibO_3.5.4_Win_x86_install_multi.msi.meta4
curl: option --metalink: is badly used here
curl: try 'curl --help' or 'curl --manual' for more information

so clearly the --metalink option does *NOT* work with URLs but is meant
for local files as the help also states:
~ # curl --help | grep metalink
      --metalink Process local Metalink file and use mirrors

calling curl with the URL only results in printing the received metalink
data to STDOUT

calling curl with:
curl -O URL
or
curl -o localfile URL
results in the 3-stage acting as already described above:
1) download the metalink data and store it in a local file
2) parse this file with help of libmetalink
3) download the file which the metalink data points to
so this proofes that metalink download plugs in automatically once curl
gets a header 'application/metalink+xml' or 'application/metalink4+xml' ...

and finally to make it absolutely clear:
this issue I see with downloading files from CGI links is a common issue
and in 1st place not related to metalink support! I'm sure we discussed
this problem already in the past, and I also believe we came to the
conclusion that we should cut off anything from URL beginning with the
'?' if we use it as local filename with the -O option, but as it seems
we didnt code the conclusion ... ;-)

sorry for the lengthly reply!

Gün.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-06-20