On Mon, May 14, 2012 at 12:18 PM, <curl-users-request_at_cool.haxx.se> wrote:
> Date: Tue, 15 May 2012 01:18:07 +0900
> From: Tatsuhiro Tsujikawa <tatsuhiro.t_at_gmail.com>
> To: the curl tool <curl-users_at_cool.haxx.se>
> Subject: Re: Metalink support patch for curl
> Message-ID:
>        <CAPyZ6=KAgQv1gnp2QOPUUQKUcEm3yuRJmehy0JXbSxY=XTN2HA_at_mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Sun, May 13, 2012 at 9:28 PM, Tatsuhiro Tsujikawa
> <tatsuhiro.t_at_gmail.com> wrote:
>> On Sat, May 12, 2012 at 12:09 AM, Tatsuhiro Tsujikawa
>> <tatsuhiro.t_at_gmail.com> wrote:
>>> On Fri, May 11, 2012 at 5:01 AM, Anthony Bryan <anthonybryan_at_gmail.com> wrote:
>>>> On Tue, May 8, 2012 at 11:28 AM, ?<curl-users-request_at_cool.haxx.se> wrote:
>>>>> Message: 1
>>>>> Date: Wed, 9 May 2012 00:28:30 +0900
>>>>> From: Tatsuhiro Tsujikawa <tatsuhiro.t_at_gmail.com>
>>>>> To: the curl tool <curl-users_at_cool.haxx.se>
>>>>> Subject: Re: Metalink support patch for curl
>>>>> Message-ID:
>>>>> ? ? ? ?<CAPyZ6=L1At3YREO_y21VtVgYqwt=bEPECBXWpkZuqu_jTmLZzw_at_mail.gmail.com>
>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>
>>>>> On Mon, May 7, 2012 at 1:18 AM, Tatsuhiro Tsujikawa
>>>>> <tatsuhiro.t_at_gmail.com> wrote:
>>>>
>>>>> I included above change in the attached patch.
>>>>> I also fixed the issue when content-type has parameters. Now you can download
>>>>> http://openoffice.mirrorbrain.org/stable/3.3.0/OOo-SDK_3.3.0_Linux_x86-64_install-deb_en-US.tar.gz.metalink
>>>>
>>>> thanks, that works for me!
>>>>
>>>> could you also sanitize <file name=""> because I noticed I could use
>>>> <file name="../foo"> or <file name="/root/bar"> and traverse
>>>> directories.
>>>>
>>>> is it possible to have this sanitizing in libmetalink? then it would
>>>> only need to be done once there for any app that uses it. or maybe it
>>>> is better suited to these curl patches, I don't know.
>>>>
>>>> from http://tools.ietf.org/html/rfc5854#section-4.1.2.1
>>>>
>>>> ? ? ?Security Note: The path MUST NOT contain any directory traversal
>>>> ? ? ?directives or information. ?The path MUST be relative. ?The path
>>>> ? ? ?MUST NOT begin with a "/", "./", or "../"; contain "/../"; or end
>>>> ? ? ?with "/..".
>>>>
>>>
>>> I agree to make this sanitizing in libmetalink. Good idea.
>>>
>>
>> Fixed in libmetalink trunk.
>>
>
> I fixed the bug that causes segmentation fault if name attribute of
> file element in Metalink is invalid (e.g., name starts with "../") and
> as a result there are nothing download.
> I also fixed the same bug when the number of resources is 0.

awesome, good catch!

> In addition to the above fixes, I added the code to always create
> directory hierarchy for Metalink download.
> The directory hierarchy creation is needed because name attribute of
> file element in Metalink contains directory information. The next
> libmetalink release will ensure that name element is relative and must
> not contain directory traversal directives.

great! thanks for all the libmetalink work and also for mentoring the student!

one possibly last thing, do we want to support transparent usage,
where metalinks are advertised with a Link header?
although this is mostly used with .meta4 files, it might be nice to
have in place for when libmetalink supports .meta4 (which seems like
it will be soonish) :)

from Daniel:
> The existing hash code is not exposed to the client code but is only libcurl
> internal so far. We do have some code that we still re-use in curl from
> libcurl under the hood that isn't using the API - just in the name of avoiding
> code duplication - and there might be reasons to do this for some hash
> functions as well. Although I'm not sure they are that easily shared like
> this. It needs to be investigated closer.

Daniel, how do you suggest hash checking should be implemented?
do you want hash checking implemented before merging?

Tatsuhiro also asked if there were some tests suitable as a template
to add Metalink test?

& should the patch be cumulative or individual (like they are now)?

to everyone else:

maybe we should have introduced things earlier, but if you haven't
heard of Metalink, it's a way to provide mirrors, hashes, signatures,
and other information for download clients.

it's used in a good number of places (KDE, Xfce, LibreOffice,
OpenOffice.org, Fedora, Ubuntu, openSUSE, sugar, XBMC, FSF, etc)

a good server to test from, you can click on "Details" for any file
here to see the metalink and other information, or just append
".metalink" to any filename.
http://download.documentfoundation.org/libreoffice/stable/3.5.3/deb/x86/

please help us test these patches!

usage is:
local metalink: curl --metalink example.metalink
remote metalink: curl http://example.com/example.metalink -O

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html