On Tue, May 22, 2012 at 8:52 AM, Anthony Bryan <anthonybryan_at_gmail.com> wrote:
> On Mon, May 21, 2012 at 12:57 PM,  <curl-users-request_at_cool.haxx.se> wrote:
>> Date: Tue, 22 May 2012 01:57:26 +0900
>> From: Tatsuhiro Tsujikawa <tatsuhiro.t_at_gmail.com>
>> To: the curl tool <curl-users_at_cool.haxx.se>
>> Subject: Re: Metalink support patch for curl
>> Message-ID:
>>        <CAPyZ6=JkQ3EX9Gv5Rr1CBuATDWxDCrpU2a7eo2-5SOqoF--VCg_at_mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> On Tue, May 15, 2012 at 11:39 AM, Anthony Bryan <anthonybryan_at_gmail.com> wrote:
>>> On Mon, May 14, 2012 at 12:18 PM, ?<curl-users-request_at_cool.haxx.se> wrote:
>>>> Date: Tue, 15 May 2012 01:18:07 +0900
>>>> From: Tatsuhiro Tsujikawa <tatsuhiro.t_at_gmail.com>
>>>> To: the curl tool <curl-users_at_cool.haxx.se>
>>>> Subject: Re: Metalink support patch for curl
>>>> Message-ID:
>>>> ? ? ? ?<CAPyZ6=KAgQv1gnp2QOPUUQKUcEm3yuRJmehy0JXbSxY=XTN2HA_at_mail.gmail.com>
>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>
>>>> On Sun, May 13, 2012 at 9:28 PM, Tatsuhiro Tsujikawa
>>>> <tatsuhiro.t_at_gmail.com> wrote:
>>>>> On Sat, May 12, 2012 at 12:09 AM, Tatsuhiro Tsujikawa
>>>>> <tatsuhiro.t_at_gmail.com> wrote:
>>>>>> On Fri, May 11, 2012 at 5:01 AM, Anthony Bryan <anthonybryan_at_gmail.com> wrote:
>>>>>>> On Tue, May 8, 2012 at 11:28 AM, ?<curl-users-request_at_cool.haxx.se> wrote:
>>> great! thanks for all the libmetalink work and also for mentoring the student!
>>>
>>> one possibly last thing, do we want to support transparent usage,
>>> where metalinks are advertised with a Link header?
>>> although this is mostly used with .meta4 files, it might be nice to
>>> have in place for when libmetalink supports .meta4 (which seems like
>>> it will be soonish) :)
>
> did you happen to add metalink detection from Link header?
>
>  # curl -sI http://download.services.openoffice.org/files/stable/3.3.0/OOo-SDK_3.3.0_Linux_x86-64_install-deb_en-US.tar.gz
> ..
> Link: <http://download.services.openoffice.org/files/stable/3.3.0/OOo-SDK_3.3.0_Linux_x86-64_install-deb_en-US.tar.gz.meta4>;
> rel=describedby; type="application/metalink4+xml"
> ..
>
> 2 helpful servers to test with:
> http://download.services.openoffice.org/
> http://go-oo.mirrorbrain.org/
>

I did not have Metalink/HTTP in mind in this patch set, but thanks to
GSOC, KGet and Wget(?) receive this feature so it would be good to
bring it into curl too.
The good side of Metalink/HTTP is it does not need libmetalink (XML
parser thing).
First I'll do some refactoring of the current patch to reuse code
between Metalink/XML and Metalink/HTTP.
Current code depends on the structs which come from libmetalink all
the way down to the path.
To share the code cleanly, I'll introduce new structure and copy the
needed data from libmetalink structs to our new structs. We use this
new structs in tool_operate.c. The good side effect of this move is
that it reduces #ifdefs
since our new struct does not depend on libmetalink.

Since we need to parse Link header field, my --dump-header fix patch
should be landed before actually doing
Metalink/HTTP work.

Best regards,

Tatsuhiro Tsujikawa

>
>>> from Daniel:
>>>> The existing hash code is not exposed to the client code but is only libcurl
>>>> internal so far. We do have some code that we still re-use in curl from
>>>> libcurl under the hood that isn't using the API - just in the name of avoiding
>>>> code duplication - and there might be reasons to do this for some hash
>>>> functions as well. Although I'm not sure they are that easily shared like
>>>> this. It needs to be investigated closer.
>>>
>>> Daniel, how do you suggest hash checking should be implemented?
>>> do you want hash checking implemented before merging?
>>>
>>
>> I added checksum checking feature in the attached patch set (still
>> cumulative...).
>> The behind story of this move follows:
>> I felt, without checksum checking, Metalink support is somewhat less
>> attractive, so I invested my time
>> to check the hash function usage in libcurl.
>> According to my research, hash functions used are md4 and md5.
>> They are not directly used from curl (tool) source code.
>> I looked the code in curl_md5.h and md5.c and found that it has
>> abstraction layer for several crypto libraries.
>> I extended this abstraction so that it is not MD5 hash function dependent.
>> The extended version is in tool_metalink.{h,c}.
>> I'm used to libgcrypt/nettle/OpenSSL, so currently, my abstractions
>> support those libraries.
>> No code added for NSS.
>>
>> The current implementation supports sha-256, sha-1 and md5. Of course,
>> the "strongest" hash function is used if there are several functions
>> available.
>
> thank you, this worked very well for me.
>
> --
> (( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
>   )) Easier, More Reliable, Self Healing Downloads
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-users
> FAQ:        http://curl.haxx.se/docs/faq.html
> Etiquette:  http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-05-25