curl-library
Re: Hashing while downloading
Date: Tue, 20 Jan 2015 00:23:20 +0100 (CET)
On Mon, 19 Jan 2015, Leon Winter wrote:
> a friend of mine is working on "modernizing" Apt of debian. Apparently there
> were running some obscure hand-written HTTP code resulting in very low
> performance. Not to mention potential security issues and lacking features.
> He therefore now rewrites Apt to use curl.
Cool!
> in order to do this one needs to link against a library providing a hash
> function. Curl however already has such dependencies and even has a small
> abstraction layer for Md5. However this is not exported
Right. I'm generally very careful with adding new APIs, especially such that
aren't strictly transfer-related and I would say MD5 isn't about transfers.
All new functions take their share of added maintenance and work.
> It is noteworthy that this copy'n'pasting already happened inside curl
> to some extend:
>
> lib/md5.c
> src/tool_metalink.c (albeit abstracting over more hash functions)
Well yes, but those two are in the library and in the tool, pretty much for
the same reason you bring up here!
> While looking into this I also noticed that the metalink code does the
> verification _after_ the download, which Daniel also mentions [0]. In the
> mentioned RFCs about the headers and XML format I found no mention of the
> time of the hash processing. Why not do it while downloading?
I don't think there's any good reason other than it hasn't been done. Possibly
because nobody has cared enough to actually do the work.
> Should we either export the awesome abstractions curl offers for hashes or
> possibly also TLS (the VTLS layer) to outside?
I would prefer to have the entire VTLS part of libcurl turned into a library
of its own that libcurl could use (although it hasn't happen because there's
just not enough desire from anywhere to drive such a change). I don't think it
is libcurl's job to offer neither crypto nor hashing functionality outside of
transfers.
> Shouldn't the metalink implementation make use of the MD5 abstraction
> already in place?
The Metalink code is not in libcurl.
-- / daniel.haxx.se ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.htmlReceived on 2015-01-20