Hi Jani,

On Sunday 31 of January 2010 10:51:59 Jani Kajala wrote:
> First I'm sorry to jump in the middle of discussion but instead of
> wildcards, wouldn't it be better to implement that on client side and just
> have only functionality to request a list of files of a single directory
> from libcurl? It's simple to request a list of files and loop the subdirs
> recursively anyway, right?

thanks for your insight! You do have a very valid point. There should be
a way to get the parsed file list out of libcurl. An application can do
whatever on its own then. Anyway I don't think it's in a conflict with
the wildcard download. The basic idea behind is to do the wildcard matching
on the remote side for protocols where such approach is known to work. We
are in fact not sure in case of FTP for now, but it would be great. Imagine
a huge directory with a few files you are asking for - the speedup may be
significant. There can also be done some optimizations on the protocol
level, e.g. you enter the directory only once, not for each file, etc.

> Also, error handling becomes more difficult when wildcards are involved,
> since if error happens, you probably want to know a) which files were
> processed b) which file caused error, so it supports the idea that
> application would keep things "under control" anyway. You can of course
> provide some kind of callback to get similar result, but you get more messy
> implementation instead of handling the recursion fully on client side.

It depends on the cause of error. If the error occurs during file-list
getting/processing, then the game is over anyway. If the error occurs during
a file download then you can decide on the application level what to do
in both cases.

> Besides, simple * or ? patterns might not be enough for application anyway,
> and I can imagine many users preferring regular expressions, which again
> would work nicely if app implements the functionality...

I think * and ? are mostly sufficient and kind of default for file-oriented
tools. If the application requires more, we should introduce a way to move
the matching out of libcurl when necessary.

> Disclaimer: Sorry if I commented on already closed topics (just ignore my
> comments if I did), just my 2c on the topic.

I think now it's the best time to come with such suggestions. It may be more
difficult to address them later when some work has been done.

Kamil
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-01-31