Here is a question that has probably come up before but I can't seem to
find it in the mail archives. What if I wanted to scp a set of files on
a remote server and pathname into a memory buffer?

Getting a single file, if I know the name of the file, seems fairly cut
and dry. But what if there are a whole set of files and all I know is a
pathname of a directory on the remote server and a few characters in the
filenames?

Like so :

# pwd
/export/home/debug

# ls -lo get_things_from_here/*
-rw-r--r-- 1 debug 1833 Apr 26 17:14 get_things_from_here/file_000.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_001.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_002.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_003.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_004.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_005.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_006.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:13 get_things_from_here/file_007.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:14 get_things_from_here/file_008.dat
-rw-r--r-- 1 debug 1833 Apr 26 17:14 get_things_from_here/file_009.dat

There could be a thousand such files. They could range in size up to 8K
each or even be zero bytes. Possibly zero bytes. Just to be special :-)

With scp I can do that with a wildcard however I have to wonder how to
do the same task with libCurl. Something like this seems almost to be
reasonable :

  curl_easy_setopt ( curl, CURLOPT_URL,
    "scp://a.server.com:22/export/home/debug/get_things_from_here/f*");

Please bear in mind, I am just thinking through it before hacking away
at this. Even if that were to fetch each file in some sort of mysterious
order that could be LC_COLLATE based or who knows, do they all end up
mashed together byte by byte into the same big memory buffer? Seems as
if they would and I therefore need the file list and sizes in advance to
unpack that buffer.

Has this been looked at before?

Dennis

ps: a real world solution would need the directory list of files well in
    advance I would think as the files could be named *almost* anything.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-04-26