On Sep 21, 2012, at 1:48 PM, Bill Middlecamp wrote:

>> On Thu, 20 Sep 2012, Daniel Stenberg wrote
>>
>>> On Thu, 20 Sep 2012, Bill Middlecamp wrote:
>>>
>>>> 1) The "foo/bar" portion of the path may or may not exist, so I use MULTICWD together with the upload of a temporary file to get libcurl to create those directories if needed. Then I DELE the temporary file with a POSTQUOTE command.
>>>>
>>>> 2) Next, I change to NOCWD, and expect that this will produce a CWD to the HOME directory.
>>>
>>> Oh. I can see how this messes up things. "NOCWD" kind of implies that you don't want to do CWD commands, but since you re-use a connection that already did libcurl will not act the way you'd like it to...
>>>
>
> It works with vsftpd and a 10-year old version of Pure FTP that I thought would give me a baseline for testing. But my end user has a server I cannot get that exhibits the problem.
>
>>> Can I ask what the point is to alter the CURLOPT_FTP_FILEMETHOD method like that between requests?
>
> It's fundamentally because libcurl will only create missing directories if MULTICWD is set. I switch back to NOCWD because it's very hard if not impossible for me to know where libcurl is in the directory hierarchy when I need to do a QUOTE command, and there would have to be some tricky path manipulation if I did.
>
>>>
>>> Anyway, I think we need to handle this somehow. I just can't think of the best possible fix right now. I think a good first step would be to write up a test case for the curl test suite that repeats the above scenario so that we have something to work with.
>>>
>
> I cannot write a test case because it depends on the server. I suspect that it has to do with the format of PWD output that comes right after login.
>
>>>> How can I code this to work for all FTP servers?
>>>
>>> This problem isn't happening because of the FTP server, it does so because you tell it to NOCWD after libcurl has changed directory in a previous request.
>>>

I believe I may have found the problem. My end user has an FTP server that returns the following to the PWD command issued by libcurl immediately following login:

257 Current directory is: "<FTP user's home directory>"

This does not conform to RFC 959, and libcurl doesn't recognize it.

The following test describes what I'm trying to do. It works with conformant servers. Following the test code is the debug log of a conformant server. It seems that libcurl is doing what I need, but do you have any ideas for handling the non-conformant server in case I am forced to support it?

------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "curl/curl.h"

////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
//
// Test scenario for the effects of switching between MULTICWD and NOCWD
//
// set the MULTICWD option
// upload file to ftp://<ip>/foo/bar /* creates foo & cwd's to it */
// set the NOCWD option
// list the home directory: UPLOAD=0; DIRLISTONLY=1; URL=ftp://<ip>/
// and tack on POSTQUOTE
// rnfr foo/bar
// rnto foo/baz
//
// The directory listing of HOME is a "carrier" for the POSTQUOTE commands,
// but it may also server as a trigger for cwd to HOME. The rnfr/rnto
// commands must use relative paths in case the user's HOME directory is
// not '/'.
//
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

/* call-out functions */
size_t consume_data(char *ptr, size_t size, size_t nmemb, void *userdata);
size_t read_tmp_stub(char *ptr, size_t size, size_t nmemb, void *userdata);

int main(int argc, char *argv[])
{
        CURLcode result = CURLE_OK;
        CURL *curl = NULL;
        char *curl_errbuf = NULL;
        char base_url[256];
        char tmpfile_url[256];
        struct curl_slist *headerlist=NULL;

        if (argc != 4)
        {
            fprintf(stderr, "Usage: %s <user> <password> <target IP>\n",
                    argv[0]);
            exit(1);
        }

        sprintf(base_url, "ftp://%s:%s@%s/", argv[1], argv[2], argv[3]);
        sprintf(tmpfile_url, "%s%s", base_url, "foo/bar");

        curl = curl_easy_init();

        curl_errbuf = (char*) malloc(CURL_ERROR_SIZE);
        curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, curl_errbuf);

        curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
        curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE , 0L);
        curl_easy_setopt(curl,
                CURLOPT_FTP_FILEMETHOD,
                CURLFTPMETHOD_MULTICWD);
        curl_easy_setopt(curl,
                CURLOPT_FTP_CREATE_MISSING_DIRS,
                CURLFTP_CREATE_DIR_RETRY);
        curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_tmp_stub);
        curl_easy_setopt(curl, CURLOPT_URL, tmpfile_url);

        result = curl_easy_perform(curl);
        if (result != CURLE_OK)
        {
            fprintf(stderr, "Failed file upload: %s\n", curl_errbuf);
            exit(2);
        }

        /* DIRLISTONLY on the base URL is effectively a no-op */
        curl_easy_setopt(curl, CURLOPT_UPLOAD, 0L);
        curl_easy_setopt(curl,
                CURLOPT_FTP_FILEMETHOD,
                CURLFTPMETHOD_NOCWD);
        curl_easy_setopt(curl, CURLOPT_DIRLISTONLY, 1L);
        curl_easy_setopt(curl, CURLOPT_URL, base_url);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, consume_data);
        headerlist = curl_slist_append(headerlist, "rnfr foo/bar");
        headerlist = curl_slist_append(headerlist, "rnto foo/baz");
        curl_easy_setopt(curl, CURLOPT_POSTQUOTE, headerlist);

        result = curl_easy_perform(curl);
        if (result != CURLE_OK)
        {
            fprintf(stderr, "Failed rnfr/rnto: %s\n", curl_errbuf);
            exit(3);
        }
}

/* Just ignore information */
size_t consume_data(char *hdr_str, size_t hdr_sz, size_t hdr_nmemb, void *userdata)
{
        return (hdr_sz * hdr_nmemb);
}

/* upload a single NUL byte */
size_t
read_tmp_stub(char *ptr, size_t size, size_t nmemb, void *userdata)
{
    ptr = "";
    size = 1;
    nmemb = 0;
    return(0);
}

------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------

Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [INFO] New connection from bmbeanstalk
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] 220-=(<*>)=-.:. (( Welcome to PureFTPd 1.0.11 )) .:.-=(<*>)=-
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] 220-You are user number 1 of 50 allowed
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] 220-Local time is now 12:07 and the load is 0.00. Server port: 21.
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] 220 You will be disconnected after 15 minutes of inactivity.
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] Command [user] [xfer]
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] 331 User xfer OK. Password required
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [DEBUG] Command [pass] [<*>]
Sep 24 12:07:34 bmc6 pure-ftpd: (?@bmbeanstalk) [INFO] xfer is now logged in
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 230-User xfer has group access to: xfer
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 230 OK. Current directory is /work/z1/bmiddlec/xfer
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [pwd] []
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 257 "/work/z1/bmiddlec/xfer" is your current location
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [cwd] [foo]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [INFO] Can't change directory to foo: No such file or directory
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 550 Can't change directory to foo: No such file or directory
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [mkd] [foo]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 257 "foo" : The directory was successfully created
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [cwd] [foo]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 250 OK. Current directory is /work/z1/bmiddlec/xfer/foo
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [epsv] []
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 229 Extended Passive mode OK (|||61974|)
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [type] [I]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 200 TYPE is now 8-bit binary
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [stor] [bar]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 150 Accepted data connection
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [NOTICE] //work/z1/bmiddlec/xfer/foo/bar uploaded (0 bytes, 0.00KB/sec)
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 226-9840235.1 Mbytes free disk space
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 226 File successfully transferred
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [cwd] [/work/z1/bmiddlec/xfer]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 250 OK. Current directory is /work/z1/bmiddlec/xfer
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [epsv] []
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 229 Extended Passive mode OK (|||1101|)
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [type] [A]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 200 TYPE is now ASCII
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [nlst] []
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 150 Accepted data connection
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 226 6 matches total
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [rnfr] [foo/bar]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 350 RNFR accepted - file exists, ready for destination
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] Command [rnto] [foo/baz]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [NOTICE] File successfully renamed or moved: [foo/bar]->[foo/baz]
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 250 File successfully renamed or moved
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [INFO] Logout - CPU time spent: 0.003 seconds.
Sep 24 12:07:34 bmc6 pure-ftpd: (xfer_at_bmbeanstalk) [DEBUG] 250 Logout - CPU time spent: 0.003 seconds.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-09-24