Skip to content

FTP download fails using libcurl when TCP_FASTOPEN enabled #6252

Closed
@KlausC

Description

@KlausC
Contributor

I found, that the ftp example program, when modified by setting the TCP_FASTOPEN flag, fails.
My working environment is Fedora Linux release 33, but I think that is also for other Linux variants.

Linux klauspc 5.9.8-200.fc33.x86_64 #1 SMP Tue Nov 10 21:58:19 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[crusius@klauspc src]$ curl-config --version
libcurl 7.71.1

[crusius@klauspc src]$ cc curlftp.c -lcurl -o curlftp

[crusius@klauspc src]$ curlftp --tcp-fastopen "ftp://xmlsoft.org/libxslt/libxslt-1.1.33.tar.gz"
*   Trying 91.121.203.120:21...
* Connected to xmlsoft.org () port 21 (#0)
< 220 (vsFTPd 2.2.2)
> USER anonymous
< 331 Please specify the password.
> PASS ftp@example.com
< 230 Login successful.
> PWD
< 257 "/"
* Entry path is '/'
> CWD libxslt
* ftp_perform ends with SECONDARY: 0
< 250 Directory successfully changed.
> EPSV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||13558|).
* Could not resolve host: 
* Can't resolve new host :13558
* Closing connection 0
curl told us 15

# For comparison with unset flag:
[crusius@klauspc src]$ curlftp "ftp://xmlsoft.org/libxslt/libxslt-1.1.33.tar.gz"
*   Trying 91.121.203.120:21...
* Connected to xmlsoft.org (91.121.203.120) port 21 (#0)
< 220 (vsFTPd 2.2.2)
> USER anonymous
< 331 Please specify the password.
> PASS ftp@example.com
< 230 Login successful.
> PWD
< 257 "/"
* Entry path is '/'
> CWD libxslt
* ftp_perform ends with SECONDARY: 0
< 250 Directory successfully changed.
> EPSV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||53060|).
*   Trying 91.121.203.120:53060...
* Connecting to 91.121.203.120 (91.121.203.120) port 53060
* Connected to xmlsoft.org (91.121.203.120) port 21 (#0)
> TYPE I
< 200 Switching to Binary mode.
> SIZE libxslt-1.1.33.tar.gz
< 213 3444093
> RETR libxslt-1.1.33.tar.gz
< 150 Opening BINARY mode data connection for libxslt-1.1.33.tar.gz (3444093 bytes).
* Maxdownload = -1
* Getting file with size: 3444093
* Remembering we are in dir "libxslt/"
< 226 Transfer complete.
* Connection #0 to host xmlsoft.org left intact


[crusius@klauspc src]$ cat curlftp.c

/***************************************************************************
 *                                  _   _ ____  _
 *  Project                     ___| | | |  _ \| |
 *                             / __| | | | |_) | |
 *                            | (__| |_| |  _ <| |___
 *                             \___|\___/|_| \_\_____|
 *
 * Copyright (C) 1998 - 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
 *
 * This software is licensed as described in the file COPYING, which
 * you should have received as part of this distribution. The terms
 * are also available at https://curl.se/docs/copyright.html.
 *
 * You may opt to use, copy, modify, merge, publish, distribute and/or sell
 * copies of the Software, and permit persons to whom the Software is
 * furnished to do so, under the terms of the COPYING file.
 *
 * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
 * KIND, either express or implied.
 *
 ***************************************************************************/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>

/* <DESC>
 * Get a single file from an FTP server.
 * </DESC>
 */

struct FtpFile {
  const char *filename;
  FILE *stream;
};

static size_t my_fwrite(void *buffer, size_t size, size_t nmemb, void *stream)
{
  struct FtpFile *out = (struct FtpFile *)stream;
  if(!out->stream) {
    /* open file for writing */
    out->stream = fopen(out->filename, "wb");
    if(!out->stream)
      return -1; /* failure, can't open file to write */
  }
  return fwrite(buffer, size, nmemb, out->stream);
}


int main(int argc, char**argv)
{
  CURL *curl;
  CURLcode res;
  struct FtpFile ftpfile = {
    "curl.tar.gz", /* name to store the file as if successful */
    NULL
  };

  curl_global_init(CURL_GLOBAL_DEFAULT);

  long tcpfast = 0L;
  if( argc >= 3 && strcmp(argv[1], "--tcp-fastopen") == 0) {
    tcpfast = 1L;
    argc -= 1;
    argv += 1;
  }
  if (argc < 2) exit(1);

  curl = curl_easy_init();
  if(curl) {
    /*
     * You better replace the URL with one that works!
     */
    curl_easy_setopt(curl, CURLOPT_TCP_FASTOPEN, tcpfast);
    curl_easy_setopt(curl, CURLOPT_URL, argv[1]);

    /* Define our callback to get called when there's data to be written */
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, my_fwrite);
    /* Set a pointer to our struct to pass to the callback */
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ftpfile);

    /* Switch on full protocol/debug output */
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    res = curl_easy_perform(curl);

    /* always cleanup */
    curl_easy_cleanup(curl);

    if(CURLE_OK != res) {
      /* we failed */
      fprintf(stderr, "curl told us %d\n", res);
    }
  }

  if(ftpfile.stream)
    fclose(ftpfile.stream); /* close the local file */

  curl_global_cleanup();

  return 0;
}

Activity

changed the title [-]FTP download fails using libcurl[/-] [+]FTP download fails using libcurl when TCP_FASTOPEN enabled[/+] on Nov 26, 2020
KlausC

KlausC commented on Nov 26, 2020

@KlausC
ContributorAuthor

With the current version (7.74), compiled with -g -Og, the same run fails earlier:

 curlftp --tcp-fastopen "ftp://xmlsoft.org/libxslt/libxslt-1.1.33.tar.gz"
*   Trying 91.121.203.120:21...
* Connected to xmlsoft.org () port 21 (#0)
< 220 (vsFTPd 2.2.2)
* Send failure: Operation already in progress
* Closing connection 0
curl told us 55
bagder

bagder commented on Nov 26, 2020

@bagder
Member

I don't see how TCP Fast Open will help for an FTP transfer so I would recommend you don't enable that, as a rather safe work-around.

KlausC

KlausC commented on Nov 26, 2020

@KlausC
ContributorAuthor

I think we will disable that in our downstream library wrapper. See Julia: JuliaLang/Downloads.jl#76

added a commit that references this issue on Nov 28, 2020
f0fde36
added 2 commits that reference this issue on Nov 29, 2020
1c5989b
bagder

bagder commented on Dec 1, 2020

@bagder
Member

That was wrong, the commit only fixed #6262, this should rather be fixed by #6265 ...

reopened this on Dec 1, 2020

3 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Participants

      @bagder@KlausC

      Issue actions

        FTP download fails using libcurl when TCP_FASTOPEN enabled · Issue #6252 · curl/curl