Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with active FTPS connection after updating to libcurl 7.87.0/7.88.0 #10666

Closed
SandakovMM opened this issue Mar 3, 2023 · 5 comments
Closed

Issue with active FTPS connection after updating to libcurl 7.87.0/7.88.0 #10666

SandakovMM opened this issue Mar 3, 2023 · 5 comments
Assignees
@icing
Labels
FTP regression TLS

Comments

@SandakovMM
Copy link

Hello
I am experiencing issues with FTP connection after updating to libcurl 7.87.0 and 7.88.0. Unable to reproduce the issue on 7.86.0.

I did this

There is a simple C utility to reproduce the issue:

int debug_curl(CURL* handle, curl_infotype type, char* data, size_t size, void* userptr)
{
    switch (type) {
    case CURLINFO_TEXT:
        printf("Curl output: * %s\n", data);
        break;
    case CURLINFO_HEADER_IN:
        printf("Curl output: < %s\n", data);
        break;
    case CURLINFO_HEADER_OUT:
        printf("Curl output: > %s\n", data);
        break;
    case CURLINFO_DATA_IN:
    case CURLINFO_DATA_OUT:
    case CURLINFO_SSL_DATA_IN:
    case CURLINFO_SSL_DATA_OUT:
    case CURLINFO_END:
        break;
    }
    return 0;
}

int result(char *buf, size_t size, size_t nmemb, void *context)
{
    printf("result: %s\n", buf);
    return 0;
}

int main(int argc, char **argv)
{
    char *url_with_user = argv[1];
    char *password = argv[2];

    CURL *curl = curl_easy_init();
    struct curl_slist *m_headers = NULL;
    if (!curl) {
        printf("unable to init curl");
        return 1;
    }

    curl_easy_setopt(curl, CURLOPT_TIMEOUT, 300);
    curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 30);
    curl_easy_setopt(curl, CURLOPT_FTP_RESPONSE_TIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPALIVE, 1);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPIDLE, 60);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPINTVL, 60);

    curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, debug_curl);
    curl_easy_setopt(curl, CURLOPT_DEBUGDATA, NULL);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    curl_easy_setopt(curl, CURLOPT_FTPPORT, "-");
    curl_easy_setopt(curl, CURLOPT_FTP_USE_EPRT, 1L);

    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0);
    curl_easy_setopt(curl, CURLOPT_FTP_SSL, CURLFTPSSL_TRY);
    curl_easy_setopt(curl, CURLOPT_SSLVERSION, CURL_SSLVERSION_MAX_TLSv1_3);

    curl_easy_setopt(curl, CURLOPT_URL, url_with_user);
    curl_easy_setopt(curl, CURLOPT_USERPWD, password);

    curl_easy_setopt(curl, CURLOPT_DIRLISTONLY, 0L);
    curl_easy_setopt(curl, CURLOPT_NOBODY, 0);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, result);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, NULL);
    curl_easy_perform(curl);
    return 0;
}

It works fine with 7.86.0, but when I build it with 7.87.0 or 7.88.0, the connection hangs for a wait timeout and closes without output.
There is the output of the program with libcurl 7.87.0:

Curl output: *   Trying 10.69.41.135:21...

Curl output: * Connected to 10.69.41.135 (10.69.41.135) port 21 (#0)

Curl output: < 220 ProFTPD Server (ProFTPD) [10.69.41.135]

Curl output: > AUTH SSL

Curl output: < 234 AUTH SSL successful
PD) [10.69.41.135]
Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Client hello (1):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Server hello (2):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Certificate (11):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, CERT verify (15):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Finished (20):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Finished (20):

Curl output: * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

Curl output: * Server certificate:

Curl output: *  subject: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  start date: Feb 28 06:04:14 2023 GMT

Curl output: *  expire date: Feb 28 06:04:14 2024 GMT

Curl output: *  issuer: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  SSL certificate verify result: self signed certificate (18), continuing anyway.

Curl output: > USER tftp

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Curl output: * old SSL session ID is stale, removing

Curl output: < 331 Password required for tftp
.69.41.135]
Curl output: > PASS 1qazxsw2

Curl output: < 230 User tftp logged in
 tftp
Curl output: > PBSZ 0

Curl output: < 200 PBSZ 0 successful

Curl output: > PROT P

Curl output: < 200 Protection set to Private

Curl output: > PWD

Curl output: < 257 "/home/tftp" is the current directory

Curl output: * Entry path is '/home/tftp'

Curl output: * Request has same path as previous transfer

Curl output: > EPRT |1|10.69.41.80|38691|

Curl output: * ftp_perform ends with SECONDARY: 1

Curl output: < 200 EPRT command successful
nt directory
Curl output: * Connect data stream actively

Curl output: > TYPE A

Curl output: < 200 Type set to A
ccessful
Curl output: > LIST

Curl output: < 150 Opening ASCII mode data connection for file list

Curl output: * Maxdownload = -1

Curl output: * Preparing for accepting server on data port

Curl output: * Checking for server connect

Curl output: * Ready to accept data connection from server

Curl output: * Connection accepted from server

Curl output: * Operation timed out after 300001 milliseconds with 0 bytes received

Curl output: * Closing connection 0

Curl output: * [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS alert, close notify (256):

The issue disappears when I remove SSL parts of the code. So likely the problem somewhere in this part. Unfortunately, I can't see where exactly.

I expected the following

I expect to see the line:

result: drwxr-xr-x   2 tftp     root         4096 Feb 28 10:46 tftpboot

Received from the other side.
For example this is the utility output built with libcurl 7.86.0:

Curl output: *   Trying 10.69.41.135:21...

Curl output: * Connected to 10.69.41.135 (10.69.41.135) port 21 (#0)

Curl output: < 220 ProFTPD Server (ProFTPD) [10.69.41.135]

Curl output: > AUTH SSL

Curl output: < 234 AUTH SSL successful
PD) [10.69.41.135]
Curl output: * TLSv1.3 (OUT), TLS handshake, Client hello (1):

Curl output: * TLSv1.3 (IN), TLS handshake, Server hello (2):

Curl output: * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

Curl output: * TLSv1.3 (IN), TLS handshake, Certificate (11):

Curl output: * TLSv1.3 (IN), TLS handshake, CERT verify (15):

Curl output: * TLSv1.3 (IN), TLS handshake, Finished (20):

Curl output: * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

Curl output: * TLSv1.3 (OUT), TLS handshake, Finished (20):

Curl output: * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

Curl output: * Server certificate:

Curl output: *  subject: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  start date: Feb 28 06:04:14 2023 GMT

Curl output: *  expire date: Feb 28 06:04:14 2024 GMT

Curl output: *  issuer: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  SSL certificate verify result: self signed certificate (18), continuing anyway.

Curl output: > USER tftp

Curl output: * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Curl output: * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Curl output: * old SSL session ID is stale, removing

Curl output: < 331 Password required for tftp
.69.41.135]
Curl output: > PASS 1qazxsw2

Curl output: < 230 User tftp logged in
 tftp
Curl output: > PBSZ 0

Curl output: < 200 PBSZ 0 successful

Curl output: > PROT P

Curl output: < 200 Protection set to Private

Curl output: > PWD

Curl output: < 257 "/home/tftp" is the current directory

Curl output: * Entry path is '/home/tftp'

Curl output: * Request has same path as previous transfer

Curl output: > EPRT |1|10.69.41.80|39219|

Curl output: * ftp_perform ends with SECONDARY: 1

Curl output: < 200 EPRT command successful
nt directory
Curl output: * Connect data stream actively

Curl output: > TYPE A

Curl output: < 200 Type set to A
ccessful
Curl output: > LIST

Curl output: < 150 Opening ASCII mode data connection for file list

Curl output: * Maxdownload = -1

Curl output: * Preparing for accepting server on data port

Curl output: * Checking for server connect

Curl output: * Ready to accept data connection from server

Curl output: * Connection accepted from server

Curl output: * Doing the SSL/TLS handshake on the data stream

Curl output: * SSL re-using session ID

Curl output: * TLSv1.3 (OUT), TLS handshake, Client hello (1):

Curl output: * TLSv1.3 (IN), TLS handshake, Server hello (2):

Curl output: * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

Curl output: * TLSv1.3 (IN), TLS handshake, Finished (20):

Curl output: * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

Curl output: * TLSv1.3 (OUT), TLS handshake, Finished (20):

Curl output: * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

Curl output: * Server certificate:

Curl output: *  subject: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  start date: Feb 28 06:04:14 2023 GMT

Curl output: *  expire date: Feb 28 06:04:14 2024 GMT

Curl output: *  issuer: C=CH; L=Schaffhausen; O=Plesk; CN=Plesk; emailAddress=info@plesk.com

Curl output: *  SSL certificate verify result: self signed certificate (18), continuing anyway.

Curl output: * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

Curl output: * old SSL session ID is stale, removing

result: drwxr-xr-x   2 tftp     root         4096 Feb 28 10:46 tftpboot <------------------------------- THE LINE

Curl output: * Failure writing output to destination

Curl output: * TLSv1.3 (IN), TLS alert, close notify (256):

Curl output: * TLSv1.3 (OUT), TLS alert, close notify (256):

Curl output: * Closing connection 0

curl/libcurl version

libcurl 7.87.0 and 7.88.0

operating system

Ubuntu 20
icing added a commit to icing/curl that referenced this issue Mar 3, 2023
@icing
ftp active mode with SSL, add the damn filter
2996627 
- since 7.87.0 we lost adding the SSL filter for an active
  FTP connection that uses SSL. This leads to hangers and timeouts
  as reported in curl#10666.
@icing icing mentioned this issue Mar 3, 2023
Closed
@icing
Copy link
Contributor

icing commented Mar 3, 2023

Would you be able to check if #10669 resolves the issue? Thanks!

@icing icing self-assigned this Mar 3, 2023
@SandakovMM
Copy link
Author

Hi,
I've checked the patch, and it helps. Thank you very much, @icing.
Also, I've checked FTP in passive mode, and seems it doesn't work with this patch. To check it just remove curl_easy_setopt(curl, CURLOPT_FTPPORT, "-"); from code I mentioned earlier:

int debug_curl(CURL* handle, curl_infotype type, char* data, size_t size, void* userptr)
{
    switch (type) {
    case CURLINFO_TEXT:
        printf("Curl output: * %s\n", data);
        break;
    case CURLINFO_HEADER_IN:
        printf("Curl output: < %s\n", data);
        break;
    case CURLINFO_HEADER_OUT:
        printf("Curl output: > %s\n", data);
        break;
    case CURLINFO_DATA_IN:
    case CURLINFO_DATA_OUT:
    case CURLINFO_SSL_DATA_IN:
    case CURLINFO_SSL_DATA_OUT:
    case CURLINFO_END:
        break;
    }
    return 0;
}

int result(char *buf, size_t size, size_t nmemb, void *context)
{
    printf("result: %s\n", buf);
    return 0;
}

int main(int argc, char **argv)
{
    char *url_with_user = argv[1];
    char *password = argv[2];

    CURL *curl = curl_easy_init();
    struct curl_slist *m_headers = NULL;
    if (!curl) {
        printf("unable to init curl");
        return 1;
    }

    curl_easy_setopt(curl, CURLOPT_TIMEOUT, 300);
    curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 30);
    curl_easy_setopt(curl, CURLOPT_FTP_RESPONSE_TIMEOUT, 60);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPALIVE, 1);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPIDLE, 60);
    curl_easy_setopt(curl, CURLOPT_TCP_KEEPINTVL, 60);

    curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, debug_curl);
    curl_easy_setopt(curl, CURLOPT_DEBUGDATA, NULL);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    // Option was here <-------------------------------------------------------------------
    curl_easy_setopt(curl, CURLOPT_FTP_USE_EPRT, 1L);

    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0);
    curl_easy_setopt(curl, CURLOPT_FTP_SSL, CURLFTPSSL_TRY);
    curl_easy_setopt(curl, CURLOPT_SSLVERSION, CURL_SSLVERSION_MAX_TLSv1_3);

    curl_easy_setopt(curl, CURLOPT_URL, url_with_user);
    curl_easy_setopt(curl, CURLOPT_USERPWD, password);

    curl_easy_setopt(curl, CURLOPT_DIRLISTONLY, 0L);
    curl_easy_setopt(curl, CURLOPT_NOBODY, 0);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, result);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, NULL);
    curl_easy_perform(curl);
    return 0;
}

I believe we could disable your changes for FTP in passive mode:

  DEBUGF(infof(data, "ftp InitiateTransfer()"));
-  if(conn->bits.ftp_use_data_ssl &&
+  if(conn->bits.ftp_use_data_ssl && data->set.ftp_use_port &&
      !Curl_conn_is_ssl(conn, SECONDARYSOCKET)) {
    result = Curl_ssl_cfilter_add(data, conn, SECONDARYSOCKET);
    if(result)

It works for me, but there might be better solutions since I'm not familiar with curl source code good enough.

@icing
Copy link
Contributor

icing commented Mar 6, 2023

@bagder: you understand the ftp flags better than me. Does the proposed fix sound good to you?

@bagder bagder changed the title Issue with FTP connection after updating to libcurl 7.87.0/7.88.0 Issue with active FTPS connection after updating to libcurl 7.87.0/7.88.0 Mar 6, 2023
@bagder
Copy link
Member

bagder commented Mar 6, 2023

It seems like a good fix. The InitiateTransfer function gets called in both passive and active mode, so it needs to check data->set.ftp_use_port to make sure the condition only applies to active.

icing added a commit to icing/curl that referenced this issue Mar 7, 2023
@icing
ftp active mode with SSL, add the damn filter
dc77a64 
- since 7.87.0 we lost adding the SSL filter for an active
  FTP connection that uses SSL. This leads to hangers and timeouts
  as reported in curl#10666.
@icing
Copy link
Contributor

icing commented Mar 7, 2023

Added this to #10669.

@bagder bagder closed this as completed in a26418c Mar 7, 2023
bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
@icing @bch
ftp: active mode with SSL, add the damn filter
fc72199 
- since 7.87.0 we lost adding the SSL filter for an active
  FTP connection that uses SSL. This leads to hangers and timeouts
  as reported in curl#10666.

Reported-by: SandakovMM on github
Fixes curl#10666
Closes curl#10669
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@icing icing

Labels
FTP regression TLS
Milestone
No milestone
Development

No branches or pull requests
3 participants
@icing @bagder @SandakovMM