curl-library
Re: Windows select fails with 7.34.0 and 7.35.0 using multi_perform
Date: Sat, 15 Feb 2014 16:25:20 -0800 (PST)
I have been able to reproduce the issue with multi-single.c
System and compile info:
Windows 7-64 pro
libCurl compiled using Makefile.m32 and MinGW G++ 4.8.0
App compiled with G++ 4.8.0 and created a 32bit app.
Here is the logging output from Msys: Note I have pasted the modified multi-single.c after the logging output
Please help me understand what is going wrong. This modified multi-single.c works with 7.33.0 and fails with 7.34.0 and 7.35.0
//----------------- Start of logging output
* Hostname was NOT found in DNS cache
running: 1
maxfd: -1
* Trying 193.166.3.2...
maxfd: 240
* Connected to ftp.funet.fi (193.166.3.2) port 21 (#0)
maxfd: 240
maxfd: 240
< 220---------- Welcome to Pure-FTPd [privsep] ----------
< 220-You are user number 47 of 1000 allowed.
< 220-Local time is now 03:11. Server port: 21.
< 220-Only anonymous FTP is allowed here
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 30 minutes of inactivity.
> USER anonymous
maxfd: 240
< 331-Welcome to the FUNET anonymous ftp archive
< 331-
< 331-Serving freely distributable files with FTP since 1990
< 331-
< 331-FTP.FUNET.FI is a service of the Finnish Academic and Research Network
< 331-FUNET, and is located at CSC,the Finnish IT center for science,
< 331-in Keilaranta 14, Espoo, Finland
< 331-
< 331-This is a Dell R710 server with 72GB of RAM. It has two QuadCore
< 331-Nehalem EP processors (Intel 5540) with hyperthreading running under
< 331-the Solaris operating system. Main storage is a fully redundant EMC CX700
< 331-fiber channel RAID array currently with over 10TB of space for public
< 331-files.
< 331-
< 331-We have a 10Gbit/s connection to Funet which in turn connects
< 331-with 10Gbit/s to FICIX, NORDUnet, Geant and Internet2.
< 331-
< 331-This archive is also reachable over the IPv6 network. WWW-browser
< 331-users can also use http://www.nic.funet.fi/pub/ to browse the archive.
< 331-
< 331-See the README file for more information about this archive.
< 331-
< 331-All anonymous FTP transactions will be logged for possible later analysis
< 331-and statistics. If you don't like this policy, please disconnect now!
< 331-
< 331-Please mail to problems@nic.funet.fi in case of problems
< 331-
< 331-Monday 15-19 finnish time is the regular Funet network service window
< 331-We may have service breaks earlier on Mondays or some other days if necessary
< 331-
< 331-NOTE: 16.10.2013 07:00 UTC onwards there will be a service break. All services
< 331-may be unavailable for several hours and there may be multiple reboots.
< 331 Any password will work
> PASS ftp@example.com
maxfd: 240
< 230 Any password will work
> PWD
maxfd: 240
< 257 "/" is your current location
* Entry path is '/'
> CWD pub/standards/RFC
* ftp_perform ends with SECONDARY: 0
maxfd: 240
< 250-*====================================================================*
< 250-* *
< 250-* This directory is maintained by the RFC Editor. If you experience *
< 250-* any problems, please report them to rfc-editor@rfc-editor.org. *
< 250-* *
< 250-*====================================================================*
< 250 OK. Current directory is /.m/mirrors1l/nic.nordu.net/in-notes
> EPSV
* Connect data stream passively
maxfd: 240
< 500 Unknown command
* Failed EPSV attempt. Disabling EPSV
> PASV
maxfd: 240
< 227 Entering Passive Mode (193,166,3,2,142,139)
* Hostname was NOT found in DNS cache
* Trying 193.166.3.2...
* Connecting to 193.166.3.2 (193.166.3.2) port 36491
maxfd: 240
select() return = -1. WSAGetLastError() = 10038
* Closing connection 0
//----------------- Start of multi-single.c
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) 1998 - 2011, Daniel Stenberg, <daniel@haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at http://curl.haxx.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
***************************************************************************/
/* This is a very simple example using the multi interface. */
#include <stdio.h>
#include <string.h>
/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>
/* curl stuff */
#include "curl/curl.h"
#include <iostream>
using namespace std;
// Get data call back
size_t getDataCB(void* ptr, size_t size, size_t nmemb, void* pBuffer) {
// Calculate number of characters to transfer
size_t numChar = size*nmemb;
// For this test just drop on the floor
return numChar;
}
// Simply download a FTP file.
int main(void) {
CURL *easy_handle;
CURLM *multi_handle;
CURLMcode multi_rCode;
int still_running; /* keep number of running handles */
curl_global_init(CURL_GLOBAL_WIN32);
easy_handle = curl_easy_init();
curl_easy_setopt(easy_handle, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(easy_handle, CURLOPT_FTP_FILEMETHOD, CURLFTPMETHOD_SINGLECWD);
curl_easy_setopt(easy_handle, CURLOPT_WRITEFUNCTION, getDataCB);
curl_easy_setopt(easy_handle, CURLOPT_WRITEDATA, (void*)malloc(100000));
curl_easy_setopt(easy_handle, CURLOPT_HEADERFUNCTION, NULL);
curl_easy_setopt(easy_handle, CURLOPT_WRITEHEADER, NULL);
curl_easy_setopt(easy_handle, CURLOPT_TRANSFERTEXT, 0L);
curl_easy_setopt(easy_handle, CURLOPT_FTP_CREATE_MISSING_DIRS, 0L);
curl_easy_setopt(easy_handle, CURLOPT_UPLOAD, 0L);
curl_easy_setopt(easy_handle, CURLOPT_NOBODY, 0L);
curl_easy_setopt(easy_handle, CURLOPT_URL, "ftp://ftp.funet.fi/pub/standards/RFC/rfc959.txt");
/* init a multi stack */
multi_handle = curl_multi_init();
/* add the individual transfer */
curl_multi_add_handle(multi_handle, easy_handle);
/* we start some action by calling perform right away */
while((multi_rCode = curl_multi_perform(multi_handle, &still_running)) == CURLM_CALL_MULTI_PERFORM);
if (multi_rCode != CURLM_OK) cout << "perform: " << multi_rCode << endl;
cout << "running: " << still_running << endl;
do {
struct timeval timeout;
int rc; /* select() return code */
fd_set fdread; FD_ZERO(&fdread);
fd_set fdwrite; FD_ZERO(&fdwrite);
fd_set fdexcep; FD_ZERO(&fdexcep);
int maxfd = -1;
long curl_timeo = -1;
/* set a suitable timeout to play around with */
timeout.tv_sec = 1;
timeout.tv_usec = 0;
multi_rCode = curl_multi_timeout(multi_handle, &curl_timeo);
if (multi_rCode != CURLM_OK) {
cout << "timeout: " << multi_rCode << endl;
break;
}
if(curl_timeo >= 0) {
timeout.tv_sec = curl_timeo / 1000;
if(timeout.tv_sec > 1)
timeout.tv_sec = 1;
else
timeout.tv_usec = (curl_timeo % 1000) * 1000;
}
/* get file descriptors from the transfers */
multi_rCode = curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
if (multi_rCode != CURLM_OK) {
cout << "fdset: " << multi_rCode << endl;
break;
}
cout << "maxfd: " << maxfd << endl;
// Windows select does not look at maxfd. So if == -1 just sleep instead
if (maxfd >= 0) {
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
} else {
rc = 0;
Sleep(200);
}
int le;
switch(rc) {
case -1:
/* select error */
still_running = 0;
le = WSAGetLastError();
cout << "select() return = -1. WSAGetLastError() = " << le << endl;
break;
case 0:
default:
/* timeout or readable/writable sockets */
while((multi_rCode = curl_multi_perform(multi_handle, &still_running)) == CURLM_CALL_MULTI_PERFORM);
if (multi_rCode != CURLM_OK) cout << "perform: " << multi_rCode << endl;
break;
}
} while(still_running);
curl_multi_remove_handle(multi_handle, easy_handle);
curl_easy_cleanup(easy_handle);
curl_multi_cleanup(multi_handle);
curl_global_cleanup();
return 0;
}
________________________________
From: Alan <jkabeal-curl@yahoo.com>
To: "curl-library@cool.haxx.se" <curl-library@cool.haxx.se>
Sent: Saturday, February 15, 2014 4:33 PM
Subject: Windows select fails with 7.34.0 and 7.35.0 using multi_perform
I am having an issue with libcurl versions 7.34.0 and 7.35.0
Note that 7.33.0 works fine for me.
I am looking for some information that will help point me in the proper direction in order to resolve this issue. Thanks in advance.
Also, from the debugging below, libCurl knows that the first transfer is still active and tries to use it but then starts a new transfer.
And 7.33.0 also starts a new transfer even though it knows that the first transfer is still active. Just does not fail in select.
Here is a short description of my flow:
1. Start all the multi stuff: global init and create multi handle.
2. Create an easy handle and set options for PWD command.
3. Add easy handle and start the multi_perform loop.
4. When command completes, grab the CWD
5. Remove the easy handle above.
6. Create new easy handle and set options for reading a file.
Note that the file is in the same folder as the command above.
7. Add easy handle and start the multi_perform loop.
8. Select returns -1
9. Report error and close everything.
Here is some debug output from 7.35.0
Note that there is less debug output in 7.35.0 than 7.34.0.
----------- From the PWD transfer that completed:
* Hostname was NOT found in DNS cache
fdset maxfd: -1
* Trying 206.214.166.233...
fdset maxfd: 936
select return: 1
* Connected to ftp.winnercomm.com (206.214.166.233) port 21 (#0)
fdset maxfd: 936
select return: 1
< 220 ftp.winnercomm.com X2 WS_FTP Server 6.0(24874977)
> USER ...
fdset maxfd: 936
select return: 1
< 331 Enter password
> PASS ...
fdset maxfd: 936
select return: 0
fdset maxfd: 936
select return: 1
< 230 User logged in
> PWD
fdset maxfd: 936
select return: 1
< 257 "/..." is current directory
* Entry path is '/...'
* ftp_perform ends with SECONDARY: 0
* Remembering we are in dir ""
> PWD
< 257 "/..." is current directory
* Connection #0 to host ftp.winnercomm.com left intact
Removing from multi_stack: 0x2ed8268
PWD string: "257 "/...""
easy cleanup: 0x2ed8268
------------- Create new easy handle to ftp read single file and add to multi handle
* Found bundle for host ftp.winnercomm.com: 0x2e7aa40
* Re-using existing connection! (#0) with host ftp.winnercomm.com
* Connected to ftp.winnercomm.com (206.214.166.233) port 21 (#0)
* Request has same path as previous transfer
> EPSV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
fdset maxfd: 936
select return: 1
< 500 Unkwown command EPSV
* Failed EPSV attempt. Disabling EPSV
> PASV
fdset maxfd: 852
select return: 1
< 227 Entering Passive Mode (206,214,166,233,45,67).
* Hostname was NOT found in DNS cache
* Trying 206.214.166.233...
* Connecting to 206.214.166.233 (206.214.166.233) port 11587
fdset maxfd: 852
select return: -1
select SOCKET_ERROR: 0 (The 0 is from WSAGetLastError())
Alan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-16