Bugs item #1822405, was opened at 2007-10-29 23:27
Message generated for change (Comment added) made by norbertf
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1822405&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: hang
Status: Open
Resolution: Invalid
Priority: 5
Private: No
Submitted By: n.f. (norbertf)
Assigned to: Daniel Stenberg (bagder)
Summary: curlftpfs hangs after reading files with no write permission

Initial Comment:
i have already reported this problem to the curlftpfs bugzilla.

http://sourceforge.net/tracker/index.php?func=detail&aid=1809225&group_id=160565&atid=816357

i'm not sure if this is really a libcurl problem, but there is a suspicion that libcurl doesn't move into a working state after reading a file failed (because no read permissions for the ftp user).

if there is a possibility to reproduce this bug without curlftpfs please tell me and i will try (perhaps a peace of demo code, which also uses curl_multi).

----------------------------------------------------------------------

>Comment By: n.f. (norbertf)
Date: 2007-10-31 13:18

Message:
Logged In: YES
user_id=135573
Originator: YES

ok - here is another log with the important parts and some curlftpfs
debugging output.

i don't understand why the data connection from a previous (failed) RETR
attempt is still open. i thought curl_multi_remove_handle() should close
it. or perhaps libcurl expects that the server closes the connection, but
proftpd doesn't...

File Added: curlftpfs3.log

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-10-31 10:14

Message:
Logged In: YES
user_id=1110
Originator: NO

First, line 428 is completely wrong on all libcurl versions before 7.17.0.
The curl_easy_setopt man page currently says:

  "NOTE: before 7.17.0 strings were not copied. Instead the user was
forced keep
  them available until libcurl no longer needed them."

If this mistake is done once, perhaps it is made more times at other
places...

And regarding EPSV vs PASV, they're more or less identical in the client
end. They're just two different commands with slightly difference
responses, but other than those details libcurl makes no difference at all
in how it acts and communicates using either approach.

Your log is hard to follow since it is multiple requests shown in a single
log. But yes, a data connection should always be closed after the transfer.

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-31 00:42

Message:
Logged In: YES
user_id=135573
Originator: YES

AFAIK there is no firewall between my workstation and my ftp server. They
are in the same private network.

I tried a ethereal dump. Below the conversation of the control channel.
Furthermore there are three TCP connections to data port 50098. Two are
empty and one pushes back the data of the readable file lall.txt. The weird
thing is that one of the empty data connections (the second one) remains
ESTABLIHED. It seems to be the one opened for the second read protected
file (messages.3).

I don't know the EPSV protocol. Shouldn't the ftp client close this
connection?

230 User XXXX logged in.
PWD
257 "/" is current directory.
CWD old_messages
250 CWD command successful.
EPSV
229 Entering Extended Passive Mode (|||50098|)
TYPE I
200 Type set to I
SIZE messages.4
213 20628
RETR messages.4
550 messages.4: Permission denied
EPSV
229 Entering Extended Passive Mode (|||50098|)
SIZE messages.3
213 10804
RETR messages.3
550 messages.3: Permission denied
EPSV
229 Entering Extended Passive Mode (|||50098|)
SIZE lall.txt
213 15
RETR lall.txt
150 Opening BINARY mode data connection for lall.txt (15 bytes)
226 Transfer complete.
------ (hangs) -----

i might try to write a test case which resembles the behavior of
curlftpfs. but it won't be easy. :-(
perhaps you could also have a look at the ftpfs_read_chunk() function at
line 392 of

http://curlftpfs.cvs.sourceforge.net/curlftpfs/curlftpfs/ftpfs.c?revision=1.25&view=markup

maybe there is something obviously wrong... i think the problem is that
the select (line 428) loop doesn't return when reading lall.txt.

thx
norbert

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-10-30 22:26

Message:
Logged In: YES
user_id=1110
Originator: NO

That "recommendation" is worth nothing here. Yeah, EPSV might cause
problems for people who use weird NATs/firewalls etc, but there too it
doesn't cause trouble to libcurl only to users of it or perhaps the entire
app.

Can you please do a plain libcurl app that repeats the problem against
your server and show it to us? As long as you need curlftpfs to repeat it,
we will continue to suspect the bug is outside of libcurl...

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-30 21:25

Message:
Logged In: YES
user_id=135573
Originator: YES

no. i didn't test PORT mode, because curlftpfs doesn't seem to have a
switch to force PORT mode.

what i can say is that the problem occurs in EPSV but not in PASV mode.

i also found this "recommendation":

http://www.ohloh.net/projects/7238/contributors/40480/commits/11711342

and yes, the problem occurs on every hit (on my own ftp server - a
proftpd-1.2.9-1.9). actually i have to try to read two files with no read
permission and then reading the third (not read protected file) hangs.

i just tried on an external FTP server - also with EPSV - but i can't
reproduce the problem there.

----------------------------------------------------------------------

Comment By: Dan Fandrich (dfandrich)
Date: 2007-10-30 21:00

Message:
Logged In: YES
user_id=236775
Originator: NO

Are you saying the problem occurs in PORT mode? Because the log attached
to the other bug is also using passive mode, as is test case 546 (EPSV
mode, to be specific). Does this problem happen every single time this
condition is hit, or just sporadically?

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-30 20:32

Message:
Logged In: YES
user_id=135573
Originator: YES

i tried to investigate this further:

this behavior might indicate that this is a libcurl bug:
the problem goes away when i use PASV mode ( -o disable_epsv)

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2007-10-30 10:06

Message:
Logged In: YES
user_id=1110
Originator: NO

Since this doesn't seem to be a bug in libcurl...

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-30 01:18

Message:
Logged In: YES
user_id=135573
Originator: YES

thanks a lot, i will try your test-case as soon as possible.

----------------------------------------------------------------------

Comment By: Dan Fandrich (dfandrich)
Date: 2007-10-30 00:51

Message:
Logged In: YES
user_id=236775
Originator: NO

The error code from the FTP server in both cases is 550, so from libcurl's
perspective it's identical. The difference with your case is that the SIZE
command succeeds, but libcurl handled that fine as well when I tried
changing the test case to match. It's possible that there are other
differences in how curlftpfs uses libcurl, but this test case at least
gives you a known-working baseline to which you can refer.

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-30 00:33

Message:
Logged In: YES
user_id=135573
Originator: YES

thanks, but i think that's not a file-not-found problem, but a
permission-denied problem.

> RETR messages.4
* FTP 0x80658d0 state change from RETR_SIZE to RETR
< 550 messages.4: Permission denied
* RETR response: 550
...
> RETR messages.3
* FTP 0x80658d0 state change from RETR_SIZE to RETR
< 550 messages.3: Permission denied
* RETR response: 550
...

have a look at curlftplog.txt, which i attached to the curlftpfs bug
report.

after trying to read messages.4 and messages.3 (which both have no read
permissions), reading messages.2 (for which i have read permissions)
hangs:

...
> RETR messages.2
* FTP 0x80658d0 state change from RETR_SIZE to RETR
< 150 Opening BINARY mode data connection for messages.2 (84257 bytes)
* Maxdownload = -1
* Getting file with size: 84257
* FTP 0x80658d0 state change from RETR to STOP
* DO-MORE phase ends with 0
* STATE: DO_MORE => DO_DONE handle 0x808e3e4; (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x808e3e4; (connection #0)
* Conn 0 recv pipe 1 inuse 0 athead 1
* STATE: WAITPERFORM => PERFORM handle 0x808e3e4; (connection #0)

(hangs)

----------------------------------------------------------------------

Comment By: Dan Fandrich (dfandrich)
Date: 2007-10-30 00:01

Message:
Logged In: YES
user_id=236775
Originator: NO

I've created test case 546 that tests a file-not-found FTP transfer
followed by a found one using the multi interface, and it passes fine.

----------------------------------------------------------------------

Comment By: n.f. (norbertf)
Date: 2007-10-29 23:40

Message:
Logged In: YES
user_id=135573
Originator: YES

sorry - the title of this bug report should be:
curlftpfs hangs after reading files with no READ permission

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1822405&group_id=976
Received on 2007-10-31