cURL / Mailing Lists / curl-users / Single Mail

curl-users

RE: Problem using CURL behind a proxy using proxytunnel option, seemingly a bug has been introduced since version 2.27

From: <Yuri.VANHAEREN_at_ext.ec.europa.eu>
Date: Mon, 9 Sep 2013 09:14:34 +0000

 
>> * Entry path is '/'
>>> PASV
>> * Connect data stream passively
>> < 227 Entering Passive Mode (195,123,248,70,233,112)
>> * Trying 158.169.9.13...
>> * Connecting to 195.122.138.70 (158.169.9.13) port 8012
>> * Establish HTTP proxy tunnel to 195.122.138.70:59760
>> * Proxy auth using Basic with user 'proxyuser'
>> * Server auth using Basic with user 'ftpuser'
>> * Send failure: Socket is not connected
>> * Failed sending CONNECT to proxy
>> * Failure sending ABOR command: Failed sending data to the peer
>> * Closing connection 0
>> curl: (55) Send failure: Socket is not connected

> The subject confuses me - there never existed any version 2.27. I assume
> you're talking about 7.27.0.

You are absolutely correct, it's 7.27 (on Ubuntu) that works. On Windows, the most recent working version I've managed to find was 7.19.5 . If I could find Windows binaries for 7.27 or 7.28 I'd test them but I can't even find 7.29 (the version I was using) anymore.

> Are you saying that curl 7.32.0 also shows this problem?

Yes, it does, which is why I decided to post it to this mailing list.

>> Anyway, I'm assuming you will reach the same conclusion as me, that
>> somewhere after version 2.27 and before 2.29 a change was made that breaks
>> ftp behind http proxies. Perhaps there are some peculiar things about the
>> proxy config here, but I hope you're already aware of the problem. Otherwise
>> I think this counts as a new bug?

> It could be a new bug. I got lost in the flood of stuff so I'm not sure I got
> it all, but you seemed to have said that it worked on some Linux version but
> fails somehow with some option combo on Windows?

Perhaps the flood was not all that necessary. It's main purpose was to illustrate how I got to my conclusion.

Summary:
- It's all about the --proxytunnel option and the fact we have an http proxy.
        Without it a directory listing is returned as an HTML page which is a pain to parse
        With the option file transfers don't work
        Without it I can't create directories on the FTP server (using the --ftp-create-dirs option) so uploads will fail. I did workaround in a script by first doing the command using the create dirs option with proxytunnel, then doing the upload without the proxytunnel option.
- I was using 7.29 on Windows and it couldn't pass our proxy properly
- Helpdesk told me it should work and showed me their logs, were I saw that it was an older version and on unix
- Hence I tried on Ubuntu and the most recent version in the repo was 7.27 and it worked.
- But I need it on a Windows server so I looked up an older binary and found 7.19.5 which worked as well.
- I figured the bug might have been specific to 7.29 so I downloaded the most recent one, 7.32 for Win32. It still has the same problem as 7.29 .

> I don't have any easy way to test this myself so I'd like to get some further
> clarification and logs from you.

>> PS: yes I read the netiquette, sorry about the HTML but without colours and
>> indentation this is barely readable.

>I only read text emails. No HTML, no colors. And correct, it wasn't very
>readable.

Well from the summary above I concluded that the problem was in the version and not in the OS and it was still present in the most recent version so I'm here to try and get it fixed. All the logs were in the last mail but I guess they were hard to read. Would it work better if I attached them as text files? Or do you prefer separate emails for each log or something like that?

>> PS2: It's a different topic, but I would like to add I don't like the dotdot
>> removal 'feature' – some of our webservices use multiple dots for other
>> reasons then indicating paths and when we use URL rewriting, new curl mashes
>> this up.

> Then you abuse URLs as we're actually following the spec (now). I'd love to
> hear a better motivation to why you use/need this. Perhaps we need a way to
> switch it off!

Well I wouldn't really call it abuse, although it is a tad creative perhaps. I prefer to see it as proper usage in a REST way of thinking. Basically what we have is a data structure that isn't a clean hierarchy but rather a net structure and by doing ".." you say "go to preferred parent". Obviously the path you take is important in this case because it's the datastructure at the server who decides where you end up. Wouldn't that also be a problem on filesystems where you use links in creative ways?
Another example I have is where we use dots to fill in as "blank characters" to pad text fields. Thanks to URL rewriting it changes from server.com/app?node=x...&weight=2 to server.com/x.../2 to get a restful approach. In certain cases this might very well yield /../ and I like to use cURL to build automated tests so it would be nice if that kept working as intended...I guess I can work around it or stick to an older curl version but wouldn't that be a pity?

Best regards,
Yuri

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-09-09