url: Fix parsing for when 'file' is the default protocol #1124

jay · 2016-11-13T20:33:48Z

Follow-up to 3463408.

Prior to this set of changes hostnames were silently stripped and the
file scheme did not work properly as the default protocol.

Ref: https://curl.haxx.se/mail/lib-2016-11/0081.html

Also I thought the error message on file://foo/bar was confusing so I changed it

Before: curl: (3) Valid host name with slash missing in URL
After:  curl: (3) Invalid file://hostname/, expected localhost or 127.0.0.1 or none

mention-bot · 2016-11-13T20:33:54Z

@jay, thanks for your PR! By analyzing the history of the files in this pull request, we identified @bagder, @dfandrich and @captain-caveman2k to be potential reviewers.

bagder · 2016-11-13T22:14:16Z

lib/url.c

-                  protobuf, path)) &&
-     strcasecompare(protobuf, "file")) {
+  for(i = 0; i < 16 && data->change.url[i] &&
+             data->change.url[i] != '\n'; ++i) {


This function already verifies that there is no CR or LF in the URL, immediately before this code snippet so this newline check is superfluous.

Thinking about it, I suppose the check for 16 bytes scheme could instead be modified to the longest supported scheme. Which I think is a draw between TELNET and GOPHER, at merely 6 bytes.

Ok I'll work on a second draft. The reason I did the things you commented on the way I did is because I was following what is already in the function as a model.

bagder · 2016-11-13T22:15:08Z

lib/url.c

+             data->change.url[i] != '\n'; ++i) {
+    if(data->change.url[i] == ':') {
+      if(!i) {
+        failf(data, "Bad URL");


Maybe even spell out that there was no colon/valid scheme in the URL?

bagder · 2016-11-13T22:19:13Z

lib/url.c

@@ -4073,7 +4098,8 @@ static CURLcode parseurlandfillconn(struct Curl_easy *data,
      char *ptr;
      if(!checkprefix("localhost/", path) &&
         !checkprefix("127.0.0.1/", path)) {
-        failf(data, "Valid host name with slash missing in URL");
+        failf(data, "Invalid file://hostname/, "
+                    "expected localhost or 127.0.0.1 or none");


We're back to what we discussed on the mailing list. I tried to make the message also convey that there needs to be a valid hostname and slash in the URL as this message will be shown for file://locahost as well, which does have a valid host name but no slash. But sure, error messages are hard and if you think this is better then I don't mind.

jay · 2016-12-06T07:33:33Z

@bagder I made the changes you requested, however I did not change the magic numbers of 15/16 since I wanted to be consistent with the size of protobuf sscanf used elsewhere in the function.

Follow-up to 3463408. Prior to 3463408 file:// hostnames were silently stripped. Prior to this commit it did not work when a schemeless url was used with file as the default protocol. Ref: https://curl.haxx.se/mail/lib-2016-11/0081.html Ref: curl#1124

Squash this into draft1 - Support --proto-default file c:/foo/bar.txt - Support file://c:/foo/bar.txt Assisted-by: Anatol Belski

jay · 2017-01-09T07:03:38Z

@bagder can you check this again, I did a second draft to cover --proto-default file c:/foo/bar.txt and file://c:/foo/bar.txt. Note I didn't use isalpha for the drive letters since I wasn't sure it would be C locale. I could put the ('a' <= path[0] && path[0] <= 'z') || ('A' <= path[0] && path[0] <= 'Z') into a macro if preferred.

weltling · 2017-01-09T15:00:39Z

@jay the current version of your patch fully restores the backward compatibility. Please let me know, if i have to test any follow up changes later on.

Thanks.

bagder

It looks fine to me. I only have one little concern: this has no #ifdefs for windows or windows-like systems so what happens when you try this URL on a *nix system? Shouldn't it fail the URL parsing then?

jay · 2017-01-11T07:30:17Z

I only have one little concern: this has no #ifdefs for windows or windows-like systems so what happens when you try this URL on a *nix system? Shouldn't it fail the URL parsing then?

I thought it would be cleaner, and it could fail on fopen if the OS doesn't support drive letters. If you want I can wrap those areas in #if defined(WIN32) || defined(MSDOS)

bagder · 2017-01-11T12:51:24Z

I thought it would be cleaner, and it could fail on fopen if the OS doesn't support drive letters.

You're right. The biggest difference is then what error code it would return back, if we avoid the unlikely event that there actually can be a file named like that on a non-windows machine.

IMO: I think it is safer that we return "malformed URL" earlier and pay the price with the additional #ifdef.

jay · 2017-01-11T20:39:38Z

IMO: I think it is safer that we return "malformed URL" earlier and pay the price with the additional #ifdef.

Well how about immediately after the file url parsing fail if drive letter and not msdos/win. This way the flow is the same for all platforms but there's only one guard and we catch it as malformed rather than passing it around. For example see draft3.

Squash me into previous draft - Fail when a file:// drive letter is detected and not MSDOS/Windows.

bagder · 2017-01-12T07:49:32Z

That's perfectly fine with me and makes the error message even clearer. 👍

bagder reviewed Nov 13, 2016

View reviewed changes

jay added 2 commits January 9, 2017 01:40

url: Fix parsing for when 'file' is the default protocol. draft2

5857a56

Squash this into draft1 - Support --proto-default file c:/foo/bar.txt - Support file://c:/foo/bar.txt Assisted-by: Anatol Belski

jay force-pushed the fix_file_default-protocol branch from 0136932 to 5857a56 Compare January 9, 2017 06:58

jay mentioned this pull request Jan 9, 2017

BC breaching behavior change in URL parsing with file:// scheme on Windows. #1187

Closed

bagder approved these changes Jan 10, 2017

View reviewed changes

url: Fix parsing for when 'file' is the default protocol. draft3

dd99723

Squash me into previous draft - Fail when a file:// drive letter is detected and not MSDOS/Windows.

jay force-pushed the fix_file_default-protocol branch from 2d18159 to dd99723 Compare January 11, 2017 20:49

jay closed this in 1d4202a Jan 12, 2017

jay deleted the fix_file_default-protocol branch January 12, 2017 20:41

lock bot locked as resolved and limited conversation to collaborators Jan 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

url: Fix parsing for when 'file' is the default protocol #1124

url: Fix parsing for when 'file' is the default protocol #1124

jay commented Nov 13, 2016

mention-bot commented Nov 13, 2016

bagder Nov 13, 2016

bagder Nov 13, 2016

jay Nov 14, 2016

bagder Nov 13, 2016

bagder Nov 13, 2016

jay commented Dec 6, 2016

jay commented Jan 9, 2017

weltling commented Jan 9, 2017

bagder left a comment

jay commented Jan 11, 2017 •

edited

bagder commented Jan 11, 2017

jay commented Jan 11, 2017

bagder commented Jan 12, 2017

url: Fix parsing for when 'file' is the default protocol #1124

url: Fix parsing for when 'file' is the default protocol #1124

Conversation

jay commented Nov 13, 2016

mention-bot commented Nov 13, 2016

bagder Nov 13, 2016

Choose a reason for hiding this comment

bagder Nov 13, 2016

Choose a reason for hiding this comment

jay Nov 14, 2016

Choose a reason for hiding this comment

bagder Nov 13, 2016

Choose a reason for hiding this comment

bagder Nov 13, 2016

Choose a reason for hiding this comment

jay commented Dec 6, 2016

jay commented Jan 9, 2017

weltling commented Jan 9, 2017

bagder left a comment

Choose a reason for hiding this comment

jay commented Jan 11, 2017 • edited

bagder commented Jan 11, 2017

jay commented Jan 11, 2017

bagder commented Jan 12, 2017

jay commented Jan 11, 2017 •

edited