Hi all, I am really close to getting what I need from the web page( This is in windows and from a dos box) If I run the command pasted into a Dos box, I get the results that I need, But If I run it from a .bat file, I do not get the results. Here is the command that I am using.: curl -Lo mee -c cookies.txt -b cookies.txt --dump-header headers.txt -A "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729)" "http://local.live.com/default.aspx?v=2&rtp=adr.6200%20South%20Main%20Street,Ashtabula,Ohio,~adr.4338%20Lake%20Road%20West,Ashtabula,Ohio~adr.2300%20West%2019%20street,Ashtabula,Ohio" -H "Content-type:" If you cut and paste this in a dos Box, you will get the correct results If you run this from a .bat file. you will not get the correct results. To see the difference all you need to do is load the "mee" file in an editor and look for the string "Trip:"(without quotes). The command run pasted into the Dos Box will show on the next line: class="distance">20.0 km If you run it from a .bat file. the next line will show class="distance">0.0 km Why the difference? any help at all would be appreciated. Thanks, Rob ---------------------------------------- From: curl-users-request@cool.haxx.se Sent: Tuesday, March 03, 2009 5:55 AM To: curl-users@cool.haxx.se Subject: curl-users Digest, Vol 43, Issue 5 Send curl-users mailing list submissions to curl-users@cool.haxx.se To subscribe or unsubscribe via the World Wide Web, visit http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users or, via email, send a message with subject or body 'help' to curl-users-request@cool.haxx.se You can reach the person managing the list at curl-users-owner@cool.haxx.se When replying, please edit your Subject line so it is more specific than "Re: Contents of curl-users digest..." Today's Topics: 1. re: curl-users Digest, Vol 43, Issue 3 (Robert Baker) ---------------------------------------------------------------------- Message: 1 Date: Tue, 3 Mar 2009 05:54:55 -0800 From: "Robert Baker" Subject: re: curl-users Digest, Vol 43, Issue 3 To: Message-ID: Content-Type: text/plain; charset="iso-8859-1" Hello Daniel Here is the command that I am using: curl.exe -v -L -b cookies.txt -c cookies.txt "http://local.live.com/default.aspx?v=2&rtp=adr.6200%20S%20Main%20street,Ashtabula,OH~adr.4338%20Lake%20Road%20W,Ashtabula,OH" I am using 7.19.3-ssl for Windows Rob ---------------------------------------- From: curl-users-request@cool.haxx.se Sent: Tuesday, March 03, 2009 3:01 AM To: curl-users@cool.haxx.se Subject: curl-users Digest, Vol 43, Issue 3 Send curl-users mailing list submissions to curl-users@cool.haxx.se To subscribe or unsubscribe via the World Wide Web, visit http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users or, via email, send a message with subject or body 'help' to curl-users-request@cool.haxx.se You can reach the person managing the list at curl-users-owner@cool.haxx.se When replying, please edit your Subject line so it is more specific than "Re: Contents of curl-users digest..." Today's Topics: 1. Re: how to use --proxy-negotiate, exactly? (Daniel Stenberg) 2. Re: Parameters in CURLOPT_WRITEFUNCTION: (Honey Dont) 3. Re: Trying to download a filled in webpage (Robert Baker) 4. ANNOUNCE: curl and libcurl 7.19.4 (Daniel Stenberg) 5. [Security Adviosory] libcurl Arbitrary File Access (Daniel Stenberg) 6. Re: Parameters in CURLOPT_WRITEFUNCTION: (Dan Fandrich) 7. Re: Trying to download a filled in webpage (Daniel Stenberg) ---------------------------------------------------------------------- Message: 1 Date: Mon, 2 Mar 2009 14:02:21 +0100 (CET) From: Daniel Stenberg Subject: Re: how to use --proxy-negotiate, exactly? To: libcurl hacking Cc: the curl tool Message-ID: Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII On Mon, 2 Mar 2009, Brian J. Murrell wrote: This thread is now taken over to the curl-library list from curl-users. The thread so far can be viewed here => http://curl.haxx.se/mail/archive-2009-02/index.html#90 >> Assuming we can do that check for both HTTP@ and KHTTP@ we might still >> be able to get away with the single bit. > > Indeed. I had thought about this a few hours ago myself. Keeping it > simple, we could change up the: > > auth="GSS-Negotiate"; > if(!negdata->context) > /* if explicitly selected it isn't inited at this point */ > ret = Curl_input_negotiate(conn, proxy, auth); > > with: > > auth="GSS-Negotiate"; > if(!negdata->context) > /* if explicitly selected it isn't inited at this point */ > if ((ret = Curl_input_negotiate(conn, proxy, auth)) < 0) { > auth="Negotiate"; > ret = Curl_input_negotiate(conn, proxy, auth); > > } > > in addition to a "neg_ctx->server_name = NULL;" in the "if(GSS_ERROR > (major_status))" condition at the end of Curl_input_negotiate() so that > subsequent calls will fall into get_gss_name(); > > The only side effect is the printing of a: > > * gss_init_sec_context() failed: : Server not found in Kerberos database Right. I think the logic that tries both versions should rather be moved into the Curl_input_negotiate() function and that could then avoid doing that fail output while "probing". >> If not, I don't see any reason why shouldn't add a bit if it makes or >> allows (lib)curl to behave better. > > Great. We seem to be on the same page then. I'd surely like to avoid it if > possible, but am willing to accept another bit if no other clean way can be > found. Well, if we exclude SPNEGO for a while (which isn't even certain to still be working afaik) and we do the try-both approach I think we're still functional with the current single bit. Are you up to provide an patch based on what we've discussed so far? -- / daniel.haxx.se ------------------------------ Message: 2 Date: Mon, 2 Mar 2009 08:00:11 -0800 (PST) From: Honey Dont Subject: Re: Parameters in CURLOPT_WRITEFUNCTION: To: curl-users@cool.haxx.se Message-ID: <585617.73067.qm@web51406.mail.re2.yahoo.com> Content-Type: text/plain; charset=us-ascii --- On Mon, 3/2/09, Daniel Stenberg wrote: > > You're obviously using the library so I would strongly > recommend that you use the curl-library mailing list > instead! > Whoops. Sorry. Wasn't aware of the difference. I have put in for a subscription to the curl-library ml. > It doesn't take anything particular into > "account", it just tells you the size of the data > and it uses the same argument set that fwrite() offers which > is why it uses two arguments to specify the size. > Okay. Thanks for this. It makes the function a lot more clear now. But, I can't think of a situation where the size would be more than 1 for what curl is doing. ------------------------------ Message: 3 Date: Mon, 2 Mar 2009 13:57:32 -0800 From: "Robert Baker" Subject: Re: Trying to download a filled in webpage To: , Message-ID: Content-Type: text/plain; charset="iso-8859-1" Hi Pedro, When I turn on the debugging, I noticed that in the "Issue another request" it shows the correct location, But then it displays the "GET/" with /help/en-ca/browsernotsupported.htm?http%3a%2f%2f with the rest of the correct path to go to. Do you know of a way to get around this? Thanks, Rob ---------------------------------------- From: Pedro Sent: Friday, February 27, 2009 7:07 PM To: rbaker@bakersoftwaretechnologies.com, the curl tool Subject: Re: Trying to download a filled in webpage > > Newbie to curl. be gentle > I am trying to download a webpage that has been filled in automatically. > What I have done is sent the correct information to the webpage Which > in turn displays the result that I want. > But when I run this command > curl.exe -o meme > "http://local.live.com/default.aspx?v=2&rtp=adr.6200%20S%20Main%20street,Ashtabula,OH~adr.4338%20Lake%20Road%20W,Ashtabula,OH" > > > I get the following error, > > > Object MovedThis document may be found > HREF="http://maps.live.com/default.aspx?v=2&rtp=adr.62000S0Main0street,Ashtabula,OH~adr.43380Lake0Road0W,Ashtabula,OH">here > > > > any help would be greatly appreciated > > Thanks, > Rob Had a similar problem once. Don't know if it works for you, but try -L -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 4 Date: Tue, 3 Mar 2009 00:11:37 +0100 (CET) From: Daniel Stenberg Subject: ANNOUNCE: curl and libcurl 7.19.4 To: Curl Announce -- curl users , curl-announce@cool.haxx.se Message-ID: Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Hi again It is with pleasure and some agony that I present this fresh release, as it comes with good things but one of those good things is a securty related fix. See further below for details, the advisory will also be posted separately in a few minutes. Get the new curl and libcurl packages here: http://curl.haxx.se/download.html Curl and libcurl 7.19.4 Public curl releases: 110 Command line options: 132 curl_easy_setopt() options: 163 Public functions in libcurl: 58 Known libcurl bindings: 38 Contributors: 700 This release includes the following security-related fix: o CVE-2009-0037 with the curl advisory here: http://curl.haxx.se/docs/adv_20090303.html This release includes the following changes: o Added CURLOPT_NOPROXY and the corresponding --noproxy o the OpenSSL-specific code disables TICKET (rfc5077) which is enabled by default in openssl 0.9.8j o Added CURLOPT_TFTP_BLKSIZE o Added CURLOPT_SOCKS5_GSSAPI_SERVICE and CURLOPT_SOCKS5_GSSAPI_NEC - with the corresponding curl options --socks5-gssapi-service and --socks5-gssapi-nec o Improved IPv6 support when built with with c-ares >= 1.6.1 o Added CURLPROXY_HTTP_1_0 and --proxy1.0 o Added docs/libcurl/symbols-in-versions o Added CURLINFO_CONDITION_UNMET o Added support for Digest and NTLM authentication using GnuTLS o CURLOPT_FTP_CREATE_MISSING_DIRS can now be set to 2 to retry the CWD even when MKD fails o GnuTLS initing moved to curl_global_init() o Added CURLOPT_REDIR_PROTOCOLS and CURLOPT_PROTOCOLS This release includes the following bugfixes: o missing ssh.obj in VS makefiles o FTP ;type=i URLs now work with CURLOPT_PROXY_TRANSFER_MODE in Turkish locale o realms with quoted quotation marks in HTTP Digest headers o VC9 makefiles are now really included o multi interface memory leak with CURLMOPT_MAXCONNECTS set o CURLINFO_CONTENT_LENGTH_DOWNLOAD size from file:// "transfers" with CURLOPT_NOBODY set true o memory leak on some libz errors for content encodings o NSS-enabled build is repaired o superfluous wait in SFTP downloads removed o FTP with the multi interface no longer kills the control connection as easily on transfer failures o compilation halting when using VS2008 to build a Windows 2000 target o ease creation of libcurl Mac OS X Framework o CURLINFO_CONTENT_LENGTH_DOWNLOAD and CURLINFO_CONTENT_LENGTH_UPLOAD are -1 if unknown o Negotiate proxy authentication o CURLOPT_INTERFACE and CURLOPT_LOCALPORT used together This release includes the following known bugs: o see docs/KNOWN_BUGS (http://curl.haxx.se/docs/knownbugs.html) This release would not have looked like this without help, code, reports and advice from friends like these: Lisa Xu, Daniel Fandrich, Craig A West, Alexey Borzov, Sharad Gupta, Peter Sylvester, Chad Monroe, Markus Moeller, Yang Tse, Scott Cantor, Patrick Scott, Hidemoto Nakada, Jocelyn Jaubert, Andre Guibert de Bruet, Kamil Dudka, Patrik Thunstrom, Linus Nielsen Feltzing, Mark Incley, Daniel Johnson, James Cheng, Brian J. Murrell, Senthil Raja Velu, Markus Koetter, David Kierznowski, Michal Marek Thanks! (and sorry if I forgot to mention someone) -- / daniel.haxx.se ------------------------------ Message: 5 Date: Tue, 3 Mar 2009 00:13:15 +0100 (CET) From: Daniel Stenberg Subject: [Security Adviosory] libcurl Arbitrary File Access To: curl users , libcurl hacking Message-ID: Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII libcurl Arbitrary File Access ============================= Project cURL Security Advisory, March 3rd 2009 http://curl.haxx.se/docs/security.html 1. VULNERABILITY When told to follow a "redirect" automatically, libcurl does not question the new target URL but will follow to any new URL that it understands. As libcurl supports FILE:// URLs, a rogue server can thus "trick" a libcurl-using application to read a local file instead of the remote one. This is a problem, for example, when the application is running on a server and is written to upload or to otherwise provide the transfered data to a user, to another server or to another application etc, as it can be used to expose local files it was not meant to. The problem can also be exploited for uploading, if the rogue server redirects the client to a local file and thus it would (over)write a local file instead of sending it to the server. libcurl compiled to support SCP can get tricked to get a file using embedded semicolons, which can lead to execution of commands on the given server. "Location: scp://name:passwd@host/a'``;date >/tmp/test``;'". Files on servers other than the one running libcurl are also accessible when credentials for those servers are stored in the .netrc file of the user running libcurl. This is most common for FTP servers, but can occur with any protocol supported by libcurl. Files on remote SSH servers are also accessible when the user has an unencrypted SSH key. There is no known exploit at the time of this writing. The Common Vulnerabilities and Exposures (CVE) project has assigned the name CVE-2009-0037 to this issue. 2. AFFECTED VERSIONS Affected versions: curl and libcurl 5.11(!) to and including 7.19.3 Not affected versions: curl and libcurl 5.10 and earlier, 7.19.4 and later Also note that (lib)curl is used by many applications, and not always advertised as such. 3. THE SOLUTION libcurl 7.19.4 introduces a new option called CURLOPT_REDIR_PROTOCOLS, which applications can use to tell libcurl what target protocols automatic redirect followings are allowed to use. This will by default exclude FILE and SCP URLs. 4. RECOMMENDATIONS We suggest you take one of the following actions immediately, in order of preference: A - Upgrade to curl and libcurl 7.19.4 B - Apply the suitable patch and rebuild For current CVS HEAD: http://curl.haxx.se/CVE-2009-0037/curl-CVSHEAD-CVE-2009-0037.patch For curl 7.19.0: http://curl.haxx.se/CVE-2009-0037/curl-7.19.0-CVE-2009-0037.patch For curl 7.18.2: http://curl.haxx.se/CVE-2009-0037/curl-7.18.2-CVE-2009-0037.patch For curl 7.18.1: http://curl.haxx.se/CVE-2009-0037/curl-7.18.1-CVE-2009-0037.patch For curl 7.16.4: http://curl.haxx.se/CVE-2009-0037/curl-7.16.4-CVE-2009-0037.patch For curl 7.15.1: http://curl.haxx.se/CVE-2009-0037/curl-7.15.1-CVE-2009-0037.patch For curl 7.11.0: http://curl.haxx.se/CVE-2009-0037/curl-7.11.0-CVE-2009-0037.patch C - Disable automatic redirection following in your application and do the logic "manually" instead. 5. TIME LINE We were notified by David Kierznowski on Feb 6th, 2009. We discussed solutions and a first patch was written and tested on Feb 11th. Vendor-sec was contacted on Feb 12, 2009. We agreed on and coordinated the synchronous disclosure of this problem together with the curl 7.19.4 release. curl 7.19.4 was released on March 3 2009, just before this flaw was publicly disclosed. 6. CREDITS Reported to us by David Kierznowski. Thanks a lot! Daniel Fandrich researched the issue and helped with the fix. Michal Marek brought the SCP side of this issue and did a bunch of the patch backports. Daniel Stenberg wrote the primary patch and this advisory. -- / daniel.haxx.se ------------------------------ Message: 6 Date: Mon, 2 Mar 2009 21:07:39 -0700 From: Dan Fandrich Subject: Re: Parameters in CURLOPT_WRITEFUNCTION: To: curl-users@cool.haxx.se Message-ID: <20090303040739.GA29795@coneharvesters.com> Content-Type: text/plain; charset=us-ascii On Mon, Mar 02, 2009 at 08:00:11AM -0800, Honey Dont wrote: > Okay. Thanks for this. It makes the function a lot more clear now. But, I can't think of a situation where the size would be more than 1 for what curl is doing. As of today, libcurl always sets it to 1. It's that way strictly to be compatible with fwrite. >>> Dan -- http://www.MoveAnnouncer.com The web change of address service Let webmasters know that your web site has moved ------------------------------ Message: 7 Date: Tue, 3 Mar 2009 09:08:11 +0100 (CET) From: Daniel Stenberg Subject: Re: Trying to download a filled in webpage To: the curl tool Message-ID: Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed On Mon, 2 Mar 2009, Robert Baker wrote: > When I turn on the debugging, I noticed that in the "Issue another request" > it shows the correct location, But then it displays the "GET/" with > /help/en-ca/browsernotsupported.htm?http%3a%2f%2f with the rest of the > correct path to go to. Do you know of a way to get around this? Can you show us a complete command line against a public URL that shows this? What curl version on what platform are you using? -- / daniel.haxx.se ------------------------------ _______________________________________________ curl-users mailing list curl-users@cool.haxx.se http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users End of curl-users Digest, Vol 43, Issue 3 ***************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ _______________________________________________ curl-users mailing list curl-users@cool.haxx.se http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users End of curl-users Digest, Vol 43, Issue 5 *****************************************

-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-03-05