On Thu, Sep 10, 2009 at 10:04 PM, unleadedblues <unleadedblues_at_gmail.com>wrote:

> I am trying to navigate using curl through a free Web Proxy but I am not
> getting the same results I get when doing the same using a browser
> manually. To illustrate what I am trying to automate with curl just:
>
> - using a browser, navigate to www.incognitobrowse.info
> - enter URL in text input field
> - click Surf Now!
>
> That's it! But when I try to use curl I get another page from
> incognitobrowse which seems to suggest that my request wasn't ever
> passed on to the target URL.
>
> The code I am using is below. Can anyone give me a clue as to what I am
> doing wrong?
>
> ------ begin code ---------
>
> # Same User Agent String as my browser uses
> UA="Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.13)
> Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13"
>
> # Browse to index.html of Web Proxy
> curl -L -A "${UA}" http://www.incognitobrowse.info >
> www.incognitobrowse.info.html
>
> # Here I will eventually parse the page returned above to learn correct
> POST variables. For now I am just plugging them into the next line
>
> curl --max-redirs 150 -L --referer
> "http://www.incognitobrowse.info;auto" -A "${UA}" --data-urlencode
> "q=www.msn.com&hl[include_form]=on&hl[accept_cookies]=on&hl[show_images]=on&hl[show_referer]=on&hl[base64_encode]=on&hl[strip_meta]=on&hl[session_cookies]=on"
> http://www.incognitobrowse.info/index.php > anon_msn.html
>
> ------ end code ---------
>
> # Can someone please give me a clue as to what I am doing wrong?
>

I think you need to individually urlencode each variable. It certainly
seems to work when I try it like that. I suspect that the --data-urlencode
option only expects to operate on a single variable, so it encodes
everything after the first equal sign and passes it all as the "q" variable.

Ralph Mitchell

-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-09-11