cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: curl - resolve http requst on terminal but not in my program

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Thu, 11 Sep 2008 07:33:28 -0700

On Thu, Sep 11, 2008 at 05:31:37PM +0530, Sachin Gole wrote:
> hi,
>
> I tried following url on my terminal as
>
> :>curl "http://mt1.google.com/mt?x=92&y=200&zoom=8" -o "file1"
>
> it downloads file. before that i have already set http proxy.

What do you mean by "i have already set http proxy"?

> now i am tring same url in my program using libcurl functions.
>
> it gives me following error.
>
> -------------------------------------------------------------------------------------------
>
> * About to connect() to proxy 172.18.1.16 port 80 (#0)
> * Trying 172.18.1.16... * connected
> * Connected to 172.18.1.16 (172.18.1.16) port 80 (#0)
> > GET http://mt1.google.com/mt?x=92&y=200&zoom=8 HTTP/1.1
> Host: mt1.google.com
> Pragma: no-cache
> Accept: */*
> Proxy-Connection: Keep-Alive
>
> < HTTP/1.1 403 Forbidden
> < Date: Thu, 11 Sep 2008 11:37:00 GMT
> < Content-Type: text/html
> < Server: GFE/1.3
> < Content-length: 1721
> < Proxy-Connection: Keep-Alive
> < Connection: Keep-Alive
> <
> * Connection #0 to host 172.18.1.16 left intact
> ----------------------------------------------------------------------------------------------
>
> what i am thinking if curl on terminal can download file, then my program
> should do.
>
> correct me if i am wrong.
>
> following seeting i am doing
>
> curl_global_init(CURL_GLOBAL_ALL);
> curl = curl_easy_init ();
> curlinfo = curl_version_info(CURLVERSION_NOW);
> curl_easy_setopt(curl,CURLOPT_VERBOSE,TRUE);
> curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1);
> curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
> curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 10);
> ret_code = curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
> printf("\n curl_easy_setopt [ret_code] : %d\n",ret_code);
> curl_easy_setopt(curl, CURLOPT_PROXYTYPE,CURLPROXY_HTTP);
> curl_easy_setopt(curl, CURLOPT_PROXY, "http://172.18.1.16:80");
> curl_easy_setopt(curl, CURLOPT_DNS_CACHE_TIMEOUT, 60 * 5); // 5 minutes
> curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);
> curl_easy_setopt(curl, CURLOPT_FILE,fp);
>
> what parameters i am missing to add.

Your original curl command-line is about the most simple you could imagine.
The only complication is setting the proxy, and you've left out that
information. Google is very sensitive about the requests it gets and rejects
anything that it thinks is a spider. The other thing I see missing from
your log is a User-Agent, and that's definitely something that could cause
problems in this case. Try using the -v option on the curl command-line and
compare the request that curl sends with that sent by your program.

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2008-09-11