cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: FW: HTTP GET using easycurl - HTTP chunks in HTTP response.

From: Rangoli Mathur <Rangoli.Mathur_at_wnco.com>
Date: Wed, 13 Dec 2006 17:08:54 -0600

Here are details of what I was trying:

The URL I am using is http://www.yahoo.com
Code snippet:
//////////////////////////////////////////

  curl_global_init(CURL_GLOBAL_ALL );
  curlHandle = curl_easy_init();
  curl_easy_setopt(curlHandle, CURLOPT_NOPROGRESS, 1);
  curl_easy_setopt(curlHandle, CURLOPT_WRITE_FUNCTION, write_data);
  curl_east_setopt(curlHandle, CURLOPT_HEADER, 1);
  curl_easy_setopt(curlHandle, CURLOPT_URL, "http://www.yahoo.com");
  //Trying with http 1.0
  curl_easy_setopt(curlHandle, CURLOPT_HTTP_VERSION,
CURL_HTTP_VERSION_1_0);
  int i = curl_easy_perform(curlHandle);

//////////////////////////////////////////////

I tried setting the HTTP version to 1.0 and I see all chunks being
coagulated.
If I don't set the HTTP version, in which case it uses HTTP 1.1, I see
([http-chunk-length] [http-chunk])* in the response.

MOREOVER, for the 1st chunk after HTTP Header, I don't see the
http-chunk-length at all. I see http-chunk-length for subsequent chunks.

//Excerpts of the response I receive from libcurl are:
////////////////////////////////////////////////////
HTTP/1.1 200 OK^M
Date: Wed, 13 Dec 2006 22:54:34 GMT^M
P3p: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR
ADM DEV TAI
PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY
ONL UNI PUR
FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"^M
Cache-Control: private^M
Vary: User-Agent^M
Content-Type: text/html; charset=utf-8^M
Via: 1.1 ics_server.swacorp.com (ICS 2.3.0.0.16)^M
Proxy-Connection: keep-alive^M
Transfer-Encoding: chunked^M
^M
<html><head>
<title>Yahoo!</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="PICS-Label" content='(PICS-1.1
"http://www.icra.org/ratingsv02.h
tml" l r (cz 1 lz 1 nz 1 oz 1 vz 1) gen true for "http://www.yahoo.com"
r (cz 1 lz
1 nz 1 oz 1 vz 1) "http://www.rsac.org/ratingsv01.html" l r (n 0 s 0 v
0 l 0) gen
true for "http://www.yahoo.com" r (n 0 s 0 v 0 l 0))'>
<base
href="http://www.yahoo.com/_ylh=X3oDMTFkOWw2OHFqBF9TAzI3MTYxNDkEcGlkAzEx
NjYw
NDk2NzAEdGVzdAMwBHRtcGwDaW5kZXgtdA--/" target="_top">
<style type="text/css">
a{color:#16387c;}
a:link,a:visited{text-decoration:none;}
a:hover{text-decoration:underline;}
</style>
<style type="text/css" media="all">
#p{width:310px;}
form{margin:0;}
</style>
</head>
<body link="#16387c" vlink="#16387c">
<center>
<table cellpadding="0" cellspacing="0" border="0" bgcolor="#EEF3F6"
width="760">
<tr><td colspan="3">
<table cellpadding="0" cellspacing="0" border="0" width="0"><tr><td
width="0" heig
ht="7"><spacer type="block" width="0" height="7"></td></tr></table>
</td></tr>
<tr>
<td width="10" height="^M
5B4^M
0" rowspan="2"><spacer type="block" width="10" height="0"></td>
<td height="56" valign="top">
<img src="http://us.a1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"
width="232" height="
44" alt="Yahoo!" title="Yahoo">
</td>
......
.....
.....
5B4^M
/fdbg.gif" nowrap="nowrap"><font face="arial" size="-1"><a
href="r/m1">My Mail</a>
</font></td></tr></table></td>
<td width="5" height="1" bgcolor="#dde8ea"
background="http://us.i1.yimg.com/us.yi
mg.com/i/ww/tbl/fdbg.gif"><spacer type="block" width="5"
height="1"></td>
<td width="1" height="100%" bgcolor="#586B7A" nowrap><spacer
type="block" width="1
" height="100%"></td>
</tr><tr>
<td width="100%" height="1" bgcolor="#586B7A" colspan="5" nowrap><spacer
type="blo
ck" width="100%" height="1"></td>
</tr></table>
</td>
</tr></table>
</td>
</tr>
<tr><td colspan="3">
<table cellpadding="0" cellspacing="0" border="0" width="0"><tr><td
width="0" heig
ht="9"><spacer type="block" width="0" height="9"></td></tr></table>
</td></tr>
</table>
<table cellpadding="0" cellspacing="0" border="0" width="0"><tr><td
width="0" heig
ht="5"><spacer type="block" width="0" height="5"></td></tr></table>

<table cellpadding="0" cellspacing="0" border="0" width="760">
<tr>
<td align="right">
<table cellpadding="0" cellspacing="0" border="0">
<tr>
<td><font face="arial" size="-1"><a href="r/l6"><b>Sign
In</b></a></font></td>
<td width="5" height="0"><spacer type="block" width="5" height="0"></td>
<td bgcolor="#888888" width="2"><spacer type="block" width="2"
height="0"></td>
<td width="5" height="0"><spacer type="block" width="5" height="0"></td>
<td><font face="arial" size="-1"><a href="r/m7"><b>Sign
Up</b></a></font></td>
<td width="10" height="0"><spacer type="block"^M
498^M
........
.........
........
AE^M
y</a>.
</font></td></tr></table>
</td></tr></table>
</center>
</body>
</html>

^M
0^M
^M
y</a>.
</font></td></tr></table>
</td></tr></table>
</center>
</body>
</html>

^M
0^M
^M

> -----Original Message-----
> From: curl-library-bounces_at_cool.haxx.se
> [mailto:curl-library-bounces_at_cool.haxx.se] On Behalf Of Dan Fandrich
> Sent: Wednesday, December 13, 2006 4:30 PM
> To: libcurl development
> Subject: Re: FW: HTTP GET using easycurl - HTTP chunks in
> HTTP response.
>
> On Wed, Dec 13, 2006 at 02:47:50PM -0600, Rangoli Mathur wrote:
> > If I have defined a function: write_data and I have used
> > Curl_easy_setopt(curlHandle, CURLOPT_WRITE_FUNCTION, write_data).
> >
> > For HTTP1.1 chunked transfer encoding, Does the write_data function
> > get called once for every http chunk there is?
>
> Not necessarily. You shouldn't rely on that behaviour, even
> it it appears to be the case.
>
> > If so, for the body, can I safely assume that every write
> would begin
> > with the number of bytes to read in hex, followed by those
> many bytes?
>
> libcurl should be taking care of un-chunking the data for
> you. If not, then there is likely a problem with the web
> server, or, less likely, libcurl itself. Send us a debug log
> of the transaction where you're seeing this if you'd like
> more details.
>
> >>> Dan
> --
> http://www.MoveAnnouncer.com The web change of
> address service
> Let webmasters know that your web site has moved
>
Received on 2006-12-14

This message: [ Message body ]
Next message: Dan Fandrich: "Re: How to add value to the base URL"
Previous message: Jeff Pohlmeyer: "Re: How to add value to the base URL"
Maybe in reply to: Rangoli Mathur: "FW: HTTP GET using easycurl - HTTP chunks in HTTP response."
Next in thread: Dan Fandrich: "Re: FW: HTTP GET using easycurl - HTTP chunks in HTTP response."
Reply: Dan Fandrich: "Re: FW: HTTP GET using easycurl - HTTP chunks in HTTP response."

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]