cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: hanging URL

From: Fred Noz <FNoz_at_siac.com>
Date: Thu, 21 Sep 2000 11:45:23 -0400

Strictly according to the TCP and HTTP protocols it shouldn't
matter whether the http request is broken into multiple writes
or is done all in one write. However, in reality it sometimes
does. Since we can't fix all the broken Web servers and firewalls
in the world, I think it is best if curl uses what works. Also,
I think it is best if curl emulates what common web browsers like
Netscape and IE actually do. It seems that Netscape does the
entire request in one write (judging from other people's posts).

Just thought I would put my two cents in...

        - Fred Noz
        New York City, USA

Rich Gray wrote:
>
> This makes NO sense at all. TCP is a reliable protocol. Things don't (err
> shouln't) get lost. It should not matter one iota whether the sender does:
>
> send("a"); send("b"); send("c") or
> send("abc")
>
> There is no guarantee in either case that the receiver will get all the
> characters at once. It might get them in one, two or three receives.
> Because TCP is a reliable protocol, the receiver will get them, or the
> connection will fail. Perhaps what we have here is a VERY broken firewall
> that is failing to deal with multiple packets??
>
> Rich
>
> > -----Original Message-----
> > From: Priebe, Jason [mailto:priebe_at_wral.com]
> > Sent: Thursday, September 21, 2000 9:17 AM
> > To: 'curl_at_contactor.se'
> > Subject: RE: hanging URL
> >
> >
> > Sure enough -- that's what did it. I modified the sendf() call
> > on line 318 of http.c to look like this:
> >
> > sendf(data->firstsocket, data,
> > "%s " /* GET/HEAD/POST/PUT */
> > "%s HTTP/1.0\r\n" /* path */
> > "%s" /* proxyuserpwd */
> > "%s" /* userpwd */
> > "%s" /* range */
> > "%s" /* user agent */
> > "%s" /* cookie */
> > "%s" /* host */
> > "%s" /* pragma */
> > "%s" /* accept */
> > "%s\015\012", /* referer */
> >
> > Compiled the program and ran:
> >
> > curl http://www.cbs.com/
> >
> > Worked like a charm. Is this something that's worth implementing
> > in the distributed version? Is it possible to assemble the request
> > string in a buffer and then send it all at once?
> >
> > Jason Priebe
> > WRAL OnLine
> > http://www.wral-tv.com/
> >
> >
> > > -----Original Message-----
> > > From: Daniel Stenberg [mailto:daniel_at_haxx.se]
> > > Sent: Wednesday, September 20, 2000 5:10 p.m.
> > > To: Curl Mailinglist
> > > Subject: RE: hanging URL
> > >
> > >
> > > On Wed, 20 Sep 2000, Priebe, Jason wrote:
> > >
> > > > The only obvious difference I can see so far is that the
> > > first two tools
> > > > sent hex 0x0D 0x0A 0x0D 0x0A to end their requests, and the
> > > second two
> > > > only sent 0x0D 0x0A.
> > > >
> > > > This is strange, since I definitely hit "return" twice
> > > within my telnet
> > > > session. And I would think that curl would send the CR-LF
> > > combination
> > > > twice.
> > >
> > > Try this:
> > >
> > > Curl normally sends its entire request in two write()s. The
> > > first write() is
> > > everything except the last CR-LF. Then it sends the final
> > > CR-LF separately.
> > > The reason for this, is that this way enables curl to fill in
> > > custom headers,
> > > cookies and whatever in between the write()s.
> > >
> > > So what you see is the first package. The second package
> > > should come later.
> > > However, if there's a TCP problem somewhere along the "road",
> > > it wouldn't
> > > surprise me if you can get this to work by sending everything
> > > in one single
> > > shot. (You only need to modify the curl source code slightly
> > > to test this
> > > theory -- of course you'll disable a few features but this is
> > > for testing.)
> > >
> > > --
> > > Daniel Stenberg -- curl project maintainer -- http://curl.haxx.se/
> > >
> >
Received on 2000-09-21