cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Why does curl use CRLF line terminators for its --dump-header option?

From: Tom Dignan <tom.dignan_at_gmail.com>
Date: Thu, 26 Apr 2012 06:24:21 -0400

@John

Thanks for straightening that out. Learned something new today.
-
On Thu, Apr 26, 2012 at 6:00 AM, <curl-users-request_at_cool.haxx.se> wrote:

> Send curl-users mailing list submissions to
> curl-users_at_cool.haxx.se
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
> or, via email, send a message with subject or body 'help' to
> curl-users-request_at_cool.haxx.se
>
> You can reach the person managing the list at
> curl-users-owner_at_cool.haxx.se
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of curl-users digest..."
>
>
> Today's Topics:
>
> 1. RE: Program goes into infinite state. (Bill Mercer)
> 2. Why does curl use CRLF line terminators for its --dump-header
> option? (Tom Dignan)
> 3. Re: Why does curl use CRLF line terminators for its
> --dump-header option? (John Meissen)
> 4. infinite -O option (Hans J. Albertsson)
> 5. Re: infinite -O option (Dan Fandrich)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 25 Apr 2012 14:24:34 +0000
> From: Bill Mercer <bmercer_at_nccer.org>
> To: the curl tool <curl-users_at_cool.haxx.se>
> Subject: RE: Program goes into infinite state.
> Message-ID: <395CF24427BDAA4EB14F06D9ED99C25822A1976B_at_MAIL.nccer.org>
> Content-Type: text/plain; charset="us-ascii"
>
> I'd suggest looking at this from the DD-WRT side.
>
> If the files you are trying to make available via FTP are at the root
> level of the flash drive, try moving them into a subfolder, and see if that
> helps. Since the flash drive is mounted at a point that corresponds to a
> real folder in the file system if you remove the drive, the folder is still
> there. Maybe a missing directory would trigger the error you need.
>
> Also, there's an option with DD-WRT to automatically run a script when a
> drive is mounted, you might be able to use that to your advantage.
>
>
> >-----Original Message-----
> >From: curl-users-bounces_at_cool.haxx.se [mailto:curl-users-
> >bounces_at_cool.haxx.se] On Behalf Of Yuvi yuvi
> >Sent: Tuesday, April 24, 2012 2:40 PM
> >To: curl-users_at_cool.haxx.se
> >Subject: Re: Program goes into infinite state.
> >
> >Yes,
> >
> >DD-WRT router act as a FTP or SFTP server, whose storage media is USB
> >storage.
> >
> >
> >On Tue, Apr 24, 2012 at 11:56 PM, Yuvi yuvi <testandroidos51_at_gmail.com>
> >wrote:
> >
> >
> > Ok,
> >
> > Let me describe you again whole scenario.
> > There is a DD-WRT Wifi- router (having Embedded Linux OS ) having
> >USB Storage, and we can configure router to act as a server, and can
> >access files using FTP & SFTP.
> > Problem occurs when while router is ON, and we remove pen drive
> >then physicaly it is removed but I think path still remains there,
> >correct me if I am worng.
> > In that case curl is not working.
> >
> > Let consider another scenario, when there is no USB storage
> >plugged in Router, and then I switch ON Router. In this case curl
> >doesn't find file and gives proper error.
> >
> > So, is there any work around for first scenario, also why TIME OUT
> >is not working in the first scenario ?
> >
> > I have already compiled latest version of curl & libssh, if some
> >one having very good experince on Linux, he can better understand that
> >what happens when we removed usb storage by pulling it out from linux
> >machine.
> >
> >
> >
> >
> >
> > On Mon, Apr 23, 2012 at 4:47 PM, Yuvi yuvi
> ><testandroidos51_at_gmail.com> wrote:
> >
> >
> > Hi,
> >
> > Following is the log taken using curl binary file :
> >
> >
> >
> > root_at_dm37x-evm:/# ./curl -u root:LinuxDev_ -v -O
> >sftp://192.168.10.1/mnt/package.QuipC
> >
> > * About to connect() to 192.168.10.1 port 22 (#0)
> >
> > * Trying 192.168.10.1...
> >
> > % Total % Received % Xferd Average Speed Time
> >Time Time Current
> >
> > Dload Upload Total
> >Spent Left Speed
> >
> > 0 0 0 0 0 0 0 0 --:--:--
> --:--
> >:-- --:--:-- 0* connected
> >
> > * Connected to 192.168.10.1 (192.168.10.1) port 22 (#0)
> >
> > * SFTP 0x2d2888 state change from SSH_STOP to SSH_INIT
> >
> > * SFTP 0x2d2888 state change from SSH_INIT to SSH_S_STARTUP
> >
> > 0 0 0 0 0 0 0 0 --:--:--
> --:--
> >:-- --:--:-- 0* SFTP 0x2d2888 state change from SSH_S_STARTUP to
> >SSH_HOSTKEY
> >
> > * SSH MD5 fingerprint: 0c7d6453b72e8862637a2abcf76ccf95
> >
> > * SSH host check: 0, key:
> >AAAAB3NzaC1yc2EAAAADAQABAAAAgwDNhToi7hEZP5Fm3LKaAxyBx75GCYUeuU4iX9B+yvj+
> >cBUxw0lV8CD+XR77jPWpgMbJPnk0ROJmch4zwj7E98q0VwukPUxFbTL6dC4W7DC3hzqlRPMl
> >IsfiNMdt7bJ6zk14RKL25yKeiPYE6TbCJF8niWkaPIRW52j7LoSbX/zmFEyb
> >
> > * SFTP 0x2d2888 state change from SSH_HOSTKEY to
> >SSH_AUTHLIST
> >
> > * SSH authentication methods available: publickey,password
> >
> > * SFTP 0x2d2888 state change from SSH_AUTHLIST to
> >SSH_AUTH_PKEY_INIT
> >
> > * Using ssh public key file /home/root/.ssh/id_dsa.pub
> >
> > * Using ssh private key file /home/root/.ssh/id_dsa
> >
> > * SFTP 0x2d2888 state change from SSH_AUTH_PKEY_INIT to
> >SSH_AUTH_PKEY
> >
> > * SSH public key authentication failed: Unable to open
> >public key file
> >
> > * SFTP 0x2d2888 state change from SSH_AUTH_PKEY to
> >SSH_AUTH_PASS_INIT
> >
> > * SFTP 0x2d2888 state change from SSH_AUTH_PASS_INIT to
> >SSH_AUTH_PASS
> >
> > * Initialized password authentication
> >
> > * SFTP 0x2d2888 state change from SSH_AUTH_PASS to
> >SSH_AUTH_DONE
> >
> > * Authentication complete
> >
> > 0 0 0 0 0 0 0 0 --:--:--
> >0:01:39 --:--:-- 0
> >
> >
> > After this " 0 0 0 0 0 0 0 0
> --
> >:--:-- 0:01:39 --:--:-- 0"
> > nothing happens I have to manually abort execution using
> >Ctrl + c, and even no core dump is generated.
> > Please let me know if there is any work around for this
> >situation.
> >
> >
> >
> >
> > On Wed, Apr 18, 2012 at 1:42 PM, Yuvi yuvi
> ><testandroidos51_at_gmail.com> wrote:
> >
> >
> > Hi,
> >
> > Yes the program goes into the infinite loop,
> Following
> >is the log, which I have taken using strace command, core dump is not
> >able to show anything.
> > I think there is an internal retry counter, can we
> set
> >that counter value by anyhow? please suggest me any solution.
> >
> >
> >
> > open("/dev/urandom", O_RDONLY|O_NONBLOCK|O_NOCTTY)
> = 4
> >
> > fstat64(4, {st_mode=S_IFCHR|0666,
> st_rdev=makedev(1,
> >9), ...}) = 0
> >
> > poll([{fd=4, events=POLLIN, revents=POLLIN}], 1,
> 10) =
> >1
> >
> > read(4,
> >"n9\314l\270+\3066\33\366T\3657\262\322<\231\373\\\22\177"..., 32) = 32
> >
> > close(4) = 0
> >
> > getuid32() = 0
> >
> > gettimeofday({1301020875, 599243}, NULL) = 0
> >
> > SYS_289(0x3, 0x293aa8, 0x288, 0x4000, 0x28f950) =
> 648
> >
> > gettimeofday({1301020875, 600494}, NULL) = 0
> >
> > SYS_291(0x3, 0x28fa80, 0x4000, 0x4000, 0x28f950) =
> 416
> >
> > gettimeofday({1301020875, 600799}, NULL) = 0
> >
> > SYS_289(0x3, 0x293aa8, 0x90, 0x4000, 0x28f950) =
> 144
> >
> > gettimeofday({1301020875, 628418}, NULL) = 0
> >
> > SYS_291(0x3, 0x28fa80, 0x4000, 0x4000, 0x28f950) =
> -1
> >EAGAIN (Resource temporarily unavailable)
> >
> > timer_delete(0x1) = 0
> >
> > timer_delete(0x1) = 0
> >
> > timer_delete(0x1) = 0
> >
> > timer_delete(0x1) = 0
> >
> > poll([{fd=3, events=POLLIN|POLLPRI,
> revents=POLLIN}],
> >1, 1000) = 1
> >
> > gettimeofday({1301020876, 106384}, NULL) = 0
> >
> > SYS_291(0x3, 0x28fa80, 0x4000, 0x4000, 0x28f950) =
> 464
> >
> > SYS_289(0x3, 0x293aa8, 0x10, 0x4000, 0x28f950) = 16
> >
> > gettimeofday({1301020876, 149292}, NULL) = 0
> >
> > SYS_289(0x3, 0x293aa8, 0x34, 0x4000, 0x28f950) = 52
> >
> > gettimeofday({1301020876, 150085}, NULL) = 0
> >
> > SYS_291(0x3, 0x28fa80, 0x4000, 0x4000, 0x28f950) =
> -1
> >EAGAIN (Resource temporarily unavailable)
> >
> >
> >
> >
> >
> > > Hi,
> > >
> > > I have wifi DD-WRT router that has USB
> Storage
> >support.
> > > I have configured that router to
> support SFTP
> >protocol.
> > > My problem is that during downloading if
> >somebody removed the USB storage
> > > then my sample program to download file
> goes
> >into the infinite state.
> >
> >
> > what is an infinite state? do you mean an
> >infinete loop?
> >
> > I suggest you to try to run your program
> in a
> >linux box. Configure your shell so as to produce core dumps. then run
> >the program, remove USB storage to cause infinite loop. Kill your
> >program with ctrl-c. Core will be dumped, then you are ready to go with
> >launching gdb and telling us what is wrong. Probably it is nothing but a
> >pourly written router SW.
> >
> > If the text above does not mean something
> to
> >you, I do not know how you can solve this problem w/o hiring someone to
> >do it for you.
> >
> > Thanks
> >
> >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Mon, 16 Apr 2012 22:47:49 +0200
> (CEST)
> > From: Daniel Stenberg <daniel_at_haxx.se>
> > To: the curl tool <curl-users_at_cool.haxx.se
> >
> > Subject: Re: SFTP : Problem with special
> >character # on target file
> > Message-ID:
> ><alpine.DEB.2.00.1204162247010.15394_at_tvnag.unkk.fr>
> > Content-Type: TEXT/PLAIN; charset=US-ASCII;
> >format=flowed
> >
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 26 Apr 2012 02:56:07 -0400
> From: Tom Dignan <tom.dignan_at_gmail.com>
> To: curl-users_at_cool.haxx.se
> Subject: Why does curl use CRLF line terminators for its --dump-header
> option?
> Message-ID:
> <CAEE2Dhbp9Nq28eiV6M54ykcf04abLdtJ=u=bOqrvgOHHN7+LSA_at_mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> So I was just using curl and I banged my head against the wall for about 30
> minutes trying to figure out why my script wasn't working properly, and
> then I realized that --dump-header uses CRLF line terminators. Any chance
> of getting this changed to unix line endings in a future release?
>
> Curl version:
>
> tom_at_tralfamadore ~/Fun $ curl --version
>
> curl 7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e
> zlib/1.2.3.4libidn/1.22 librtmp/2.3
> Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s
> rtmp rtsp smtp smtps telnet tftp
> Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
>
> This is the latest curl in Linux Mint 12.
>
> Proof of concept
>
> #!/bin/bash
>
> curl http://google.com --dump-header foo &>/dev/null
> file foo
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://cool.haxx.se/pipermail/curl-users/attachments/20120426/4a2a5309/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Thu, 26 Apr 2012 00:57:50 -0700
> From: "John Meissen" <john_at_meissen.org>
> To: the curl tool <curl-users_at_cool.haxx.se>
> Subject: Re: Why does curl use CRLF line terminators for its
> --dump-header option?
> Message-ID: <20120426075750.4EA8317F685_at_john>
> Content-Type: text/plain; charset=us-ascii
>
> tom.dignan_at_gmail.com said:
> > So I was just using curl and I banged my head against the wall for about
> 30
> > minutes trying to figure out why my script wasn't working properly, and
> then I
> > realized that --dump-header uses CRLF line terminators. Any chance of
> getting
> > this changed to unix line endings in a future release?
>
> This has nothing to do with curl. The HTTP protocol spec (RFC2616) says
> that
> headers are terminated with CRLF. Curl is only dumping what it's getting
> from
> the server.
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 26 Apr 2012 10:52:32 +0200
> From: "Hans J. Albertsson" <hans.j.albertsson_at_branneriet.se>
> To: curl-users_at_cool.haxx.se
> Subject: infinite -O option
> Message-ID: <4F990CD0.3080401_at_branneriet.se>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Sometimes curl -L will find multiple locations for file downloads in one
> go.
>
> curl -LO will only handle one download, the next spews binary at my
> terminal window.
>
> -LOO handles two...
>
> -LOOO handles three locations
>
> Is there a "cover-all-contingents" -O form? Sort of "do -O for any and
> all downloads?
>
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 26 Apr 2012 11:11:04 +0200
> From: Dan Fandrich <dan_at_coneharvesters.com>
> To: curl-users_at_cool.haxx.se
> Subject: Re: infinite -O option
> Message-ID: <20120426091104.GA24656_at_coneharvesters.com>
> Content-Type: text/plain; charset=us-ascii
>
> On Thu, Apr 26, 2012 at 10:52:32AM +0200, Hans J. Albertsson wrote:
> > Sometimes curl -L will find multiple locations for file downloads in one
> go.
> >
> > curl -LO will only handle one download, the next spews binary at my
> > terminal window.
> >
> > -LOO handles two...
> >
> > -LOOO handles three locations
> >
> > Is there a "cover-all-contingents" -O form? Sort of "do -O for any
> > and all downloads?
>
> http://curl.haxx.se/docs/manpage.html#--remote-name-all
>
> >>> Dan
>
>
> ------------------------------
>
> _______________________________________________
> curl-users mailing list
> curl-users_at_cool.haxx.se
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
>
>
> End of curl-users Digest, Vol 80, Issue 25
> ******************************************
>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-04-26