cURL / Mailing Lists / curl-users / Single Mail

curl-users

RE: cURL and POST Large Amount of Data

From: Samantha Savvakis <samantha_at_cfs.net.au>
Date: Thu, 22 Aug 2002 08:51:51 +1000

Hi,

Thanks for replying.

Cris's suggestion did fix my problem. I'm writing to a temp file and
using the --data-binary option and setting the Content-Type header.

The size of the data does vary and can be up to 2MB in size. I know it's
crazy to be sending this amount of data around, but it's a requirement
unfortunately from clients to be sending this information around.

About the PHP exec buffer, I did try to up the memory parameters in the
PHP ini file to see if that made a difference.

I was able to get this data sending using fsockopen and PHP had no
problem sending this data using fputs to the socket. But it's easier to
use cURL as it strips out all the headers and any tailing crap from the
received message and only returns me to the data which is great.

Anyway, I've finally resolved my problem using the --data-binary option
from the example article that Cris did post. Thankyou for your tips, I
will be sure to always keep them in mind.

Thanks,
Sam

> -----Original Message-----
> From: curl-users-admin_at_lists.sourceforge.net
> [mailto:curl-users-admin_at_lists.sourceforge.net] On Behalf Of
> rmitchell_at_eds.com
> Sent: Wednesday, 21 August 2002 8:34 PM
> To: Samantha Savvakis
> Cc: curl-users_at_lists.sourceforge.net
> Subject: Re: cURL and POST Large Amount of Data
>
>
> Sam,
>
> If Cris's suggestion doesn't fix it for you, you might try
> altering the options just a tad:
>
> /usr/local/bin/curl -m 600 -d "$strXML" -L -s -S
> http://myurl.com/webpage.php
>
> The curl manpage shows all the options first, followed by the
> url(s) you're getting/posting, unless you prefix the url(s)
> with '--url'. The '-s', which you already have, tells curl
> to be silent, but that also disables error messages - adding
> '-S' re-enables error messages, should there be any.
>
> Are you passing a string array to the php exec call for it to
> return any output generated? If not, you might try it.
> Alternatively, toss in a few more options:
>
> -D xxxx.head -o xxxx.out --stderr xxxx.err -v
>
> That should get you any returned headers (-D) in xxxx.head,
> any regular output (-o) in xxxx.out and any error output
> (--stderr) in xxxx.err. The '-v' makes it verbose, so you
> get all the protocol startup, connections, redirects, etc,
> dumped into the stderr file.
>
> Cris suggested using single-quotes around your data variable.
> That would stop the shell doing something silly with it,
> such as expanding *'s to all the files in the directory, etc.
>
> Cris also suggested using some kind of shell-escape function
> to escape any weird characters in your data variable.
> Probably better than that is PHP's 'urlencode' or even
> 'rawurlencode' function, because the receiving web server may
> react unpredictably if things like spaces, &'s and ?'s show
> up in its input stream. Curl doesn't fix those up for you...
> If you already do that, apologies...:)
>
> Lastly, just how big is the chunk of XML you're trying to
> send ? It may be overflowing php's exec buffer, or possibly
> the shell's command line buffer. If I'm wrong on that
> possibility, someone please correct me...:)
>
> Just few thoughts...
>
> Ralph Mitchell
>
>
> Samantha Savvakis wrote:
>
> > HI,
> >
> > I'm using the cURL binary to perform a HTTP POST to a web
> page within
> > PHP Code. I am sending XML data to this web page.
> >
> > The issue I've come up against if the size of the data that
> I'm trying
> > to post and it looks like cURL is crapping out.
> >
> > This is the command line that I'm currently using:
> >
> > /usr/local/bin/curl -m 600 -d "$strXML"
> http://myurl.com/webpage.php
> > -L -s
> >
> > $strXML - variable
> containing the XML data. I run this command using
> > "exec" in PHP to call curl.
> >
> > The return code I receive is: 127. This means a fatal error has
> > occured with cURL but that's all I know.
> >
> > I have read the detail about using the -F option. I do not want to
> > POST the "file" as such, but rather the "DATA" in the file
> is I create
> > a file of the XML. The manpage for cURL states:
> >
> > "-F : To just get the content part from a file, prefix the
> file name
> > with the letter <. The difference between @ and < is then
> that @ makes
> > a file get attached in the post as a file upload, while the
> < makes a
> > text field and just get the contents for that text field from a
> > file."
> >
> > So I've tried going:
> >
> > /usr/local/bin/curl -m 600 -F </tmp/tempfile.xml
> > http://myurl.com/webpage.php -L -s
> >
> > /tmp/tempfile.xml - contains the XML data. This isn't working.I'm
> > getting a return code of "2" meaning: "Failed to initialize".
> >
> > I'm at a loss. I need to post this large amount of XML data to a
> > remote site and don't know how to get it there.
> >
> > Any ideas?
> >
> > Thanks,
> > Sam
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by: OSDN - Tired of that same
> old cell phone? Get a new here for FREE!
> https://www.inphonic.com/r.asp?r=sourceforge1> &refcode1=vs3390
>

-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone? Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390
Received on 2002-08-22