GOT IT!!

Cheers everyone, you know I was stupid and had this:

curl_setopt($curl, CURLOPT_FOLLOWLOCATION, false);

instead of true, gee dont know why I changed it to false earlier.

Every single one of you have helped and thanks all! very very appreciated,
great!

On Mon, Nov 10, 2008 at 7:47 AM, Jo Anonymous <jojoanon_at_gmail.com> wrote:

> Here is a user asking the same as what I want:
>
> http://curl.haxx.se/mail/archive-2007-09/0040.html
>
>
>
>
> On Mon, Nov 10, 2008 at 7:27 AM, Jo Anonymous <jojoanon_at_gmail.com> wrote:
>
>> You are right about the redirect.
>>
>> I just realised, the actual file I am getting is from a redirect after the
>> login.
>>
>> What I meant by prompts is that when I login, I am then redirected from
>> their javascript redirect code to the file.
>> When this happens, download dialog box pops up.
>>
>> What I want to do is just download that file automatically to my server
>> without any user interaction.
>> But now that I see the page is loaded from a redirect, I am wondering if
>> it even possible to download the file.
>>
>> Jojo
>>
>>
>> On Fri, Nov 7, 2008 at 12:43 AM, Thorben Thuermer <
>> t.thuermer_at_mediaclipping.de> wrote:
>>
>>> On Thu, 6 Nov 2008 08:34:42 +1300 "Jo Anonymous" <jojoanon_at_gmail.com>
>>> wrote:
>>> > Cheers for your replies..
>>> > Have tried both methods with no luck.
>>> >
>>> > Thorben, tried your method but it also didn't work for me, the results
>>> said:
>>> >
>>> > there was no no filename in the headers... Failed writing body
>>>
>>> well, either the format of the header is different and the regular
>>> expression
>>> does not match, or there is some kind of other problem... (see below)
>>>
>>> > The problem I think is that the actual file itself does not exist on
>>> the
>>> > server. It is just generated, so bascially all I need is a way to
>>> bypass the
>>> > download prompt and have it somehow let me download without any user
>>> > interaction.
>>>
>>> it does, as far as curl (or for that matter http) is concerened, not make
>>> any
>>> difference at all if the file is static or dynamically generated.
>>>
>>> > Will give it another go, any other suggestions are greatly appreciated.
>>>
>>> also, would you __PLEASE__ __CLARIFY__ what you mean by "download the
>>> file"???
>>> do you want to store it in a file on your server?
>>> do you even care about the name it would be saved as by a browser???
>>> or do you maybe want to pass it through to the caller of your php script
>>> (which is Stephen now assumes in his response).
>>>
>>> also, "download to your server without any prompts" does not make any
>>> sense
>>> at all, because curl (or php for that matter) would have no way to (even
>>> if it
>>> wanted to!) display such a dialog like a browser would.
>>> the browser shows the dialog because the server asks it to (using the
>>> Content-Dispostion header), which a non-browser client (curl, wget, ...)
>>> will simply ignore.
>>>
>>>
>>> from your current responses, it seems most likely that the server
>>> you are requesting the file from sends you a different response when you
>>> request the file with curl than when you request it with a browser.
>>>
>>> this could be due to some authentication scheme (do you need to be logged
>>> in?
>>> cookies?, maybe limited by IP?),
>>> or maybe some user-agent or similar check that is meant to prevent
>>> exactly what
>>> you are doing?
>>>
>>> another possibility would be that the server does not actually return
>>> data with a
>>> content-disposition, but returnsn a redirect to another url (maybe a
>>> static file).
>>>
>>> in the end, the only (and easy!) way to find out is to find out what
>>> response
>>> curl actually gets from the server (statuscode, headers, and body),
>>> instead of
>>> just saying that it "doesn't work", while you are not even specifying
>>> what your
>>> intended result (=definition of "works") is.
>>>
>>> and before you ask, here is some code to do just that.
>>> please run it with your url, and post the output
>>> (at least the headers).
>>>
>>> <?
>>> $handle=curl_init("
>>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/download.image.php
>>> ");
>>> curl_setopt($handle,CURLOPT_HEADER,1);
>>> curl_exec($handle);
>>> curl_close($handle);
>>> ?>
>>>
>>> - Thorben
>>>
>>> > On Wed, Nov 5, 2008 at 12:01 AM, Thorben Thuermer <
>>> > t.thuermer_at_mediaclipping.de> wrote:
>>> > > this is actually scarily related to the unanswered question i signed
>>> up to
>>> > > this
>>> > > list for...
>>> > >
>>> > > proper solution without downloading the file twice:
>>> > > (but also using global variables)
>>> > >
>>> > > <?
>>> > > // these should really be stored inside the curl context
>>> > > global $filename;
>>> > > global $outfile;
>>> > >
>>> > > function header_cb($handle,$h){
>>> > > global $filename;
>>> > > if(preg_match("/^Content-Disposition:
>>> > > .*filename\=([^\\n|\\r]+)[\\n\\r]/", $h, $m)){
>>> > > $filename=$m[1];
>>> > > echo "got filename: ".$filename."\n";
>>> > > }
>>> > > return strlen($h);
>>> > > }
>>> > >
>>> > > function data_cb($handle,$d){
>>> > > global $filename;
>>> > > global $outfile;
>>> > > if (!$outfile){
>>> > > $status=curl_getinfo($handle,CURLINFO_HTTP_CODE);
>>> > > if ($status != 200){
>>> > > echo "bad http status\n";
>>> > > return -1;
>>> > > }
>>> > > if ($filename == null){
>>> > > echo "there was no no filename in the headers...\n";
>>> > > return -1;
>>> > > }
>>> > > $outfile=fopen($filename,"w");
>>> > > }
>>> > > return fwrite($outfile,$d);
>>> > > };
>>> > >
>>> > > // testcase urls found via google:
>>> > > $handle=curl_init("
>>> > >
>>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/download.image.php
>>> > > ");
>>> > > //$handle=curl_init("
>>> > >
>>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/original.jpg
>>> > > ");
>>> > >
>>> > > curl_setopt($handle,CURLOPT_WRITEFUNCTION,"data_cb");
>>> > > curl_setopt($handle,CURLOPT_HEADERFUNCTION,"header_cb");
>>> > > $filename=null;
>>> > > if (!curl_exec($handle)){
>>> > > echo curl_error($handle)."\n";
>>> > > }
>>> > > if ($outfile) fclose($outfile);
>>> > > curl_close($handle);
>>> > >
>>> > > ?>
>>> > >
>>> > > - Thorben
>>> > >
>>> > > On Mon, 3 Nov 2008 17:44:01 -0500
>>> > > "Stephen Pynenburg" <spynenburg_at_gmail.com> wrote:
>>> > > > There's probably a way to do it in one request, but I think you
>>> could
>>> > > just
>>> > > > as easily do this:
>>> > > > <?PHP
>>> > > > $ch = curl_init();
>>> > > > curl_setopt($ch, CURLOPT_URL, "site/page.php");
>>> > > > curl_setopt($ch, CURLOPT_HEADER, 1);
>>> > > > curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
>>> > > > $res = curl_exec($ch);
>>> > > > preg_match("/filename\=(.*)\\n/", $res, $matches);
>>> > > > echo $matches[1]; //the filename
>>> > > > ?>
>>> > > >
>>> > > > Then do another request with the filename gleaned from that
>>> request.
>>> > > >
>>> > > > -Stephen
>>> > > >
>>> > > > 2008/11/3 Jo Anonymous <jojoanon_at_gmail.com>
>>> > > >
>>> > > > > Hello,
>>> > > > >
>>> > > > > I am trying to download a csv file which is generated from the
>>> url.
>>> > > > >
>>> > > > > I can login to the website using curl succesfully but I can't
>>> download
>>> > > the
>>> > > > > file I want.
>>> > > > >
>>> > > > > The download url looks something like this:
>>> > > > >
>>> > > > > http://www.websitename.com/file.aspx?type=all&download=1
>>> > > > >
>>> > > > > The website has this line in the header:
>>> > > > >
>>> > > > > header("Content-Disposition: attachment; filename=dada.csv");
>>> > > > >
>>> > > > > which brings up a download prompt.
>>> > > > >
>>> > > > > I just want to download the csv file to my server without any
>>> prompts.
>>> > > > >
>>> > > > > I've been trying for ages to figure this out, there must be a
>>> way.
>>> > > > >
>>> > > > > Many thanks in advance.
>>> > > > >
>>>
>>> _______________________________________________
>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>
>>
>>
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-11-09