On Sun, May 9, 2010 at 7:50 PM, Charles <cecrume_at_gmail.com> wrote:

> Hello;
>
> I have a little membership system that I wrote in PHP and run on an Apache
> v2.0 web server. A couple of pages are protected with .htaccess files.
>
> When a user logins into my membership system I don't want them to have to
> enter the username/password that Apache normally requires to gain access.
>
> I have made some progress using cURL and can get the protected page to
> appear -- but cURL does not transfer to the actual URL specified. The
> browser shows the page, and it's contents, but the links (the page contains
> files to download) are bad because they are relative links.
>
> 1) Login to membership system.
> 2) Click on link "Download Updates" link on main page and it runs
> http://www.charlescrumesoftware.com/members/user_downloads.php
> 3) The above PHP script runs the following cURL code (among other stuff):
>
> $Cookie_Jar = "Cookie_Jar.doc";
> $c = curl_init("http://www.charlescrumesoftware.com/downloads/dms");
> // the actual page I want
> curl_setopt($c, CURLOPT_COOKIE, 1);
> curl_setopt($c, CURLOPT_COOKIEJAR, "$Cookie_Jar");
> curl_setopt($c, CURLOPT_FOLLOWLOCATION, 1); // -L
> curl_setopt($c, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
> curl_setopt($c, CURLOPT_USERPWD, "UserName:Password");
> // curl_setopt($c, CURLOPT_RETURNTRANSFER, 1); // save cURL
> output to file
> curl_setopt($c, CURLOPT_RETURNTRANSFER, 0); // allows us to
> FOLLOW
> $myDOWN_Page = curl_exec($c);
> curl_close($c);
> 4) the browser show the contents of the "download" page correctly -- but IE
> shows http://www.charlescrumesoftware.com/members/user_downloads.php in
> the address bar.
>
> 5) If I click on the "Parent Directory" link I am taken up one level in the
> directory structure -- as I should.
>
> 6) However... All the links to files that can be downloaded are relative
> links:
>
> <a href="DMSv64Update.zip">
>
> so I get taken to a page that does not exist.
>
> http://www.charlescrumesoftware.com/members/DMSv64Update.zip
>
> Can someone shed some light on what I am overlooking here?
>
> (I realize I could save the page's contents, have PHP to alter all the
> links, then display the result -- but would prefer to just access the page
> directly, if possible.)
>

I think the bit you're overlooking is that a browser will remember the url
for the page it just fetched, and then apply that url to any relative links
links it find in the page. You can mimic that (to some extent) in curl by
using the -w option:

   EFFURL=`curl -s -S -L -w '%url_effective" -o file.html http://........`

The page will be stored in page.html and the effective url, after redirects
are followed, will be in the $EFFURL shell variable. Your script will still
need to convert the relative urls to absolute, but at least you have the
starting point.

Ralph Mitchell

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-05-10