cURL / Mailing Lists / curl-users / Single Mail

curl-users

Help Using command line cURL with an ASP site

From: Mike Juvrud <ljuvrud_at_info-link.net>
Date: Wed, 9 Jun 2004 14:33:18 -0500

I'll try my best to explain - any help appreciated!

I am looking for help in extracting records from an online database. The
data is presented as a webpage for each record - There are 5 pages per
record. I need to get each page for each of the 12000+ records.

The problem: it uses ASP and session state...and I don't know how to get
it to work with cURL.
Once I have the resulting pages downloaded, it won't be a problem for me
to reconstitute the database from the individual webpages.

I'm assuming there is an easy solution for those that know how to use
cURL with an ASP site. If you know of a better solution, I'm all ears.

I'll explain the process that I go through manually (hopefully this is
descriptive for those in the know)
The first step - you have to accept the disclaimer at the following
link:
http://morris.state.mn.us/tax/disclaimer.asp?cid=61

It then prompts for a query - I would like to get all (it accepts the
wildcard "*") to present a list of all record numbers.

Then you have to click a record link to get to the first page of a
record. There are 4 additional links per record.

The following is the basic format of the link to a record:
"http://morris.state.mn.us/tax/Parcel.asp?pid=%20[RECORD
NUMBER]&tid=[0-4]"

HERE IS AN EXAMPLE FOR RECORD NUMBER 01-0001-000
url =
"http://morris.state.mn.us/tax/Parcel.asp?pid=%2001-0001-000&tid=0"
url =
"http://morris.state.mn.us/tax/Parcel.asp?pid=%2001-0001-000&tid=1"
url =
"http://morris.state.mn.us/tax/Parcel.asp?pid=%2001-0001-000&tid=2"
url =
"http://morris.state.mn.us/tax/Parcel.asp?pid=%2001-0001-000&tid=3"
url =
"http://morris.state.mn.us/tax/Parcel.asp?pid=%2001-0001-000&tid=4"

Any examples, tips, solutions would be greatly appreciated.

Will sum,

M Juv

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.698 / Virus Database: 455 - Release Date: 6/2/2004
 
Received on 2004-06-09