On Jan 31, 2011, at 14:41, Michael Herbig wrote:
> My goal is to download all customer comments, strip away the extraneous text, and parse the fields into a .csv file. This is further complicated by the fact that there can be multiple pages of comments (via the "next" and "back" links), but just getting curl/wget to get the data would be a good start.
______________________________________________________________________

Hey Mike,

Since you were having trouble getting your data using curl I first tried dumping the url with Lynx and found the reviews right away.

Then I simply tried: curl http://www.edgeangling.com/Daiwa-SW4000-B.html

And I was able to find the complete text of the two reviews in the source. I just searched for "\bpros\b" with grep and found them right away.

(curl 7.21.2 (x86_64-apple-darwin10.4.0) libcurl/7.21.2 OpenSSL/1.0.0a zlib/1.2.5 libidn/1.19)

HTH

--
Best Regards,
Christopher Stone
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html