curl-users
Re: Using curl to download CBOE csv file quotes...help!
Date: Sat, 29 Jan 2011 10:04:54 -0500
So there are cookies involved. Doesn't curl support a way to pass cookies to a site? Any idea how to formulate the curl command with cookie data in this particular situation?
Cheers,
Dave
---- Yves PETRONIN <y.petronin_at_wanadoo.fr> wrote:
> I am very far from being an expert. I have hovever become familiar
> with such difficulties. I have found most useful to use the "Form"
> optin from Firefox, as well as the capabilities of the software to
> record the live http headers.
> In your case, the apparently simple submission of a ticker correspond
> to the following form
>
> <input
> id="ctl00_ctl00_AllContent_ContentMain_QuoteTableDownloadCtl1_txtTicker"
> name="ctl00$ctl00$AllContent$ContentMain$QuoteTableDownloadCtl1
> $txtTicker" size="10" maxlength="20">
>
> Using the capabilities to record the live transaction made by
> Firefox, when submitting the ticker "AAPL", I have several
> activities related to cookies, as usual, and the following :
>
>
> POST /DelayedQuote/QuoteTableDownload.aspx
> __EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=%
> 2FwEPDwUKLTc3NTgxOTcwNw9kFgJmD2QWAmYPZBYCAgUPZBYCAgMPZBYEAgUPZBYCAgEPZBY
> CAgcPZBYIAgEPZBYCAgIPZBYKAggPFgIeBFRleHQFgAI8SUZSQU1FIHNyYz0iaHR0cDovL2F
> kLmRvdWJsZWNsaWNrLm5ldC9hZGkvcXVvdGUuY2JvZS5kYXJ0L3F1b3RlL3F1b3RldGFibGV
> kb3dubG9hZDtzej0xMjB4NjA7dGlsZT0yO29yZD0nICsgb3JkICsgJyIgd2lkdGg9IjEyMCI
> gaGVpZ2h0PSI2MCIgdmFsaW
> ......
> ...... many line omitted
> .......
>
> kY3RsMDAkbHZFZHVGZWF0dXJlZEV2ZW50DzwrAAoCBzwrAAQACAIEZCKnBY8jkhgzLm5rm2I
> pVY41%2BYEo&__EVENTVALIDATION=%2FwEWBwKJpfylBgLnn5gNAtXW%2F%
> 2BYLApSZ7dMOArre%2Ba0MArm0noIMAui9nNYP7PowP1dq4P4JaIlPHUsY%2BhoF%2FBg%
> 3D&ctl00%24ctl00%24AllContent%24ucHeader%24ucCBOEHeaderQuoteBox%
> 24txtHeaderQuote=Quote&ctl00%24ctl00%24AllContent%24ucHeader%
> 24CBOEHeaderSearchBox%24txtHeaderSearch=Search&ctl00%24ctl00%
> 24AllContent%24ContentMain%24QuoteTableDownloadCtl1%
> 24txtTicker=AAPL&ctl00%24ctl00%24AllContent%24ContentMain%
> 24QuoteTableDownloadCtl1%24cmdSubmit=Download
>
> I suggest to explore this type of route/tests ... It looks like that
> a very simple command will not suffice to mimic all this......and
> that perhaps the site is designed in such a way that it is made very
> difficult to use other tools than a browser....
>
> Rgds
>
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-users
> FAQ: http://curl.haxx.se/docs/faq.html
> Etiquette: http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-01-29