cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: trying to get/generate cookies

From: Ralph Mitchell <ralphmitchell_at_gmail.com>
Date: Thu, 5 May 2016 22:13:49 -0400

On Thu, May 5, 2016 at 6:55 PM, bruce <badouglas_at_gmail.com> wrote:

> Hi Thanks for the eyeballs on my typos..
>
> However, running the curls.. still don't get cookies..
>
> Any thoughts?
>
> As I stated and the weird thing, I can clear cookies in FF, and use the
> 1st url, and generate cookies.. As far as I can tell, I'm replicating the
> traffic/data I see from livehttpheaders over the wire...
>
>
> echo "" > aa.lwp
> curl -vvv -A 'Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
> Firefox/38.0' -H 'Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H
> 'Accept-Language: en-US,en;q=0.5' -H 'Connection: keep-alive'
> --cookie-jar aa.lwp --cookie aa.lwp -L '
> http://www.bkstr.com/missouristatestore/home'
>

Did you look at the page being returned from the above curl call? It
contains an enormous amount of scripting, including this little gem:

        dojo.addOnLoad(function() {
                        SearchJS.init();
                        readAkamaiCookieValues();
                        enableDisableMiniCartLink();
            });

Curl doesn't attempt to process the scripting at all, it just hands you the
raw page. A browser processes the page as it goes along, executing
javascript and fetching page elements/style sheets/whatever as it renders
the page. I wouldn't be surprised to find one of the javascript files
generates cookies on the fly. Perhaps not by the readAkamaiCookieValues()
function, but by some script fragment.

Ralph Mitchell

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
<#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ: https://curl.haxx.se/docs/faq.html
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-05-06