Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Curl Help?
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: bruce via curl-users <curl-users_at_cool.haxx.se>
Date: Mon, 18 Jan 2021 17:08:12 -0500
Hi Ray.
Thanks for the reply!!
I had thought of doing a selenium/py piece of code to drive the
returned content/html.
The weird thing. Other sites that have this similar layout with the
"Term" icon, allow a simple "curl" process to generate the data.
Welcome to the world of different versions/behind the scenes
functions...
If you created a test chunk of code could you fire it to me?
I hadn't seen the 2nd level html call based on the term #.
thanks..
On Sun, Jan 17, 2021 at 6:08 PM Ray Satiro via curl-users
<curl-users_at_cool.haxx.se> wrote:
>
> On 1/17/2021 11:13 AM, bruce via curl-users wrote:
> > The test site is a Psoft site:
> > https://cmsweb.pscs.calpoly.edu/psc/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> > https://cmsweb.pscs.calpoly.edu/psp/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> >
> > I've been successful in using curl to "simulate" and drive testing a
> > few other similar sites. In this situation, the site has a "term"
> > element that's being driven by a:
> > javascript:pAction_win0(document.win0,'SLO_SS_DERIVED_STRM$prompt')
> >
> > In order to try to craft the "curl" for this. I used the following steps:
> > -Fire up Firefox v78
> > set the developer tools/Network tools
> > initiate the url;
> > https://cmsweb.pscs.calpoly.edu/psc/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> >
> > select the "calpoly" as the institution
> > clear the network traffic section
> > select the "term" icon
> > (this generates the 'popup/frame' for the term list)
>
>
> It is a web application that is implemented heavily in javascript. You
> could use a web debugger like fiddler in an attempt to view and rebuild
> the commands but I doubt that's going to be tenable long term. It looks
> like in the post that SLO_SS_DERIVED_STRM is set to the terminal number
> and then after that a get request is made to the same URL and replies
> with more html and buried in there is the actual response "No matching
> values were found".
>
> IMO curl is not the right tool for this, look into headless browser
> webscraping.
>
>
> -----------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
> Etiquette: https://curl.haxx.se/mail/etiquette.html
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-01-18
Date: Mon, 18 Jan 2021 17:08:12 -0500
Hi Ray.
Thanks for the reply!!
I had thought of doing a selenium/py piece of code to drive the
returned content/html.
The weird thing. Other sites that have this similar layout with the
"Term" icon, allow a simple "curl" process to generate the data.
Welcome to the world of different versions/behind the scenes
functions...
If you created a test chunk of code could you fire it to me?
I hadn't seen the 2nd level html call based on the term #.
thanks..
On Sun, Jan 17, 2021 at 6:08 PM Ray Satiro via curl-users
<curl-users_at_cool.haxx.se> wrote:
>
> On 1/17/2021 11:13 AM, bruce via curl-users wrote:
> > The test site is a Psoft site:
> > https://cmsweb.pscs.calpoly.edu/psc/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> > https://cmsweb.pscs.calpoly.edu/psp/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> >
> > I've been successful in using curl to "simulate" and drive testing a
> > few other similar sites. In this situation, the site has a "term"
> > element that's being driven by a:
> > javascript:pAction_win0(document.win0,'SLO_SS_DERIVED_STRM$prompt')
> >
> > In order to try to craft the "curl" for this. I used the following steps:
> > -Fire up Firefox v78
> > set the developer tools/Network tools
> > initiate the url;
> > https://cmsweb.pscs.calpoly.edu/psc/CSLOPRD/EMPLOYEE/SA/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
> >
> > select the "calpoly" as the institution
> > clear the network traffic section
> > select the "term" icon
> > (this generates the 'popup/frame' for the term list)
>
>
> It is a web application that is implemented heavily in javascript. You
> could use a web debugger like fiddler in an attempt to view and rebuild
> the commands but I doubt that's going to be tenable long term. It looks
> like in the post that SLO_SS_DERIVED_STRM is set to the terminal number
> and then after that a get request is made to the same URL and replies
> with more html and buried in there is the actual response "No matching
> values were found".
>
> IMO curl is not the right tool for this, look into headless browser
> webscraping.
>
>
> -----------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
> Etiquette: https://curl.haxx.se/mail/etiquette.html
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-01-18