Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: sort of tangent to curl -- web browser/extension
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: bruce via curl-users <curl-users_at_cool.haxx.se>
Date: Wed, 2 Jun 2021 16:26:33 -0400
Hi Petr.
Thanks so much for your time on this.
As far as I can tell. It appears that the overall sit/process uses
"blocking" tech from perimterX. This combines with recaptcha and other
processes to impact the ability to access the data. Even using the
browser/tabs too fast will generate issues.
I'm thinking a possible solution might be to go in a "different"
direction via some sort of browser extension which would access the
content and then send the content off to a extern server for
parsing/processing.
Thanks
On Wed, Jun 2, 2021 at 3:02 PM Petr Pisar via curl-users
<curl-users_at_cool.haxx.se> wrote:
>
> V Tue, Jun 01, 2021 at 02:34:33PM -0400, bruce via curl-users napsal(a):
> > Hi.
> >
> > The test page/URL I'm looking at for my use case would be:
> > https://www.bkstr.com/efollettstore/home
> >
> > Selecting the dev tools/network, and then doing an inspection of the
> > functions in the "network" tab you can see the::
> >
> > >>>GET - svc.bkstr.com - https://svc.bkstr.com/store/byName?storeType=FMS&catalogId=10001&langId=-1&schoolName=*
> > (might need to do a refresh to gen the list of functions/calls for the page)
> >
> I don't get that address (.../byName) loaded anywhere when visiting the
> (.../home) page. But that can be because I block various address, cookies,
> scripts etc. by default. Or maybe the scam-like web page is completely
> broken because it randomly returns to me "under maintanance" message error.
>
> > However I'm trying to figure out if there's an extension or a method
> > to gen/save the content of the page when I'm viewing the top level
> > URL. The process would have to then gen the content of the associated
> > URLs which comprise the page.
> >
> I'm sorry, I cannot help you.
>
> -- Petr
> -----------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
> Etiquette: https://curl.haxx.se/mail/etiquette.html
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-06-02
Date: Wed, 2 Jun 2021 16:26:33 -0400
Hi Petr.
Thanks so much for your time on this.
As far as I can tell. It appears that the overall sit/process uses
"blocking" tech from perimterX. This combines with recaptcha and other
processes to impact the ability to access the data. Even using the
browser/tabs too fast will generate issues.
I'm thinking a possible solution might be to go in a "different"
direction via some sort of browser extension which would access the
content and then send the content off to a extern server for
parsing/processing.
Thanks
On Wed, Jun 2, 2021 at 3:02 PM Petr Pisar via curl-users
<curl-users_at_cool.haxx.se> wrote:
>
> V Tue, Jun 01, 2021 at 02:34:33PM -0400, bruce via curl-users napsal(a):
> > Hi.
> >
> > The test page/URL I'm looking at for my use case would be:
> > https://www.bkstr.com/efollettstore/home
> >
> > Selecting the dev tools/network, and then doing an inspection of the
> > functions in the "network" tab you can see the::
> >
> > >>>GET - svc.bkstr.com - https://svc.bkstr.com/store/byName?storeType=FMS&catalogId=10001&langId=-1&schoolName=*
> > (might need to do a refresh to gen the list of functions/calls for the page)
> >
> I don't get that address (.../byName) loaded anywhere when visiting the
> (.../home) page. But that can be because I block various address, cookies,
> scripts etc. by default. Or maybe the scam-like web page is completely
> broken because it randomly returns to me "under maintanance" message error.
>
> > However I'm trying to figure out if there's an extension or a method
> > to gen/save the content of the page when I'm viewing the top level
> > URL. The process would have to then gen the content of the associated
> > URLs which comprise the page.
> >
> I'm sorry, I cannot help you.
>
> -- Petr
> -----------------------------------------------------------
> Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
> Etiquette: https://curl.haxx.se/mail/etiquette.html
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-06-02