Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Error 1020
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Jeremy Roussak via curl-users <curl-users_at_cool.haxx.se>
Date: Thu, 15 Apr 2021 19:29:24 +0100
First post here - inexperienced curl user!
Mac, Catalina.
I have an AppleScript which uses “do shell script” to run curl and then fiddles with the returned html. It’s been working perfectly well for quite a while but over the last couple of weeks has been failing with “error code 1020”. I get the same error if I run curl in Terminal. But if I paste the URL into a browser, the page loads fine.
The URL is “https://www.e-infin.com/uk/item/3999/sony_alpha_a7r_iv_mirrorless_digital_camera_body_only <https://www.e-infin.com/uk/item/3999/sony_alpha_a7r_iv_mirrorless_digital_camera_body_only>” but other URLs into that site behave in the same way.
I assume they’ve changed something to prevent scraping; my needs are just to look at the prices of a few items once a week, so it shouldn’t upset them too much.
Does curl have options which would make it appear to be Safari/Chrome/Firefox/other? I’ve tried —user-agent but it made no difference. Any ideas gratefully received.
Jeremy
Jeremy Roussak
Photography: jbr.is
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-04-15
Date: Thu, 15 Apr 2021 19:29:24 +0100
First post here - inexperienced curl user!
Mac, Catalina.
I have an AppleScript which uses “do shell script” to run curl and then fiddles with the returned html. It’s been working perfectly well for quite a while but over the last couple of weeks has been failing with “error code 1020”. I get the same error if I run curl in Terminal. But if I paste the URL into a browser, the page loads fine.
The URL is “https://www.e-infin.com/uk/item/3999/sony_alpha_a7r_iv_mirrorless_digital_camera_body_only <https://www.e-infin.com/uk/item/3999/sony_alpha_a7r_iv_mirrorless_digital_camera_body_only>” but other URLs into that site behave in the same way.
I assume they’ve changed something to prevent scraping; my needs are just to look at the prices of a few items once a week, so it shouldn’t upset them too much.
Does curl have options which would make it appear to be Safari/Chrome/Firefox/other? I’ve tried —user-agent but it made no difference. Any ideas gratefully received.
Jeremy
Jeremy Roussak
Photography: jbr.is
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2021-04-15