Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: I Nerd help downloading a page
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Bastian Jesuiter via curl-users <curl-users_at_lists.haxx.se>
Date: Sun, 12 Oct 2025 01:47:23 +0200
Hi,
Check the headers.
Some sites might block curls User-Agent.
Alternatively, the server might block you because curl is missing cookies
or other identifiable headers, for example authorization headers.
But we can only guess.
If you need the download links again, just check the JavaScript, or monitor
network calls from browser when you click the drop down menu.
It will be filled somewhere.
Either at initial page load, or later via xhr or fetch.
I might take a quick look later today.
Just experiment a bit though.
Use the browser network tab as help.
The browser offers copy as curl in the conxt menu for each network call.
Compare both curl calls, yours and the browsers.
Good luck,
Bastian
On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <
curl-users_at_lists.haxx.se> wrote:
> Hi All,
>
> curl-8.9.1-4.fc41.x86_64
>
> How come Firefox can see this page, but curl can not?
>
>
> First attempt:
>
> $ curl -L https://icedrive.net/apps/desktop-laptop -o eraseme.html
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:-- --:--:--
> 51058
>
>
> Second attempt (that is Brave Browser user agent.
> Firefox's did not work either):
>
> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"
> https://icedrive.net/apps/desktop-laptop/ -o eraseme.html
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:-- --:--:--
> 48422
>
>
> Yours in confusion,
> -T
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
> Etiquette: https://curl.se/mail/etiquette.html
>
Date: Sun, 12 Oct 2025 01:47:23 +0200
Hi,
Check the headers.
Some sites might block curls User-Agent.
Alternatively, the server might block you because curl is missing cookies
or other identifiable headers, for example authorization headers.
But we can only guess.
If you need the download links again, just check the JavaScript, or monitor
network calls from browser when you click the drop down menu.
It will be filled somewhere.
Either at initial page load, or later via xhr or fetch.
I might take a quick look later today.
Just experiment a bit though.
Use the browser network tab as help.
The browser offers copy as curl in the conxt menu for each network call.
Compare both curl calls, yours and the browsers.
Good luck,
Bastian
On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <
curl-users_at_lists.haxx.se> wrote:
> Hi All,
>
> curl-8.9.1-4.fc41.x86_64
>
> How come Firefox can see this page, but curl can not?
>
>
> First attempt:
>
> $ curl -L https://icedrive.net/apps/desktop-laptop -o eraseme.html
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:-- --:--:--
> 51058
>
>
> Second attempt (that is Brave Browser user agent.
> Firefox's did not work either):
>
> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"
> https://icedrive.net/apps/desktop-laptop/ -o eraseme.html
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:-- --:--:--
> 48422
>
>
> Yours in confusion,
> -T
> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
> Etiquette: https://curl.se/mail/etiquette.html
>
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-10-12