Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: I Nerd help downloading a page
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: ToddAndMargo via curl-users <curl-users_at_lists.haxx.se>
Date: Sat, 11 Oct 2025 17:03:30 -0700
>> On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <curl-
>> users_at_lists.haxx.se <mailto:curl-users_at_lists.haxx.se>> wrote:
>>
>> Hi All,
>>
>> curl-8.9.1-4.fc41.x86_64
>>
>> How come Firefox can see this page, but curl can not?
>>
>>
>> First attempt:
>>
>> $ curl -L https://icedrive.net/apps/desktop-laptop <https://
>> icedrive.net/apps/desktop-laptop> -o eraseme.html
>> % Total % Received % Xferd Average Speed Time Time
>> Time
>> Current
>> Dload Upload Total Spent
>> Left
>> Speed
>> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:--
>> --:--:--
>> 51058
>>
>>
>> Second attempt (that is Brave Browser user agent.
>> Firefox's did not work either):
>>
>> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0
>> <http://141.0.0.0> Safari/537.36"
>> https://icedrive.net/apps/desktop-laptop/ <https://icedrive.net/
>> apps/desktop-laptop/> -o eraseme.html
>> % Total % Received % Xferd Average Speed Time Time
>> Time
>> Current
>> Dload Upload Total Spent
>> Left
>> Speed
>> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:--
>> --:--:--
>> 48422
>>
>>
>> Yours in confusion,
>> -T
On 10/11/25 4:47 PM, Bastian Jesuiter via curl-users wrote:
> Hi,
>
> Check the headers.
> Some sites might block curls User-Agent.
>
> Alternatively, the server might block you because curl is missing
> cookies or other identifiable headers, for example authorization headers.
>
> But we can only guess.
>
> If you need the download links again, just check the JavaScript, or
> monitor network calls from browser when you click the drop down menu.
> It will be filled somewhere.
> Either at initial page load, or later via xhr or fetch.
>
> I might take a quick look later today.
>
> Just experiment a bit though.
> Use the browser network tab as help.
>
> The browser offers copy as curl in the conxt menu for each network call.
>
> Compare both curl calls, yours and the browsers.
>
> Good luck,
>
> Bastian
>
>
I am seeing these two under Firefox's "Network":
https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js
https://icedrive.net/assets/builds/icedrive.frontend.4029.min.js
Date: Sat, 11 Oct 2025 17:03:30 -0700
>> On Sun, 12 Oct 2025, 01:37 ToddAndMargo via curl-users, <curl-
>> users_at_lists.haxx.se <mailto:curl-users_at_lists.haxx.se>> wrote:
>>
>> Hi All,
>>
>> curl-8.9.1-4.fc41.x86_64
>>
>> How come Firefox can see this page, but curl can not?
>>
>>
>> First attempt:
>>
>> $ curl -L https://icedrive.net/apps/desktop-laptop <https://
>> icedrive.net/apps/desktop-laptop> -o eraseme.html
>> % Total % Received % Xferd Average Speed Time Time
>> Time
>> Current
>> Dload Upload Total Spent
>> Left
>> Speed
>> 100 6995 100 6995 0 0 50638 0 --:--:-- --:--:--
>> --:--:--
>> 51058
>>
>>
>> Second attempt (that is Brave Browser user agent.
>> Firefox's did not work either):
>>
>> $ curl -L --user-agent "Mozilla/5.0 (X11; Linux x86_64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0
>> <http://141.0.0.0> Safari/537.36"
>> https://icedrive.net/apps/desktop-laptop/ <https://icedrive.net/
>> apps/desktop-laptop/> -o eraseme.html
>> % Total % Received % Xferd Average Speed Time Time
>> Time
>> Current
>> Dload Upload Total Spent
>> Left
>> Speed
>> 100 7215 100 7215 0 0 48312 0 --:--:-- --:--:--
>> --:--:--
>> 48422
>>
>>
>> Yours in confusion,
>> -T
On 10/11/25 4:47 PM, Bastian Jesuiter via curl-users wrote:
> Hi,
>
> Check the headers.
> Some sites might block curls User-Agent.
>
> Alternatively, the server might block you because curl is missing
> cookies or other identifiable headers, for example authorization headers.
>
> But we can only guess.
>
> If you need the download links again, just check the JavaScript, or
> monitor network calls from browser when you click the drop down menu.
> It will be filled somewhere.
> Either at initial page load, or later via xhr or fetch.
>
> I might take a quick look later today.
>
> Just experiment a bit though.
> Use the browser network tab as help.
>
> The browser offers copy as curl in the conxt menu for each network call.
>
> Compare both curl calls, yours and the browsers.
>
> Good luck,
>
> Bastian
>
>
I am seeing these two under Firefox's "Network":
https://icedrive.net/assets/frontend/lib/jq/jquery-3.3.1.min.js
https://icedrive.net/assets/builds/icedrive.frontend.4029.min.js
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2025-10-12