Fetching from sites reverse proxied by CloudFlare?

Greetings,

I suppose that if I was goingt o be maximally polite, I would just
go off and try this first, before asking about it here, but I'm not
generally know for maximal politness, so please allow me to just ask...

Has anyone ever tried fetching, say, binary data files that are sitting
on some web site that is itself being reverse proxied by Cloudflare,
specifically?

I know that when one first visits (with some ordinary browser) any
given web site that is being proxied by Cloudflare, it is often
(always?) the case that before you even get to see the web site,
Cloudflare will first show you a page that says something like
"Checking out your browser, please wait a moment".

I really don't know all of the particulars about this, but I am quite
sure that it has something to do with Cloudflare's anti-DDoS protection
features. I'm just wondering if those feature would significantly
interfere with being able to use curl to fetch files from a site being
protected by Cloudflare.

Does anyone have any experience trying to do this? If so, how did it go?

Regards,
rfg

P.S. Yes, I'd like to create a web site and put some binary data files
on it someplace, and then put the whole thing behind Cloudflare. But
also, yes, I'd like to still have people be able to fetch those files,
e.g. from a shell script, using either curl, or wget, or perhaps even
lynx, if worse comes to worse.

Oh yea, and I'd like to have the people doing the fetching be required to
authenticate thmselves (somehow) also. I don't reckon that this added
little wrinkle would likely cause any special problems that would not
already be present without the authentication, but I mention it just for
the sake of completeness.
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2020-01-03