I'm guessing you're looking at scraping a website to extract data, and
want to simulate what a browser session would look like to give the
appearance of an automated user...

The simple answer, of course. If you want to have a single
instance/hit to a server, you can examine all that a real browser
session would send/receive from a server, and craft a curl process to
replicate this..

This would work for simple cases.

Anything more complex, and you're going to increase the complexity of
the simulated software process to look like an auto browser session.

On Sun, Nov 2, 2014 at 2:30 PM, Jeremy Nicoll - ml curl users
<jn.ml.crlu.36_at_letterboxes.org> wrote:
> Can curl send a request to a web server in a way that makes it impossible
> for the web server to know that it's from curl, and not a user doing a
> manual webpage request in a browser?
>
> I mean, I presume there's some way to send the equivalent of a browser
> id/agent string but is there anything more subtle that would tell a server
> curl's involved?
>
> --
> Jeremy C B Nicoll - my opinions are my own.
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-users
> FAQ: http://curl.haxx.se/docs/faq.html
> Etiquette: http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-11-02