cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: is it possible to curl browser rendered pages ?

From: Erik Ronström <erik.ronstrom_at_doremir.com>
Date: Wed, 9 Dec 2015 01:09:44 +0100

Seems like that page is just a container, and the real content is loaded with ajax. Curl just gets you the javascripts, but unless you actually run them, you probably won’t be able to fetch the contents.

However, if you are interested in some specific sub-content of the page, you may be able to track the ajax requests from the browser and repeat them with curl. But you are probably in for a lot of work, if it is at all possible.

Erik

> 9 dec 2015 kl. 00:05 skrev home_at_node.de:
>
> If I try to curl this page, then I get only a few lines of source code inside curl instead of the whole page with content.
>
> Are there special parameters needed to get the whole page source with curl ?
>
> this page as an example, does not load the whole source with curl:
> http://www.immobilienscout24.de/anbieter/suchen/Baden-Wuerttemberg/Baden-Baden?geocodeid=1276001002&focustype=2&order=ALPHABETICAL
>
>
> Any suggestions ?
>
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-12-09