On 3/21/2017 1:59 PM, Claudio Piga wrote:
> Hi Nicholas, sorry if I reply this way, but I do not seem to be able
> to find a "reply to all" button in the mailing list page on the browser.
> However your command
> curl -sLvvvD- -o example.html "<url>"
>
> produces a file with some content ( a list of hotels in the right
> destination), but not the same content that is shown when the url is
> directly used on a browser, especially in terms of the dates indicated
> in the link, which I report here again for convenience:
>
> http://www.booking.com/searchresults.html?src=index&nflt=&lang=en-gb&checkin_monthday=14&checkin_year_month=2017-6&checkout_monthday=15&checkout_year_month=2017-6&&group_adults=2&group_children=0&dest_type=city&dest_id=-1454990&selected_currency=GBP
>
> So somehow curl is not mimicking the browser environment.

Please do not top-post it makes the conversation harder to follow. That
website does not show the hotel listings when you use the curl user
agent. I can show the results using another user agent and setting the
language, like this:

curl -v -L -H "Accept-Language: en-US,en;q=0.8" -A "Mozilla/5.0
(compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)" -o out
"URL"

What user agent you use can determine how much detail is returned, for
example IE9 user agent above booking doesn't return GPS coordinates. I
would ask them the best way to get information like that. You could
retrieve all 600k and parse the DOM but I suspect there's probably some
JSON you could tap into if they allow it.

[1]: https://curl.haxx.se/mail/etiquette.html#Do_Not_Top_Post

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-03-21