cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: Robots protocol

From: vivekanandan <vivekanandan_at_comodo.com>
Date: Tue, 25 Apr 2006 11:53:02 +0530

Yeah ... I am refering the robots.txt and also some sites are using meta
tag for robots is it possible find this..

Daniel Stenberg wrote:

>On Tue, 25 Apr 2006, vivekanandan wrote:
>
>
>
>>how can we find a site which is using robots protocol or not..
>>
>>
>
>Are you referring to the robots.txt file?
>
>
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2006-04-25