What is a robots.txt file?
A robots.txt file is that file in which you tell search engine robots or web crawlers to index only specific web pages that belong to your website or blog.
Updated : 4 May 2017
A robots.txt file is that file in which you tell search engine robots or web crawlers to index only specific web pages that belong to your website or blog.
Updated : 4 May 2017