It is a text file that a search robot do for your site Search engines will index pages.


In other words, the search engine that will go in search of The robot checks that sometimes, you Some of the changes in the site or whether the update Checking the deeds of the text file robots Help him with the information.


But information that search engine not crawled give by you, Must be set before robots.txt File create.


What is Robots.txt
Item Reviewed: What is Robots.txt 9 out of 10 based on 10 ratings. 9 user reviews.