A question about robots.txt file

Tommy Matalino Level 1

October 22, 2012, 9:26 am

Write the reason you're deleting this FAQ

A question about robots.txt file

Do we really need to manually create the Robots.txt file?

Comments

Please login or sign up to leave a comment

Join

Beverly

It's possible, you could use a premade robots.txt file, such as a template used by another site and change any relevant details to what your site needs. That would almost be automatic but would allow you to have input.

12 years ago

Are you sure you want to delete this post?

Linksdepth

It's possible, you could use a premade robots.txt file, such as a template used by another site and change any relevant details to what your site needs. That would almost be automatic but would allow you to have input.

Can U suggest me giving some example please........

12 years ago

Are you sure you want to delete this post?

Corzhens

I guess what you mean about the robot.txt file is if you need that for the search engines to crawl on your site. From what I understand about the robot.txt file it is a guide for the search engine crawlers for the direction which parts of the web page that it should see and which parts that the crawlers should not see. But without the robot.txt file the crawler will check on everything that is on the web page which is the standard. In other words you don’t need the robot.txt file.

6 years ago

Are you sure you want to delete this post?

SEOClerks

A question about robots.txt file

A question about robots.txt file

Comments

Please login or sign up to leave a comment