SEOClerks

Clear
Sign In Join

A question about robots.txt file

  • Tommy Matalino Level 1
    • 3

    Write the reason you're deleting this FAQ

    A question about robots.txt file

    Do we really need to manually create the Robots.txt file?

    #robots Website Management
    Share post:

    Comments

    Please login or sign up to leave a comment

    Join
    Beverly

    Beverly

    It's possible, you could use a premade robots.txt file, such as a template used by another site and change any relevant details to what your site needs. That would almost be automatic but would allow you to have input.
    12 years ago
    1970-01-01T00:00:00+0000

    Are you sure you want to delete this post?

    Linksdepth

    Linksdepth

    It's possible, you could use a premade robots.txt file, such as a template used by another site and change any relevant details to what your site needs. That would almost be automatic but would allow you to have input.


    Can U suggest me giving some example please........
    12 years ago
    1970-01-01T00:00:00+0000

    Are you sure you want to delete this post?

    Corzhens

    Corzhens

    I guess what you mean about the robot.txt file is if you need that for the search engines to crawl on your site. From what I understand about the robot.txt file it is a guide for the search engine crawlers for the direction which parts of the web page that it should see and which parts that the crawlers should not see. But without the robot.txt file the crawler will check on everything that is on the web page which is the standard. In other words you don’t need the robot.txt file.
    6 years ago
    1970-01-01T00:00:00+0000

    Are you sure you want to delete this post?