How to Enable and Customize Blogger Robots.txt File?

Hi ! In this tutorial I am traveling to appearance you how to adapt your Blogger blog robots.txt file. Webmasters use custom robots.txt book to ascendancy seek engine web robots (also accepted as Web Wanderers, Crawlers, or Spiders) to clamber some directories and web pages or links of website or blog. if we yield settings robots.txt book in blogger again seek engine automatically basis or abolish pages from seek engine according to the settings. 

By absence anniversary website allows the Seek engines robots about if you would like to restricts the robots either to not clamber any apprenticed directory, book or the complete website again you may wish the robots.txt book in which you've got to address instructions for seek engine bots.

Steps to edit robots.txt on blogger

Your Site Settings › Search preferences › Crawlers and indexing  

Here you will able to see two options Custom robots.txt and Custom robots attack tags.These two options would action you the adaptability to customise your robot.txt file.In the endure column i told you about 

Custom robots attack tags.

Now columnist adapt button which is present afterwards 'Custom robots.txt' option. Afterwards acute on adapt button you can see a bulletin " Enable custom robots.txt content? " so columnist "Yes" and advance to next step.

Now you can see  text area, type the content which you want to exclude a content from crawling.
Click on Save Changes button.

You are done!
How to block Link from search engines?
No, you have to write the URL. For example, if your want to stop robots from crawling this URL
(www.bloggerknown.blogspot.com/p/about.html) then, in the robot.txt file you will enter this command.

User-agent: *Disallow: p/about.html

Your robots.txt file located under your main blogspot directory as for Blogger Known its located on the following url:

http://www.bloggerknown.blogspot.com/robots.txt

By default blogger robots.txt contains the following content:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://bloggerknown.blogspot.com/feeds/posts/default?orderby=UPDATED

I know this post is not a new thing but I hope it might help many newbies to understand the importance of robots.txt.