Introduction to Custom Robots.txt

The custom Robots.txt is a simple textual content record that instructs seek engine crawlers & robots on which pages, sections, folders, and URLs of a website are allowed to be crawled/traversed and which aren't.

Before a web-robotic or crawler wants to go to a URL it first visits robotic.txt and assessments whether it's far approved to move slowly or now not.

It is not an awesome SEO practice to permit crawlers and web robots to needless crawl admin-side folders, URLs, and public URLs with search labels. it may impact the performance of your Blogger weblog.

How to add Robots.txt code

If you are a beginner and want to add a robot.txt code to your blogger web then follow the following steps.

1. Go to your blogger account and click on the settings on the left side menu.

2. scroll down you will find the crawler and indexing portion.

3. In the crawler and indexing portion you have to enable the custom robots.txt. make sure that the toggle button in front of 'enable the custom robots.txt' is green.

4. Now you have to click on custom robots.txt.

5. After clicking on custom robots.txt a new portion will open here you have to add your custom robots.txt code.

6. If you have no experience with how to make the robots.txt code you can add the code below without any worry.

User-agent: *

Allow: /
Sitemap: http://<subdomain_name>.<domain_name>.<tld>/sitemap.xml

Note: remember that you have to cut the site map in the upper code and paste the site map of your own site.

↠If you don't know about the site map click on the link below this is an auto-site map generator. You can also generate the whole custom robots.txt code on this web

Site map generator

In the auto site generator paste the link of the home page of your site.

Then click on Generate XML Sitemap.

It will automatically generate your website map and also the robots.txt code you just have to copy the code except for the first two lines and paste it to the new portion opened after clicking robots.txt in blogger settings.

For example