Add URLs to robots.txt file

If you would like to add URLs to the robots.txt file you can customize it. If you have found that you have pages listed in the sitemap.xml file that you want to block from search engines, then you can add them into your robots.txt file.

 

1. In the section URLs in the XML sitemap, click the link You can fully control which pages add or block and change their priorities here.

2. You can select URLs (press Ctrl or Shift button to select several pages at the same time) and Click the Block button. The URLs will be added to robots.txt to be blocked from search engines.