Site structure

Yandex.Webmaster shows site sections that contain at least 10 pages and make up more than 1% of the total number of pages loaded into the search database. Sections are displayed in a tree hierarchy according to the structure of the site's URLs. Information is available for each site section on the number of pages that were visited by the Yandex robot and that participate in the search.

The data presented helps you determine if the robot downloads the website sections that are meant for the users. You can prohibit loading pages from certain website sections using the robots.txt file.

Note. If you see in Yandex.Webmaster any sections that are not present on the site, it's likely that links to them are placed on other sites. To stop the robot from crawling the links, prohibit indexing the site sections with the Disallow directive in the robots.txt file.

Adding a section

You can get indexing information about any website section, even if it doesn't make up 1% of all pages loaded to the search engine database. To get this data, click the Add section button and specify the section starting with "/". For example, "/catalog1".

You can use the * operator to specify a non-exact match to the name of the site section:


catalog1*	Adds all sections with URLs beginning with "catalog1"
*catalog1	Adds all sections with URLs ending with "catalog1"
catalog1	Adds all sections with URLs containing "catalog1"

Changes will take effect after a few days.

FAQ

What does “loaded” mean?

The number of pages on your website that the Yandex robot accessed. This may include both indexed pages and pages that returned an error.

What does “pages available in search” mean?

The pages that are included in Yandex search results. A page may be missing from the search results if it duplicates the content of another page, contains spam or viruses, redirects to another URL, or if there was a downloading error.

To find out why the page is missing in the Yandex search, use the Check URL status tool.

Why aren't all pages in a section indexed?

Most likely, not enough time has passed since the robot discovered the website. Update the Sitemap file and wait until the robot indexes it.

How can I prohibit the Yandex robot from visiting pages in a particular section?

To prohibit indexing a website section, enter the following in the robots.txt file:

User-agent: Yandex
Disallow: /section/

At the next site indexing, the robot will take this into account and remove the section. Exceptions for the robots.txt file are described in the Yandex robots in server logs section.

Tell us what your question is about so we can direct you to the right specialist:

Yandex.Webmaster shows site sections that contain at least 10 pages and make up more than 1% of the total number of pages loaded into the search database.