Troubleshooting the site's position in the search
This section helps you independently determine the reason why your site changed its position or isn't displayed in Yandex search results anymore.
Step 1. Check to see which response was received by the robot before the site status in search was changed.
-
-
Go to the Crawl statistics page in Yandex Webmaster.
-
In the list of changes, find the URL of the page that changed its position or disappeared from the search. For example:
In the example, the
https://example.com/comment/articles/ks/
page changed its status and became unavailable for the robot due to the indexing prohibition. If the page isn't available to the robot, it is excluded from the search. Therefore, the page stops being displayed in search results for queries or the site's position is lowered.The page may become inaccessible to the robot for several reasons:
The page is closed from indexing.The HTTP response code is other than 200 OK.You can close a page from indexing using one of the methods:
- Using the Disallow directive in the robots.txt file. To remove the ban, make changes to
robots.txt
and check the correctness of the entries in the file using the Robots.txt analysis tool. - Using the
noindex
ornone
directive in therobots
metatag in the source code of the site page or in theX-Robots-Tag
HTTP header. Learn more. To remove the ban, make changes to the page code or change the HTTP header.
- Using the Disallow directive in the robots.txt file. To remove the ban, make changes to
-
Step 2. Check what response the robot gets after making changes to the site.
-
- In Yandex Webmaster, go to the Server response check page.
- In the field, enter the URL of the page whose response you checked in step 1.
If the server response matches the 200 OK HTTP code, it means that the page will appear in the search results within two weeks.
See the recommendations depending on the HTTP response code, as well as explanations of other messages when checking the server response.
After fixing the error send the page for reindexing.
Recommendations didn't help
If the page doesn't appear in the search and the site's position continues to go down, check out other reasons for position changes and excluding pages from search.