Possible issues

This section contains solutions to common problems from the “Possible” category identified during site diagnostics in Yandex Webmaster. Issues in this category may affect the quality and speed of site indexing.

Tip

Configure notifications about the results of site monitoring.

Robots.txt file not found

The indexing robot requests the robots.txt file several times a day and updates information about it in its database. It the robot cannot load the file, a warning appears in Yandex Webmaster.

In the service, check the availability of the robots.txt file. If the file is still unavailable, add it. If you can't do this yourself, contact your hosting provider or domain name registrar. After the file is added, the data in Yandex Webmaster will be updated within a few days.

Errors were detected in the robots.txt file

Check the robots.txt file of your site. To fix errors, see the directive descriptions.

No Sitemap files are currently used by the indexing bot

The Sitemap file it is an auxiliary tool for indexing a site. It allows you to regularly inform the robot about new pages on the site. This warning appears if the robot is not using any Sitemap files for the site.

For the robot to start using the created file, add it to Yandex Webmaster and wait for the robot to process the file. This can take up to two weeks. After that, the warning will disappear.

Errors found in Sitemap files

Check the Sitemap file of your site. The check may detect the error “Unknown tag”. This means that the file contains elements that are not supported by Yandex. The unsupported elements are ignored by the robot during Sitemap processing, but data from supported elements is still taken into account. You don't have to change the contents of the file. Learn more about supported Sitemap elements.

Incorrect display settings for non-existent or deleted files and pages

Probably, the site is incorrectly configured to return the 404 HTTP code (resource not found/Not Found). In this case, the search bot may receive a 200 OK response when accessing the site via any link and index a non-existent page. This will lead to a large number of duplicate pages, complicating and slowing down the indexing of the site and increasing the server load.

To check the server response, use the Page status check tool. Enter the address of a page that does not exist on your site. You can use any combination of letters and numbers, for example, https://example.com/kugbkkrfck.

If it is impossible to set the recommended 404 response code:

  • Set any other 4xx code, except 429.

    We do not recommend using 429 and 5xx response codes, as they tell the bot that the server is experiencing difficulties. This may slow down the indexing of the site.

  • Use the noindex directive or set up a 301 redirect.

    When using noindex and 301 redirect, error notifications may appear in Yandex Webmaster, but the pages will not be included in the search, and it will not affect the site’s ranking. These methods are the least preferable, as they increase the server load and slow down the indexing of the site.

The title element and the description metatag are missing

The title element and description metatag are used for generating a good site description in search results. For details, see Snippet.

If either or both of these elements are missing from your website, add them to the HTML code of the pages and save the changes. If the elements have already been added, wait for the robot to recrawl the pages. The error message will disappear after reindexing. Learn more

Pages have the same titles and descriptions

This problem appears if the title or description is duplicated on many pages of the website. The title element and the description meta tag should be informative and appealing and reflect the page content. This will help users find the search results they are looking for.

See examples of pages with duplicate titles or descriptions that Yandex Webmaster found while the robot crawled the website. To fix them, follow the recommendations for:

The problem will stop being displayed when the robot learns about changes on the website. To make this happen faster, send the most important pages to be reindexed or configure page crawling with a Yandex Metrica tag.

The favicon file is not accessible to the robot

If this issue is displayed for your site, it means the favicon is not appearing in search results. In Yandex Webmaster, go to Website optimization → Site diagnostics (Possible problems section) and check the reason why the indexing bot couldn't load the file, then follow the instructions:

Error

Solution

The file responds with an HTTP code other than 200 OK

Check the server response. The response should be 200 OK. For other response statuses, see Server response check.

The file redirects to another URL

Incorrect data type

Check the type parameter value in the file link. It should match the file format.

How do I install a favicon?

Contact support

Tell us what your question is about so we can direct you to the right specialist:

Check out the recommendations. If the file is available to the robot and can be downloaded in Yandex Webmaster, but the issue still appears, fill out the form:

See the recommendations above. If the file was added more than 2 weeks ago, but the message doesn't disappear, fill out the form:

See the recommendations above. If there are no errors in the file, but the message still appears, fill out the form:

See the recommendations above. If the server response matches 200 OK and the value of the type parameter matches the file format, but the message doesn't disappear, fill out the form:




You can also go to

A small image displayed in the snippet in Yandex search results, next to the site address in the browser's address bar, and next to the site name in Favorites or Bookmarks in the web browser.