Conducting experiments

The Yandex Advertising Network provides partners with convenient statistical analysis tools to improve the effectiveness of their ad blocks. You can also experiment with the blocks yourself to improve the efficiency of your advertising and increase revenue from your website.

Experimentation is a controllable method for researching some kind of phenomenon. Results of conducted experiments serve as a confirmation or disproof of your hypothesis.

Note the following:

  • When conducting one experiment, it is best to test no more than one hypothesis at a time (for example, only use a new block configuration or only one change in placement location).

  • Select a time period in which no significant changes are occurring to the site (for example, in the site's design or layout).

  • Try to select equivalent time periods when comparing any kind of indicators. For example, it would be better to compare impression dynamics for January of this year with the impression dynamics for January of the previous year rather than for February of this year (for more information,  see Seasonal statistical fluctuations).

  • Statistical data is only worthwhile if a large volume of data has been accumulated.

Here is an experiment you can conduct right now.

Hypothesis: using icons for advertisers' sites in ad blocks could increase CTR.

Test method: select a section of the site where you're going to conduct your experiment. Create new ad blocks in the visual code designer, add the advertisers' icons to the ad blocks, and re-embed the code on the site.

Results analysis: after two weeks, compare the CTR of the ad blocks in the tested cross-section with the CTR of the ad blocks before the ad code was modified. Compare the tested CTR with the CTR of ad blocks in other sections of the site (if you created statistical cross-sections).

Conclusion: the ad block option that showed the highest CTR should be selected as the permanent ad block.

Other experiment ideas:

  • Change the number of ads in an ad block.
  • Test how enabling and disabling behavioral targeting affects your site revenue.
  • Find out how blocking competitors affects ad effectiveness and site traffic.
  • Change the location of blocks on a page and see if this changes the number of bounces and the time spent on your website (according to Yandex.Metrica).