FAQ and troubleshooting
- Can I run experiments with JS scripts?
- There's a significant difference between the test and control variants in my redirect experiment
- The experiment data and the Yandex Metrica report data differ
- I was sorted into different variants of one experiment, but Yandex Metrica recorded only one session
- What if I want to use conditions that are not available in the Varioqub interface?
- I'm trying to create an experiment, but all the fields are locked
- Visual editor doesn't work
- Experimental changes don't appear on the page immediately
- How do I disable a specific experiment type for an entire site or page?
- An element loads after the page. How do I include it in the experiment?
- My site has dynamic element IDs. Can I still use the visual editor?
- Can I add a redirect to a different domain?
- My monetization statistics in Yandex Metrica and the partner interface are different
- I've run an A/A experiment, and the test variant differs from the control one
- Audience discrepancy in experiments with redirects
Can I run experiments with JS scripts?
Yes, you can do it in the visual editor. To do this, enable the enableJS parameter in setConfig.
There's a significant difference between the test and control variants in my redirect experiment
In Varioqub, a control page loads first, and then experimental changes are applied to it. If the change is a redirect, the page quickly changes, causing flickering. This might affect user behavior.
You can hide the display of the original page until the redirect is applied. However, this can increase the page load time, which may also impact user behavior.
We recommend a different solution: add a redirect to the control variant. Create a copy of the original page and specify the link to it in the control variant.
The experiment data and the Yandex Metrica report data differ
Yandex Metrica reports and experiment reports are generated differently.
Common causes for discrepancy:
-
The URL includes GET parameters.
-
Some sessions aren't sorted into in any samples because of the experiment conditions.
I was sorted into different variants of one experiment, but Yandex Metrica recorded only one session
When a user enters an experiment, they're identified by the first-party cookie named __ymab_param. However, Yandex Metrica reports apply more sophisticated identification methods.
If you delete cookies on your device, you may be sorted into a different sample. But Yandex Metrica will still consider you the same user.
What if I want to use conditions that are not available in the Varioqub interface?
You can use custom parameters. They're available in the extended version.
I'm trying to create an experiment, but all the fields are locked
Check your access rights to the current Yandex Metrica tag.
You need edit access in order to create, launch, and stop experiments.
Visual editor doesn't work
This experiment type may not work on some sites.
This happens if the server sets the Cross-Origin-Opener-Policy header with the value same-origin. This setting prohibits data exchange between the Yandex Metrica interface and the site.
There is currently no workaround for this technical limitation.
Experimental changes don't appear on the page immediately
In Varioqub, a control page loads first, and then experimental changes are applied to it. Because of this, you might see one version of the page quickly replacing another.
To ensure the experimental variant loads instantly, hide the control variant before starting the experiment.
How do I disable a specific experiment type for an entire site or page?
Set the appropriate parameters to false in setConfig.
An element loads after the page. How do I include it in the experiment?
Enable the enableWatch parameter in setConfig.
This parameter enables the page change monitoring mode.
My site has dynamic element IDs. Can I still use the visual editor?
-
In the visual editor, click in the editing mode selection window.
-
Disable the ID of the element in the selector option, then click Save.
Can I add a redirect to a different domain?
We don't provide such functionality for Redirect links experiments for security reasons. Use flags in the code to switch between domains. For more information, see the GitHub repository.
My monetization statistics in Yandex Metrica and the partner interface are different
The values may differ due to different post-processing of data.
To judge an experiment's success, we compare the test and control variants in Yandex Metrica. Post-processing affects them the same way.
I've run an A/A experiment, and the test variant differs from the control one
In A/A testing, this scenario is acceptable. Its probability is equal to the significance level alpha = 0.05. This means that one out of 20 experiments may show statistically significant differences.
If your A/A experiments show statistically significant differences more often, check the experiment configuration for potential problems. If this doesn't help, contact Support.
Audience discrepancy in experiments with redirects
If you see a discrepancy in the audience of an experiment with redirects, make sure that the Varioqub code is correctly added on every page participating in the experiment, including the redirect target pages.