Verification using the opinion of the majority (Majority vote) defines how well the user's responses correspond to the responses of other users.For example, if a task has an overlap of 5 and three users selected the same answer, this is considered indirect confirmation of the correct response.
The skill level is calculated using the formula:
is the number of the user's responses that matched the majority.
is the number of the user's tasks where the majority all answered identically.
To use verification by the majority:
Create a skill on the Skills page in the production version of Toloka.
Set up a quality control rule on the pool page (click Add Quality Control Rule → Majority vote).
Field or group of fields
The number of matching responses that is considered the “majority vote” (for example, 3). The threshold should be less than or equal to the overlap.
A condition for performing the action in the then field. For example: “Number of responses ≥ 10”.
Action to perform for the condition:
Add a skill filter on the pool page (in the Users filter section). For example: “The myskill skill is missing or ≥ 60”.