How does Toloka work?
To place a task in Toloka, create three different entities: a project, a pool, and the tasks to be uploaded. Each of these entities is responsible for its own task settings block. This section explains why Toloka works this way and what its advantages are.
A project contains instructions, task interface settings, and a specification for input and output data. You can also set up quality control rules that apply to the entire project.
Projects make it easier for you to post similar tasks in the future, because you don't have to re-configure the interface. You can also view aggregate statistics for all tasks in the project.
A pool is a set of tasks to be assigned to performers simultaneously. Main pool settings: pricing, selection of performers, overlap, and quality control.
Select various performers for tasks.
For example, text in tasks for comment moderation can be in different languages: some comments in Russian and others in English. You can split these comments into different pools and use filters to specify the language that performers should know.
Set a price, the overlap, and configure quality control based on the complexity of tasks.
For example, tasks for transcribing audio files that are 10-20 seconds and 50-60 seconds long should have a different price.
Manage task priority by assigning one pool before another.
Group tasks to view separate statistics for each group.
A task is the data you need to mark up. Each image or comment is a separate task.
There are different types of tasks: regular, control (with correct responses), and training (with hints).
You upload tasks to Toloka as a TSV file. The file structure must match the input and output data specification that you set up in the project.
Tasks are assigned to performers page by page. A page may contain one or several tasks. You specify their number when uploading tasks in the pool.
- A performer's response will only be registered after they complete all the tasks on the page.
- You specify the price per task suite. Performers are only paid for the entire page of completed tasks.
Some performers have a slow internet connection, so adding multiple tasks lets them update a page less often, which means they'll complete your tasks faster.
However, don't include too many tasks on a page: quality will be worse and performers will get tired. Place 5-10 simple tasks or 1-4 complex ones on a single page.
What does it look like for performers?
When you start a pool, performers can see it in the Tasks section. They select the pool, read the instructions, and complete tasks.
Each task in a task suite is part of a web page that can include various objects, such as images, text, and input fields.
For more information, see How performers see pools.
If you need to frequently post similar tasks, automate this process with the Toloka API.
If you want more flexibility when posting tasks, the Toloka API has additional features that are not yet available in the interface. For example, use the API to specify overlap for individual tasks and add multiple correct responses to control tasks.