Creating a pool

It is better to test the pool and task suites settings in the sandbox, and then transfer them to the Toloka production version (see Workflow).

To create a pool with main tasks and control tasks:

  1. Prepare the data for tasks.

    If the tasks have images, they need to be available in the internet. For example, they can be stored on Yandex.Disk.

  2. Create a TSV-файл with tasks input data.

    You can include control task answers into the TSV file or enter them later (at step 6) in task editing mode (to do it you have to use “smart mixing”).

  3. Click Add pool on the project page.

  4. Set the pool parameters and click Save.

  5. Load a TSV file with tasks.

    Click Upload on the pool page and choose a method to split the pool into task suites:

    • By empty row.

      Set task suites in the TSV file. After each taskk suite in TSV file, add an empty line.

    • Set manually.

      Define the number of tasks in a task suite. Task suites will be formed from the tasks in the order they are placed in the TSV file.

    • Smart mixing.

      Specify how many tasks of each type should be in a task suite. For example, 10 main tasks and one control task. Tasks will be distributed in the specified proportion. Users will not get repeated tasks. After you load tasks with “smart mixing” you can edit them.

    To see what the tasks look like, click Preview.

  6. If you used “smart mixing”, you can markup the tasks.

To send tasks out for completion, click on the pool page.

Tasks markup

Tasks markup lets you add correct answers and hints to the tasks, as well as edit them. For example, you can turn main task into a training task by adding correct answer and a hint.

Task markup is available only for training pools and main pools, uploaded with “smart mixing” option.

To markup the tasks:

  1. Stop the pool.
  2. Click the Edit button in the Pool tasks section.
  3. Add or edit the correct answer and hint.
  4. Tick the answers that should be checked. (The list of the output data fields is on the right.)
Example of task markup

In this example, the correct answer and a hint are added to the task. Only the value chosen in the "result" field (Cat/ Dog/ Other animal) will be checked.