TL;DR
- The answer came from interviews, not dashboards: core users treat Toloka as a primary source of income, so predictability and duration of a project matter more than its rate.
- Three things followed – bonuses tied to volume, key projects prioritized in the task list, and explicit communication of each project’s conditions – each with a numeric hypothesis written before the work.
- Retention rose 65% against a 10% hypothesis; tasks per worker +21% after bonuses; key-project labeling time −34% after prioritization.
- The acquisition budget fell 26% with revenue unchanged, and operations FTE fell 70%.
Situation
Toloka is a crowdsourcing platform built inside Yandex – spun out in 2025 with Nebius Group (ex-Yandex) as its majority shareholder – that connects businesses and researchers with a distributed crowd to complete various micro-tasks, such as data labeling, image recognition, and content moderation.
Toloka's revenue is connected with client requests' fulfillment, which workers perform. The number of completed tasks per day and the number of days the worker stays active on the platform are critical variables for business margins.
As Senior Marketing Manager, B2C, I was responsible for the acquisition, engagement, and retention of these workers.
Task
The task was to increase Toloka's margins through new workers' acquisition budget optimization. The significant subtask was strengthening product-market fit to grow workers' engagement and retention.
Action
To meet the challenges, I've decided to focus on the following areas:
- find the metrics to evaluate worker's efficiency and calculate workers' unit economy;
- increase workers' retention to key projects;
- forecasting and planning upcoming requests fulfillment.
Here, we will dwell more on actions and results in area 2. We'll review areas 1 and 3 in separate cases.
Area 2 included the following subtasks:
- discover current workers' retention (average for each cohort);
- identify the users with the highest retention rate and average number of performed tasks;
- conduct qualitative interviews with those users and find out their reasoning for that particular behavior;
- adjust marketing and product strategy based on the findings.
To accomplish the mentioned tasks and to lead the whole project, I brought together the efforts of the User Acquisition, Analytics, Attribution, Product, and Development teams.
After I figured out the average retention rate (for each cohort), I performed the following analysis:
- for each cohort, broke down retention by percentile;
- checked the volume of performed tasks dynamics over time;
- checked which share of tasks is performed by relatively new and existing users.
Also, it became clear that we should consider the task complexity from the worker's perspective.
I interviewed core users to understand workers' values and motivation. The key takeaways were:
- core users consider Toloka as the primary or significant source of income and are willing to work more in case they receive extra bonuses;
- predictability and duration of the projects are vital information for core users, with a much higher probability that they will work on a big project.
After that, I formulated the following hypothesis:
- shifting a significant share of the budget from user acquisition to bonus payouts will increase workers' performance by at least 15%, because for core users Toloka is the only significant source of income;
- key project prioritization in the task list will increase the speed of order fulfillment probability by at least 30%, since the average number of impressions drops significantly for each subsequent project in the list;
- clear communication regarding projects' conditions and volume will increase workers' conversion and retention rate on this project by at least 10%, since the worker's income would be much more predictable.
Result
- Retention into the projects with clear (from the worker's perspective) conditions and volume increased by 65%:
- the average number of tasks each worker performs increased by 21% after we introduced the bonus system;
- key projects' labeling time decreased by 34% after projects' prioritization.
- The user acquisition budget for major language segments decreased by 26%, while Toloka's revenue remained unchanged.
- Decreased FTE for operations related to user acquisition by 70%, which helped focus on the optimization of current campaigns.
- As a result, I strengthened the product-market fit for Toloka's product for workers.
Results in numbers
| Metric | Target | Result | How measured |
|---|
| Retention on projects with clear conditions | ≥ 10% | +65% | Retention by cohort on the project |
| Tasks performed per worker | ≥ 15% | +21% | Average per worker after the bonus system; the hypothesis was worded as worker performance |
| Key-project labeling time | ≥ 30% faster fulfillment | −34% | Time to label after prioritization in the task list; the hypothesis was worded as fulfillment speed |
| Acquisition budget, major language segments | – | −26% | Budget against revenue; revenue held flat |
| Operations FTE on user acquisition | – | −70% | FTE on acquisition-related operations |
Table 4. Targets are the three hypotheses stated before the work; results are as stated in this case.
"Maria achieved great results in the acquisition, retention, and engagement of ML data makers thanks to her strategic thinking, data analysis skills, project management skills, understanding of marketing tools and technologies, strong work ethic and attention to detail, and continuous improvement mindset."
– Dmitry Stepanov, Founder & GP at AAL VC · Forbes 30 Under 30 · Senior to Maria at Toloka
Written by Maria Atamanova – Senior Growth & Performance Marketing Manager and applied mathematician; twelve years growing businesses – apps, e-commerce, EdTech, AI data platforms – on numbers, not gut feeling. The three hypotheses in this case were written before the work and are reproduced above unchanged, including the ones that undershot the result.
More cases
