Situation
Toloka is a crowdsourcing platform developed by Yandex that connects businesses and researchers with a distributed crowd to complete various micro-tasks, such as data labeling, image recognition, and content moderation.
Toloka's revenue is connected with client requests' fulfillment, which workers perform. The number of completed tasks per day and the number of days the worker stays active on the platform are critical variables for business margins.
As Senior Marketing Manager, B2C, I was responsible for the acquisition, engagement, and retention of these workers.
Task
The task was to increase Toloka's margins through new workers' acquisition budget optimization. The significant subtask was strengthening product-market fit to grow workers' engagement and retention.
Action
To meet the challenges, I've decided to focus on the following areas:
- find the metrics to evaluate worker's efficiency and calculate workers' unit economy;
- increase workers' retention to key projects;
- forecasting and planning upcoming requests fulfillment.
Here, we will dwell more on actions and results in area 2. We'll review areas 1 and 3 in separate cases.
Area 2 included the following subtasks:
- discover current workers' retention (average for each cohort);
- identify the users with the highest retention rate and average number of performed tasks;
- conduct qualitative interviews with those users and find out their reasoning for that particular behavior;
- adjust marketing and product strategy based on the findings.
To accomplish the mentioned tasks and to lead the whole project, I brought together the efforts of the User Acquisition, Analytics, Attribution, Product, and Development teams.
After I figured out the average retention rate (for each cohort), I performed the following analysis:
- for each cohort, broke down retention by percentile;
- checked the volume of performed tasks dynamics over time;
- checked which share of tasks is performed by relatively new and existing users.
Also, it became clear that we should consider the task complexity from the worker's perspective.
I interviewed core users to understand workers' values and motivation. The key takeaways were:
- core users consider Toloka as the primary or significant source of income and are willing to work more in case they receive extra bonuses;
- predictability and duration of the projects are vital information for core users, with a much higher probability that they will work on a big project.
After that, I formulated the following hypothesis:
- shifting a significant share of the budget from user acquisition to bonus payouts will increase workers' performance by at least 15%, because for core users Toloka is the only significant source of income;
- key project prioritization in the task list will increase the speed of order fulfillment probability by at least 30%, since the average number of impressions drops significantly for each subsequent project in the list;
- clear communication regarding projects' conditions and volume will increase workers' conversion and retention rate on this project by at least 10%, since the worker's income would be much more predictable.
Result
- Retention into the projects with clear (from the worker's perspective) conditions and volume increased by 65%:
- the average number of tasks each worker performs increased by 21% after we introduced the bonus system;
- key projects' labeling time decreased by 34% after projects' prioritization.
- The user acquisition budget for major language segments decreased by 26%, while Toloka's revenue remained unchanged.
- Decreased FTE for operations related to user acquisition by 70%, which helped focus on the optimization of current campaigns.
- As a result, I strengthened the product-market fit for Toloka's product for workers.
"Maria achieved great results in the acquisition, retention, and engagement of ML data makers thanks to her strategic thinking, data analysis skills, project management skills, understanding of marketing tools and technologies, strong work ethic and attention to detail, and continuous improvement mindset."
– Dmitry Stepanov, Founder & GP at AAL VC · Forbes 30 Under 30 · Senior to Maria at Toloka
More cases
