Policy labelling in seven East African and Philippine languages

The client runs a social media and messaging platform with tens of millions of users in East Africa and the Philippines. Its trust and safety teams write the policies on what may be posted, and use classifiers to find posts that break them, so that fewer people see those posts and fewer moderators have to.

It came to us in April 2022 because its classifiers performed much worse in Kiswahili, Sheng, Somali, Amharic, Filipino, Taglish and Cebuano than in English, and its own reviewers in those languages were too few to produce training data at the volume it needed.

Much of what breaks a policy is not obvious. Hate speech uses coded terms that look like ordinary words; threats are framed as jokes; scams are written as job offers or prize notices; incitement can sit inside a proverb. Meaning depends on local context, particularly around elections, and Sheng changes from month to month. A labeller who does not live in the language will miss most of it.

The platform’s policy is written in English for the whole world and changes several times a year, and every label has to follow the version in force. And the material itself is distressing: violence, hate, sexual content and self-harm come up regularly, which is why the project is graded at content tier S2.

Team

About 110 labellers, five quality reviewers and two team leads, in two separate teams, one at each centre, on the open floor but apart from other projects, with a policy lead in Toronto. As at every tier S2 project, the work is never done through Rungwell at Home. Labellers join after at least a year at Rungwell, a week of training on the policy and a conversation with a counsellor.

Tools

Posts arrive with their text, any images and up to sixty seconds of video, with the author’s identifying details removed by the platform before they reach us. In Rungwell Tasks a labeller chooses the policy area that applies, or “no violation”, highlights the words or the part of the image that decide it, and writes a short reason in English. Images and video open greyscaled and blurred, and sound is off, until the labeller chooses to reveal them. Every task card shows the content warning and tier, and any task with a content warning can be declined with Not for me.

Guidelines

With the platform’s policy team we built a local annex for each language: coded terms and slurs with their meanings and the contexts in which they are and are not abusive, recurring scam scripts, and political terms that matter around elections. The annexes are updated every month. The Kiswahili and Sheng annex has about 1,900 entries.

Quality

The gold set is agreed with the platform’s own in-language experts. Reviewers label 10% of posts a second time, independently, and the platform’s experts label a monthly sample so that agreement can be measured. Both teams calibrate every Thursday, and once a month with the platform’s policy specialists. Material that appears to show the abuse of a child is never assigned for labelling: if it reaches a queue it is removed at once and escalated under the client’s reporting procedure, and the labeller who saw it is offered a counsellor that day.

Wellbeing

Four hours of wellbeing time a month in the shift, counsellors on site during both shifts, a quarterly check-in for everyone on the project, and the right to step away from it at any time without losing pay for the shift. Nobody spends more than six months at a time on the project without at least a month on other work, and there is a daily limit on video items for each labeller.

We have labelled 15,195,800 posts for the platform since April 2022. On the monthly samples, our labels matched those of the platform’s in-language experts on 93.4% of posts in 2026, highest in Filipino and Kiswahili and lowest in Amharic and Sheng.

The platform tells us that, in these seven languages, the share of violating posts its systems found before any user reported them rose from 38% in 2022 to 71% in 2025. Our teams also sent the platform 214 notes on its policy in 2025, cases where the policy did not fit what they were seeing; it changed its guidance in 31 of them.

The two teams include former teachers, a journalist and a pastor, chosen for their judgment as much as for their languages, and most have been with Rungwell for three years or more. In 2025 the Manila team’s cohort representatives asked for a daily limit on video items, and the limit now applies to both teams.

They do some of the hardest work we have. They take their wellbeing time in the shift, have a counsellor on site, and can leave the project at any time, for any reason, without being asked to explain.