Unprofane, the Perspective API, OpenAI moderation and 4 npm filters on the same 9,029 texts in 17 languages, labelled blind by AI judges. Perspective and OpenAI return a score, so each is also read where it flags as much clean text as Unprofane does, and then the one that catches more toxic text is the better filter.
The private test set, read once on 2026-09-28, engine 2026-09-29. The breakdowns further down are from the development set and are labelled so.
Toxic text caught, Unprofane
89.0%
flagging 7.0% of clean text, balanced, standard policy
Perspective, same false-alarm rate
58.4%
TOXICITY or PROFANITY at 0.479, flagging 7.0% of clean text
OpenAI moderation, same false-alarm rate
40.8%
its largest abuse category at 0.689, flagging 7.0% of clean text
Each group reads every system at one share of clean text flagged: Unprofane's own, then each competitor's as it ships. Toxic text caught, on the private test set.
7.0% of clean text flagged
Unprofane's standard policy
Thresholds: Perspective at 0.479, OpenAI moderation at 0.689.
20.6% of clean text flagged
OpenAI moderation, its own flagged verdict
1.0% of clean text flagged
Perspective, TOXICITY or PROFANITY at 0.7 (shipped by Coral and Discourse)
Unprofane cannot be read this low: the texts its rules block outright, whatever the model says, already flag 1.6% of clean text.
The private test set, read once, totals only. Clean flagged and toxic caught are the main draw; the other columns are the rest of the set.
| System | Clean flagged | Toxic caught | Rude acted on | Clean usernames flagged | Toxic usernames caught | Disguises caught | Look-alikes flagged | All clean flagged |
|---|---|---|---|---|---|---|---|---|
| Unprofanebalanced, standard policy | 7.0% | 89.0% | 66.8% | 1.9% | 76.5% | 86.0% | 5.5% | 5.5% |
| Unprofanefast: the rules alone, standard policy | 3.6% | 53.4% | 8.2% | 1.0% | 57.4% | 61.9% | 0.8% | 2.8% |
| Unprofanebalanced, read at the false-alarm rate of OpenAI moderation, its own flagged verdict | 20.5% | 96.4% | 96.8% | 16.0% | 93.9% | 96.3% | 26.3% | 19.2% |
| PerspectiveTOXICITY or PROFANITY at 0.7 (shipped by Coral and Discourse) | 1.0% | 33.0% | 1.8% | 0.0% | 20.0% | 22.9% | 0.0% | 0.7% |
| Perspectiveat 0.5 | 6.4% | 56.5% | 10.9% | 0.8% | 31.6% | 49.3% | 0.7% | 4.8% |
| Perspectiveat 0.479, Unprofane's false-alarm rate | 7.0% | 58.4% | 12.3% | 1.1% | 32.5% | 51.7% | 0.8% | 5.3% |
| OpenAI moderationomni-moderation-latest, its own flagged verdict | 20.6% | 62.2% | 35.4% | 0.7% | 18.8% | 45.5% | 3.8% | 14.8% |
| OpenAI moderationomni-moderation-latest, largest abuse category at 0.689, Unprofane's false-alarm rate | 7.0% | 40.8% | 9.1% | 0.0% | 9.0% | 26.9% | 0.5% | 5.0% |
| obscenity 0.4.6its default list, flagged or not | 4.2% | 8.4% | 2.3% | 1.5% | 10.7% | 7.2% | 0.3% | 3.4% |
| bad-words 4.1.5its default list, flagged or not | 1.3% | 9.7% | 1.8% | 0.1% | 4.1% | 4.0% | 0.5% | 0.9% |
| leo-profanity 1.9.0its default list, flagged or not | 1.1% | 5.1% | 0.4% | 0.0% | 2.0% | 2.5% | 0.0% | 0.8% |
| 2toad-profanity 3.3.0its default list, flagged or not | 0.7% | 6.1% | 0.9% | 0.1% | 3.2% | 3.1% | 0.2% | 0.5% |
The development set, where the engine was tuned, so these rates flatter it. The share of clean text flagged, by kind, across all of the set's clean text, with Perspective and OpenAI moderation each read at Unprofane's overall rate.
| Kind of clean text | Texts | Unprofane | Perspective | OpenAI moderation |
|---|---|---|---|---|
| Abuse quoted or reported | 1,007 | 11.5% | 4.3% | 15.7% |
| Ordinary text, nothing hard about it | 3,115 | 8.9% | 1.6% | 3.3% |
| An innocent word that contains a profane string | 229 | 7.9% | 3.1% | 0.0% |
| Counter-speech: pushing back on abuse | 1,557 | 5.1% | 6.6% | 11.3% |
| A slur reclaimed by the group it names | 1,387 | 4.5% | 10.4% | 2.6% |
| A mention of an identity | 1,761 | 2.2% | 1.0% | 1.7% |
| A violent idiom ("killing it") | 1,610 | 1.4% | 8.9% | 13.7% |
| The title of a film, book or song | 882 | 0.9% | 1.1% | 0.0% |
| A place name | 1,847 | 0.8% | 1.3% | 0.0% |
| A surname | 1,268 | 0.5% | 0.3% | 0.0% |
| Where the clean text came from | Texts | Unprofane | Perspective | OpenAI moderation |
|---|---|---|---|---|
| The main draw from open datasets | 3,477 | 5.0% | 5.0% | 5.0% |
| Usernames | 1,459 | 2.0% | 0.5% | 0.1% |
| Clean look-alikes of disguised spellings | 598 | 4.9% | 0.8% | 0.5% |
| Abuse quoted or rebutted, names set in sentences, built from the datasets | 3,123 | 4.7% | 1.7% | 2.6% |
| Reclaimed words, idioms and identity mentions, written from templates | 5,428 | 1.4% | 5.1% | 6.5% |
| Real text a separate filter picked as likely abuse, first round | 520 | 10.6% | 2.5% | 5.2% |
| Real text a separate filter picked as likely abuse, second round, never tuned on | 656 | 25.0% | 3.2% | 13.7% |
The development set. Every group of texts where Perspective or OpenAI moderation, read at Unprofane's overall false-alarm rate, is right on more texts than Unprofane is. Every one is clean text that Unprofane flags and the competitor lets through. On toxic text no competitor catches more at the same rate, in any language, category, source, username or disguise group.
| Group | Against | Texts | Only the competitor is right | Only Unprofane is right |
|---|---|---|---|---|
| Ordinary text, nothing hard about it | Perspective | 3,115 | 253 | 26 |
| Ordinary text, nothing hard about it | OpenAI moderation | 3,115 | 237 | 65 |
| Real text a separate filter picked as likely abuse, second round, never tuned on | Perspective | 656 | 154 | 11 |
| Abuse quoted or rebutted, names set in sentences, built from the datasets | Perspective | 3,123 | 125 | 30 |
| Real text a separate filter picked as likely abuse, second round, never tuned on | OpenAI moderation | 656 | 127 | 53 |
| Abuse quoted or reported | Perspective | 1,007 | 99 | 26 |
| Abuse quoted or rebutted, names set in sentences, built from the datasets | OpenAI moderation | 3,123 | 136 | 69 |
| Real text a separate filter picked as likely abuse, first round | Perspective | 520 | 48 | 6 |
| Clean usernames | OpenAI moderation | 1,459 | 28 | 0 |
| Real text a separate filter picked as likely abuse, first round | OpenAI moderation | 520 | 49 | 21 |
The 10 largest of 40 groups, by the difference. The rest are smaller.
The development set's main draw. Each system is read at Unprofane's overall false-alarm rate, so a language's own rate varies. Perspective has no model for Hebrew or Turkish and lets that text through.
| Language | Toxic caught | Clean flagged | ||||
|---|---|---|---|---|---|---|
| Unprofane | Perspective | OpenAI moderation | Unprofane | Perspective | OpenAI moderation | |
| Arabic ar | 87.8% | 13.7% | 20.8% | 12.0% | 2.4% | 6.8% |
| German de | 91.0% | 42.7% | 39.9% | 5.7% | 3.8% | 8.1% |
| English en | 96.4% | 73.2% | 58.0% | 8.4% | 8.4% | 2.4% |
| Spanish es | 94.3% | 74.6% | 42.0% | 1.8% | 2.3% | 9.6% |
| French fr | 95.2% | 65.3% | 58.1% | 4.2% | 2.1% | 6.3% |
| Hebrew he | 91.7% | 0.0% | 18.3% | 4.1% | 0.0% | 0.0% |
| Hindi hi | 89.2% | 56.8% | 13.6% | 7.7% | 12.7% | 3.2% |
| Indonesian id | 93.4% | 44.6% | 25.3% | 8.0% | 2.9% | 3.8% |
| Italian it | 91.1% | 46.1% | 42.9% | 5.8% | 3.1% | 5.3% |
| Japanese ja | 91.2% | 62.6% | 29.7% | 1.6% | 4.1% | 0.0% |
| Korean ko | 84.1% | 36.5% | 10.3% | 2.4% | 2.4% | 0.0% |
| Dutch nl | 94.2% | 67.6% | 69.1% | 0.9% | 9.3% | 13.2% |
| Polish pl | 92.3% | 49.4% | 42.1% | 2.4% | 4.3% | 7.1% |
| Portuguese pt | 94.0% | 69.4% | 55.2% | 2.1% | 6.4% | 4.3% |
| Russian ru | 96.7% | 71.1% | 31.1% | 3.7% | 0.7% | 0.7% |
| Turkish tr | 81.3% | 0.0% | 7.7% | 6.8% | 0.0% | 1.1% |
| Chinese zh | 91.6% | 61.4% | 39.1% | 3.5% | 14.2% | 5.3% |
The development set's main draw: toxic text caught, by what the judges said it is, each system at Unprofane's overall false-alarm rate.
| Category | Texts | Unprofane | Perspective | OpenAI moderation |
|---|---|---|---|---|
| harassment | 109 | 90.8% | 52.3% | 35.8% |
| hate | 1,075 | 92.6% | 55.0% | 56.2% |
| insult | 975 | 91.1% | 51.7% | 32.6% |
| profanity | 996 | 96.9% | 70.9% | 29.6% |
| sexual | 346 | 90.5% | 48.8% | 26.9% |
| threat | 536 | 95.2% | 66.2% | 71.6% |
As measured on the development set's run of 2026-09-28. Each figure is measured differently, and the notes under the table say how.
| System | p50 ms | p99 ms | $ per 1,000 |
|---|---|---|---|
| Unprofanebalanced, standard policy | 310 | 546 | $0.29 |
| Unprofanefast: the rules alone, standard policy | <1 | 3 | $0.29 |
| Perspective | 236 | 298 | free |
| OpenAI moderationomni-moderation-latest | 914 | 1471 | free |
| obscenity 0.4.6 | <1 | <1 | free |
| bad-words 4.1.5 | <1 | <1 | free |
| leo-profanity 1.9.0 | <1 | <1 | free |
| 2toad-profanity 3.3.0 | <1 | <1 | free |
The texts come from open third-party datasets whose licences allow evaluation, in 17 languages, up to the same number from each language and kind of text: comments, chat and general text, clean and toxic, with the clean text that fools filters sampled on purpose (quoted abuse, counter-speech, reclaimed words, violent idioms, identity mentions, places, surnames and titles). Usernames and disguised spellings are measured apart, each disguise beside a clean look-alike made the same way. The development set adds clean text built to be hard and real text picked by a separate filter as likely abuse.
Each text is labelled blind by two AI judges (Claude) from the text, its language and its context alone, under a written rubric (rubric version 2); a third judge decides where they disagree. No judge sees where a text came from or what any system answered. Toxic means a standard policy for a general-audience platform acts on it for profanity, insults, harassment, hate, threats or sexual content.
The texts are split in two. The development set, 19,584 texts, is where Unprofane was tuned, and on it the same engine flags 5.0% of clean text and catches 91.8% of toxic text. The private test set, 9,029 texts, was read once, on engine 2026-09-29 as released, and gives the headline. It is read as totals only.
Unprofane runs its standard policy in balanced mode: the rules read every text, and the model reads every text the rules do not block outright. Perspective is read as TOXICITY or PROFANITY at a threshold; OpenAI moderation as its own flagged verdict, or as its largest abuse category at a threshold. Their answers were fetched once and kept, and the npm filters ran in process with their default lists. Every system answered every text; a text one system could not answer is dropped for all. Other systems appear only where their terms allow publishing a benchmark.
For disguised spellings and innocent words in depth, with a set you can download and run yourself, see the Scunthorpe Test.