Skip to main content
Every platform draws its own lines. A gaming community and a tutoring product can run the same detectors and reasonably want different thresholds, different escalation paths, and different definitions of “act now”. Policy automation lets you encode those decisions as server-side rules that evaluate on every detection, so the judgement lives with your safeguarding policy rather than scattered through your application code.
Available on Assure and Enterprise.

What a rule looks like

A rule is a set of conditions and an action. When a detection matches the conditions, the action is applied and returned to you in policy_action.
Conditions can combine a minimum risk score, a minimum severity, specific categories, and age groups. Endpoints scopes the rule to particular detectors, or ["*"] for all of them. Actions are one of: Rules are evaluated in priority order, so you can put narrow, high-stakes rules above broad ones.

Managing rules

The same operations are available as MCP tools if you drive moderation from an agent.

Test before you enforce

POST /api/v1/policy/automation/evaluate runs a hypothetical detection result through your rule set and returns what would have happened, changing nothing. Combined with log_only, this gives you a safe path to introducing a rule: write it as log_only, watch what it would have caught for a week, then switch it to block or escalate once you trust it. Tightening a moderation threshold without knowing its false-positive rate is how platforms end up suppressing ordinary users, and this is the way to avoid it.

Where the result appears

Every detection response carries the outcome:
rule_id and message appear only when a rule matched and, for message, only when that rule defines one. On /analyse/multi, each result in results carries its own policy_action, from the rules that name that result’s detector. The same content gets the same answer through either route. If no rule matches, action is allow and your existing handling applies unchanged. Adding the policy engine to a running integration is therefore non-breaking: rules do nothing until you write one.

Incident logging

What happens to a detection after a rule fires.

Webhooks

Delivering notify and escalate outcomes to your systems.