How It Works

​

The detection pipeline

​

1. Content ingestion & language detection

1. Explicit code — If you pass a language parameter, it is used directly.
2. Trigram detection — If no explicit language is given, the API runs trigram-based analysis on the input text.
3. LLM confirmation — The LLM also identifies the content language during analysis. When the LLM’s detection is a supported language, it takes precedence — this ensures correct detection for closely related languages like Norwegian, Swedish, and Danish.

​

2. Multi-model classification

​

3. Context engine

• Linguistic intent — Is “I want to kill myself” an expression of frustration over a video game, or a genuine crisis signal? Tuteliq analyzes surrounding context, tone, and conversational history to distinguish the two.
• Relationship dynamics — A single message may appear harmless. The context engine tracks multi-turn escalation patterns — compliments, then secrecy requests, then isolation attempts, then boundary violations — that only become visible across a conversation. Every conversation-aware endpoint returns a message_analysis array that shows exactly how risk escalates message by message, with individual risk scores and detected tactics for each entry.
• Platform norms — Teen slang, gaming culture, and social media language evolve fast. The context engine recognizes that “I’m literally dead” in a group chat has a fundamentally different risk profile than the same phrase in a private message to a younger child.
• Adversarial fragmentation — Bad actors sometimes try to evade detection by spreading a phrase across many one-character or two-character messages. Conversation-aware endpoints reassemble runs of consecutive same-role short messages before analysis, so "d","o","n","t","t","e","l","l" is evaluated as "donttell". Reassembled fragments score at a lower severity than the same intent written naturally (the underlying signal is weaker), but recommended_action still escalates when the verdict is medium or above. Branch alerting on recommended_action, not on severity === 'critical' — that way reassembled grooming attempts still reach a moderator.
• Long conversation handling — detect_grooming is designed for conversation windows up to about 20 turns; this is where the multi-turn risk-trajectory analysis is sharpest. For longer threads, pass the response’s continuation_token back on the next call and analyse the next chunk — server state stays on the customer’s side and the trajectory carries across windows. If a conversation is submitted that exceeds the engine’s processing window, the response returns analysis_status: "engine_error" with recommended_action: "flag_for_moderator" rather than a generic failure — escalate to a human moderator regardless of the risk score in that case.

​

4. Age-calibrated scoring

​

5. Response generation

​

Context fields

​

Crisis support resources (support_threshold)

{
  "text": "I don't want to be here anymore",
  "context": { "ageGroup": "13-15" },
  "options": { "support_threshold": "medium" }
}

• unsafe (boolean, legacy endpoints) or detected (boolean, new detection endpoints) — Clear yes/no for immediate routing decisions. Legacy endpoints return unsafe; newer detection endpoints use detected instead.
• categories (array) — Which KOSA harm categories were triggered.
• severity (string) — low, medium, high, or critical, calibrated to the age group.
• risk_score (float, 0.0–1.0) — Granular score for threshold-based automation.
• confidence (float) — Model confidence in the classification.
• rationale (string) — Human-readable explanation of why the content was flagged. Useful for trust & safety review and audit trails.
• message_analysis (array, conversation-aware endpoints) — Per-message risk breakdown, returned when conversation_history is provided. Each entry contains message_index, risk_score, flags, and summary, making the escalation sequence visible for dashboards and reporting. Available on grooming, social engineering, app fraud, romance scam, mule recruitment, gambling harm, coercive control, vulnerability exploitation, and radicalisation endpoints.
• recommended_action (string) — Suggested next step, such as “Escalate to counselor” or “Block and report.”
• language (string) — Resolved language code (ISO 639-1) used for analysis, auto-detected or explicit.
• language_status (string) — "stable" for English, "beta" for all other supported languages.

​

Beyond detection

​

Action plan generation

• For children — Gentle, reading-level-appropriate language explaining what happened and what to do next.
• For parents — Clear explanation of the detected risk with suggested conversations and resources.
• For trust & safety teams — Technical summary with recommended platform actions and escalation paths.

​

Incident reports

​

Architecture principles

​

Next steps