Mistral AI has released a new Moderation API targeted at increasing content safety with scalable and resilient LLM-based systems that handle different languages and diverse applications.
Mistral said the API, which supports moderation in its Le Chat chatbot platform, can be adapted to individual applications and safety criteria.
The Moderation API is based on the same structure as Mistral AI’s Le Chat platform, which also enables moderation. It offers customers a versatile tool that can be customised to meet specific safety standards and application requirements.
As the demand for large language model (LLM)–based moderation systems increase, Mistral AI’s offering aims to deliver a scalable and robust solution.
“Safety plays a key role in making AI useful,” Mistral’s team said in announcing the release. “At Mistral AI, we believe that system-level guardrails are critical to protecting downstream deployments.”
The new moderation service, which uses a fine-tuned version of Mistral’s Ministral 8B model, is intended to detect potentially problematic content in nine areas, including sexual content, hate speech, violence, dangerous activities, and personally identifiable information. The API supports both raw text and conversational content analysis.
The API includes an LLM classifier that can categorise text inputs into nine different categories. It has endpoints for both raw text and conversational content, which allows it to classify messages within certain conversational settings.
The concept supports a wide range of languages, including Arabic, Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish, making it accessible to a global audience.
We earlier reported that Mistral has released Pixtral 12B, its first multimodal model that can analyse text and images.
The model will soon be available for testing on Mistral’s chatbot and application programming interface platforms, Le Chat and Le Platform, according to a post made on X by Sophia Yang, head of Mistral developer relations.
