Mistral, a French AI startup, has released Pixtral 12B, its first multimodal model that can analyse text and images.
The model will soon be available for testing on Mistral’s chatbot and application programming interface platforms, Le Chat and Le Platforme, according to a post made on X by Sophia Yang, head of Mistral developer relations.
Newsng gathered that Nemo 12B, a text model created by Mistral, serves as the foundation for Pixtral 12B.
Like existing multimodal models like Anthropic’s Claude and OpenAI’s GPT-4, the new model has a 400-million-parameter vision adapter that enables users to input images alongside text for tasks like image captioning, counting objects in an image, and image categorisation.
Images can be supplied as base64 encoded files or via URLs. Pixtral 12B processes photos by partitioning them into 16 by-16-pixel patches, which makes it better suited to handle high-resolution images.
The model’s visual encoder makes use of 2D RoPE (Rotary Position Embeddings), which helps it comprehend spatial relationships in the supplied images.
With a 24 GB weight, the model is freely available for usage, modification, and commercialisation under the Apache 2.0 license.
Although working online demos aren’t yet available, developers can obtain them from Hugging Face and GitHub.
Pixtral 12B comes after Mistral closed a $645 million fundraising round sponsored by General Catalyst, valued at $6 billion.
Mistral, which is a minority-owned subsidiary of Microsoft, is often regarded as Europe’s answer to OpenAI despite its youth.
We earlier reported that Mistral has unveiled Codestral, the first generative AI model designed only for coding.
