Mistral AI and NVIDIA today launched a new next-generation language model, Mistral NeMo 12B, which developers can easily customize and deploy for enterprise applications that support chatbots, multilingual tasks, coding, and summarization.
By combining Mistral AI’s training data expertise with NVIDIA’s optimized hardware and software ecosystem, the Mistral NeMo model delivers high performance for a variety of applications.
“We are fortunate to collaborate with the team at NVIDIA, leveraging their world-class hardware and software,” said Guillaume Lample, co-founder and chief scientist at Mistral AI. “Together, we have developed a model with unprecedented accuracy, flexibility, high efficiency, and enterprise-grade support and security through the implementation of NVIDIA AI Enterprise.”
Mistral NeMo was trained in the NVIDIA DGX Cloud Artificial intelligence platform that offers dedicated, scalable access to the latest NVIDIA architecture.
NVIDIA TensorRT-LLM for accelerated inference performance on large language models and NVIDIA NeMo A development platform was also used to create custom generative AI models to advance and optimize the process.
This collaboration underscores NVIDIA’s commitment to supporting the modeling ecosystem.
Offering unprecedented precision, flexibility and efficiency
Excelling at multi-turn conversations, math, common sense reasoning, world knowledge, and coding, this enterprise-grade AI model delivers accurate and reliable performance across a variety of tasks.
With a context length of 128K, Mistral NeMo processes large and complex information more consistently and accurately, ensuring contextually relevant results.
Mistral NeMo is a 12 billion parameter model released under the Apache 2.0 license, which encourages innovation and supports the broader AI community. Additionally, the model uses the FP8 data format for model inference, which reduces memory size and speeds up deployment without degrading accuracy.
This means that the model learns tasks better and handles diverse scenarios more effectively, making it ideal for enterprise use cases.
Mistral NeMo comes packaged as a NVIDIA NIM Inference microservice that delivers performance-optimized inference using NVIDIA TensorRT-LLM engines.
This containerized format allows for easy deployment anywhere, providing greater flexibility for diverse applications.
As a result, models can be deployed anywhere in a matter of minutes, rather than several days.
NIM has enterprise-grade software that is part of NVIDIA AI for Businesswith dedicated feature branches, rigorous validation processes, and enterprise-grade support and security.
Includes comprehensive support, direct access to an NVIDIA AI expert, and defined service level agreements, delivering consistent, reliable performance.
The open model license allows companies to seamlessly integrate Mistral NeMo into commercial applications.
Designed to fit the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090, or NVIDIA RTX 4500 GPU, the Mistral NeMo NIM offers high efficiency, low compute costs, and enhanced security and privacy.
Development and customization of advanced models
The combined expertise of Mistral AI and NVIDIA engineers has optimized training and inference for Mistral NeMo.
Trained with Mistral AI’s expertise, especially in multilingual, multi-turn code and content, the model benefits from accelerated training on the full NVIDIA stack.
It is designed for optimal performance, utilizing efficient model parallelism, scalability, and mixed-precision techniques with Megatron-LM.
The model was trained using Megatron-LMpart of NVIDIA NeMowith 3072 80GB Tensor Core H100 GPUs on DGX Cloud, powered by NVIDIA AI architecture, including accelerated computing, network fabric, and software to increase training efficiency.
Availability and deployment
With the flexibility to run anywhere—cloud, datacenter, or RTX workstation—Mistral NeMo is ready to revolutionize AI applications across multiple platforms.
Experience Mistral NeMo as NVIDIA NIM today via ai.nvidia.comwith a downloadable NIM coming soon.
See warning regarding software product information.
