Mistral 7B from Mistral.AI - FULL WHITEPAPER OVERVIEW
Mistral 7B, a language model with 7 billion parameters designed for superior performance and efficiency. Mistral 7B surpasses the performance of the best open 13B model (Llama 2) across all evaluated benchmarks. It also outperforms the best released 34B model (Llama 1) in reasoning, mathematics, and code generation. The model utilizes grouped-query attention (GQA) for faster inference and sliding window attention (SWA) to handle sequences of arbitrary length efficiently.
Mistral 7B - Instruct, a fine-tuned model that outperforms Llama 2 13B - chat model on both human and automated benchmarks. The models are released under the Apache 2.0 license.