TII Introduces Falcon Mamba 7B AI Language Model
The Falcon Mamba 7B is recognized as the top-performing open-source State Space Language Model (SSLM) by Hugging Face.
The Technology Innovation Institute (TII) in Abu Dhabi has released the Falcon Mamba 7B, a new model in its Falcon series of large language models. The Falcon Mamba 7B is recognized as the top-performing open-source State Space Language Model (SSLM) by Hugging Face.
This model adopts an SSLM architecture instead of the traditional transformer-based approach. Falcon Mamba 7B surpasses Meta’s Llama 3.1 8B, Llama 3 8B, and Mistral’s 7B models on new benchmarks and will be the first model on Hugging Face’s upcoming leaderboard.
SSLMs are effective at processing complex, time-evolving information without needing additional memory, making them suitable for tasks like estimation, forecasting, and control. Like transformer models, they also perform well in natural language processing tasks such as machine translation, text summarization, and audio processing.
The Falcon series has been downloaded over 45 million times. The Falcon Mamba 7B will be available under TII Falcon License 2.0, promoting responsible AI use.
Recently, G42's Inception launched JAIS 70B, a large language model (LLM) designed to enhance Arabic natural language processing (NLP). JAIS 70B, with 70 billion parameters, aims to support the adoption of generative artificial intelligence (AI) services across various sectors, improving customer service, content creation, and data analysis.
In May, TII unveiled the latest project of its large language model (LLM) series, Falcon 2, with two versions: Falcon 2 11B, an efficient LLM with 11 billion parameters trained on 5.5 trillion tokens, and Falcon 2 11B VLM, featuring vision-to-language capabilities.
Falcon 2 11B VLM stands out as TII’s first multimodal model, boasting image-to-text conversion capabilities. Tested against competitors, including Meta’s newly launched Llama 3, Falcon 2 11B surpasses the performance of the former while matching Google’s Gemma 7B.
Both Falcon 2 11B models are open-source, providing developers worldwide with unrestricted access to AI technology. TII plans to broaden the Falcon 2 series, incorporating advanced machine learning capabilities such as 'Mixture of Experts' (MoE) to elevate performance.