Mohamed bin Zayed University Unveils K2-65B LLM

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) has introduced the K2-65B large language model (LLM). Developed in partnership with Petuum, K2-65B is a cost-effective, sustainable LLM designed to redefine standards in open-source AI development.

This unveiling marks a significant expansion of the LLM360 framework, an initiative aimed at simplifying the creation of open-source large language models. The framework, launched last December, seeks to streamline the costly process of LLM training while enhancing transparency and reproducibility.

K2-65B utilizes 35% fewer resources than its predecessor, Llama 2 70B, while delivering comparable performance to higher-capacity models like Llama 3. Trained on 1.4 trillion tokens using 480 NVIDIA A100 Tensor Core GPUs, K2-65B showcases robust reasoning and text generation capabilities across various domains, including medicine, coding, and mathematics.

Moreover, MBZUAI has bolstered its commitment to open-source collaboration by releasing additional resources under the LLM360 framework. These include the LLM360 Research Suite, Developer Suite, Pretraining Suite, and Model Performance and Evaluation Collection, providing a comprehensive toolkit for researchers, developers, and AI practitioners.

With K2-65B now available globally under the Apache 2.0 license, MBZUAI aims to catalyze knowledge sharing and technological advancement in the field of AI.