DBRX

DBRX
DevelopersMosaic ML and Databricks team
ReleaseMarch 27, 2024
LicenseDatabricks Open Model License
Websitehttps://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Repositoryhttps://github.com/databricks/dbrx

DBRX is a large language model (LLM) developed by Mosaic under its parent company Databricks, released on March 27, 2024 under the Databricks Open Model License. It is a mixture-of-experts transformer model, with 132 billion parameters in total. 36 billion parameters (4 out of 16 experts) are active for each token. The released model comes in either a base foundation model version or an instruction-tuned variant.

At the time of its release, DBRX outperformed prominent models such as Meta's Llama 2, Mistral AI's Mixtral, and xAI's Grok-1, in several benchmarks ranging from language understanding, programming ability and mathematics.

It was trained for 2.5 months and reported using on 3,072 Nvidia H100s connected by 3.2 terabytes per second bandwidth (InfiniBand), for a training cost of US$10M.[non-primary source needed]

References

  1. ↑ "Databricks Open Model License". Databricks. 27 March 2024. Retrieved 6 August 2025.
  2. ↑ "Databricks Open Model Acceptable Use Policy". Databricks. 27 March 2024. Retrieved 6 August 2025.
  3. 1 2 "Introducing DBRX: A New State-of-the-Art Open LLM". Databricks. 2024-03-27. Retrieved 2024-03-28.
  4. ↑ Avidon, Eric (March 27, 2024). "New Databricks open source LLM targets custom development". Tech Target. Retrieved 2024-03-28.
  5. ↑ Ghoshal, Anirban (2024-03-27). "Databricks' open-source DBRX LLM beats Llama 2, Mixtral, and Grok". InfoWorld. Retrieved 2024-03-28.
  6. 1 2 "A New Open Source LLM, DBRX Claims to be the Most Powerful – Here are the Scores". GIZMOCHINA. Mar 28, 2024.
  7. ↑ Wiggers, Kyle (2024-03-27). "Databricks spent $10M on new DBRX generative AI model". TechCrunch. Retrieved 2024-03-29.
  8. ↑ "Data and AI company DataBrix has launched a general-purpose large language model (LLM) DBRX that out." Maeil Business Newspaper. 2024-03-28. Retrieved 2024-03-28.
  9. 1 2 Knight, Will (March 27, 2024). "Inside the Creation of the World's Most Powerful Open Source AI Model". Wired. ISSN 1059-1028. Retrieved 2024-03-28.