Databricks Releases DBRX, a State-of-the-Art Generative AI LLM, Under a Semi-Open Source License
Data-lake specialist Databricks has announced the release of a semi-open source large language model (LLM), DBRX, which it claims sets a “new standard” for generative artificial intelligence (gen AI) — and that, by the company’s own testing, outperforms rivals including Llama2, Mixtral, Grok, and OpenAI’s GPT-3.5.
“Databricks’ mission is to deliver data intelligence to every enterprise by allowing organizations to understand and use their unique data to build their own AI systems,” the company claims in its announcement of the new LLM. “Today, we are excited to advance our mission by open sourcing DBRX, a general purpose large language model (LLM) built by our Mosaic Research team that outperforms all established open source models on standard benchmarks. We believe that pushing the boundary of open source models enables generative AI for all enterprises that is customizable and transparent.”
Databricks has launched DBRX, a large language model it claims can beat the competition at a range of tasks. (📷: Databricks)
Based on a mixture-of-experts (MoE) model created using the company’s open source MegaBlocks library, DBRX is claimed to offer improved performance by splitting itself into chunks depending on requirements — with the model itself being sized at an impressive 132 billion parameters, but only using 36 billion parameters at any given time to boost the throughput in tokens per second.
Despite this, Databricks claims the model outperforms its competition at a range of tasks — using, admittedly, its own Gauntlet benchmark suite. Testing on language understanding, programming, and math tasks, DBRX is claimed to beat rival open source models Llama2-70B, Mixtral, and Grok-1, as well as OpenAI’s GPT-3.5 — nearly doubling the latter’s score for programming tasks.
“[We] believe that open source LLMs will continue gaining momentum,” Databricks claims in support of its launch. “In particular, we think they provide an exciting opportunity for organizations to customize open source LLMs that can become their IP, which they use to be competitive in their industry.”
Using a mixture-of-experts approach, DBRX is claimed to be able to deliver more tokens per second than rivals despite having more parameters. (📹: Databricks)
DBRX has been released under the custom Databricks Open Model License, which allows for reproduction and distribution but which specifically excludes using DBRX, derivatives, or outputs of same “to improve any other large language model” — and which includes a limit of 700 million monthly active users, after which a license must be requested at unspecified cost.
The company also requires DBRX users to agree to an acceptable use policy, which includes a moratorium on, among other things, using the model to provide medical advice “that is intended to be a substitute for professional medical advice, diagnosis, or treatment” or to “generate or disseminate information and place the information in any public context without expressly and intelligibly disclaiming that the information and/or content is machine generated.”
If the restrictive covenants of the “open” license aren’t a deal-breaker, DBRX is available on GitHub and Hugging Face now; more information on the model is available in Databricks’ technical blog post.
