Ai2, a nonprofit AI research organization founded by Paul Allen, has recently introduced a new family of AI models called OLMo 2. This new family, part of the OLMo series, is unique in that it can be reproduced from scratch. OLMo stands for “Open Language Model,” and the release of OLMo 2 marks a significant milestone for Ai2.
The Open Source Initiative has defined open source AI, and OLMo 2 meets this criterion by providing open and accessible training data, open-source training code, reproducible training recipes, transparent evaluations, and intermediate checkpoints. This level of transparency allows the open-source community to explore new and innovative approaches in the field of AI.
The OLMo 2 family consists of two models: OLMo 7B with 7 billion parameters and OLMo 13B with 13 billion parameters. These parameters are indicative of a model’s problem-solving abilities, with models containing more parameters generally performing better. The OLMo 2 models are capable of various text-based tasks such as answering questions, summarizing documents, and writing code.
To train these models, Ai2 utilized a vast dataset of 5 trillion tokens, sourced from websites, academic papers, Q&A discussion boards, and math workbooks. The models were trained on this diverse dataset to ensure high-quality performance across different tasks. Ai2 claims that the OLMo 2 models are competitive with other open models, such as Meta’s Llama 3.1 release.
The OLMo 2 models and their components are available for download on Ai2’s website under the Apache 2.0 license, allowing for commercial use. Despite concerns about the potential misuse of open models, Ai2 believes that the benefits of open models outweigh the risks. By promoting technical advancements, ensuring verification and reproducibility, and reducing power concentration, open models like OLMo contribute to the development of more ethical and equitable AI models.
In conclusion, Ai2’s release of the OLMo 2 family of models represents a significant step forward in the field of AI research. With its commitment to transparency, reproducibility, and accessibility, Ai2 is paving the way for the development of innovative and ethical AI solutions. The OLMo 2 models are not only powerful in performance but also contribute to the democratization of AI technology.
