Tag: DeepMind
Interpreting LLMs with Sparse Autoencoders: DeepMind’s Breakthrough
Large language models (LLMs) have been advancing rapidly in recent years, but the challenge of understanding how they work remains. Researchers at artificial intelligence...
Scaling Language Models with Millions of Tiny Experts: DeepMind’s PEER Approach
DeepMind has introduced a new approach called Parameter Efficient Expert Retrieval (PEER) to address the limitations of current Mixture-of-Experts (MoE) techniques used in scaling...