GaLore: Breakthrough Technique Enables Large AI Model Training on Regular Laptops

Researchers have introduced GaLore, a novel method that dramatically reduces the memory required to train large language models, making it feasible to fine-tune billion-parameter models on consumer-grade GPUs like the NVIDIA RTX 4090. Traditional full-parameter training demands vast memory for gradients and optimizer states, often exceeding the capacity of typical hardware. GaLore leverages a memory-efficient low-rank projection of gradients, allowing models with up to 7 billion parameters to be fine-tuned on a single 24GB GPU without sacrificing performance. This approach could democratize AI development by lowering hardware barriers, enabling more researchers and hobbyists to work with state-of-the-art models. The technique is particularly promising for parameter-efficient fine-tuning and has been validated across various model architectures and tasks.