Someone has finally broken the memory bottleneck of LLM reasoning from an incremental perspective; now online learning scenarios can run more efficiently.

Research proposes Δ-Mem, an online memory system designed for large language models. By only storing and updating incremental changes in activations, abandoning the complete activation state, it significantly reduces memory usage. Experiments show memory consumption can decrease by up to 70%, with minimal loss in output quality, enhancing online inference and continual learning capabilities in resource-constrained environments.