Sparse Delta Memory: Linear RNNs with Sparse Memory Scale State Size Substantially9. July 2026AI ModelsSparse Delta Memory significantly increases the state capacity of linear RNNs without raising computational costs, thereby improving long-context and reasoning performance. Share on:
Hybrid LLMs Lose Long-Context Capabilities Through CoT Fine-Tuning10. June 20264. July 2026AI ModelsCoT fine-tuning degrades long-context retrieval in hybrid LLMs by distorting query-key projections; QK-Restore fixes this without additional training. Share on:
