Kimi K3 as an open frontier model with native vision, million-token context, and 2.5× better scaling efficiency compared to K2, with all weights released.
LongStraw enables RL training on 2.1 million tokens using Group Relative Policy Optimization (GRPO) on eight H20 GPUs by optimizing memory accesses and compressing computational graphs through response-branch replay.
Self-Guided TTT improves long-context processing by having the model itself identify relevant text passages before parameter adaptation, rather than selecting spans randomly.
Sparse Delta Memory significantly increases the state capacity of linear RNNs without raising computational costs, thereby improving long-context and reasoning performance.
FlashMorph converts Transformers into hybrid attention models by optimally determining which layers require full attention and which can be replaced with linear attention.
InfoKV combines attention scores with uncertainty signals for KV-cache compression, outperforming pure attention-based methods on long reasoning tasks by measurable margins.
MSA reduces attention computation for million-token contexts by a factor of 28.4 through blockwise sparse selection and achieves practical speedups via co-design of algorithm and GPU kernel.
LSA predicts relevant context sections in advance and retains only these in GPU memory, compressing the KV-cache by over 86 percent without sacrificing accuracy.
LCLMs compress KV-caches through encoder-decoder architecture up to 1:16 more efficiently than previous methods while reducing peak memory consumption and processing time.
Encoder-decoder compressors with adaptive expansion improve KV-cache compression methods in speed and memory efficiency without significant quality loss.
