In a nutshell: LongStraw enables RL training on 2.1 million tokens with Group Relative Policy Optimization (GRPO) on eight H20 GPUs by optimizing memory accesses and compressing computational graphs through response-branch replay.
Researchers have developed LongStraw, an execution architecture for Reinforcement Learning (RL) with up to 2.1 million token contexts using fixed GPU resources. The system addresses the growing gap between inference lengths and RL post-training, where models have so far remained limited to 256K tokens.
The increasing disparity between inference context lengths and RL post-training capacity is becoming critical for AI agents: inference systems achieve million-token contexts, while post-training workloads typically stagnate at 256K tokens or below, relying on length generalization during deployment. In agents, observations, tool outputs, documents, and prior decisions accumulate over long trajectories.
LongStraw is an architecture-aware execution layer instantiated with Group Relative Policy Optimization (GRPO). The implementation evaluates the shared prompt without autograd overhead, retains only model-specific states for subsequent tokens, and replays response branches sequentially—reducing the active training graph but requiring additional recompute time. The approach was implemented for Qwen 3.6-27B (Hybrid Recurrent + Full Attention) and GLM-5.2 (Compressed Attention Mixture-of-Experts).
On eight H20 GPUs, LongStraw performs scoring and backward pass for Qwen at 2.1 million positions; increasing group size from 2 to 8 adds only 0.21 GB peak memory. A separate stress test achieves 4.46 million positions. On 32 H20 GPUs, the end-to-end path for a 2.1-million-token prompt was validated across all 78 layers of GLM-5.2. The experiments primarily demonstrate execution capacity; the captured prompt state is detached, and some distributed forward and gradient composition paths remain incomplete.
Source: arxiv.org · Published 15 July 2026
Lumi AI News — AI-assisted curation pursuant to Article 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.7.3.