Loading
Life sciences · Preprint
arXiv · August 12, 2026
Early or partial results. Treat as a signal, not a conclusion.
This preprint introduces GCPO, a constrained policy optimization method designed to stabilize reinforcement learning post-training of LLMs by controlling subspace geometry of weight updates. Empirical results on three task categories show improvements over GRPO and related baselines on two small models (Qwen3-8B, GLM4-9B), but the work remains unpublished and lacks independent replication or broader deployment validation.
Comparative algorithm evaluation on fixed benchmarks. Large language models (Qwen3-8B and GLM4-9B) evaluated on mathematical reasoning, code generation, and tool-use tasks.. Intervention: GCPO (Geometrically Constrained Policy Optimization) with hard bilateral orthogonal projections to constrain updates to complementary subspaces.. Compared with: GRPO, DAPO, GSPO, and base models..
GCPO improved over base models by up to 27.69 points across mathematical reasoning, code generation, and tool-use tasks. GCPO improved over strongest baseline (GRPO variant) by up to 2.37 points. GCPO eliminated response-length inflation and stabilized policy entropy compared to GRPO.
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is an unreviewed preprint proposing a novel method (GCPO) with empirical results on benchmark tasks, but lacks the peer review, independent validation, and clinical/deployed-setting evidence needed for stronger evidence grades.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
On-policy rollout methods such as GRPO are central to post-training of large language models, yet they frequently suffer from training instabilities, cross-task capability degradation, and response-length inflation. Although prior work has characterized the subspace geometry of aggregate updates, the stepwise variation of this geometry and its relationship to model performance remain unclear. We introduce Principal-Subspace Overlap, a dimension-corrected measure of individual rollout updates relative to the dominant singular subspaces of pretrained weights. Despite low average overlap, transient spikes often precede performance degradation. To address this, we propose GCPO (Geometrically Constrained Policy Optimization), which applies hard bilateral orthogonal projections to constrain updates to the complementary subspaces, preventing such excursions by construction. Across mathematical reasoning, code generation, and tool-use tasks on Qwen3-8B and GLM4-9B, GCPO consistently outperforms GRPO and recent variants, including DAPO and GSPO, improving over the base models and the strongest baseline by up to 27.69 and 2.37 points, respectively. Furthermore, GCPO preserves general capabilities, eliminates response-length inflation, and stabilizes policy entropy. Our findings provide a new diagnostic lens and a principled design perspective for stable reinforcement learning post-training.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.