From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks

The paper traces seven open-source RL post-training frameworks (verl, TRL, OpenRLHF, NeMo-RL, ROLL, slime, AReaL) through their commit histories and links them to 6,939 screened arXiv papers from January 2025 to June 2026. Of 1,598 papers proposing a general RL training method, 37 reached at least one framework. Across five quarterly cohorts, the number included within 180 days stayed between 5 and 7 while cohort sizes grew from 72 to 386 papers. Included papers more often changed coefficients, clips, masks or weights in existing losses (29% vs 9%). A public tracker updates weekly.

Why it matters

Frameworks act as a gate between new RL methods and practitioners, so knowing which methods get included, and how quickly, shapes what people can easily run.

Read with care

Inclusion measures what a framework offers on its main branch, not whether practitioners actually use a method; the source's own discussion notes this gap.