A new training framework, APIVIS, combines direct and searched responses to enhance the mathematical reasoning capabilities of large language models using reinforcement learning with verifiable rewards.
Reinforcement learning with verifiable rewards (RLVR) has substantially improved the mathematical reasoning capabilities of large language models.
we propose APIVIS, a training-time framework that adapts finite-budget Gumbel search to chunk-level mathematical reasoning
Experiments on widely recognized mathematical reasoning benchmarks and different model scales demonstrate substantial improvements over competitive search-based methods, validating the effectiveness of APIVIS.
Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the mathematical reasoning capabilities of large language models. Recent work introduces search into RLVR rollouts to increase trajectory diversity, but diversity alone does not ensure that the search-induced rollout policy improves upon the current policy. To address this gap, we propose APIVIS, a training…
