ReFeri: Training-free LLM Verification via Recycling Few-shot Examples

News

Overview

Response selection is a central bottleneck in test-time scaling. LLMs can generate multiple plausible reasoning paths for the same query, but majority voting is limited to outputs that can be normalized and aggregated, while learned verifiers require additional supervision and training.

ReFeri recycles the few-shot examples already available in the prompt as a training-free verifier. The same demonstrations used to guide generation are reused to evaluate which candidate follows the intended task while avoiding demonstration-specific shortcuts.

ReFeri combines two complementary likelihood signals. The forward confidence score measures how well a candidate follows the few-shot guidance, while the backward reconstruction penalty identifies candidates that are overly specific to the demonstrations. Their difference provides a lightweight verification signal without task-specific verifier training.

Results

ReFeri consistently improves training-free response selection. Across three generation LLMs and seven reasoning benchmarks, ReFeri achieves an 8.2% average relative gain over random selection and a 2.6% average relative improvement over the strongest prior training-free baseline.

ReFeri scales with the candidate pool. As the number of responses increases from 1 to 20, accuracy improves from 75.8 to 79.4 on MATH500, 41.4 to 45.5 on GPQA, and 75.6 to 86.0 on MuSR-ta, while several competing training-free selectors saturate or degrade.

GPT-4o-mini generates N = 1–20 candidates with Few-shot CoT; each method selects one.

Analysis

The backward term captures demonstration mimicry. Ranking candidates by the raw backward term and reusing each as a one-shot demonstration, accuracy decreases with rank: candidates with larger backward terms better reconstruct the original few-shot answers, which is exactly the behavior ReFeri penalizes.

Demonstration fragments inflate the backward term. Prepending the first ρ% of the few-shot examples to each candidate raises the raw backward term (solid) much more sharply than the forward score (dashed).

This penalty keeps selection robust to injected artifacts. When demonstration fragments are prepended to a low-forward incorrect candidate, forward-only selection drops from 45.3% to 33.3%, while ReFeri reaches 47.5%.

Selection accuracy (%) when the first ρ% of the retrieved demonstration answer is prepended to a low-forward incorrect candidate (GPT-4o, GPT-4o-mini, and LLaMA-3.1-8B generators; MATH500, GPQA, MuSR-ta).

Robustness and Efficiency

ReFeri remains effective across estimation models. Holding candidate pools fixed, ReFeri improves over Few-shot CoT with every tested estimator across model families and scales: LLaMA-3.2-1B, Qwen2.5-7B, LLaMA-3.1-8B, and LLaMA-3.1-70B.

A lightweight estimator provides a favorable cost-accuracy trade-off. ReFeri with a 1B estimator remains on a strong trade-off frontier and runs approximately 2.6× faster than CoT-WP with an 8B estimator.

Citation

@article{lee2025training,
  title={Training-free LLM Verification via Recycling Few-shot Examples},
  author={Lee, Dongseok and Hong, Jimyung and Kim, Dongyoung and Kim, Jaehyung},
  journal={arXiv preprint arXiv:2506.17251},
  year={2025}
}