11,000 papers. 10 worth reading. How do you find them?
PapersWithELO is not an AI reviewer. It is an open, incremental attention ranking protocol — it tells you what you can safely skip.
A filter, not a chooser
Distribution of ground-truth decisions across fused percentile, ICLR 2024 (10% sample, seed 42, n = 730).
We are good at filtering low-quality work, not picking the best. The top decile is still 69% accepted — as it should be; science is noisy.
How it works
6 papers per batch. The LLM never sees an absolute scale.
The judge only outputs an ordering. Ranking is the only output we trust.
Bayesian Plackett–Luce with MAP + Laplace uncertainty: every paper gets θ ± σ.
Two judges (mimo, deepseek), z-scored and fused. New entries rank against fixed incumbents.
LLMs are too kind for absolute scores — everything is “above average, 7/10”. We never ask for scores. We force comparisons, and fit a Bayesian model to the rankings.
Three independent tournaments
Each year is ranked separately — θ is only comparable within a year; percentile is comparable across years.