top of page
Search


Benchmarks and beyond: measuring progress in LLMs.
Paper 1 defines the problems of benchmark saturation as the loss of reliable discriminative power among top-performing models under comparison and identifies benchmark properties usually associated with saturation after an analysis of 60 benchmarks, and makes concrete anecdotally known characteristics of benchmarks, with implications for benchmark lifecycle management. Paper 2 questions the benchmark culture itself, and how this is perpetuating structural harms towards less r

WAI CONTENT TEAM
4 days ago2 min read
bottom of page
