winogrande_5shot · dvbench

WinoGrande, 5-shot score (%).

UNVERIFIED · Qwen2 Technical Report · arXiv:2407.10671 · Tables 2, 4

Participate

measurement = experiment.measure(checkpoint)

@measurement.evaluate(params=0, batch=1)
def evaluate(params, batch):
    ...

@measurement.aggregate
def aggregate(outputs):
    ...

score = measurement.run(checkpoint, batches)
measurement.submit("winogrande_5shot", score)

# Next steps? Getting started docs

Results

10 runs

Result trend


literature/qwen2_7b	self reported	77.00	2026-09-04 19:49:26
literature/qwen1.5_7b	self reported	71.30	2026-09-04 19:49:26
literature/llama_3_8b	self reported	77.40	2026-09-04 19:49:26
literature/gemma_7b	self reported	79.00	2026-09-04 19:49:25
literature/mistral_7b_v0.2	self reported	78.40	2026-09-04 19:49:25
literature/qwen2_72b	self reported	85.10	2026-09-04 19:49:25
literature/qwen1.5_110b	self reported	83.50	2026-09-04 19:49:25
literature/qwen1.5_72b	self reported	83.00	2026-09-04 19:49:25
literature/llama_3_70b	self reported	85.30	2026-09-04 19:49:24
literature/mixtral_8x22b	self reported	85.00	2026-09-04 19:49:24