UNVERIFIED literature results. Source: Qwen2 Technical Report, Tables 2 and 4.
Academic and scientific knowledge evaluations.
mmlu_5shot vs. estimated ISO FLOPs
9 matching runs · 6ND; missing tokens estimated at 20× parameters
All 10 filtered runs
| Run | Scale (B) | ISO FLOPs | Training data | mmlu_5shot | gpqa_5shot | mmlu_pro_5shot | theorem_qa_5shot |
|---|---|---|---|---|---|---|---|
| literature/qwen2_72b | ~72.000 | ~6.2e23 | 84.20 | 37.90 | 55.60 | 43.10 | |
| literature/qwen1.5_110b | ~110.000 | ~1.5e24 | 80.40 | 35.90 | 49.40 | 34.90 | |
| literature/llama_3_70b | ~70.000 | ~5.9e23 | 79.50 | 36.30 | 52.80 | 32.30 | |
| literature/mixtral_8x22b | — | Unknown | 77.80 | 34.30 | 49.50 | 35.90 | |
| literature/qwen1.5_72b | ~72.000 | ~6.2e23 | 77.50 | 36.30 | 45.80 | 29.30 | |
| literature/qwen2_7b | ~7.000 | ~5.9e21 | 70.30 | 31.80 | 40.00 | 31.10 | |
| literature/llama_3_8b | ~8.000 | ~7.7e21 | 66.60 | 25.80 | 35.40 | 22.10 | |
| literature/gemma_7b | ~7.000 | ~5.9e21 | 64.60 | 25.70 | 33.70 | 21.50 | |
| literature/mistral_7b_v0.2 | ~7.000 | ~5.9e21 | 64.20 | 24.70 | 30.90 | 19.20 | |
| literature/qwen1.5_7b | ~7.000 | ~5.9e21 | 61.00 | 26.70 | 29.90 | 14.20 |