LLM benchmark
BAA benchmark: the same prompt, thirty models.
BAA compares thirty model outputs from the same assignment: a short 3D arcade game about herding sheep.
- Results
- 30
- Playable or partial
- 11
- Available demos
- 29
- Method
- Same prompt
Full ranking
Results and playable demos
The default ranking reflects the practical result. Filters keep the ranking intact and only hide runs that do not match.
No model matches the selected filters.
| Preview | Model | Type | Parameters / memory** | Result | Bugs | Note | Demo |
|---|---|---|---|---|---|---|---|
|
1Claude Fable 5 | Closed | - | Minimal, a few minor issues | Reasoning effort 4/6, not maximum. Small Mac viewport fix. |
Demo | |
|
2Kimi K3 | Open-weight | 2.8T 1.54 TB - 4.90 TB | Minimal, minor issues | Generation cost: USD 7.31. | Demo | |
|
3Claude Opus 5 | Closed | - | No issues found | - | Demo | |
|
4GPT 5.6 Sol | Closed | - | Few issues | - | Demo | |
|
5Claude Opus 4.8 | Closed | - | Few issues | - | Demo | |
|
6Grok 4.5 | Closed | - | No obvious serious issue | - | Demo | |
|
7Gemini 3.5 Flash | Closed | - | Few issues | Required 1 corrective prompt. | Demo | |
|
8Qwen 3.6 27B ThinkingCap | Open-weight, GGUF | 27B 18 GB - 50 GB | No serious bugs | ThinkingCap modification by Bottlecap. Generated in 17 minutes; no corrective prompt. | Demo | |
|
9Gemini 3.6 Flash | Closed | - | No serious bugs | - | Demo | |
|
10Qwen 3.5 397B A17B | Open-weight | 397B (17B active) 224 GB - 400 GB | Many serious bugs | - | Demo | |
|
11MiniMax M2.7 | Open-weight | - | Many serious bugs | Characters barely move. | Demo | |
|
12GPT 5.5 | Closed | - | Many bugs | - | Demo | |
|
13MiniMax M3 | Open-weight | 428B (23B active) 240 GB - 480 GB | Many bugs | - | Demo | |
|
14Claude Sonnet 4.6 | Closed | - | Several issues | Generation took unusually long, over an hour. | Demo | |
|
15GLM 5.2 | Open-weight (OpenRouter) | 744B (40B active) 416 GB - 816 GB | Many bugs | - | Demo | |
|
16DeepSeek V4 Flash | Open-weight (OpenRouter) | 284B (13B active) 160 GB - 304 GB | Many bugs | - | Demo | |
|
17Qwen 3.7 Max | Closed | - | Very many bugs | Required 1 corrective prompt. | Demo | |
|
18DeepSeek V4 Pro | Open-weight (OpenRouter) | 1.6T (49B active) 896 GB - 1.37 TB | Many bugs | Required 1 corrective prompt. | Demo | |
|
19Big Pickle | Unknown | - | Many bugs | - | Demo | |
|
20Kimi 2.7 Code | Open-weight (OpenRouter) | 1T (32B active) 560 GB - 880 GB | Many bugs | Required 3 corrective prompts. | Demo | |
|
21Qwen 3.6 27B | Open-weight | 27B 18 GB - 50 GB | Many bugs | - | Demo | |
|
22Qwen 3.5 122B A10B | Open-weight | 122B (10B active) 72 GB - 176 GB | Very many bugs | Both the sheep and dog fly around. | Demo | |
|
23GLM 4.7 Flash | Open-weight (OpenRouter) | 31B (3B active) 20 GB - 50 GB | Many bugs | Required 1 corrective prompt. | Demo | |
|
24Qwen 3.6 35B A3B | Open-weight | 35B (3B active) 22 GB - 52 GB | Many bugs | Required 1 corrective prompt. | Demo | |
|
25MiMo v5 | Open-weight | 310B (15B active) 176 GB - 336 GB | Many bugs | Required 1 corrective prompt. | Demo | |
|
26Gemma 4 31B | Open-weight | 31B 20 GB - 58 GB | Many bugs | Required 1 corrective prompt. | Demo | |
|
27Mistral Small 4 | Unknown | - | Very many serious bugs | Required 3 corrective prompts. | Demo | |
|
28Ornith 1.0 35B | Open-weight | 35B 22 GB - 64 GB | Many bugs | Required 2 corrective prompts. | Demo | |
|
29Qwen 3.5 A3B | Open-weight | 30B (3B active) 20 GB - 50 GB | Many bugs | - | Demo | |
| 30Nemotron 3 Ultra | Open-weight | 550B (55B active) 320 GB - 864 GB | — | - | — |
**Memory is only an approximate estimate for Q4 weights: the first value estimates 0K context, the second 100K context. It is not a measurement or exact sizing for a specific architecture.
Notes refer to corrective prompts needed to launch the game (for example a JavaScript error) and to any run settings such as reasoning effort. No corrective prompt was used to tune the game itself.
On mobile or tablet, Demo opens a version with an added arrow and A/B control panel. The original benchmark code remains unchanged.
2Kimi K3
Open-weight
3Claude Opus 5
Closed
4GPT 5.6 Sol
Closed
5Claude Opus 4.8
Closed
6Grok 4.5
Closed
7Gemini 3.5 Flash
Closed
8Qwen 3.6 27B ThinkingCap
Open-weight, GGUF
9Gemini 3.6 Flash
Closed
10Qwen 3.5 397B A17B
Open-weight
11MiniMax M2.7
Open-weight
12GPT 5.5
Closed
13MiniMax M3
Open-weight
14Claude Sonnet 4.6
Closed
15GLM 5.2
Open-weight (OpenRouter)
16DeepSeek V4 Flash
Open-weight (OpenRouter)
17Qwen 3.7 Max
Closed
18DeepSeek V4 Pro
Open-weight (OpenRouter)
19Big Pickle
Unknown
20Kimi 2.7 Code
Open-weight (OpenRouter)
21Qwen 3.6 27B
Open-weight
22Qwen 3.5 122B A10B
Open-weight
23GLM 4.7 Flash
Open-weight (OpenRouter)
24Qwen 3.6 35B A3B
Open-weight
25MiMo v5
Open-weight
26Gemma 4 31B
Open-weight
27Mistral Small 4
Unknown
28Ornith 1.0 35B
Open-weight
29Qwen 3.5 A3B
Open-weight30Nemotron 3 Ultra
Open-weightMethod
Assignment used for the comparison
The prompt is preserved in its original wording. Evaluation is based on actually launching the output, playability, bugs, and constraints of each run.
Prompt used for the comparison
Chcel by som vytvorit jednoduchu arkadovu hru, trocha akoby to bola nintendo hra ale v 3D s pohladu z tretej osoby ale trocha vzdialena kamera. Hra bude v prostredi luk v horach na kopcovitom vlnenom terene kde sa pasu ovce, Vacsina plochy budu luky, v pozadi hory a lesy a nejake domy. Hrac bude hrat za ovciarskeho alebo ineho psa ktory sa pouziva pri zahanani oviec, nech si vie vybrat z par plemien ktore sa budu lisit niektorymi vlastnostami. Hrac bude mat moznost sa pohybovat sipkami, medzerou alebo nejakym vhodnym key na klavesnici bude moct brechat co bude moct pouzit aj na ovce teda ak to realne pomaha pri zahanani, ale aj na odplasenie inych zvierat, napr. inych psov, vlkov, medveda, ludi okrem pastiera ktory sa bude tiez nachadzat na luke. Ovce nech reaguju na psa cize budu sa pohybovat v stade ale budu sa pohybovat tak ako ich pes zazenie, nech beekaju (baa), ale nech to je arkadove. Podstata hry ma byt zaviest ovce do kosiara cize, z bodu A do bodu B a celit nastraham ako su obcasne vypady inych psov, vynimocne vlkov, alebo medveda. Vlci a medved nech su nieco ako bosovia alebo proste vacsie prekazky v dalsich levelov. Vymysli si mensie lahsie prekazky v prvych leveloch. Levelov nech je nejaky rozumny pocet, mysli na to ze toto ma byt kratka hra na pobavenie na par minut max par desiatok minut mozno 1 hodina ak by sa to hralo do konca do posledneho levela. Na mape by sa mohli objavovat aj ine zvierata, navhrni napr. kravy, lisky obcas niekde nejake vtaky atd. proste priroda. Nech su okrem luk aj nejaka ta polna cesta, obcas nejake prekazdky ci uz v podobe ohrad, utesov, krikov cez ktore ovce neprejdu. Grafika nech je pekna, farebna, arkadova, trocha nech pripomina akoby to bolo nintendo hra ale nekopiruj ziadnu konkretnu hru respektive jej dizajn. Hru napis v html/javascript/
Standalone demo
Final BAA game
The manually completed playable version is separate from the model ranking and serves as a standalone demo of the original idea.
- Playable
BAA - final game
A short 3D arcade game about a sheepdog herding a flock across meadows into a pen. This is the playable version; the benchmark above shows how each model handled the same assignment.