25 AI models tested on real mixing decisions

Mixroadie measures your tracks itself. The AI’s job is narrower: it reads those measurements and decides the numbers for each move, like how much to cut at 300 Hz on the bass or what ratio to give the vocal compressor. A full mix asks it several dozen of these questions.

You can pick which AI does that. So we wanted to know which one does it best, and we didn’t want to go by general leaderboards. We tested them on Mixroadie’s own questions.

Updated 9 October 2026: we re-ran the test with 25 models instead of six, including DeepSeek, Gemini, Qwen, Llama, Grok and the newest budget models from Anthropic and OpenAI, plus the smaller built-in models for Macs and PCs with less memory.

How we tested

We ran Mixroadie on a real project and recorded every question it asked the AI: 48 decisions, covering EQ, de-essing, compression, levels, and working out what each track is. Then we replayed exactly those questions against each model, through the same code the app uses, including its retries.

The local models ran on a Mac with an M5 Max chip. The cloud models were reached through OpenRouter, one account for all of them, at the providers’ list prices. The run was on 9 October 2026.

We measured four things:

• Time per request. How long “mix my song” takes from start to finish. Mixroadie asks up to eight questions at once, so this is the time you actually wait, not the time for one answer.
• Cost per request. Tokens counted on every call, priced at the list price, and scaled up to the biggest request in the recording.
• Same answer twice. Ask the identical question again, get the same numbers? This matters more than it sounds. When you tick moves and press Apply, Mixroadie works the plan out again before writing it. A model that changes its mind can write a different move from the one you approved.
• Honest reasons. Every move comes with a reason, like “the vocal measures −17.7 LUFS against a target of −28.1”. We checked whether the numbers in those reasons actually appeared in what the model was given, or whether it made them up.

On your computer

Model	Where it runs	Time per request	Same answer twice	Honest reasons
Qwen3 4B (built-in, under 14 GB of memory)	Your computer	23 s	100%	100%
gpt-oss-20b	Your computer, via LM Studio	23 s	88%	96%
Qwen3 30B, 8-bit (built-in, 64 GB or more)	Your computer	28 s	100%	100%
Qwen3 8B (built-in, 16–24 GB)	Your computer	32 s	100%	100%
Qwen3 30B, 4-bit (built-in, 32–48 GB)	Your computer	34 s	100%	100%

All of them are free and send nothing anywhere. The times are on a fast Mac; a smaller chip takes longer, and the small models are the ones that run on smaller computers.

In the cloud

Model	Time per request	Cost per request	Same answer twice	Honest reasons
Gemini 2.5 Flash-Lite	8 s	under 1¢	100%	96%
DeepSeek V4.1 Flash	12 s	1¢	100%	99%
Gemini 2.5 Flash	14 s	2¢	100%	100%
GPT-5.6 Luna (Mixroadie’s ChatGPT default)	17 s	1¢	100%	92%
GPT-5 mini	18 s	2¢	100%	85%
Claude Haiku 5.5 (Mixroadie’s Claude default, and our recommendation)	21 s	1¢	100%	100%
GPT-5.2	22 s	10¢	100%	80%
Gemini 3.1 Flash-Lite (Mixroadie’s Gemini default)	24 s	1¢	100%	91%
Qwen3.7 Flash (Mixroadie’s OpenRouter default)	25 s	under 1¢	100%	99%
Claude Haiku 4.5	27 s	6¢	100%	90%
Claude Opus 5	48 s	38¢	100%	94%
Grok 4.3	83 s	11¢	75%	100%
Llama 3.3 70B	84 s	1¢	100%	100%
DeepSeek V3.2	99 s	1¢	100%	98%
GPT-5 nano	185 s	under 1¢	88%	90%

Every model answered every question, except DeepSeek V3.2, which missed one in 48. When a model doesn’t answer, Mixroadie falls back to its built-in rules for that move. We also measured DeepSeek V4 Flash, GLM-5.3 Flash, Qwen3.8 Flash and the cloud version of Qwen3 30B. They gave steady answers but took 33 to 98 seconds, so we left them out of the table. Mistral Small was rate-limited by the service in between on almost a third of its questions, so its result says more about that route than about the model, and we left it out too.

What we took from it

Almost every model now gives the same answer twice. In September, the cloud models changed their numbers on 29% to 46% of repeated questions. This time almost none did. The difference is a change we made after the first test: Mixroadie now tells the model what “subtle”, “moderate” and “strong” mean in decibels and ratios for each kind of move, and that pins the answers down. The two runs used different recordings, and the repeat test is only eight questions per model, so read this as “much steadier”, not as a precise number.

“Thinking” was the biggest slowdown. Several of the newer budget models think before every answer unless told not to. With thinking on, DeepSeek V4.1 Flash took over two minutes per request; with it off, 12 seconds, with the same consistency. Claude Haiku 5.5 went from 31 to 21 seconds, GPT-5.2 from 38 to 22. Mixroadie now switches thinking off wherever it knows how to: Claude, ChatGPT, DeepSeek, and the main thinking models on OpenRouter.

The small built-in models hold up. Qwen3 4B and 8B, the models for computers with less memory, were as consistent and as honest as the big one on this test. They didn’t change the plan they made, and they only quoted figures they had been given.

The cheap cloud models beat the expensive ones here. Claude Haiku 5.5 and Gemini 2.5 Flash were consistent, never quoted a number they weren’t given, and finished in about 20 seconds or less, at 1 to 2 cents a request. Claude Opus 5 cost 38 cents, was slower, and was less careful with its numbers. That’s not a verdict on the big models in general. These questions are short and numeric, and a large model doesn’t have much room to show what it can do.

Honest reasons is where the cloud models still differ. GPT-5.2 put a number in about one reason in five that wasn’t in the measurements. That’s the one sentence you read to decide whether to trust a move, so we care about it.

How we pick the defaults

We count a model as good enough for the job when it answers every question, gives the same answer twice, changes its answer when you ask for “moderate” instead of “subtle”, quotes real figures in at least nine reasons out of ten, and finishes a full mix in under a minute. For each provider, Mixroadie starts on the cheapest model that passes. The setup recommends the cheapest cloud model that came top on every one of those measures, and that was Claude Haiku 5.5.

Provider in Mixroadie	Starts on	Cost per song
Claude (recommended)	Claude Haiku 5.5	about 1¢
ChatGPT	GPT-5.6 Luna	about 1¢
Gemini	Gemini 3.1 Flash-Lite	about 2¢
DeepSeek	DeepSeek Flash (V4.1)	about 2¢
OpenRouter	Qwen3.7 Flash	under 1¢

The cost per song is list prices times the tokens we counted in this test, for a full mix the size of the one we recorded. It’s the estimate the setup shows next to each company. A song with more tracks costs more. Gemini 2.5 Flash-Lite and GPT-5 nano are cheaper still, but the first ignored “subtle” and “moderate” in a third of our tests and the second took three minutes per mix.

What changed in Mixroadie because of this

• Claude is now the recommended cloud option in the setup, replacing ChatGPT. It was the cheapest cloud model with top marks on every measure.
• New starting models for the cloud options: Claude starts on Claude Haiku 5.5 (was Haiku 4.5), ChatGPT on GPT-5.6 Luna (was GPT-5 mini), Gemini on Gemini 3.1 Flash-Lite (was 2.5 Flash), DeepSeek on DeepSeek Flash, and OpenRouter on Qwen3.7 Flash (was Gemini 2.5 Flash). You can still pick any model in Settings.
• The cost per song in the setup is measured now. It used to be an estimate from before the first test; it now uses the tokens we counted on every call.
• The DeepSeek option starts on DeepSeek’s current model. DeepSeek has retired deepseek-chat, the model name Mixroadie used to start on.
• Thinking is switched off for the models that would otherwise think before every answer, where the provider lets us.
• OpenRouter works with the newest Claude and GPT models. Mixroadie asked them for a setting they don’t accept, and OpenRouter refused the request instead of ignoring it.

Which one should you use?

• Apple Silicon Mac or Windows PC: the built-in model. It’s free, it runs offline, nothing leaves your computer, and every size of it was fully consistent in this test.
• Intel Mac: gpt-oss-20b or Qwen3 30B in LM Studio or Ollama, if your computer has the memory for it. Otherwise a cloud key.
• You’d rather use the cloud: Claude, on Claude Haiku 5.5. It was the most consistent cloud model, at about a cent a song. If you’d rather use a different company, each one starts on its cheapest model that did well here. DeepSeek V4.1 Flash was faster still at a similar price, but DeepSeek’s own service runs in China. Cloud models get numbers, track names and what you typed. They never get audio.

What this test doesn’t tell you

• It’s one project, 48 decisions and one run. A different song would move the numbers a little.
• There is no “right” compressor ratio, so we didn’t score whose mix sounded best. We scored the things we can check: speed, cost, consistency and honesty.
• Cloud times were measured through OpenRouter. Using a provider’s own API key, as most people do in Mixroadie, can be a little faster or slower.
• We haven’t measured the built-in models on Windows yet.
• Prices are list prices on 9 October 2026, and they change.

We’ll re-run the benchmark when new models come out and update this post. To see which AI options work on your computer, check the requirements page.