Why Running Two LLMs on a Mini PC Is a Waste of Time

I benchmarked every combination. The shared DDR5 bus ruins everything, but MoE architecture already solved the problem.

I wanted to run two models at once. A big one for reasoning, a small one for quick tasks. The mini PC has 96GB of RAM, so why not?

Because physics. Specifically, memory bandwidth physics. I have the benchmarks to prove it.

The Hardware

Minisforum UM790Pro. Ryzen 9 7940HS with an integrated 780M GPU. 96GB of DDR5-5600. The iGPU shares system memory with the CPU through a 48GB GPU-accessible pool. Total memory bandwidth: about 80 GB/s, shared between everything.

That "shared between everything" part is where all the problems come from.

I tested four models on Ollama: the qwen3.6:35b (a 36B parameter Mixture of Experts model), gemma4 at 4.6B, qwen3:4b-instruct, and the tiny qwen2.5:1.5b.

Solo Performance Is Actually Great

Running one model at a time, the numbers are solid. The 35B MoE model does 17.8 tokens per second on the iGPU. That's genuinely usable for chat and coding. Gemma hits 42.9 tok/s. The 1.5B model on CPU: 53.4 tok/s. Fast enough for anything.

So I figured, why not run the big model on GPU and a small helper on CPU at the same time?

The Dual-Model Disaster

I ran four test combinations. Every single one made things worse.

Both models on GPU: The 35B dropped 26% to 13.1 tok/s. Gemma dropped 41%. They're fighting over the same compute units and the same memory bus.

Big model GPU, tiny model CPU (best case): The 35B only dropped 16% to 14.9 tok/s. But the 1.5B model got cut in half, from 53.4 to 26.2 tok/s. This was the least bad option.

Big model GPU, medium model CPU: Forcing the 4.6B Gemma to CPU didn't help. Enough memory traffic from CPU inference to compete with GPU reads. Both models dropped over 25%.

The worst case: The 4B instruct model with its 256K context window. Its KV cache ballooned to 24.2 GB. Combined with the 35B model's allocation, we maxed out the memory bus. Both crawled at 11 tok/s. A 35% drop on the big model.

Why This Happens

On a discrete GPU, you get dedicated GDDR6 with 300+ GB/s of bandwidth. The CPU has its own memory bus. They don't interfere with each other.

On an APU like the 7940HS, the CPU and iGPU drink from the same straw. DDR5-5600 gives you about 80 GB/s total. When two models are streaming weights simultaneously, they're competing for every byte of that bandwidth.

This isn't a compute problem. The GPU has plenty of CUs. The CPU has plenty of cores. It's pure memory bandwidth starvation.

Agent Frameworks Don't Save You

I was building toward running an agent framework with a planner model and an executor model. Big model thinks, small model acts. Sounds great on paper.

But agent workflows are sequential. The planner generates tokens while the executor sits idle. Then the executor runs while the planner waits. They take turns. At no point are both models generating tokens simultaneously.

So you're paying the memory tax of having two models loaded, getting worse throughput on both, and gaining zero parallelism. The second model is just wasting RAM that could go toward a bigger context window for the first model.

MoE Already Solved This

Here's the part that made me feel dumb for running all these tests.

The qwen3.6:35b is a Mixture of Experts model. 256 experts, 8 active per token. For any given token, it does the compute of a 4-5B model while pulling from the knowledge of a 36B model. The big model already IS the small model in terms of cost per token.

17.8 tok/s for 35B-class reasoning. On a mini PC. With no discrete GPU. That's fast enough for everything I throw at it. Adding a second model only makes the first one slower.

What You Should Do Instead

Run one model. Pick the biggest MoE model that fits your VRAM. Give it all the memory. Use the rest for a massive context window.

On my 96GB UM790Pro, that means the 35B model gets the full 48GB GPU pool with room for 128K+ context. Way more useful than splitting RAM between two models that take turns doing nothing.

The era of "just throw more models at it" doesn't work on shared-memory hardware. One good model, one big context window. That's the play.

Tested on: Minisforum UM790Pro, AMD Ryzen 9 7940HS, 96 GB DDR5-5600, Ollama v0.9.x, Ubuntu Linux