What Games Can We Build with a Small Model (10B Active Parameters)?

Here's the thing everyone assumes: if you want good game dialogue, you reach for GPT-4. And sure, GPT-4 writes great dialogue. But it costs money every single second it runs, the latency stretches from a few hundred milliseconds all the way up to whole seconds, and every time an NPC opens its mouth, you're shipping player behavior data off to a cloud server. That's a privacy problem. Small language models exist precisely to solve all three of those headaches. So let's talk about what a model in the ten-billion-parameter range can actually pull off inside a game.

Quick version up front: models like Mistral 7B, Gemma 9B, or Llama 3.2 11B can run locally — on a consumer graphics card like an RTX 4090, or on an Apple Silicon Mac — at somewhere between twenty and fifty tokens per second. That's fast enough for real-time NPC conversation. Where they shine is clear, tightly-constrained tasks. Where they stumble is complex reasoning and staying consistent over a long stretch. And the whole trick of designing a game around them is working *with* those limits instead of against them.

So what do we actually mean by "small model" here? We're talking seven to thirteen billion active parameters. Mistral 7B and its bigger sibling Nemo at twelve billion are prized for raw inference efficiency. Gemma 9B from Google is really strong at following instructions. Llama 3.2 11B from Meta brings multilingual support and even a multimodal version. And if you want to go even smaller, Microsoft's Phi-3.5 Mini sits at under four billion — it trades some quality for speed. The nice part: once you apply four-bit quantization, these models only need roughly four to eight gigabytes of memory. That runs on a mid-range consumer GPU, or on the unified memory of an M2 or M3 Mac.

Now, what can they do in a game? Let's go through it.

First, and this is the most mature use right now — dynamic NPC dialogue. Traditionally, RPG conversations are pre-written as a branching tree. The player just picks from a menu of options. A small model lets you have a genuinely free-form conversation instead. Picture this: the player says, "I heard you know something about the missing children." And the NPC, generated live, leans in and says, "Keep your voice down. The guards rotate at midnight — that's when I can talk. Ask me now, and I know nothing." That texture, that in-character evasiveness, comes out of the system prompt. And designing that prompt is the real craft. It has to carry the character's background — their personality, their secrets, how they speak. It has to know the current scene state — how much the player trusts them, the time of day, the location. And it has to encode world constraints — what this character actually knows and, just as importantly, what they don't.

Second use: procedural narrative generation. A small model can spin up short story fragments on the fly, reacting to what the player's been doing. Think of a roguelike where, every time you step into a new area, the game writes you a fresh description — the history of this abandoned dungeon, a clue scratched into the wall by the last explorer who came through. There's actually a 2025 arXiv paper on exactly this, showing that small models can approach large-model quality on short, well-contextualized creative content — and with far more variety than you'd ever get from rigid, rule-based generation.

Third: adaptive content. Same mission, but the hint language shifts depending on the player's skill level. Personalized mission briefings. Different narration branching off the choices you made earlier.

And fourth — this is where small models genuinely shine — interactive fiction and text adventures. Give the model a clear world setup, and let it drive the story forward based on what the player types. There's an experimental project called the Painter Game from Godoka that does exactly this — an interactive painting narrative running on a small model.

Let me walk you through how this actually fits together in a game. On one side you've got the game engine, holding two things: the game state — player position, inventory, relationship values — and the event trigger, which fires when the player types something or walks up to an NPC. That event, plus the current state, flows into a system prompt builder. That builder assembles the character, the state, and the constraints into one prompt. That goes to the local model — running on something like llama.cpp or Ollama. The model's output then passes through a filter that checks content safety and validates the format. And only then does it flow back to update the game state and show up on screen. So it's a loop: trigger and state in, prompt built, model generates, output filtered, screen and state updated.

On the tooling — llama.cpp is the workhorse local inference engine, and because it's C++, you can drop it right into a game engine. Ollama gives you a simple HTTP API, which is lovely for quick prototyping. And both Unity and Unreal have community-built packages that wire llama.cpp in for you.

Okay, but let's be honest about the gap versus the big models, because it's real. On speed, a local 10B model does twenty to fifty tokens a second; GPT-4o does fifty to a hundred — but remember, that number doesn't include the network round trip. And that's where local wins big: latency under a hundred milliseconds for a direct local call, versus anywhere from three hundred milliseconds to two full seconds once you account for the network. On cost, the small model is a one-time hardware purchase; the API charges you roughly five to fifteen dollars per million tokens, forever. On privacy, local data never leaves the device; the API sends everything to OpenAI's servers. Where the big model pulls ahead is long-range consistency and complex reasoning — both noticeably stronger. And on short creative generation, the small model very nearly matches it.

That long-range consistency point is the single biggest practical limitation, so sit with it for a second. Once a conversation runs past a few thousand tokens, a small model starts to *forget* — it loses the character setup or the plot details you established earlier. The fix is not to hope the model remembers. The fix is to hold all your important state *outside* the model, in a game database, and re-inject it into the context on every single call. Don't rely on the model's memory. Manage the memory yourself.

So let me leave you with three things to hold onto.

One: as of 2025, a 10B model is genuinely good enough for real-time NPC dialogue, short procedural text, and interactive narrative. This isn't theoretical anymore.

Two: it is *not* a GPT-4 replacement, and you shouldn't frame it as one. Think of it as your entry ticket into a whole new category — real-time, free, on-device language generation.

And three: design around the constraints, not in spite of them. Short context, clear constraints, explicit state management. The games that lean into those limits might end up with mechanics that feel genuinely fresh — interesting precisely *because* they were built inside the box, not outside it.