What a matrix game is
Before the rest of it makes sense, the form. A matrix game is a structured way of arguing about what happens next — a tool the professional wargaming world has used for decades to think through situations too contingent, and too human, for a spreadsheet model. Chris Engle formalized it in 1988, and defense and policy shops have leaned on it ever since. The mechanics are deliberately plain: each player takes one actor in the situation and proposes an action, then makes the case for why it would work — the leverage behind it. The other actors argue back, for or against. A referee weighs the competing arguments into a modifier — a thumb on the scale, from heavily against to heavily for — and where the outcome is genuinely contested, a dice roll settles it. There are no hidden combat tables and no math you have to take on faith; there is only stated reasoning, adjudicated in the open. What comes out is not a prediction. It’s an account of how a situation could go, with the argument for every turn left on the table where you can read it.
What I was actually testing
A simulation like this makes one promise: show me the range of how a decision could go, not a guess about how it will. The hard part isn’t generating outcomes — it’s trusting them. So I did the most demanding check I could think of: I found a crisis that real experts had already played, in the open, with a documented result, and I asked whether my simulation’s space contained what they found.
The reference is the US Army War College’s 2015 ISIS Crisis matrix game, facilitated by Rex Brynen, in which expert teams played the actors in the conflict — the United States, Iran, ISIL, the Iraqi government, the Sunni factions, the Kurds — and reached a specific ending: ISIL contained but not defeated, Iraq fractured into three de facto regions, Iran deployed on the ground, and a quiet, unspoken cooperation between the US and Iran. One expert path through the crisis, written down.
This came out of Possible Futures, the decision-simulation program in my lab. Possible Futures takes a real decision and runs it forward many times, every stakeholder modeled as a separate AI agent arguing its own case, with every move, ruling, and outcome written in plain language you can read back afterward — the same matrix-game form, but played by agents instead of people in a room, and at a scale a room can’t reach. For this test I ran the scenario 126 times: 42 games each on three different language models, seven actors, four turns apiece, at about four cents a game.
First, the result everyone expects
About 70% of the runs ended messy: no single actor winning, several powers left standing, nothing resolved. That is the boring, correct answer, and it matters that the engine produced it. It matches what the human experts found — their session didn’t crown a winner either — and it matches what anyone who lived through that period knows the region actually was. A simulation that returned tidy victories for a crisis this tangled would be the one I’d distrust. This one returns the mess, because the mess was real.
If that were the whole finding, it wouldn’t be worth your time. The expected result confirms the engine isn’t lying to me. The interesting result is what’s underneath it.
The decisive edges
“No clear winner” hides structure. So I stopped asking who won and asked a sharper question: what happened to ISIL — the engine of the whole crisis? In the 42 runs I’d configured to match the War College’s exact table, the answer was strikingly symmetric. In about one run in five, ISIL was decisively crushed — beaten down to a fraction of its starting strength. In about one run in five, ISIL won outright — consolidated, triumphant, the coalition in disarray. The rest sat in the contested middle.
That symmetry is the thing the single human session could never show. The experts played the crisis once and reached one outcome; their result, by design, sits in the partition zone — ISIL intact, Iraq fractured. They never saw the run where a coalition forms early and breaks ISIL by turn three, and they never saw the run where ISIL gets an opening and runs the table. The map has both. Holding the rare decisive outcomes next to the common messy one — and being able to say how often each occurs — is the entire reason to run a thing 126 times instead of once.
What separates them
So what divided the run where ISIL collapsed from the run where it won? I read the decisive games move by move — this is where being able to read every step earns its keep — and one axis kept surfacing: where the Sunni factions threw their weight.
When the Sunnis broke with Baghdad early — refusing the government’s overtures, suspending cooperation — ISIL tended to win; that break ran ahead of six of the eight runs where ISIL took the table. The collapse runs were more varied. In some, the Sunnis settled with Baghdad early and the coalition consolidated around a contained ISIL — in one of those the Iraqi government finishes stronger than in any other game in the set. In others the Sunnis hardened against Baghdad too, and ISIL collapsed anyway, because its own offensives kept failing. So the Sunni position isn’t a switch that flips the outcome. It’s one of the axes the whole crisis pivots around — and the most legible one, the place a reader can actually watch the fork happen.
I didn’t script the Sunni position as a pivot; it emerged from agents arguing their own interests. And it lines up with an axis the human session turned on too: in the War College game, US policy pushed the Sunnis toward ISIL, and that drift is a load-bearing part of how their crisis unfolded. What my 126 runs add is the branch the experts never played — the version where the Sunnis settle instead, and ISIL is left with no ground to stand on. Playing once, the experts saw one side of that axis. The simulation maps both. That isn’t the engine matching the humans; it’s the engine showing what a single playthrough structurally can’t.
About that 12%
Now the number the earlier analysis got stuck on. The experts’ full, specific ending — all four of its features at once — showed up whole in 5 of those 42 runs. Twelve percent. The first pass treated that as a disappointment: only 12% matched.
It is exactly backwards. Twelve percent is the result I was hoping for. If the experts’ precise outcome had turned up in 70 or 80% of runs, that would not have validated the engine — it would have condemned it, because it would mean the tool collapses a live, open crisis into a single foregone conclusion. That’s a forecast, and a forecast of a thing this contingent is a fiction. A possibility map should contain the expert outcome — proof it’s a real, reachable path — without being anchored to it, because the expert outcome was only ever one of many things that could have happened. Twelve percent says exactly that: their ending is in here, it’s reachable, and it is one island in an archipelago. The map is working. The earlier read measured a map against the standard of a forecast and marked it down for not being one.
What this doesn’t show
The limits are real, and I’d rather state them than have you find them.
One scenario. This is one crisis, checked against one human reference. I have evidence the method holds up within this scenario; I have no evidence yet that it transfers to a different one. Until I’ve run a second documented crisis against its own human playthrough, this is a rigorous result on a single case, not a validated system. That second scenario is what I’m building toward.
A thin yardstick. The human record I’m comparing against is sparse — roughly a line per turn, the outcome without the reasoning behind it. My engine records every actor’s full argument on every move; the humans’ transcript can’t. That asymmetry is exactly why I want a better-documented human game to benchmark against next — one with the players’ reasoning preserved — so the comparison can go deeper than outcomes.
No decision rode on it. This shows the engine maps a crisis richly and legibly. It does not show that anyone made a better decision because of it. That — whether running a real choice through this changes what a real person does — is the test that matters most, and it’s the one I haven’t run yet.
What drives each outcome is a study of its own. I read the decisive runs closely enough to see that the Sunni alignment is one of the axes they pivot on — but a real causal account (how much is the Sunni move, how much is ISIL’s own military luck, how the picture shifts across the three models) is analysis I haven’t finished. I can show you the edges of the map and one of the axes; I can’t yet hand you the full mechanism. That’s the next note, not this one.
What’s next
A second locked scenario with a richer human record to benchmark against. And the first client-facing runs of the method, which I’m preparing now — written up the same way, including the parts that don’t work.
If you’re holding a decision with this shape — several actors, a date on the calendar, real stakes, and genuine uncertainty about how the key people will move — the Brief is where that conversation starts.