Listen to this article:

Hi, my name is Allen Wu. To introduce myself, I’m a Silver-level pro and I have played in six Pro Tours over the past two years, cashing in three of them. The Magic accomplishment I’m most proud of is perhaps recently hitting 1970 rating on Magic Online, playing exclusively Martyr-Proc.

I’m finishing a master’s degree in decision analysis at Stanford at the moment, so competitive Magic has taken a backseat to my studies. I still enjoy playing and thinking about Magic though, and recently wrote a blog post about applying Monte Carlo simulation to Magic that’s garnered a lot of attention and discussion.

This article is inspired by that blog post. I expected only a few close friends to read my post, so I didn’t think to fully explain my methods, what I was trying to accomplish, or how the numbers should be interpreted. But since a lot of people seem to be interested, I’d like to use this article to better explain the concepts in my blog post and how you should apply them.

Prologue

I was getting back into Magic Online after selling my collection and wanted to try learning Eldrazi Tron because it was the best-performing Modern deck that I had no experience with. I read Sam Pardee’s primer on the deck, which says to keep hands that do something unfair, without being too picky about what unfair thing. Sam suggests looking for hands with Tron, Chalice of the Void, or Temples and Eldrazi. I asked my friend Collin Rountree for advice, since he’d had a lot of success with Eldrazi Tron, and he agreed. Collin said he would keep mulliganing until he found a broken hand, and that he’d won plenty of games on four and five cards.

This mulligan strategy was unintuitive for me. I’m most comfortable with control and midrange decks that keep most of their seven-card hands, so I couldn’t help but push back.

Mulligan Math Basics

Consider the following hand, on the draw:

It’s useful to break mulligan situations down into outs and draw steps. First, you need to figure out what cards you would need to draw in order to be happy with your hand.

For example, this hand is an Urza’s Tower or Eldrazi Temple away from being one of the best possible hands for Eldrazi Tron. If you draw either land on any of the first three draw steps, you have a great hand. Even in the worst-case scenario of drawing Temple on turn 3, you still cast TKS ahead of schedule.

If you draw Expedition Map on turn 1, that’s as good as Urza’s Tower. If you draw Expedition Map on turn 2, you can still play Expedition Map, crack it for Tower on turn 3, then use the Tower to cast Matter Reshaper. At the end of the day, you  have Tron, a decent body in play, and TKS or Endbringer to follow up.

If you draw Chalice of the Void or Mind Stone on either of the first two draw steps, then your hand isn’t great but it’s certainly fine. You either curve Chalice into Reshaper into TKS, or play TKS on turn 3. These aren’t insane draws, but I’d feel pretty comfortable keeping if I had a Chalice or Mind Stone instead of the Sea Gate Wreckage.

So overall, I’d be happy with Expedition Map, Chalice of the Void, or Mind Stone in the first two draw steps, and Urza’s Tower or Eldrazi Temple in the first 3. I have two Mind Stones in my deck and four of each of the other cards, so I’m looking for eighteen outs in my first two draw steps and eight outs in my third draw step. I’m okay if I draw multiple Towers, so I want to find the probability that I don’t hit any of my outs. There are 53 cards left in my deck, so the chance that I miss is:

Then the chance I get there is 1 minus that, or 0.625=62.5%.

If you’re familiar with statistics, this calculation is analogous to the hypergeometric distribution, only the number of successes and failures in the population change from draw step to draw step.

This probability doesn’t tell the whole story, but it’s a good baseline for evaluating how strong a hand is.

The Simulation

When I showed the hand above to Collin, he thought for a couple seconds and said that he’d mulligan. We talked through the math, but Collin wasn’t convinced. So I asked him what six card hands he’d prefer to this 62.5% shot. Ultimately, he determined that he’d rather have any hand with Tron, any hand that already had a Chalice, a hand that could cast Reshaper on turn 2, or a hand with 2 Temples and any spell.

There was some back and forth here. For example, I asked Collin whether he’d really prefer a six-land hand with Tron, and he said that he’d made it a policy to never mulligan a hand with Tron. Ultimately, to help determine whether the hand was a mulligan, I decided to simulate how frequently the hands Collin preferred showed up.

Monte Carlo simulation is a statistical tool that’s useful to find the probabilities of complicated events. For example, it’s easy to figure out how likely you are to draw Chalice of the Void in six cards. You can just use a hypergeometric calculator. But how can you calculate the probability of having Tron? It’s certainly possible, but with a computer it’s much easier to simply draw a couple thousand hands and check how many of them have Tron.

After simulating for 500,000 hands, I got the following tables:

You can see from the tables that if you know your scry after mulliganing to six, you actually have a 63.85% chance of getting one of the hands Collin preferred . If you don’t count the scry, you have a 50.66% chance. The real probability you want is somewhere between those, but biasing toward 63.85% makes sense because you additionally gain a second opportunity to mulligan. You also don’t account for the chance that you get a similar hand on six cards. For instance, the example hand would much rather have a scry than either the Sea Gate Wreckage or the Ghost Quarter.

Before running the simulation, I was sure that it would support my argument for keeping. The notion of mulliganing a 62.5% chance at a good hand when the fail case is a mediocre hand was preposterous to me. But after simulating, I was convinced that Collin was right and I should have mulliganed.

Conclusions

Now, this isn’t the whole story. The hands we get when we hit with the seven-card hand are better than the hands we looked for when simulating. Some decks rely on quantity more than quality. It’s possible that the metagame is 40% Jund, significantly increasing the probability of turn-1 Thoughtseize. These are all relevant factors, and realistically the decision to keep or mulligan the hand above is so close that you’re not making a big mistake either way.

While you have to accept that you can’t make perfect decisions or build perfect models, you can always do better. These numbers should only be one factor in your decision-making process, but they should factor into it. I originally thought that the example hand was an easy keep, and now I recognize that it represents a very close decision and lean toward mulliganing.

Simulating like this can inform not only how you mulligan, but also how you build your decks. For example, I also learned that Eldrazi Tron is less than 15% to have Tron in a seven-card hand. In my view, that makes playing cards like Karn Liberated and Wurmcoil Engine that are difficult to cast without Tron preposterous. I ran similar simulations for Affinity and discovered that Welding Jar and Memnite have surprisingly similar effects on the consistency and explosiveness of the deck.

Although this analysis was limited to Eldrazi Tron, I’ve found myself mulliganing more often in general. I significantly underestimated the power of mulliganing and the value of the scry from the Vancouver mulligan rule up until these simulations, and am doing my best to adjust myself.

Notes

If you have any questions, ask on Twitter @nalkpas or in the comments below. If you’d like to read more of my writing, you can find some old tournament reports and miscellaneous rambling on my blog. If you’re curious about the simulation code, you can find it on GitHub.

I’m also considering writing some more articles on useful heuristics or statistical techniques, so let me know if you’d be interested in seeing more articles like this.