Teaching a machine to understand Magic is harder than I thought
A friend who works in machine learning asked how Manayard understands cards. Does it use a language model to turn rules text into something structured? Does it learn an embedding where we can group similar cards together? I've has tried versions of both in the month since creating Manayard, and think it makes sense to reflect.
How it works now
First, I parse each card's rules text from Scryfall into effects, triggers and costs. Next is a process of "linking" cards where one card's effect is related to another. These links and card facts get turned into "themes" which I have labeled. I also use deck lists and counts from a collection of cubes, drafts, and commander tournament decks, although these typically indicate popularity instead of mechanical relevance. All of this data is used to as part of a pipeline of filtering and ranking cards based on their fit for any given list of cards, in any format. Why Manayard doesn't start with popularity explains that last part.
How I got here
The first version of Manayard was close to my friend's idea: it was a visual map of every Commander card based on text embedding. Embeddings are good at "similar" and bad at "cares about a game event". How do you find cards that work well with Mayhem Devil? The embedding model I had was unable to answer this question, so I stopped focusing on this approach.
Next, I tried to teach an LLM to build entire Commander decks by focusing on some basic "rules" of Commander (every deck needs ramp, card draw, removal, etc) and giving it a few search tools. This was a little bit successful, but the decklists were far from functional. I also realized that the cost of hosting a tool like this would be too high. So instead, I focused on ways to keep my costs down.
In mid-October I completely rewrote the system to focus on understanding every rule of Magic. Most of the work since has gone into closing gaps like that one. I compared my work with XMage, an open-source Magic rules engine. In XMage, each card is implemented in source code. This helped me build an understanding of how a computer can model all the Magic mechanics. The card Reanimate is implemented with about ten lines of code, instead of the two sentences you see on the card. But in Magic, there are hundreds of exceptions to the rule. Cards on XMage can have fully custom implementations when they break the core rules.
Since I'm not trying to build a full Magic engine, what I've built uses the XMage implementations of cards as a reference, but then adds information in layers on top. For example, the Manayard Reanimator theme tracks every card that has an effect _like_ the card Reanimate, and uses an LLM to rank the fit score for each card.
What's next
I keep improving the engine behind Manayard as I find more mechanics to represent. For example, I recently added support for understanding the card Illusionist's Bracers. That card grants a copying ability to an equipped creature, so it works well with any creature that has activated abilities. Then it's up to you whether you like this connection when searching.