A software engineer in Austin has hooked up Animal Crossing to an AI and breathed new and disturbing life into its villagers. Using a Large Language Model (LLM) trained on Animal Crossing scripts and an RSS reader, the anthropomorphic folk of the Nintendo classic spouted new dialogue, talked about current events, and actively plotted against Tom Nooks predatory bell prices.The Animal Crossing LLM is the work of Josh Fonseca, a software engineer in Austin, Texas who works at a small startup. Ars Technica first reported on the mod. His personal blog is full of small software projects like a task manager for the text editor VIM, a mobile app that helps rock climbers find partners, and the Animal Crossing AI. He also documented the project in a YouTube video.Fonseca started playing around with AI in college and told 404 Media that hed always wanted to work in the video game industry. Turns out its a pretty hard industry to break into, he said. He also graduated in 2020. Im sure youve heard, something big happened that year. He took the first job he could find, but kept playing around with video games and AI and had previously injected an LLM into Stardew Valley.Fonseca used a Dolphin emulator running the original Gamecube Animal Crossing on a MacBook to get the project working. According to his blog, an early challenge was just getting the AI and the game to communicate. The solution came from a classic technique in game modding: Inter-Process Communication (IPC) via shared memory. The idea is to allocate a specific chunk of the GameCube's RAM to act as a mailbox. My external Python script can write data directly into that memory address, and the game can read from it, he said in the blog.He told 404 Media that this was the most tedious part of the whole project. The process of finding the memory address the dialogue actually lives at and getting it to scan to my MacBook, which has all these security features that really dont want me to do that, and ending up writing to the memory took me forever, he said. The communication between the game and an external source was the biggest challenge for me.Once he got his code and the game talking, he ran into another problem. Animal Crossing doesn't speak plain text. It speaks its own encoded language filled with control codes, he said in his blog. Think of it like HTML. Your browser doesn't just display words; it interprets tags like <b> to make text bold. Animal Crossing does the same. A special prefix byte, CHAR_CONTROL_CODE, tells the game engine, The next byte isn't a character, it's a command!But this was a solved problem. The Animal Crossing modding community long ago learned the secrets of the villagers language, and Fonseca was able to build on their work. Once he understood the games dialogue systems, he built the AI brain. It took two LLM models, one to write the dialogue and another he called The Director that would add in pauses, emphasize words with color, and choose the facial animations for the characters. He used a fine-tuned version of Googles Gemini for this and said it was the most consistent model hed used.To make it work, he fine-tuned the model, meaning he reduced its input training data to make it better at specific outputs. You probably need a minimum of 50 to 100 really good examples in order to make it better, he said.Results for the experiment were mixed. Cookie, Scoot, and Cheri did indeed utter new phrases in keeping with their personality. Things got weird when Fonseca hooked up the game to an RSS reader so the villagers could talk about real world news. If you watch the video, all the sources are heavily, politically, leaning in one direction, he said. I did use a Fox news feed, not for any other reason than I looked up news RSS feeds and they were the first link and I didnt really think it through. And then I started getting those resultsI thought they would just present the news, not have leanings or opinions.Trumps gonna fight like heck to get rid of mail-in voting and machines! Fitness obsessed duck Scoot said in the video. I bet hes got some serious stamina, like, all the way in to the finish linezip, zoom!The pink dog Cookie was up on her Middle East news. Oh my gosh, Josh :)! Did you see the news?! Gal Gadot is in Israel supporting the families! Arfer, she said, uttering her trademark catchphrase after sharing the latest about Israel.In the final part of the experiment, Fonseca enabled the villagers to gossip. I gave them a tiny shared memory for gossip, who said what, to whom, and how they felt, he said in the blog.The villagers almost instantly turned on Tom Nook, the Tanuki who runs the local stores and holds most of Animal Crossing's inhabitants in debt. Everythings going great in town, but sometimes I feel like Tom Nook is, like, taking all the bells! Cookie said.Those of us with big dreams are being squashed by Tom Nook! We gotta take our town back! Cheri the bear cub said.This place is starting to feel more like Nooks prison, yknow? Said Scoot.Why do this to Animal Crossing? Why make Scoot and Cheri learn about Gal Gadot, Israel, and Trump?Ive always liked nostalgic content, Fonscesca said. His TikTok and YouTube algorithm is filled with liminal spaces and music from his childhood thats detuned. Hes gotten into Hauntology, a philosophical idea that studiesamong other thingspromised futures that did not come to pass.He sees projects like this as a way of linking the past and the future. When I was a child I was like, Games are gonna get better and better every year, he said. But after 20 years of playing games Ive become a little jaded and Im like, oh there hasnt really been that much innovation. So I really like the idea of mixing those old games with all the future technologies that Im interested in. And I feel like Im fulfilling those promised futures in a way.He knows that not everyone is a fan of AI. A lot of people say that dialogue with AI just cannot be because of how much it sounds like AI, he said. And to some extent I think people are right. Most people can detect ChatGPT or Gemini language from a mile away. But I really think, if you fine tune it, I was surprised at just how good the results were.Animal Crossings dialogue is simple and that simplicity makes it a decent test case for AI video game marks, but Fonseca thinks he can do similar things with more complicated games. Theres been a lot of discussion around how what Im doing isnt possible when theres like, tasks or quests, because LLMs cant properly guide you to that task without hallucinating. I think it might be more possible than people think, he said. So I would like to either try out my own very small game or take a game that has these kinds of quests and put together a demo of how that might be possible.He knows people balk at using AI to make video games, and art in general, but believes itll be a net benefit. There will always be human writers and I absolutely want there to be human writers handling the core, he said. I would hope that AI is going to be a tool that doesnt take away any of the best writers, but maybe helps them add more to their game that maybe wouldnt have existed otherwise. I would hope that this just helps create more art in the world. I think I see the total art in the world increasing as a good thingnow I know some people would say that using AI ceases to make it art, but Im also very deep in the programming aspect of it. What it takes to make these things is so incredible that it still feels like magic to me. Maybe on some level Im still hypnotized by that.