No rules, no problem: DeepMind’s MuZero masters games while learning how to play them

DeepMind has made it a mission to show that not only can an AI truly become proficient at a game, it can do so without even being told the rules. Its newest AI agent, called MuZero, accomplishes this not just with visually simple games with complex strategies, like Go, Chess, and Shogi, but with visually complex Atari games.

The success of DeepMind’s earlier AIs was at least partly due to a very efficient navigation of the immense decision trees that represent the possible actions in a game. In Go or Chess these trees are governed by very specific rules, like where pieces can move, what happens when this piece does that, and so on.

The AI that beat world champions at Go, AlphaGo, knew these rules and kept them in mind (or perhaps in RAM) while studying games between and against human players, forming a set of best practices and strategies. The sequel, AlphaGo Zero, did this without human data, playing only against itself. AlphaZero did the same with Go, Chess, and Shogi in 2018, creating a single AI model that could play all these games proficiently.

But in all these cases the AI was presented with a set of immutable, known rules for the games, creating a framework around which it could build its strategies. Think about it: if you’re told a pawn can become a queen, you plan for it from the beginning, but if you have to find out, you may develop entirely different strategies.

This helpful diagram shows what different models have achieved with different starting knowledge.

As the company explains in a blog post about their new research, if AIs are told the rules ahead of time, “this makes it difficult to apply them to messy real world problems which are typically complex and hard to distill into simple rules.”

The company’s latest advance, then, is MuZero, which plays not only the aforementioned games but a variety of Atari games, and it does so without being provided with a rulebook at all. The final model learned to play all of these games not just from experimenting on its own (no human data) but without being told even the most basic rules.

Instead of using the rules to find the best-case scenario (because it can’t), MuZero learns to take into account every aspect of the game environment, observing for itself whether it’s important or not. Over millions of games it learns not just the rules, but the general value of a position, general policies for getting ahead, and a way of evaluating its own actions in hindsight.

This latter ability helps it learn from its own mistakes, rewinding and redoing games to try different approaches that further hone the position and policy values.

You may remember Agent57, another DeepMind creation that excelled at a set of 57 Atari games. MuZero takes the best of that AI and combines it with the best of AlphaZero. MuZero differs from the former in that it does not model the entire game environment, but focuses on the parts that affect its decision-making, and from the latter in that it bases its model of the rules purely on its own experimentation and firsthand knowledge.

Understanding the game world lets MuZero effectively plan its actions even when the game world is, like many Atari games, partly randomized and visually complex. That pushes it closer to an AI that can safely and intelligently interact with the real world, learning to understand the world around it without the need to be told every detail (though it’s likely that a few, like “don’t crush humans,” will be etched in stone). As one of the researchers told the BBC, the team is already experimenting with seeing how MuZero could improve video compression — obviously a very different problem than Ms. Pac-Man.

The details of MuZero were published today in the journal Nature.

Pre-orders for the Analogue Pocket retro portable game console start August 3, ships May 2021

Analogue has repeatedly proven that it’s the gold standard when it comes to retro gaming, delivering extremely faithful, but modern hardware to play original NES, SNES, Sega cartridges and more. The company revealed its forthcoming Analogue Pocket last October, and now it’s about to kick off pre-orders for the portable classic console, which can play Game Boy, Game Boy Color and Game Boy Advance games out of the box, and works with even more classic handheld game systems via adapters.

The Analogue Pocket will be available to pre-order for $199.99 on August 3, starting at 8 AM PST (11 AM EST). The actual ship date is quite a while after that, however: Analogue estimates that the hardware should actually start to be delivered to customers in May, 2021. That’s due to “the unfortunate global state of affairs and supply chain challenges outside of our control,” according to the company, and they’re hardly the only indie hardware outfit feeling the pinch of the ongoing COVID-19 pandemic and its impact on tech suppliers.

Image Credits: Analogue

The good news is that so long as you’re patient, the Pocket will almost certainly deliver the goods. Analogue isn’t new to this, having successfully shipped multiple products in the past, including the Nt mini, the Super Nt and the Mega Sg. Each of these more than delivered on their promises, offering fantastic performance in bringing classic games to modern TVs and displays – without relying on emulation.

Analogue Pocket has changed a bit since it was originally introduced last year, with the start and select button relocated to the base of the front of the device, a design change designed for “optimal comfort” according to the company. The Dock that you can use to connect the Pocket to your TV for a big-screen gaming experience also now features a recessed USB-C port to make the connection more stable.

True to form in terms of combining classic gameplay with modern conveniences, Analogue has designed Pocket with a sleep and wake function that’s much more like what you’d expect from today’s smartphones and tablet: Press the power button once and the console enters a low-power suspended state – press it again and it wakes to right where you left off. That’s an awesome perk for games that often lack their own internal save mechanisms.

Image Credits: Analogue

The Analogue Dock ($99.99) can support up to four controllers at once, using either wired, Bluetooth or 2.4ghz wireless connectivity. You can also use separately available multilink cables to connect up to four Pockets for local multiplayer action.

Analogue is also offering a range of other accessories for the Pocket, including a transparent hard case for storage and transportation, a USB-C fast-charging power brick, adapters to provide compatibility with Game Gear, Neo Geo Pocket Color and Atari Lynx games, and MIDI and Analog sync cables for connecting to Mac, PC and music peripherals for use with the company’s Nanoloop music creation software.

Image Credits: Analogue

The company has also revealed some new software features for the Pocket, including ‘Original Display Modes’ which provides faithful representations of the displays (quirks and all) of the original hardware consoles these games where available for. The display itself is made of Gorilla Glass for extra resilience, and offers variable refresh rates and 360-degree custom rotation control.

Analogue Pocket has a 4,300 mAh built-in rechargeable battery that offers between 6 and 10 hours of play time, and over 10 hours of sleep when not in active use.

This definitely looks like Analogue’s most impressive product yet, and one that will be truly amazing for portable console gaming.

Growing up with the Atari 2600, my first gaming crush

Growing up with the Atari 2600, my first gaming crush

I’m 4 or 5 years old, and I’m fixated on the black garbage bag in the back of my parents’ closet.

My ’80s Wall Street dad is gone by this point, officially separated from his married life and settling in for a divorce fight that would last more than a decade. He abandoned a lot of his stuff when he left, probably because a mix of embarrassment and guilt triggered his lizard brain to GTFO as quickly as possible. 

Among the stuff he left was that heavy, black garbage bag. It contained what would become my first game console: an Atari 2600.

Now, decades later, I don’t remember the exact first moment I played a video game. Memories from so long ago come back in flashes, frozen scenes. The one that feels like the first is Pole Position. I can summon up a foggy picture of my mom’s bedroom, a shag carpet, and a tiny version of myself gazing rapturously at the brightly colored virtual race unfolding on our clunky, old TV screen. Read more…

More about Atari, Summer Of Gaming, Entertainment, and Gaming