Facebook Claims to have Developed a Board Game AI that is “Human-Level”

They refer to the capability of the object, not the likelihood that it will manipulate or desert you.


In the board game Diplomacy, which is notable for being based on human interaction rather than moves and manoeuvres, like chess, Facebook, or as we’re supposed to call them now, Meta, announced earlier today that their CICERO artificial intelligence had achieved “human-level performance.”

Here is a trailer :

If you’ve never played Diplomacy, why is it a big deal? In that case, it would be a board game first published in the 1950s and primarily played by people settling disputes while seated at a table or entering other rooms. Dice or cards decide nothing about the game; instead, it is decided by human contact.

Therefore, it is a bold claim for an AI developer to say that it is playing at a “human level” in a game like this! The Meta supports this by stating that CICERO operates on two different levels, one of which crunches the game’s progress and status. The other is trying to interact and communicate with us on a human level.

To bolster their claims, Meta has joined “Diplomacy World Champion” Andrew Goff. He claims, “In contrast to many human players, CICERO never softens its stance or begins to become motivated by retaliation. It simply presents the situation as it is. It is therefore ruthless in carrying out its strategy, but not in a way that irritates or frustrates other players.”

That seems ideal, but as Goff points out, it might be too ideal. This shows that CICERO could play more proficiently, even though it keeps up with humans. CICERO “occasionally generates inconsistent dialogue that can undermine its objectives,” as Meta themselves state in a blog post. I complain that every illustration they give of its communication (such as the one below) portrays it as a psychopathic office worker terrified that if they don’t end every sentence with “!!!” you’ll think they’re a bad person.


Of course, winning board games isn’t the program’s ultimate objective. It’s merely “advancing human-AI interaction” by using Diplomacy as a “sandbox”:

Although CICERO can only play Diplomacy, the technology that made this achievement possible has many practical uses. For instance, RL and planning-based natural language generation could reduce communication barriers between humans and AI-powered agents. Today’s AI assistants, for example, are excellent at simple question-answering tasks like providing weather information, but what if they could hold a sustained conversation to teach you a new skill? Instead, consider a video game where non-player characters (NPCs) could organize and communicate just like real people do, understanding your intentions and tailoring their responses to assist you in your mission to storm the castle.