Interesting talk from Soren Johnson. We grapple with many of these same issues in designing the AI for the Age games so it is neat to see a similar perspective on a different (and more complex game). It does remind me how many, many hours I've lost to Civ.... when's my Civ 5 coming, eh?
Playing to Lose: AI in Civilization
Soren Johnson, EA
www.designer-notes.com
AI programmer for Civ 3, Civ 4, now working on Spore.
Good vs Fun AI. Good = beat the player at own game, a human substitute. Fun = algorithms are the content of the game. Not necessarily looking to beat you, but rather to provide an interesting experience. Good AI = Chess, Deep Blue. Checkers is a “solved” game. Oldschool wargaming often falls in this mold because it is sometimes hard to find human opponents. Fun AI is different: Black and White, Sims. Desktop Tower Defense – the game is about pathfinding. Focus for Fun AI is on the player.
WoW: Aggro is “Fun” AI. Tanks / Healers / DPS. Aggro controls AI – heal the tank, but not too much. DPS the mob, but not too much. Trivial AI problem to “solve”. Aggro is puzzle, not intelligence? WoW mods that show aggro. Commodotized, like HP or some other core game mechanic. Not trying to make it unpredictable.
Most strategy games fit along this spectrum – Chess vs DTD. Starcraft a bit closer to Chess. Partly because it is a symmetrical game – all players trying to basically do the same thing under basically the same rules. No real diplomacy. Heroes of Might and Magic a little closer to the “fun” side of the scale. A bit more asymmetrical, RPG aspects. Civilization is right around the middle of the scale, which actually causes some problems. Deep diplomacy, but crossed with symmetrical game design.
Rules over time: Fixed vs Evolving. DTD changing frequently, SC changing infrequently. Symmetrical vs Asymmetrical. Best suited for MP vs suited for SP. Tactics available to the AI? Everything vs intentionally limited. Measuring performance: objective vs subjective (winning vs having a good time). Passing the Turing test: “Good” = pass, “Fun” = irrelevant.
Good AI = playing to win, Fun AI = playing to lose? Where does Civ fit in that? Win or Lose? Mix of fixed and evolving rules. Symmetrical. Single player focused. Limited options for the AI – a lot of tactics the AI simply won’t use, particularly in the “diplomacy problem”. Common strategy for players is to take all the techs of a defeated AI then break your deal and crush them anyways, something that would be very frustrating for the AI to do to a human. Objective testing? A little unclear, but there are some basic objective measures. Used automated testing during Civ development to see how the AI was performing. Flat out fails the Turing test – not even trying. So Civ is somewhere in the middle. This is the art of writing strategy game AI, to find that best balance.
Lots of different players: Challenge, Sandbox, Narrative.
Narrative players: aim for personality, maintain memory, fall for traps. Tokugawa plays different from Washington, Gandhi vs Montezuma. Personalities who are isolationist, even though this is clearly a suboptimal strategy. It’s OK to make the player feel clever. We want players to win – ideally right at the last minute. But we’d rather err on the side of them winning. If they do lose, they need to understand why very clearly.
Sandbox vs Challenge: Difficulty levels. Sandbox players can get off easy, but how to ratchet up the challenge and include new tactics? What about AI cheating? Good AI should never cheat – who wants to play the chess AI that can teleport his pieces around. Fun AI has irrelevant cheating. You are playing the game in that world, it is just consistent in that way so not thinking about cheating. So what happens in the middle?
Civ 4 AI production modifiers: +50% settler, to -40% deity. Noble is even. Then 5 / 10 / 15 / 20 as ascending the scale. Noble does have other cheats though: animal / barb combat, inflation, unit support, war weariness – the AI just needs help in these areas. The player is in control of the game so they can quit / load, etc. They do what they want and we are trying to help them have an interesting game. That freedom is things where the AI can just never do – like leaving all their cities empty because they know they can reload or know they can trust their neighbors. If the AI did this it would be a disaster. So the balance needs to take that into account. To make something where it is 100% even is just an academic exercise, it doesn’t make a better game for the player. Cheats don’t need to be linear, sometimes the AI needs more help in other areas.
But it should never “feel” unfair. Civ 1 and 2 had issues with this: free Wonders, ganging up on humans. Sid just wrote it that way, probably didn’t realize the level of scrutiny the game would come under given its success. People know it was cheating, but this cheat felt wrong. It’s not much different to say it is half cost since they can build wonders, but just getting something out of the blue is much worse. Civ 1 literally always declared war on the human in 1900 if they were ahead.
Civ 3 and 4 tried to put in human-blind diplomacy so that the human is treated the same as other players, same for testing the “value” of a tech. There are information cheats in the game, however. Basically comes down to limited AI development resources – fog of war is a difficult AI problem and the benefits didn’t justify it vs to a few information cheats. These cheats can backfire though! AI choosing cities to invade looking at defenders – and players realized that the AI would always invade at a lone undefended city, so they could “juggle” the AI. So Civ 4 does some random updates of assault targets, and no longer looks at temporary data like nearby units since the player can manipulate it too easily.
The “Diplomacy” problem. When is it ok for an AI to ally with another AI? Why not just have them all gang up on the human? Should the human be trusted / feared like AIs, even though they are less reliable? Should the AI do anything to try and win?
Cheating is relative – the “tech trading problem”. 1. The AI must trade techs, 2. The AI must trade fairly, 3. The Human can trade techs cheaply. But only 2 of these can be true at the same time. Same as the “problem of evil” (theodicy) – if god is all powerful, and good, why does evil exist? So should the AI sell techs cheaply? Civ 3 probably went too far in this, too aggressively trading techs with other AIs and really trying to extract value out of its techs. Made it feel like the game was rocketing forward too fast and that all the AIs were at the same tech levels due to arbitrage. In Civ 4, AI can undersell by 33% but tries to make up difference in gold, and only trades with other AIs on random turn intervals. Uses same “refuse trade with” logic as with a human. Not necessarily solved the issue, but maybe closer? Art not science.
Remember that the point of cheating is not to write the “best” AI or beat the human or be fair. Cheating is for challenge, which is one source of fun.
How do we design for the AI? Design help for AI: global unit support, closed borders, collateral damage, enforced peace treaties, maintenance vs infinite city spam. But make sure not to design just for AI – all these decisions made sense for humans too, like really enjoying closed borders.
Civ IV AI: all civ AIs been written by the designers. 10 files (game, team, player, city, unit). 25,000 lines of code. Soft-coded decision making. Tiny slice of the overall code base. Traditional testing fails, so need automated testing and hardcore fans to analyze – 1.5 yr closed beta (peaked at 100 users). No Ai scripts (warrior, then temple, etc.). No enums. Everything is made generic. Less brittle code, less predictable AI, works well with mods and expansions.
Probablisitc reasoning: Assign weights to various factors, adjust to current situation, add some noise and then go. Allows a lot of data driven mods: “Double Your Pleasure” for Civ 3 – tons of extra units and buildings. Civ 4 Fall from Heaven.
Game / AI code released on April 06, standalone project compiles into a DLL that replaces game code. 100% independent of engine, not even platform specific. Very powerful, FFH added a whole magic system. Extends game functionality.
Ai projects: community enhancement project, genetic AI project, Blake’s better AI (civ 4 better AI on source forge). Eventually Blake actually joined to work on the Civ 4 expansion.