Wow.  This was *easily* the best presented and most inspirational / thought-provoking talk I've been to at GDC this year (and maybe in a number of years).  Very good stuff, Clint said he's going to put the talk up on his blog later (you can see it in the blogroll to the left) and I recommend checking it out, though given his style the slides alone won't convey the story.

My interest level in Far Cry 2 shot up about 1000x after this talk, even though he went out of his way to not talk about the game itself.

i-fi: Immersive Fidelity in Games

Clint Hocking

Here to talk about immersion.  Why immersion?  Creative Director on Far Cry 2 but not really talking about that today.  3 yrs ago doing initial conception.  What would be the core of the game?  Identified a half-dozen possible pillars.  FORMID(able). 

Freedom of Gameplay

Open World

Realism

Meaning

Dynamism (drama management?)

Immersion

DRFMO instead?  Figured by now gold master would be out, easy to show immersion.  If you think you understand quantum mechanics, you missed something.  Same with immersion.  In addition to being complex, it is very divisive – smart people with conflicting opinions. 

Immersion: the basics.  Not a property of a game, but a player state.  Evoked in the player, not put in the game.  Similar to other states?  PlayerImmersed=1.  Doesn’t seem to have an analog range – immersed or not.  Not partially immersed any more than partially dead?  Rapidly switch in and out, which may feel that way.

So if it is a state what can draw him into that state?  Formal: left brain, Sensual: right brain.  Logical vs Sensory. 

Sensual Immersion, via the right brain.  Passionate, emotionally charged.  Hamlet on the Holodeck – 4 properties of digital envrionemtn “procedural, participatory, encyclopedic, spatial”.  Last two = immersion.  Painting immersion as a sensual experience.  Movie theater – sense deprivation and getting all our senses from a movie.  Any sense alone can maybe do this – master painting, miles davis, pure chocolate.  Sex – total immersion in the tactile.  Incense = smell immersion?  Well, look at dogs…

What are we trying to do?  Close the gap between the player, avatar, and game world.  Not applicable only to these kind of games but it gets cumbersome to use the more formal language and want to talk about concrete, not abstract.  Answer?  Trespasser!  The promise was to be the most immersive game ever made:  sprite impostors, normal and specular maps.  First game to offer many features that are standard today.  Needs driven AI (if barely functional).  Only recently achieved (Sims).  Robust physical simulation (Half Life 2 used a clever fiction to dispel need for arm). 

Drill in on two: IK driven animations, removal of all HUD elements.  IK was very ambitious for the time, trying to do something very ambitious.  Another decade before a game would succeed at this (AC).  Asserting their failure doesn’t enlighten us – look at why those decisions were made in the first place. 

Goals of animation system?  1:1 between player input and avatar response.  Higher fidelity than queueing up canned animation.  More procedural, so more reliant on inherent values of the mechanism.  Success of the Wii similar parallelism?  Canned response to motion feels bad, but direct motion correspondence causes immersion.  Trigger like buttons on the Xbox controllers to dominance of shooters.  Innovations like skate driven this way.  Guitar Hero using  Guitar Hero like controller.  GH on a pad is worse than even Simon.  Improve immediacy and responsiveness, and make 1:1 correspondence between avatar and player.  What about player and avatar connection to the game world itself?  Back on Chaos Theory, vivid memory of reaching out and taking hands in co-op.  Almost felt as if he had reached through, vivid feeling of physical presence.  This was with canned and cued animation, but maybe with a fully procedural solution we can make that work – hand always grab the hand.  Very human when achieved.  Increase the connectivity of the animation.

What about removal of the HUD?   A lot of that information is needed, like ammo.  Vocal feedback for state of ammo, health on a tattoo on Anne’s left breast.  Not just about the tits here – most games would show you those AND give a health bar.  What were they trying to do?  Every game needs to communicate state to the player.  Chess or Pac-man, you see all the pieces and have a precise game state.  Civ requires exploration and scouting, so making decisions about what you need to know now.  In a first person view we are taking the most limiting camera, and thus limiting that information.  So guys shouting out “reloading” when they are around the corner.  Makes it seem like enemy is communicating with allies, but really is saying “now charge me and kill me before I reload”.  Convert every meter in the game into something that is perceived directly by the player’s sense.  UI goes through a mediating filter of a HUD.  Direct experience facilitates sensory immersion. 

Left brain: Games vs Digital Environments.  Bioshock – compelling sense of place and graphics / audio / script.  More immersive than “Norbit” or other movies on the shelves.  While immersive it is a different kind of immersion – those qualities that are unique to games.  Games as film.  Immersion that is fundamental to a game, what does that even mean?

Formal Immersion.  Sensory immersion is a tool but it is not unique to game.  Is chess not immersive because it has no sound or smell?  You can play chess like a list of text of course, so it is not immersive?  Of course it is, perhaps even moreso than things that just require us to immerse our senses.  So what is going on with our brains?  They are sophisticated pattern matching machines, so good at it that they have constructed in their likeness another: the computer.  We just sort automatically.  Drawn in by looking through particular types of patterns.  Sorting “small things” is a few moments, but games are complex and require deep recursion.  Fractal leaves.  Uncharted new areas, exploring deeper in known games.  Exploring aimlessly is like doing math.  Playing a game with a goal is a process of seeking good patterns.  Good patterns persist, optimal patterns win.  Chess openings, these have been very heavily recursed.  Lets look at Guitar Hero, a simple game – clear power of recursive patterns. 

Three axes: implication, complexity, tolerance.  Implication is number of elements implicated in the pattern -- # of frets used.  Higher degree of implication = more frets = more difficulty.  Implication in chess – 4 for Anastasias Mate.  Complexity is what the player much be able to do with the implicated elements.  GH, 1 implicated element at a time at first, but over times more chords and sustains, moving up and down the fretboard, finally juxtaposition of rapid movement and plucking.  Complexity in chess – the comprehension of simultaneous offense and defensive nature of pieces. For each piece how is it implicated for you and opponent.  Tolerance – requirement of precision and accuracy.  Literally has a meter for this in GH – how fast it fills and drains.  Chess – the competence of your opponent.  Easy opponent is very tolerant for Kasparov, but Deep Blue was not.

So, if well used how does this create immersion?  Start on easy, seeing patterns of notes.  Variations on a pattern within a song.  Different themes and patterns within a level, but more or less the same level of implicity, complexity, and toleration.  Easy: slightly increasing complexity.  Medium: implication and tolerance immediately jump.  Continues recursively through hard and expert.  These well tuned forces acting in concert create immersion.  Unlike sensual immersion, this kind of immersion is universal to games.

Now what?  We have these two kinds of immersion.  Which is better?  Clearly they must fight!  Critics of sensual immersion: not fundamental to games, so at best a supporting aesthetic goal.  But total  sensual immersion is a critical convergence point?  Critics of formal immersion say that those patterns, while delightful, are not mass market.  Grandma may play GH at Thanksgiving but she doesn’t recurse it.  Love and fear and sacrifice not challenges of dexterity.  Rock god, not Simon expert.  The formal structure of GH doesn’t cause this, it is the skin and the controller.  Sensual immersion the domain of film?  If we want to compete for cultural dominance, don’t attack on their home turf.  Movies have a lead, so play to our own strengths?  Recursion through game systems have a divisive effect, leaving behind those who cannot keep up so as games become more complex they inherently become less accessible.  Imagine if every mystery movie you watched increased your ability to watch mystery movies, by today they would be incomprehensibly complex. 

The opposite of one profound truth may very well be another profound truth.  Quantum vs Relativity.  WTF? 

Reconciliation:  How can we get them to work together and “flirt”?  Mix those elements and make them more than reflex honing.  Chess vs MacBeth – similar structure, can we theme one up as the other?  Bioshock leverages its sensual immersion by daring us to rip out the little sisters hearts, while challenging us to recurse our patterns of saving them.  Recurse through love and sacrifice.  While experiencing love while immersed in both directions, we will be able to recurse through them.  What if instead of mystery complexity it was nuances of human emotion?  What if we trained not for playing notes but bringing people together?

Coming together of these two forms of immersion is the future.  All media can bring people together.  Each new medium can obliterate the dominance of previous forms.  We are not on the verge of edging out films.  We are just starting and when we finish we will obliterate them – games to film as films are to radio. 

Q:  People don’t “level up” in mystery.  But they do, difficult by your 20^th novel its tough for the author to trick you.  Yes, that’s an oversimplification. But they don’t get explosive the way you do when you recurse patterns.  You don’t get a little bit better at GH – Expert vs Easy is orders of magnitude more difficult.  Linear media we get better at by degrees because we don’t have that feedback loop.

Q: Voice in immersion?  AC: world is believable but spoken voice was disrupting.  Also relevant in narrative of SC (on blog).  Difficult for avatar to talk, speech is very specific unlike throwing a grenade.  When I say something I mean what I say, so if my avatar is speaking for me there is a big lack of connection.  One technique – in the X-Men movies Wolverine comes in and makes fun of them but in fact what he is doing is acting as the bridge of the audience into this movie, expressing their incredulity to ultimately accept this because we followed Wolverine’s journey.  Unfortunately can make your avatars cynical.  Silence maybe better to not break immersion.

Q:  Learning is a real barrier to immersion but those are not really mass media – millions don’t play chess.  But Chess is mass media – more people play it than GH.  More people play sports than videogames.  But those things are not about media, and those kinds of sensual immersion experiences.  Q: In reality, we don’t have to learn the details of physics to operate.  We need to leverage that sensual immersion but then recurse them – it’s not enough to just build them, we have to attach the warp drives as well.  Bioshock is the best example of this.

Q:  Is IK less of a 1:1 relationship because you’re not jumping?  Difficult choice to strip away a lot of the intent that players can express – pressing button to dive through a wall or off of a stall.  The problem with that approach is the recursive problem.  We started makng games where the controls are too complex, so many things you need to do that it becomes inaccessible.  By making movement where I want to go, but abstracting the jumps and leaps, you keep it abstract. 

Q:  Competitive MP can be very immersive.  If we want to enhance that we need to do that by displaying as much information as possible as accurately as possible.  Doesn’t losing the life bars hurt games like Street Fighter?  Philosophical answer:  MP and SP are two different problems.  In MP tolerance for not having that information is very very low.  In SP we have become addicted to giving more information than we need.  Information that is mostly junk.  258 bullets, or I just need to reload?  More interesting question is how do you ask when you *do* need to know?  What is that threshold?  Ways to give the player better sensory experience than relying on a HUD.   Consistency between MP and SP is always a challenge, hard thing to balance.

Q:  Intersection of formal and sensory immersion.  Started talking about love and doubt and sacrifice.  Is empathy a sense we can engage via immersion?  If we postulate synthetic emotion, then we can recurse.  What does that look like?  An example starting to build:  my girlfriends in GTA San Andreas.  Give mechanical bonus but there is no sensual immersion in that relationship.  Maybe Hot Coffee would have increased that immersiveness?  If managing that relationship as a running component of the game and sensually invested and can really “feel” that bond in a non vacant and stupid way (cinematic like currently), then we can explore it on a super-accelerated timeframe.

Q:  HUD as mediator of sensory immersion – isn’t an avatar this as well, since it isn’t you?  Avatar can be a way of physicalizing certain elements of the HUD.  The Tattoo was just copy and paste onto the avatar so it doesn’t really work.  Done in 3^rd person games frequently these days, seeing wounds and blood and stance.  Avatar itself can be a meter but if expressed physically can feel different.  Q:  does even presenting the avatar cause that disconnect?