Thursday, 20 April 2017

Evoking Presence

Playing a videogame can put you in a state where the borders between your self and the character gets blurry. This is one of the major differences that sets games apart from other mediums such as films and literature. When creating games, evoking this feeling of presence is worth trying to achieve. 

Before starting on the concept of presence, I need to discuss why it's so important to dig deeper into these aspects of games. This is not really needed in order to explain presence, but I think it is vital to know why it is so crucial to gain this deeper understanding.

I have talked about the concept of an "idea space" and how developing a game is basically about navigating this space. The most important concept that I want to get across is that developing a game is like going on a journey. You have a starting point and an idea of where you want to end up. When making a narrative game, having a clear focus on the goal is extremely important as there'll be many occasions when you need to go against what has the greatest gameplay benefit in the short term in order to reach a better end result. But given that you can't choose your next step based on what gives the largest boost to "fun", what do you base your decision on? How can you achieve a high degree of certainty that you've made the right choice?

You do this by having rules and principles that you follow. A simple example of such a principle is to ensure that gameplay makes sense within the story. It might be more "fun" from a short-term perspective to give the player a flying unicorn, but if that seems silly within the story then this is a bad decision. However, it's not always that clear-cut, and since you can't simply "follow the fun" you need other things to guide you.

Evoking the feeling of presence is such a principle.

So what exactly is presence? Well, it isn't the most well-defined term, but for our needs we can define it as something like "How much the player feels like they are present inside the game's virtual world". One way to measure this is to test the player's unconscious reflexes and see if these react to events in the game. For instance, does the player flinch if an object comes flying towards the screen? It's simple, but not the only way to measure presence. A more important aspect, in my opinion, is to evaluate to what degree the player feels they are their on-screen character. If the player views an in-game threat as something that is bad for them personally, then it means the sense of presence is high.

Silent Hill 2 (2001)

I remember playing Silent Hill 2 with my wife 6 years or so ago. As the intro sequence was over and she headed into the woods, she started to feel quite shaken. She went on for a minute or so and then eventually exclaimed that she couldn't play it any longer. The game was just too scary. So I took over the controller and suddenly she didn't feel as scared any more. I now decided to conduct an experiment and handed her back the controller. The moment she started controlling the main character she got scared, and again refused to play for more than a minute or so. This is quite interesting. Her feelings towards the game were quite different depending on whether or not she was holding the controller.

This is a great example of presence. When my wife held the controller she was no longer just a spectator of a scary narrative, she was the protagonist in a horror world. This sense of presence changed her view of the game drastically, and I believe this is what makes it a core component of creating good interactive storytelling [1]. So, understanding this phenomena is paramount in becoming better at making this sort of games.

To understand what it is that happens here, let us take a look at an experiment.

In order to conduct this experiment you will need a screen, a rubber hand and a hammer. You let your subject place their hand on a table and then place the rubber hand next to it. The screen is placed between the two so that the subject can only see the rubber hand.



You now start to stroke the rubber hand and the subject's real hand in the same place at the same time. Once you have done this for a while the subject will start to feel as if the rubber hand is their own. You can now test this sensation by quickly grabbing the hammer and slamming it on the rubber hand. The subject will now, as an unconscious reflex, pull their real hand out of the way. You can see a video of it all in action here:



This is quite astounding. By just using some very simple manipulation you are able to change a person's mind in such a way that they think of a rubber hand as their own. You don't even have to use a hammer to test it. You can even threaten the rubber hand with a knife and see that the galvanic skin response (palm sweat basically) is the same as if it was the real hand that was threatened. There has really been a change in how a person perceives their body.



A bunch of similar experiments have been made by Henrik Ehrsson, above, who has managed to get people to have out of body experiences, by putting them in the bodies of mannequins and using very similar techniques to the ones explained earlier.

So why does this happen? In order to understand this you first have to understand a bit of how the brain works.

It is common to intuitively think that inside our heads sits a little man, a homunculus, who receives all of the input picked up by our eyes, ears and other sensory organs. When you start pondering this idea, it's obvious it's not the case - it just begs the question of how the little man is able to see, and you end up in an infinite regression. What actually happens is that there are a bunch of different modules in your brain that collect and process various data. This data is then sent onward for more processing or used as a means for decision making. There is no one thing that controls the brain. It's all controlled by a bunch of different computational systems, each receiving different input and being able to give certain output. Marvin Minsky's "society of mind" is a very good description of how it all works.



So what happens in the rubber hand illusion is that the input you get from your eyes overrides the kinesthetic sense. There is a sort of feedback loop going on between the constant sensation of being stroked, combined with the visual confirmation of seeing it being stroked. This provides a slight conflict with the kinesthetic sense, but the brain has to make a decision and decides to treat the hand as actually being the rubber hand.

Your sense of self is not set in stone. It's something that is highly malleable and is under constant evaluation. At any moment, the brain relies on the information that it has available in order to form the concept of your self. The entity that you refer to as "yourself" is really just a mental construct that's useful in making sense of the world, navigating it, and taking decisions. Most of the time it's fairly accurate and gives the right picture, but as we've seen, it's not always the case. It can be hacked.

This is where games enter the stage - because this sort of self-hacking is exactly what games do. When your current mental model of your self incorporates your in-game character, an approaching monster will make you feel afraid. This is extremely powerful and something that makes games very special. When you press down the button or stick that makes the player move forward, you instantly get confirmation that you are making a character move. Volition turned into action becomes a feedback loop and this causes your brain to change its view of your self.

In books and movies there is no such feedback loop. Information is only presented to you. In these media you are a spectator that watches as events unfold. But in a game you are an active participant who causes events and where things happen to you personally.

This is what presence is all about! And for me this is the core reason why interactive storytelling is so exciting. You are no longer just a passive audience but an active and present participant in the narrative. Being able to achieve a strong presence is a fundamental building block in an interactive narrative.



So does this mean that Virtual Reality is the ultimate device for doing interactive storytelling? Well, it is true that VR has a lot of potential to create presence. For one, it adds two senses, balance and peripheral vision, to the mix, It also allows a natural feedback loop to occur by looking and seeing the view move about. There is no denying that VR does things that games on your standard TV or monitor cannot. However, what is crucial with presence in games is what sort of activities it allows you to be present in. VR can increase the sense of presence when it comes to just standing and looking around. But I am not as convinced that VR will be as suitable for more complex narrative actions. For instance, a drawback of VR is that the game can't really seize control of the camera - something that allows many games to provide contextual animations. This and other tricks are things that are effective in making the player feel present in story events. We used this a lot in SOMA in order to give philosophically complex events, such as the body swap, a more visceral feel.

Obviously if you use the VR medium to its advantage you can elicit responses that wouldn't be possible otherwise. But what it all comes down to is that different mediums can do different things well, and that it's not a case for VR always being better at conveying presence.

I am not bringing this up to be dismissive of Virtual Reality - I think it's a very exciting field. I go over this in order to make it clear that a sense of presence is not just about recreating our normal way of being as accurately as possible. Ways to convey presence can take many forms, but what they all have in common is that they hack our brain into believing it's partaking in something it's not.

This also answers the common question whether or not a first person mode is better at generating presence: sometimes it is, and sometimes it isn't. Sure, we normally see our lives from where our eyes are situated. But this is really just a helpful mental model. Remember, there is no small man in your head that is witnessing everything. It is all just modules that process information in various ways. If it wanted to, your brain could construct your sense of reality from a third person perspective. This is in fact what happens during out of body experiences. The reason we don't use this version during everyday life is because it is not the most optimal one. We cannot see everything that happens around us, and therefore it makes more sense to just let vision be modelled as if seen through the front of our face.



So really, the output from a game is just a stream of data that gets injected into our brains. The brain will then process the data and model the world accordingly. A third person viewpoint is just another way to be presented with that data. Sometimes it can be advantageous compared to a first person one - for instance when showing damage to the protagonist. This is something we normally get as a signal of pain, but as that is not possible [2] in a game, you can trick the brain by having the onscreen character limp and show big disgusting wounds on their legs. So again, it all comes down to different approaches being suited for different things.

So the sense of presence is a brain hack, and it can be done in different ways. Then what exactly are these different ways?

I will now go over a few basic principles that will help you maximize your sense of presence. There is a lot more to be said about these, but right now I'll just summarize the most important aspects. I'll go over each of these in more detail in another blog post later on.


Intuitive controls
The most important principle is to make sure that the controls can't be overly complicated. What we want to achieve is a feedback loop where the player thinks of something and then sees it happening. We want to make a connection between the onscreen character and the player by connecting volition with action. This won't happen if the player is too focused on pressing specific buttons.

In order to achieve a strong sense of presence, the controls need to be established as early as possible and be used for all future actions. Every time the player has to learn new ways to control their character, or has to look down on the controller to make sure they are doing an input correctly, the feedback loop is broken and presence is weakened.

A good example of a game doing this correctly is Limbo (and the more recent Inside). The player is taught the controls during the very first minutes of gameplay, and from then no new controls are needed. Instead the existing control scheme is used in intuitive ways to provide many different sorts of actions. This makes the player-protagonist connection very strong and I think it is one of the game's big success factors.


Constant Feedback
Once you have the player-protagonist feedback loop up and running it is important to keep it up. If the player just sits with the controller in their lap watching as things happen, the feedback loop will be broken and their self will no longer be extended. So it is important to keep the player busy. It is especially good if you also make sure the input has a good correlation with the movements that you are doing. For instance, moving the mouse to look around creates a nice feedback loop, but if you just press a button in order to accomplish a complex manoeuvre you will not feel as present.

Experiments show this clearly. As soon as you stop stroking that rubber hand the illusion start fading away.

This is one of the reasons why we have the physical interaction in Amnesia. Not only does it allow for some interesting analog actions (such as peeking out from a closet), it also makes sure that the players are feeling a sense of presence as they open the door, pulling a lever and so forth.


Consistency
In order to hack our brains there needs to be a good pattern to follow. The key feedback loop has to do with desiring something and then seeing it happen. But in order for this to actually work, the thing that you want to happen must actually happen. If you press the jump button and the character doesn't jump then there is not a connection any more. In fact, this becomes a negative stream of data to the brain which brings about the conviction that you are in fact not controlling the on screen character. The same rule also applies to interactions. The player will base their actions upon what they are currently seeing and what they know about the world. And if that set of beliefs is not accurate, then the player's volition will fail.

This doesn't just apply to actions that you input, but also to those where you don't. It is really annoying if the character does some movement without you having provided any input for it. A game that does this the right way is Assasin's Creed. Here the player's character jumps without the player providing input, but it feels good because you press down on a button as it happens (hence willing the action) and jumping is sensible and handled in a consistent manner. Presence is maintained. However, the game also fails miserably at times and the characters can start jumping when you really just wanna walk close to a wall. In these cases the sense of presence is severely weakened.


Realism
Finally, it also very important that things feel real. By this I don't mean that things should be photorealistic. But it is good if things happen according to how they are anticipated to happen, and that forms take a shape that make sense to us. In that way, the brain can more easily process the data using existing methods. For instance, presence works better when your character is walking like a normal human and not running around like some freak (as is the case in games like good ol' Doom). In the same way it's positive if as many of the actions as possible feel close to the ones we experience in everyday life.

Experiments can clearly express this by showing that it is much harder to make the subject feel as if the table belongs to them, than it is with the rubber hand. However, a super detailed hand doesn't matter as much. The most important aspect is that it looks pretty close to a real hand.

Exactly what sort of level of realism required is hard to say. A good thumb of rule is that you should try and have enough room for the brain to fill out the details. If you go with overly photorealistic there will be too much focus on that, and you'll end up with a problem similar to the uncanny valley [3].

---

Back to where I started. Now we have four new principles that we can follow in order to navigate the space of ideas. So instead of just trying to go with ideas that give us the most fun gameplay, we can instead try and get as much presence as possible. What is good about having principles like these is that we don't have to be able to directly playtest the amount of presence added, we can instead just rely on making sure the game fulfils the requirements for creating lots of presence.

Obviously you'll need to test at some point. A principle is not an absolute truth. But it allows you to plan further ahead and gives you more confidence to tread into uncharted territory. If you can see that a certain path through idea space means that the underlying goals of evoking presence is met, then that's a good indicator you're moving in the right direction. Of course, presence is not the only thing you need to work on, but it's a fundamental part of creating an engaging narrative experience. If your game is going in a direction where the principles are not met, then you might be undermining any other features intended to accomplish interactive storytelling

That's it for now on presence. There are more details to be explored, but those will be brought up in later blog posts. Next week, I will be going over something called the Mental Model. This will go deeper into how we as humans create a virtual representation of both our selves and the world, and how this can be exploited for making better narrative games.


Footnotes:
[1] This is how I see it and where I personally want to take games. There are other ways to approach digital storytelling. For instance, you can see the player's role as someone who controls how the plot and pacing flows, and so forth. There is really no best way of doing it. But in order to get somewhere I need to take a stance, and games that puts the player in the shoes of another character are the ones that I find most interesting. Therefore, this is the direction that I want to explore. People who feel otherwise are welcome and encouraged to follow other paths.

[2] At least not without special equipment and not being afraid of a painful gaming experience.

[3] This is a huge subject and very interesting, but will have to cover it in a future blog post.


12 comments:

  1. The first principle seems like the easiest to achieve. Right now even popular AAA franchises like CoD have relatively complex controls (Far Cry 3 has sprint, jump, crouch, melee, shoot, aim, grenade, first aid, throw rock, context sensitive action, hold breath, weapon scrolling & shortcuts for different weapons\items etc without feeling overcomplicated). Comparatively, the controls for Penumbra\Amnesia\Soma are quite simple. Besides, if the controls are too complex, the gameplay would suffer, so even from a classical gameplay perspective someone would be able to spot the problem.

    ReplyDelete
    Replies
    1. This is a good point!

      However, I would argue there is a few crucial differences between a game like Inside and your average AAA shooter.

      - Unless you played a similar game, it takes a while before you actually get an intuitive grasp. In a narrative heavy game, you need the player to be onboard as soon as possible.

      - Shooters tend to have a lot of contextual actions where there is constantly a button prompt popping up. Whenever a game needs to do this, it is a sign that the controls are not optimized to make intuitive sense.

      - There is a certain "messiness" to modern shooter controls where it is less about maximizing intuitiveness and more about cramming in as many features as possible. I guess this is highly subjective, but at least for me there is a very big difference.

      That said, I do think that modern shooters do a pretty good job (at least given that you practice a bit) at letting you feel like you are not pushing buttons to produce results, but actually are willing action into being.

      Modern shooters are in fact pretty good at many of the basic presence-creating aspects. So if you want to do a narrative focused game, checking out a game like Call of Duty to get ideas on controls is not a bad start

      Delete
    2. Let's see the arguements one by one:

      "Unless you played a similar game, it takes a while before you actually get an intuitive grasp. In a narrative heavy game, you need the player to be onboard as soon as possible."

      That's true, but I think the fact that many gamers nowadays are used to complex control schemes makes the process of adaptation easier for them. Complexity isn't something that they are unfamiliar with, it doesn't "scares" them.

      The problem still stands though, since new controls need time to be learnt.

      ---

      "Shooters tend to have a lot of contextual actions where there is constantly a button prompt popping up. Whenever a game needs to do this, it is a sign that the controls are not optimized to make intuitive sense."

      It's not a problem if all the possible context sensitive actions are showcased early. So in Far Cry 3 for example, the context sensitive command is used for many things: Talk, buy stuff, enter vehicle, open door, take new weapon\replace current weapon with the one on the floor, search enemy corpse for loot, disable alarm, take plants, skin animals, open chests.

      Now if the game is constantly giving you new ways in which you can in interact with the same button, that would be a problem, but the game shows it all in the first 15 minutes or so.

      Also important to notice that there is little to no ambiguity to how the character would act in most of these contexts. When you are near a door, you know that the button would be used for opening it. When you are in front of an alarm device, what else may want to do than disable it? Likewise, if you see a weapon you can take it etc. (The "search corpse" command is less obvious though, since you might want to hide the corpse instead of looting it).

      ---

      "There is a certain "messiness" to modern shooter controls where it is less about maximizing intuitiveness and more about cramming in as many features as possible. I guess this is highly subjective, but at least for me there is a very big difference."

      There is a modern trend in gaming to mix different styles of gameplay, even when they clash with the storytelling, or the overall theme. So in the example of Far Cry 3, you can freely ignore the main quest in order to explore the island, do crafting, climb radio towers etc even if narrative-wise you must hurry because your friends are in the hands of ruthless pirates. Or regarding the second case (gameplay vs theme), you just can't put extensive platforming mechanics in a horror game, it would feel out of place. There are also people who felt that the focus on crafting in Fallout 4 didn't fit with the action RPG gameplay, it's the same issue.

      So I would say that is not in complexity where the problem lies, but in certain parts of gameplay not getting well with the overall theme/storytelling.

      Would you say that Thief: The Dark Project or FEAR feel messy or needlessly complex for example? Complexity and intuitivity aren't mutually exclusive in my view.

      Delete
  2. I want to add two more ways in which the effect of presence can be created. The first one can be used in basically every type of game (at least in the convential sense of the term) while the second is more genre-specific.



    1. Having to beat the game.

    Or having to "escape from the haunted mansion" and the like, if we are talking for story-oriented games. Giving the player a simple goal and then letting him figure out how to achieve it is very effective for creating engagement, because in order to "win", one needs to focus, forgetting the outside world. Similarly, if we examine in what context we are really focused in every day life, that would be when we are trying to solve a problem or when we are engaged in an activity that requires a certain level of skill. That's how games always worked, and it remains the most effective way of creating engaging, "catching" experiences.

    2. The concept of "something bad can happen".

    Having in mind that there are things which can damage your well-being is basically the #1 thing that makes horror games scary. Of course something must eventually happen, or the player would realize that the danger is just in his head (especially if the person in question is familiar with the horror game genre).

    ReplyDelete
  3. Thanks for an interesting read!

    About VR, horror experiences and presence, one could argue that technically VR is capable of emulating all other mediums, so even if you make a game that isn't about the player experiencing things from a first person perspective you can still create it for VR and unlock immersive features that wouldn't be possible if they weren't in VR.

    There was a game called "Alone" on the Oculus Rift that was interesting because it was essentially about you playing a game within the game. Basically you played an old school alone-in-the-dark-esque game on a TV-screen while sitting in a virtual room.

    As you progressed in the 2D-game, things happened in the "real" room, reflecting things that happened in the 2D-game such as something running past a window or the sound of footsteps coming from the upper floor.

    https://www.youtube.com/watch?v=3MUfm1uXhfY

    It seems that when you focus on a screen or something else that draws attention in virtual reality, it's easy to 'forget' that the rest of the world is virtual, so the atmosphere becomes more tense and you feel less safe when things goes on around you since you are so immersed in what happens on the screen. It becomes like an extra atmospheric layer that adds to the on-screen experience.

    This kind of VR meta-horror is very interesting I think. For example, how about a game or experience where you are reading a book and the world around you is affected by what you're reading. When reading the diary of a sailor, sounds of the ocean could be heard or water could start leaking in from under the doors and so on.

    I think the power of VR lies in invoking different emotions using virtual environments that surrounds the player, but the "active" part of the game itself doesn't necessary have to be in VR.

    This isn't meant as criticism by the way. Just some general thoughts I had when reading the article.

    ReplyDelete
    Replies
    1. "About VR, horror experiences and presence, one could argue that technically VR is capable of emulating all other mediums, so even if you make a game that isn't about the player experiencing things from a first person perspective you can still create it for VR and unlock immersive features that wouldn't be possible if they weren't in VR."

      I don't think so. Take for example how the early survival horror games were switching to different camera angles for dramatic effect. For this to work, one needs to make the player unable to move freely the camera. In VR though, you can always move around your head to look around. See the problem?

      Delete
    2. Also the approach used in Alone deliberately breaks the fourth wall. The Meta-horror aspect is interesting, but it's unsuitable for more standard horror.

      Delete
    3. What I mean is that if you want to create a game that you intend to be experienced on a flat screen then you can create a flat screen in VR and use that in the same way you would a physical flat screen. Normally this wouldn't really be neccessary since you already have a physical screen, but thinking outside the box you can create interactive VR environments that enhances the experience in different ways.

      I'm not saying all games needs to be VR, but I think it's interesting to explore the concept behind first person perspective apart from the "you are in the game" aspect. There is so much you can do with VR and we have only started to scratch the surface.

      Delete
  4. Great article, Thomas. Just say when you will start playing "Gothic"? It does everything right what modern RPGs doesn't.
    You only need some community patches installed in this order:

    1. PlayerKit 1.08k
    2. PlayerKit 2.8
    3. SystemPack 1.7

    https://forum.worldofplayers.de/forum/threads/1340357-Release-Gothic-½-—-SystemPack-(ENG-DEU)

    And you should know how the controls are working:

    1. Forget the mouse
    2. Actions like talking to NPCs, climbing a ladder or take items etc. you have to press "Ctrl" and one of the Arrow keys (depending on the direction).

    That's all.
    I would like to know what you as a professional game designer would think about this good old RPG classic :-)

    ReplyDelete
  5. Is the analogy correct though? I mean, in the rubber hand illusion, the effect breaks after you hit the fake hand. In a video game, if your character is being attacked, the effect doesn't break until the character is dead.

    Just a thought that popped in my head.

    ReplyDelete
    Replies
    1. This is a really good question! Here are some thoughts

      1) In a game you do get feedback from getting attacked, such as health being lowered, the controlling rumbling, movement being harder for a second and so forth. But in the rubber hand illusion you don't get anything of the sort. That means that games can keep up the illusion, while the rubber hand cannot.

      2) That said, there are cases when being attack = sense of presence is broken. For instance in a horror game when the monster actually attacks you and you realize it is not as bad as you thought it would be. This is a reason why you often want as little player death as possible in a game that heavily relies of presence.

      Delete
  6. I think that there are two separate phenomena here, which are very easily to be perceived as one. The sense of being inside the game's world is not the same as the sense of being the playable character.

    For example in a game where the protagonist is constantly commenting on his surroundings, the player will separate himself from the playable character. But, he will still feel like he is inside the game's world (eg. the player doesn't want the main character to die, and in general he takes the game's world seriously).

    So I think that the sense of presence inside a virtual world isn't dependant to whether the player identifies with the playable character.

    ReplyDelete