Player - Avatar Symbiosis

In a recently released paper, Jeroen D. Stout (creator of Dinner Date) proposes an interesting theory on the relashionship between player and avatar. It is related to the things that have been discussed previous post about immersion, so I felt it was relevant to bring it up. The full paper can be gotten from here. I will summarize the ideas a bit below, but I still suggest all to read the actual paper for more info!

Most modern theorists of the mind agree that it is not single thing, but a collection of processes working in unison. What this means is that there is no exact place where everything comes together, but instead the interaction between many sub-systems give rise to what we call consciousness. The most clear evidence of this is in split brain patients, where the two brain-halves pretty much form two different personalities when unable to communicate.

This image of a self is a not fixed thing though and it is possible to change. When using a tool for a while it often begins to feel like an extension of ourself, thus changing ones body image. We go from being "just me" to be being "me with hammer". When the hammer is put down, we return to the old previous body image of just being "me". I have described an even clearer example of this in a previous post, where a subject perceives a sense of touch as located at a rubber hand. Research have shown that this sort of connection can get quite strong. If one threatens to drop a heavy weight or similar on the artificial body part (eg the rubber hand), then the body reacts just like it would to any actual body part.

What this means for games is that it is theoretically possible for the player form a very strong bond with the avatar, and in a sense become the avatar. I discuss something similar in this blog post. What Jeroen now purposes is that one can go one step further and make the avatar autonomously behave in a way that the players will interpret has their own will. This is what he calls symbiosis. Instead of just extending the body-image, it is the extension of the mind. Quite literally, a high level of symbiosis means that part of your mind will reside in the avatar.

A simple example would be that if player pushes a button, making the avatar jump, players feel as if they did the jumping themselves. I believe that this sort of symbiosis already happens in some games, especially noticeable when the avatar does not directly jump but has some kind of animation first. When the player-avatar symbiosis is strong this sort of animation does not feel like some kind of cut scene, but as a willed action. Symbiosis does not have to be just about simple actions like jumping though, but can be more complex actions, eg. assembling something, and actions that are not even initiated by the player, eg. picking up an object as the player pass by it. If symbiosis is strong then the player should feel that "I did that" and not "the avatar did that" in the previous examples. The big question is now how far we can go with this, and Jeroen suggests some directions on how to research this further.

Having more knowledge on symbiosis would be very useful to make the player feel immersed in games. It can also help solving the problem of inaccurate input. Instead of doing it the Trespasser way and add fine-control for every needed body joint, focus can lie on increasing the symbiosis and thus allowing simply (or even no!) input be seen by players as their own actions. This would make players feel as part of a virtual world without resorting to full-body exo-skeletons or similar for input. Another interesting aspect of exploring this further is that it can perhaps tell us something about our own mind. Using games to dig deeper into subjects like free will and consciousness is something I feel is incredibly exciting.

Tech feature: Light Masking

So just wanted to give a quick info on a brand new feature: light box masks.

When placing lights in some rooms, it is common that light bleeds through walls, and show up in other rooms close by. The obvious way to fix this is to add shadows, but shadows can be pretty expensive (especially for point lights), so it is not often a viable solution. In Amnesia we solved this through careful placement, yet bleeding can be seen in some places.

To fix this I added a new feature that is able to limit the lights range with a box. This way the light can cast light as normal but is cut off before reaching an adjacent area. This pretty much does the job of shadows, but is much cheaper.

It turned out to be pretty simple to implement as well. In the renderer, different geometrical shapes are used to render lights (spheres for point lights and pyramids for spots) which make sure the light only affects needed pixels. To implement the masking, these shapes where simply exchanged for a box and then with some small shader changes it all worked.

Without masking:

With mask:

Bye, bye Pre-Pass lighting

I have an announcement to make.

I am dumping pre-pass lighting.

A couple of weeks ago I started to remaking the renderer from a deferred shader to a pre-pass lighting one. Directly after implementing it, I wrote this post. At first, pre-pass lighting sounded great: faster light rendering and more variation in materials. Having seen that companies such as Crytek and Insomniac Games used it, I thought it would be the next logical step to take.

However, even as implemented it, the problems began. The first one was that specular lighting has to be made through hacks or something that makes it closer to deferred lighting. The next was that implementation become more messy. I suddenly needed to redraw all objects in two separate passes and this made the material and shader code harder to maintain. Normal deferred shading has this nice design where all material info is rendered in one pass to one buffer. But in pre-pass lighting, this spread out and makes more annoying to add new stuff and to update existing.

Still, I stuck to it, because I was sure that the speed and material variety would make up for it. One of the features I was looking forward to was making more interesting decals, with normals and such. Since only the light data is written to an accumulation buffer I thought this would allow me to easily put more effects to the decals. However, I quickly realized that I had been quite foolish and not considered that pretty much every interesting part of a materials is added when lighting it. The surface normals, specular, etc are all baked into the light data. So I ended up doing tricks that I could actually work with normal deferred shading.

So what ended up with was lighting of worse quality, compared deferred shading, and with no more room for special effects. Still, this rendering is much faster right? Well, I did some checks which I collected in this post. It turns out that pre-pass is actually slower unless in very specific situations. None of the improvements I was hoping for turned out to be true.

Still, I stuck to it. I am not sure why, but I guess I did not want to face the truth after having put so much time and effort into it. Going back to the old renderer was something I did not want to consider.

Then last week, as I was starting making undergrowth for the terrain, it suddenly happened. I realized that I had to render the vegetation twice, creating more overdraw and making it a lot more cumbersome to implement. At this point I decided that I should seriously consider going back to the old deferred renderer. What I was most worried about about was that it would exclude us from consoles, but I found out that games like Burnout Paradise used a deferred shader too, and assuring me that consoles would still be possible to do.

This post by Adrian Stone, with an in-depth discussion on the subject, sealed the deal for me and I got to work with going back to deferred shading. I had actually come across Adrian's post before when implemented pre-pass lighting, but never read it carefully. I guess it would not had made me stop then since I wanted to check it out myself, but it is interesting to see how one can convince oneself that something is correct, to the point of avoid contradictory sources. This is a very important lesson to learn and one should always be prepared to reconsider and "kill your darlings".

Right now I have fully implemented the deferred shader again and even updated it a bit too. For one thing, I fixed so the decals support all the feature I had in the pre-pass lighting shader. Since we are aiming for a little higher specs (shader model 3 or 4) for our next game, I took that into account and was able to add some other fun stuff. Examples are colored specular and saving the emission in the g-buffer (allowing to cheaply to a variety of effects).

I am really happy to back to the old renderer and now that I am adding new features things are going a lot smoother. The pre-pass renderer was not all in vain though. I cleaned up the rendering code a lot and it also made me rethink how some features could be added. Last but not least, it also reminded me that I should never get too attached to an idea.