Friday, 7 September 2012

Tech Feature: HDR Lighting


Introduction

Hello my name is Peter and I’m the new graphics and engine programmer. New is not really the correct word since I have been working at Frictional for a year now. During this time I have updated the engine and added a lot of new graphic features. This will be the first of my blog posts descripting the changes that have been made.


HDR


One of the biggest changes to the the new engine is the introduction of HDR (High Dynamic Range) Lighting. This is a technique to increase the detail of the lighting system. The benefit of using HDR is that bright things can be really bright, dark things can be really dark, and details can be seen in both.

In nature there is no limit to how bright something can be. The difference between a 60 W light bulb and sunshine hitting the earth is around 10 000 luminance (cd/m^2). This means that we need a way to store high intensity values while keeping the quality and precision of the dark areas. Thankfully there already exists a method for storing such values - by using floating point numbers.

We use a 16-bit fp RGBA buffer to store our lighting. This gives us enough of a dynamic range without taking up too much memory.


Tone mapping

A normal computer monitor can display 8-bit colors between the value of [0..1]. Because of this the monitor can not display a 16-bit HDR image directly. To be displayed the image will have to be converted to 8-bit while keeping as much of the details as possible.

Tone mapping is the process of converting an image of dynamic range to one with a clamped range between [0..1].

The simplest method for doing this is to use the Reinhard tone mapping algorithm.

vec3 color = x / (x + 1.0)

No matter how high x gets the final value will always stay between [0..1]. The problem with Reinhard is that it desaturates your dark colors and removes contrast. In the brighter parts of the image Reinhard produces great result with its soft highlights.

What you want to have is an algorithm that preserves the saturation of the color in the dark areas and that keeps as much of the contrast as possible.

I ended up choosing an algorithm created by John Hable for Uncharted 2.

vec3 Uncharted2Tonemap(vec3 x)
{
float A = 0.15;
float B = 0.50;
float C = 0.10;
float D = 0.20;
float E = 0.02;
float F = 0.30;

return ((x*(A*x+C*B)+D*E)/(x*(A*x+B)+D*F))-E/F;
}
Left: Long curve, Right: S-curve near dark values



This algorithm is based on a filmic tone mapping algorithm created by Kodak. It keeps the nice highlights from Reinhard while using a slightly S-shaped curve too keep the dark colors saturated.



Exposure

If you have been sitting in a dark room for some time and walk out into the sun, your eyes will not be ready for the bright light and you will have to squint. After a while your eyes will have adjusted to the light and it will not bother you anymore.

Exposure is a way to control the intensity of light that gets passed through the lens. In the eye this is controlled by the size of the pupil and in a camera it is done by selecting for how long the sensor should be active.

There are a few different ways to control exposure in a game. It can be controlled automatically by storing the average luminance over a few frames and calculating a exposure from that value.
We choose to go with a much simpler method that lets the artists control the exposure by dividing the level into areas that have different exposures. So in a dark area the exposure can be increased and in an outdoor area the exposure can be decreased.

vec3 color = Uncharted2Tonemap(scene_color * exposure)


White Point

A white point is used to increase the contrast of the image. This is the value that is selected to be the brightest any pixel can be. Pixels brighter than the white point will be clamped to 1.0.

vec3 color = Uncharted2Tonemap(scene_color * exposure) / Uncharted2Tonemap(white_point)

HDR Bloom

When a pixel is brighter than the white point it can be used to generate an additional post effect called HDR Bloom. Pixels that get too bright should start bleeding over to other nearby pixels, producing a halo around them. It is a subtle effect that adds realism to the image.

To solve this I use a few render passes. First a bright pass is applied to the image which removes all pixels that are below the white point and scales down all the pixels that are above it. The result is then blurred to create the halos. The blurred image is then added to the original image. The HDR Bloom effect must be performed before the tone mapping.


Final Thoughts

I would say that HDRL and filmic tone mapping is the most important part of any rendering pipeline. It greatly increases the quality of lighting and will make your game look much more realistic.

But HDR and tone mapping is all for nothing if your calculations are not done in linear color space. My next tech feature will focus on gamma correction and the linear color space.

References


Monday, 20 August 2012

The Self, Presence and Storytelling

At GDC last week I gave a talk called "The Self, Presence and StoryTelling". There will be a version of it up on the GDC Vault in 2 - 3 weeks I think and hopefully it will be free like last years' talk. Before that comes up I will put up a pdf version of the talk containing a bit more information (something I promised at the end of my talk). You can get hold of that here:

Download "The Self, Presence and Story-telling"

Get the mobi version here (thanks to Tomasz Rozanski for creating this).

The paper is basically a summary about a lot of the stuff that has been written on this blog. It is an attempt to define a new design approach that makes it easier to make games that can deal with a wider range of themes and let you play all the way through. I would be very interesting any feedback you got as I would like to keep the document updated and revised if there are any new insights or any info in it that is no longer true.

EDIT: Proof-read the paper a bit and made a little bit more readable.
EDIT2: Mobi version now available.


Wednesday, 4 July 2012

Horror Tip: Slender

Slender is a short horror game really similar to Hide. You walk (or run) around in a monotone environment looking for notes, while there is a scary monster hunting you. Like Hide it is interesting to see how a simple setup can create a really spooky experience. The graphics are nothing special, the music is simple moody droning and the sound effects are of no great quality. Still, taking all together and put in an interactive space, it gets a lot more immersion than what you would expect. I think this is a great testament to the power of interaction to create a strong sense of presence in way that is much harder to accomplish in any other media.

The game only lasts for few minutes, which is good because there is so little to do that any longer would most certainly break the spell. This makes me wonder what would have to be needed to make an experience like that last for one to two hours. Adding too much would likely decrease the sense of dread, so one would have to be careful in lengthening it. Hope someone attempts to do this.

Some other stuff worth noting:

- The game hides the mechanics that govern how the monster hunts you down and what makes you eventually get killed. I think this was a good move as you are free to make up for yourself what happened. (Bound to only work on the first play-through though, but that does not have to be a bad thing).

- A lot of the creepiness is induced through sensory deprivation. You mostly only see the same vague shapes over and over and it is not long until you start to imagine things.

- It is interesting how effective the tunnel vision created by the flashlight is. There is just something about having these large chunks of your vision pitch black that make you unnerved.

Recommend you all to give it a go!

Downloads:
Mac  (alternative)
(there is no official website as far as I am aware)


Sunday, 24 June 2012

Thoughts of The Walking Dead (ep1)




Introduction
I played through the first episode of The Walking Dead recently and few stuff popped up that I thought was worth discussing. For those of you who do not know what The Walking Dead is, it is a horror adventure game based upon a comic book (which is now also tv-series) featuring Heavy Rain inspired gameplay. It is developed by Tell Tale (makers of the Sam and Max reboot, etc) and is released on an episodic basis. I was unsure if Tell Tale could deliver a game with this kind of atmosphere, but having played I have to say that it is quite successful. The first episode is not a master piece by any means, but contain a few things worth bringing up.

Graphics
The comic-book inspired art direction combined with so-so animations does not look all that inviting and immersive. But when you play the game, it works very well and does the job. This even though the drama of the game is mostly about close-up dialog and relationships. I think this is a very important lesson about not having to have photo realistic graphics even though the game is meant to focus on human emotions. I have a hard time saying that Heavy Rain, which as a lot more gloss on its visual, managed to elicit any more emotions from me. However, watching trailers, Heavy Rain seems a lot better in this aspect and I thought Walking Dead looked downright horrible at times. But in-game it turned out not be really matter. This is also lesson in taking care of how you present the game in trailers and such, and make sure that the feelings you get from actually playing the game comes across.

Choices
Just like in Heavy Rain, a big feature is to make hard decisions throughout the game. You must choose who to save, whether to lie or not, etc. Most of these are made using timed dialog choices, where you only have a short time to decide what to do. On paper I think it sounds okay, but I just do not like how it feels when actually playing. There is just something that bothers me in knowing that all of these choices are prefabricated and that I the choice I did not make might have been better. And it does not really matter that the choices do not affect the game mechanically (eg like in Mass Effect where bad choice might mean less gain), there is just something holding me back from playing along with it.
I think a big problem is that it is way too obvious that you are actually making a choice. Supporting this hypothesis is that the game by default gives a pop-up hint of the consequences of a choice (eg that a character trust you less), and removing this makes it a lot better (but still not good enough). A few choices are made in a sort of "Virtua Cop"-like manner where you have to point a cross-hair over a target and then choose an action. It is not always clear that these are actual choices, in part because it is much more analog (not just choosing from a list of options) and partly because it is less clear that you can only can choose a ONE of the presented alternatives. These sequences did not bother me at all as much as the dialog options.

Pixel hunting
While the game does a lot to remove annoying adventure game features and make a smoother experience, it also falls back upon some annoying aspects of the genre. The most obvious is that of pixel hunting. There are only really two major adventure-game like puzzles in the game and both of these has the player searching for one or several objects, non-obviously located, in the environment. The worst of these is a remote control that is hidden in  drawer which is not accessible until you have done certain unrelated actions. This caused me to wander aimlessly in the scene for far too long.
I think it is really important to try and minimize this sort of things as it makes you go about exploring the scenes in a very unnatural way. Best is if the player can sort a puzzle out without having to search every nook and cranny for items.

Repetition
Once you get caught up wandering without any real goal, like I mentioned above, you start doing the same things over and over. This is when you start noticing the slim output of lines that characters have. When asked the same question, they just repeat the same line they gave before. This is especially jarring when it amounts to longer exchange between the protagonist and a supporting character. Repeating canned responses like this really breaks the sense of immersion for me. I just simply cannot role-play when I am subjected to this sort of repetition.
I think the game should have removed hot-spots, given leading answers from characters (especially the protagonist), etc. Anything to push me in the direction and to keep up the make-belief that it is real characters inhabiting the virtual world. Now they just come of as cardboard signs the moment you start wandering off the intended path.

End Notes

I'd say that The Walking Dead is worth playing and being just over 2 hours of gameplay in the first episode it just not that much wasted time in case you end up hating it. While the game did not blow me away, I was pleasantly surprised and am intrigued to see how the next episode will turn out.

If anyone else has played the game, I would love to hear your thoughts on the following.:

- What did you think of the graphics? Was the discrepancy between trailer and in-game also large?
- What did you like the choices? Did it feel like you could roleplay or was it hard to put as side that there was a better choice?
- How did you feel about the repeated lines? Not bothering at all, or a nail in eye each time they were encounter?


Sunday, 17 June 2012

Thoughts on Lone Survivor


Introduction
I just a finished Lone Survivor and because there is so much interesting stuff going on in it, I thought it was worth to write a blog post about it. At first glance Lone Survivor might look like some kind of 2D Silent Hill ripoff*, but there is a lot more to it than what is perceived at first sight.
In summary, Lone Survivor is basically about surviving in a world where most people been turned into monsters. You play as a guy that have been holed up alone in an apartment for quite some time and who is no longer sure what is real and what is not. It is viewed from a 2D side-scrolling perspective with pixel the size of fists, forcing you to imagine what most objects really look like. The story and atmosphere is very Lynchian in tone and filled all kinds of wonderful strangeness. Most of the gameplay is about conserving various resources, exploring the environment (with a map straight out of Silent Hill) and shooting / avoiding bad guys. I thought the first half of the game was really good, but later on it becomes a bit repetitive and fails to be as engaging as it was starting out. The game is truly a diamond in the rough though and implements some truly innovative features that are well worth discussing.


One large world
A striking feature of the game is that you can always go back to the room you started (and most other places you visit). In fact it is vital for your survival in the game. You need to go back to save the game, cook food,  check radio messages, etc. This creates a constant need to come back to your home base and it helps build up a solid sense of place. Also, many locations have characters and objects that reward you when revisited, adding to the feel of a persistent and real world.
This is not something new for videogames (many rpgs and some adventure games feature similar mechanics), but I cannot name a single horror game that use it. By allowing the protagonist to always have his own place to go back to, it increase the survival aspect of the game immensely. The simple act of forcing you to cook your food at home greatly enhances this feeling, and is so much more immersive eating what you find on the spot. Mechanics like that also make the environment seem more real and objects like stove pop out from the background to become something with a purpose.


Backtracking
There is a bad side to the open world design though, and that is backtracking. I have covered this subject earlier, and Lone Survivor brings out some new aspects to problem. The intial reaction to backtracking is that it is an annoyance to the player, but on some further consideration it is clear that is quite important. As mentioned in the previous paragraph, having the player revisit areas makes the game's world feel much more alive. Therefore, backtracking in Lone Survivor is a crucial element, and not something that one should simply try and minimize. However, while the backtracking helps build the world of the game, it backfires very easily. Traversing the same section of the map easily becomes repetitive, and you as a player quickly try and find ways to get it over with faster. When you get into this mind set, you are pulled out of the world, thus counteracting the building of presence it should be doing in the first place!
For instance, Lone Survivor feature a system to travel instantly back and forth between the apartment and last place you visited by looking into a mirror. This is a neat way of fixing quick traversal as it works with the story of the game, keeping the player's sense of presence. However, once the need arise to start going back and forth a lot, you stop roleplaying along with the mirror gazing, and it decays into an abstract tool for accessing abstract mechanics of the game such as saving. The same happens when trying to sneak past an enemy multiple times or even just walking the same path over and over. The travels become an monotonous chore and the feeling of immersion fades.
So how can this be fixed? First of all, a solution is not to try and remove the backtracking. As mentioned earlier, this is a crucial part of the experience. What should be focused on instead is to make the backtracking more varied and make sure the player does not feel inclined to (or is discouraged from) doing it too many times in a row. The experience should be designed in such a way that even though the player is going back a familiar path, the act of traveling should be in itself engaging, and not just a means to an end to use some kind of mechanic. This does not mean that it must be fun, it can actually be boring, but just that it is presented in such a manner that the player sees the journey as an important aspect of the experience. This is not an easy thing to accomplish especially in an open world design, where the designer has much less control over the flow of the game. But if considered from the start I do not see any problem with achieving it, and to have other mechanics work with it. Which brings me to next topic.


Combat
Combat is a quite a big part in Lone Survivor and comes with a bunch of problems. I think the main issue is that it overexposures you to the monsters of the game. Because forcing the player to deal with enemy encounter is such a big part of the game, one very quickly becomes used to the look of the creatures. This is quite a shame, because the ambiguous pixel graphics are quite good at depicting them in a moody and disturbing way. But after staring at a monster for the tenth time in a row most of the effect is lost. I think the encounters should have been briefer and more sparse.
A problem that Lone Survivor does avoid is that of combat detracting from the focus of the experience. Normally in horror games, having combat means that the player looks forward to monster encounters, since fighting them is so gratifying. Not so here. You only have a single gun as a weapon and shooting is quite clumsy, but simplistic and mechanically stable enough for it to be easily used as intended. It is not something that you have very fun doing though and mostly you do it simply because it is the easiest way of bypassing a threat (which works great with the story).
There are more problems to the combat though, and all are connected to the backtracking. First off, the game features a standard "die and retry" death mechanic that I do not think fits very well. Whenever you die, you almost always have to go back the exact same path that you did before and repeat the same sequence of actions. This essentially requires the player to do even more (non-engaging) backtracking, something that is already lessening immersion for the player. Given the strange story and gameplay based on resource management, I think death could have been implemented in numerous other ways without resorting to the standard trial and error way. For instance the player restart from the scene of death, but lose precious resources, gain some kind of injury or just be transported back to bed with any progress intact.
Having to sneak past enemies is also part of the game's combat mechanic, and at first it works quite well. However, because of the backtracking it soon becomes necessary to sneak past the enemy over and over again. This repetition not only makes the whole gameplay boring, it also pulls the player out of the immersive atmosphere and sneaking becomes an abstract mechanistic obstacle. A big part of the game is to choose whether to use lethal force or not, but choosing the stealth option is far less appealing, not only because it is quickly becomes a chore, but because it pulls you out of the experience.
In Lone Survivor combat is quite important for both story and gameplay reasons, so I do not think simply removing it would be an option. Instead, I think that having more dynamic enemy encounters could make most problems go away. This way the player does not always have to use repeat a section of stealth when backtracking. Monster exposure could also be better controlled this way. By keeping track of how the player is doing, the game could spawn enemies as needed.


Dynamic systems
What was I liked the best of about Lone Survivor was the dynamic nature of the game. The player consantly needs to keep track of health/hunger, sanity, ammunition along with a few other things. These are not just abstract meters the player need to keep in balance, but things that affect how the game plays out. For instance, the game triggers some scenes not based on where the player is located, but on how the current stats are. So for instance, running out of ammunition has a special scene associated with it. The protagonist also constantly comments on his current state in various ways which I thought really helped in building atmosphere.
Mot having strictly laid out events makes Lone Survivor feels so much more alive. I only wish that the game would have gone full out with it. The path the player needs to take to the game is still quite static, and have little dynamic elements to it. Because the story is so strange and fragmented the game could easily have had a random order of the important scenes. Instead of forcing a strict order on puzzles, items and major events, these could have depended on how the way the player played the game. This could also solve some backtracking problems and make sure the player found certain things in a specific order no matter how they chose to go. Solutions to puzzles could also depend on what the player currently had available. This kind of of design does not have to require any sophisticated algorithms either, but could use simple means and still be very much designed (as opposed to completely random).
Having a world that shapes according to the world is something I think can be very useful for story focused games. While Lone Survivor does not fully implement something like this, the element that it does have is a good indicator of what could be done.


Importance of text
Another thing I really liked in game was the great usage of text. In our age of crisp graphics and high quality voice overs, I think good old text is much underused. Because of the low fidelity graphics in Lone Survivor, writing plays a crucial part in giving feedback to the player. That does not mean that text does all of the work though, rather it is complemented by both sound and graphics, forming a nice synthesis. Here are some examples:

  • When using the radio, a brief sound sample of radio static is heard as the text of the transmission is displayed. This simple sound effect really sets the mood for how the text should be read and greatly adds to the experience. Another great thing about using text in these instances is that it handles repetition much better. Reading the same text a few times is not nearly as repetitive as hearing the same voice-over repeated. 
  • It is through text that most of the feelings of the protagonist are shown. Sometimes this is accompanied by audio/visuals, but mostly it is just presented as pure text. This is yet another great application for text, and while it can be a bit annoying to have to press a button for it to go away, overall it really helps to make the character's mood come across. Also, like with the radio message, repetition is a lot less problematic what a voice-over would be.
  • One point int game, the player is standing on balcony with nothing more than a house front seen in the background. But if you interact at a certain spot, the protagonist describes what he sees looking out over the city. I thought this worked nicely and really fitted with the protagonists situation of being locked inside an apartment complex. I actually think having had some image shown of the cityscape would have been a lot less effective, but when you have a lot of resources at your disposal  it is hard to forget that less might sometimes be more.
This is just a few samples of the great usage of text in Lone survivor. Of course it is far from the only game  to do this, but I think it is a fine example of its potential.
Text is much easier to autogenerate and to transform in various ways, lending it to dynamic systems a lot better than voices. It also leaves more to the imagination and can work great when combined with sound and graphics. It is important not to forget about and consider having it as an integral part of the game. As seen in Lone Survivor much of the text based stuff works because so integrated into the experience.


Multiple endings
The last thing I want to discuss, is the multiple endings of Lone Survivor. Not the actual story content of the endings, but of how the overall structure is designed. Basically, the player gets one of three endings based on how you have performed during the game. In doing so, it takes a lot of different variables into account (which are actually shown at the end of the game). So the game collects very personal data from you, and yet there are only three (not very personal) prefabricated endings to be seen.
When finishing a game with multiple endings I always get a feeling of having been cheated. A game with a single ending, even if it is not that good, almost always feel better than a game where I know there are more endings to be seen. Suddenly the game sets up a sort of competitive goal that I was not asking for: "see if you can find all endings" or "now try and get the proper ending" and I get the feeling that I am missing out on the full experience or that what I had was not the proper one.
I think that a better way of doing it would have been to have a dynamic final sequence tailored based upon the choices that you have. This would have made the ending much more personal and it would not be possible to simply look it up on YouTube. But that is just in theory of course, it is hard to say how it would work in practice.
Heavy rain does similar thing to do this, and have a certain number of smaller clips that are chosen based upon certain options during the play-through. I did not find this very satisfying either, and the problem here is that it is very easy to look up each of the individual clips. So to avoid that, the final scenes must be quite dynamic and details based on how the game was played.
Actually, I think the best choice is to not have an ending sequence at all and have the game play all the way to the credits. It is actually a bit strange that having played a game to the very end, we are rewarded with a non-interactive cut-scene. Pretty much every story based game works like this. But if the interaction continued all to the end, I think you could have a lot of differences and it would leave one feeling a lot less cheated.

A final note on this: In Amnesia we tried to have multiple endings, with the idea that each ending should fit the playing style of the player. So a player that played very aggressively would get a that kind of ending and so on. However we only collected data for very few (two or so actually) events, making any guess on the player style of play close to random. In the end, the endings (or the final sequence for that manner) were not received very well. Even though having multiple endings often sounds good in theory, I do not think we will be used it again, because it has a such a high risk of backfiring.


End notes
Lone Survivor is in no way a perfect game, but it is filled with lots of great and original ideas. If you are the slightest interested in horror games you should really give it a go. It is easily one of the most original and fresh horror games I have played for a couple of years.


*The designer actually did make a so called demake of SH2 called "Soundless Mountain II"


Thursday, 31 May 2012

Humble Indie Bundle V

We at Frictional Games are yet again part of a Humble Indie Bundle and this time it is quite the pack. Our contribution is Amnesia: The Dark Descent and it is joined by Limbo, Bastion, Psychonauts and Superbrothers: Sword and Sworcery. This pack of games is offered to a price of your own choosing, and part of the revenue, you chose how much, will go to charity. This video will explain more:





I think the all of the games that Amnesia share the bundle with are quite special, and are all really worth playing if you haven't yet.

Sword and Sworcery was my top from last year and there is a lot to learn from playing it. Foremost, it has this magical blend of music, gameplay and graphics that really shows of the strengths of the medium. It is clear that this is not a game that has been created to be some kind of competition, but to create a special, mystical experience. This is something we need more of.

Limbo manages to create a very tense atmosphere, but more importantly it how to use a very simplistic control scheme to create great variety. Nowadays, game often require you to learn tons of buttons, but in Limbo you understand all required controls after a few minutes of play. And yet the game manages to give you varied activities from start to finish. It also never takes away control from the player, allowing for a highly interactive narrative experience.

Bastion is excellent in the way it builds up its fiction. The constant voice over gives meaning to your actions and also let you see the world in a quite different way than you would have otherwise. As the flowing narrative unfolds, enemies stop being cannon fodder and instead feel like proper denizens of the world. By small means a  lot of depth is added to the game, and I highly recommend it in a lesson on how to do story exposition.

Finally, Psychonauts paint up a vibrant world filled with quirky, interesting and yet human characters. What I liked best about it is how the core theme of the game, to enter people's minds, really merges both gameplay and storytelling. Actually playing the game does not simply servers as some filler between plot points, but actually adds depth to the narrative.

Also, note that the sound tracks of all games comes along with the bundle! I mean, the awesome sound track from Sword &  Sworcery is enough reason to buy the bundle alone. The music from Bastion is not bad either...