People of Frictional: Rasmus Gunnarsson

Fifth post in the series. You can read the other posts here.

Who am I?

Hello! My name Rasmus Gunnarsson and I started working for Frictional Games as a freelance artist back in 2010 to help out with the final stages of Amnesia's development. Later, I became an intern, and that landed me full-time employment as a concept artist / 3D artist / level artist / whatever’s needed artist.                                   

I work from an elevated desk where I stand all day. 

Background

I've always drawn a lot since I was a child. I never took it that seriously, which is something I regret, but then again perhaps if I had I wouldn't have spent most of my free time playing games. I used to buy those magazines that came with demo disks, and then use all my savings to either get a cool game from the budget bin or save it up for a larger release. I played everything from Megaman to Silent Hill to Counter Strike to Fallout to Starcraft to Monkey Island.

The time comes when you look at your life and where you are going; soon you'll be out of school and in need of a job. I asked myself what I wanted to do. If I was going to have to work for most of my life, I’d love it to be something that I cared about and enjoyed. This started a long and hard journey to where I am today.

I started out with all the resources I could find on the internet and practised all day instead of just playing - I was drawing, painting, reading tutorials and watching educational materials. When I did play games or watch movies, I tried to be more observant of things art-related. But just working on my own wasn't really going to cut it; even when you improve with lots of practice, you won't have any real practical tests of your skill and it’s easy to fall into patterns that feel comfortable.

I enrolled in a school for Industrial Design in my home town. I'd read that many successful concept artists had started out there, and it was the best way I could find of getting practical experience - or so I thought at the time. It turned out that it wasn’t a good idea at all. All it offered were some practical opportunities for that specific field, and no real meaty general design courses regarding aesthetics and so on -  we were left to figure that out on our own. As I was already doing that myself, I just quit, and looked for a course that might give me a chance for real practical experience.

This search took me to The Game Assembly, a school with close links to the games industry. So not only were there many more practical opportunities to be had, but there was also a chance to meet people in the industry. I didn't expect to be spoon-fed anything and still focused on the work and practice I did on my own.

I got to work on many small games during the course, but the big turning point was when I just got lucky. One of the guys I befriended happened to be doing ex for Frictional. Due to the busy schedule at the school, he had trouble keeping up with everything he needed to do and asked me to help out. While I was quite busy myself, I wouldn't pass up on this opportunity.

Painting made in my spare time.

A personal painting much inspired by Beksinski.

When I started working on Amnesia I had no idea what Frictional had done in the past, so of course I decided to check it out. After playing Penumbra, I realised quite how good Frictional were, and felt under pressure to perform well. I really enjoyed Penumbra and thought it did a lot of new things with old gaming ideas that I loved in a way that got me really excited. During school I hadn't wanted to end up working on AAA games as I felt they didn't manage to create new things that could rival many of the old great classic games. But after playing Penumbra I knew that I'd do anything to keep working with Frictional.

Concept art for the Justine suitor enemy.

After completing the freelance work for Amnesia I got an internship at Frictional where I, among other things, worked on the Justine expansion. After the internship I got full employment.

What do I do?

I create concept art and sketches for objects and environments. I also create 3D models and work on levels like a normal 3D artist when that takes a higher priority.

Sometimes an area might need a redesign, and doing a paintover is the quickest way of evaluating what needs to be changed and to set a clear goal; at other times we might need to fix a bunch of objects both technically and aesthetically.

I really love being able to have a finger in everything, making sure that the concepts are translated correctly, and then having the freedom to explore and tweak things, making every area or object as good as it can be instead of simply contributing art and hoping that it becomes something good in the end.

I use Photoshop and a Wacom for drawing and Modo for creating 3D assets.

My normal workflow is to interpret the initial design doc by Thomas (the Lead Designer), using his description and simple layout to try and capture the right mood and give the artists building the level something to go on.

Concepts for the teaser.

Above is an example from the teaser video we recently released. We chose "hero shots" of the environment where there is something special going on, so you get the most mileage out of a single sketch.

Another thing that happens in the process of designing an area is that the initial sketches tend to act as a visual playtest. Something that might've been problematic to communicate visually can easily slip through when you are only designing something in your head or on paper. In the end, nothing is final and will always evolve in each stage it goes through.

New Screenshot and Intercepted Radio Message

Click to view large version.

Transcription of radio message intercepted at XX/XX/XXX.

UNKNOWN: Look, this is the short version. -- [CORRUPT DATA] -- gone rogue. We're not sure what it’s up to, but it looks like it’s messing with the help. Are you hurt in any way, are you guys okay?

AMY: -- [CORRUPT DATA] -- are turning on us! We need to get out of here.

UNKNOWN: Hold on, don't panic. Carl, tell Amy to calm down. You guys can’t have much left to do before evac.

CARL: Calm down, Amy.

AMY: You calm down!

An alarm is heard in UNKNOWN’s mic.

UNKNOWN: Shit. I got to go. Stay the course, ok? Stay the --

Static outburst after which the signal is lost.

Tech Feature: SSAO and Temporal Blur

Screen space ambient occlusion (SSAO) is the standard solution for approximating ambient occlusion in video games. Ambient occlusion is used to represent how exposed each point is to the indirect lighting from the scene. Direct lighting is light emitted from a light source, such as a lamp or a fire. The direct light then illuminates objects in the scene. These illuminated objects make up the indirect lighting. Making each object in the scene cast indirect lighting is very expensive. Ambient occlusion is a way to approximate this by using a light source with constant color and information from nearby geometry to determine how dark a part of an object should be. The idea behind SSAO is to get geometry information from the depth buffer.

There are many publicised algorithms for high quality SSAO. This tech feature will instead focus on improvements that can be made after the SSAO has been generated.

SSAO Algorithm

SOMA uses a fast and straightforward algorithm for generating medium frequency AO. The algorithm runs at half resolution which greatly increases the performance. Running at half resolution doesn’t reduce the quality by much, since the final result is blurred.

For each pixel on the screen, the shader calculates the position of the pixel in view space and then compares that position with the view space position of nearby pixels. How occluded the pixel gets is based on how close the points are to each other and if the nearby point is in front of the surface normal. The occlusion for each nearby pixel is then added together for the final result. 

SOMA uses a radius of 1.5m to look for nearby points that might occlude. Sampling points that are outside of the 1.5m range is a waste of resources, since they will not contribute to the AO. Our algorithm samples 16 points in a growing circle around the main pixel. The size of the circle is determined by how close the main pixel is to the camera and how large the search radius is. For pixels that are far away from the camera, a radius of just a few pixels can be used. The closer the point gets to the camera the more the circle grows - it can grow up to half a screen. Using only 16 samples to select from half a screen of pixels results in a grainy result that flickers when the camera is moving.

Grainy result from the SSAO algorithm

Bilateral Blur

Blurring can be used to remove the grainy look of the SSAO. Blur combines the value of a large number of neighboring pixels. The further away a neighboring pixel is, the less the impact it will have on the final result. Blur is run in two passes, first in the horizontal direction and then in the vertical direction.

The issue with blurring SSAO this way quickly becomes apparent. AO from different geometry leaks between boundaries causing a bright halo around objects. Bilateral weighting can be used to fix the leaks between objects. It works by comparing the depth of the main pixel to the depth of the neighboring pixel. If the distance between the depth of the main and the neighbor is outside of a limit the pixel will be skipped. In SOMA this limit is set to 2cm.

To get good-looking blur the number of neighboring pixels to sample needs to be large. Getting rid of the grainy artifacts requires over 17x17 pixels to be sampled at full resolution.

Temporal Filtering 

Temporal Filtering is a method for reducing the flickering caused by the low number of samples. The result from the previous frame is blended with the current frame to create smooth transitions. Blending the images directly would lead to a motion-blur-like effect. Temporal Filtering removes the motion blur effect by reverse reprojecting the view space position of a pixel to the view space position it had the previous frame and then using that to sample the result. The SSAO algorithm runs on screen space data but AO is applied on world geometry. An object that is visible in one frame may not be seen in the next frame, either because it has moved or because the view has been blocked by another object. When this happens the result from the previous frame has to be discarded. The distance between the points in world space determines how much of the result from the previous frame should be used.

Explanation of Reverse Reprojection used in Frostbite 2 [2]

Temporal Filtering introduces a new artifact. When dynamic objects move close to static objects they leave a trail of AO behind. Frostbite 2’s implementation of Temporal Filtering solves this by disabling the Temporal Filter for stable surfaces that don’t get flickering artifacts. I found another way to remove the trailing while keeping Temporal Filter for all pixels.

Shows the trailing effect that happens when a dynamic object is moved. The Temporal Blur algorithm is then applied and most of the trailing is removed.

Temporal Blur 

(A) Implementation of Temporal Filtered SSAO (B) Temporal Blur implementation 

I came up with a new way to use Temporal Filtering when trying to remove the trailing artifacts. By combining two passes of cheap blur with Temporal Filtering all flickering and grainy artifacts can be removed without leaving any trailing. 

When the SSAO has been rendered, a cheap 5x5 bilateral blur pass is run on the result. Then the blurred result from the previous frame is applied using Temporal Filtering. A 5x5 bilateral blur is then applied to the image. In addition to using geometry data to calculate the blending amount for the Temporal Filtering the difference in SSAO between the frames is used, removing all trailing artifacts. 

Applying a blur before and after the Temporal Filtering and using the blurred image from the previous frame results in a very smooth image that becomes more blurred for each frame, it also removes any flickering. Even a 5x5 blur will cause the resulting image to look as smooth as a 64x64 blur after a few frames.

Because the image gets so smooth the upsampling can be moved to after the blur. This leads to Temporal Blur being faster, since running four 5x5 blur passes in half resolution is faster than running two 17x17 passes in full resolution. 

Upsampling

All of the previous steps are performed in half resolution. To get the final result it has to be scaled up to full resolution. Stretching the half resolution image to twice its size will not look good. Near the edges of geometry there will be visible bleeding; non-occluded objects will have a bright pixel halo around them. This can be solved using the same idea as the bilateral blurring. Normal linear filtering is combined with a weight calculated by comparing the distance in depth between the main pixel and the depth value of the four closest half resolution pixels.

Summary 

Combining SSAO with the Temporal Blur algorithm produces high quality results for a large search radius at a low cost. The total cost of the algoritm is 1.1ms (1920x1080 AMD 5870). This is more than twice as fast as a normal SSAO implementation.

SOMA uses high frequency AO baked into the diffuse texture in addition to the medium frequency AO generated by the SSAO.

Temporal Blur could be used to improve many other post effects that need to produce smooth-looking results.

Ambient Occlusion is only one part of the rendering pipeline, and it should be combined with other lighting techniques to give the final look.

References

1. http://gfx.cs.princeton.edu/pubs/Nehab_2007_ARS/NehEtAl07.pdf 
2. http://dice.se/wp-content/uploads/GDC12_Stable_SSAO_In_BF3_With_STF.pdf 

Code Appendix

 // SSAO Main loop

//Scale the radius based on how close to the camera it is

 float fStepSize = afStepSizeMax * afRadius / vPos.z;
 float fStepSizePart = 0.5 * fStepSize / ((2 + 16.0));    

 for(float d = 0.0; d < 16.0; d+=4.0)
 {
        //////////////
        // Sample four points at the same time
        vec4 vOffset = (d + vec4(2, 3, 4, 5))* fStepSizePart;
        

        //////////////////////
        // Rotate the samples
        vec2 vUV1 = mtxRot * vUV0;
        vUV0 = mtxRot * vUV1;

        vec3 vDelta0 = GetViewPosition(gl_FragCoord.xy + vUV1 * vOffset.x) - vPos;
        vec3 vDelta1 = GetViewPosition(gl_FragCoord.xy - vUV1 * vOffset.y) - vPos;
        vec3 vDelta2 = GetViewPosition(gl_FragCoord.xy + vUV0 * vOffset.z) - vPos;
        vec3 vDelta3 = GetViewPosition(gl_FragCoord.xy - vUV0 * vOffset.w) - vPos;

        vec4 vDistanceSqr = vec4(dot(vDelta0, vDelta0),
                                 dot(vDelta1, vDelta1),
                                 dot(vDelta2, vDelta2),
                                 dot(vDelta3, vDelta3));

        vec4 vInvertedLength = inversesqrt(vDistanceSqr);

        vec4 vFalloff = vec4(1.0) + vDistanceSqr * vInvertedLength * fNegInvRadius;

        vec4 vAngle = vec4(dot(vNormal, vDelta0),
                            dot(vNormal, vDelta1),
                            dot(vNormal, vDelta2),
                            dot(vNormal, vDelta3)) * vInvertedLength;


        ////////////////////
        // Calculates the sum based on the angle to the normal and distance from point
        fAO += dot(max(vec4(0.0), vAngle), max(vec4(0.0), vFalloff));

}

//////////////////////////////////
// Get the final AO by multiplying by number of samples

fAO = max(0, 1.0 - fAO / 16.0);

------------------------------------------------------------------------------ 

// Upsample Code

 
vec2 vClosest = floor(gl_FragCoord.xy / 2.0);
vec2 vBilinearWeight = vec2(1.0) - fract(gl_FragCoord.xy / 2.0);

float fTotalAO = 0.0;
float fTotalWeight = 0.0;

for(float x = 0.0; x < 2.0; ++x)
for(float y = 0.0; y < 2.0; ++y)
{
       // Sample depth (stored in meters) and AO for the half resolution 
       float fSampleDepth = textureRect(aHalfResDepth, vClosest + vec2(x,y));
       float fSampleAO = textureRect(aHalfResAO, vClosest + vec2(x,y));

       // Calculate bilinear weight
       float fBilinearWeight = (x-vBilinearWeight .x) * (y-vBilinearWeight .y);
       // Calculate upsample weight based on how close the depth is to the main depth
       float fUpsampleWeight = max(0.00001, 0.1 - abs(fSampleDepth – fMainDepth)) * 30.0;

       // Apply weight and add to total sum
       fTotalAO += (fBilinearWeight + fUpsampleWeight) * fSampleAO;
       fTotalWeight += (fBilinearWeight + fUpsampleWeight);
}

// Divide by total sum to get final AO
float fAO = fTotalAO / fTotalWeight;

-------------------------------------------------------------------------------------

// Temporal Blur Code

//////////////////
// Get current frame depth and AO
vec2 vScreenPos = floor(gl_FragCoord.xy) + vec2(0.5);
float fAO = textureRect(aHalfResAO, vScreenPos.xy);

float fMainDepth = textureRect(aHalfResDepth, vScreenPos.xy);   

//////////////////
// Convert to view space position

vec3 vPos = ScreenCoordToViewPos(vScreenPos, fMainDepth);

/////////////////////////
// Convert the current view position to the view position it 

// would represent the last frame and get the screen coords
vPos = (a_mtxPrevFrameView * (a_mtxViewInv * vec4(vPos, 1.0))).xyz;

vec2 vTemporalCoords = ViewPosToScreenCoord(vPos);
       
//////////////
// Get the AO from the last frame
float fPrevFrameAO = textureRect(aPrevFrameAO, vTemporalCoords.xy);
float fPrevFrameDepth = textureRect(aPrevFrameDepth, vTemporalCoords.xy);

/////////////////
// Get to view space position of temporal coords
vec3 vTemporalPos = ScreenCoordToViewPos(vTemporalCoords.xy, fPrevFrameDepth);
       
///////
// Get weight based on distance to last frame position (removes ghosting artifact)
float fWeight = distance(vTemporalPos, vPos) * 9.0;

////////////////////////////////

// And weight based on how different the amount of AO is (removes trailing artifact)

// Only works if both fAO and fPrevFrameAO is blurred
fWeight += abs(fPrevFrameAO - fAO ) * 5.0;

////////////////
// Clamp to make sure atleast 1.0 / FPS of a frame is blended
fWeight = clamp(fWeight, afFrameTime, 1.0);       

fAO = mix(fPrevFrameAO , fAO , fWeight);
   

------------------------------------------------------------------------------

People of Frictional: Marcus Johansson

This is the fourth part in a series. See all of them here.

Who am I?
Hi! My name is Marcus Johansson and I'm a 3d artist who's been with Frictional Games since January 2009. When I think of that date it doesn't sound that long, but then I realize it's actually 5 years ago.
I started as an intern for 6 months, and after that went straight on to full time.

For most of this 5 year span I've had to work in a corner of the living room, but I finally moved into a 3-room apartment a year ago. Having a dedicated office really helps at home as my other half doesn't share the same working hours as me.

My workspace. (Yes, I made that ugly foot-stand from a old table piece.)

Background

Meetup fun on big toys!

I've always been curious about how things work. At a young age seeing my step dad fixing anything from broken electronics to cars really got me hooked on figuring things out and trying to fix and create stuff myself.

I didn't have a gaming system or computer at home when I was small, but I played NES/SNES at my friend's house every chance I got. It wasn't until I was 12 that I saved up to buy my first gaming system, a Playstation that I spent most of my time playing FF VII and THPS2 on.

At 15, after finding odd jobs here and there and managing to save up enough cash, I bought my first computer, managing to build it from parts without breaking anything. Although not a single physical part is in my current computer it still feels like it's the same one, as I've made incremental updates ever since.

In high school I managed to make a few games in Flash by myself, but later teamed up with a programmer friend and we managed to make a few multiplayer games. We had a Flash gaming site with a working community system up and running for a while, even managing to get a few hundred members, but we didn't take it much further.

 I found these old screenshots from our old game community.

After school I found a course at a large university targeted towards game development. As I didn’t know much about gamedev aside from my Flash/web experiences, I jumped in with both feet without doing much research. Turned out that the course was just an assembly of modules the university had been running for years already; they'd slapped on the game development tag to lure students in. I wasn't learning anything that would get me into the industry.

I realised my mistake and did some proper research, finding a much better programme called Gamemaker that had the art and programming aspects separated. I applied to the art course and got accepted.

I started off in September 2007 and focused on modelling and texturing. The course and teachers were good but it was up to you to make the most out of it. It would have been easy just to pass, but I wanted more than that, so along with a few other students I stayed at the school late at night trying to get good at this thing. The fact that I'd wasted almost a year of my time and student loan on a bad course made me more serious about getting it right this time.

All this hard work finally landed me an internship at Frictional Games after graduating and I've been here ever since.

Since I got my foot into Frictional's basement I haven't touched my portfolio at all. It's still up at www.hadex.se with the stuff I showed to apply for the internship. There are only 4 images, but I figured that it's better to show a few good things than lots of average ones. Keep it short and honest - at least that's what I'd want to see. Same with the letter you apply with; show that you want the position but don't use too much grease or it will feel cheesy and dishonest.

(Looking at these images today with a lot more experience, I wouldn't keep any of them for my portfolio!)

So - if you are looking to get into the industry make sure you find a course that has actual game-making modules on it!

What do I do?
I'm one of the 3D artists at Frictional Games and I do a little bit of everything on the visual side - except that I can’t draw. If there's someone else out there that can’t draw like me, you can still make it as a 3D artist as long as you have a good eye for things and are willing to learn and work hard.

I remember that back when I started here I thought some of the deadlines for the early Amnesia development were very stressful. If I could talk to that version of me today I would tell him to enjoy that nice and relaxing time. But I guess he would be too arrogant and wouldn't listen anyway!

Back then I'd get a task to create a batch of 2-4 different models in the same style along with their measurements and sketches. An example task might be to create a large table, a small table and a chair all as a set. Then I'd just have to model and texture those, and then move on to the next batch.

Although I'm still modelling and texturing smaller props it's now only a small portion of my responsibilities. My main job is to build levels based on an initial design. This usually means that I end up creating modular assets like walls, ceiling, floors, pipes, cables and so on; creating gameplay and story-driven objects; adding detail objects; creating particle systems; creating decals; and making smaller rigs and animations. I send out requests to our outsourcers for extra models and 2D art that I know I won't have time to do if I want to get the level built. I also talk with the animators that we contract to for larger animations.

It can be stressful to try and deal with all of this at the same time, especially as our levels are bigger and more fleshed out than ever before. But this range of tasks is what makes the job so varied and fun for me; your work can be completely different from month to month.

At Frictional we start by focusing on the most important story and gameplay aspects of each level, but at the same time I like to get in a bit of lighting and add some particles in order to spice up the visuals whenever I can. You don't want to leave all of the polishing work until the last few weeks!

Thomas puts a ton of work into designing the levels, but it's up to us 3D artists to kick off this vision and make it into an environment that you can walk around in. You start with a huge level design document and are not really sure where to begin at first, but once you get going and the level comes alive piece by piece it's a great feeling. You get the 'this is my level' feeling after a while even if the initial design isn't yours. I prefer it this way as each level ties perfectly in with each other and the story, instead of having each level builder also be the level designer.

Here is an example of making a room come to life following design and concept.
(click to see big version)

There's a quick insight into what I do in the company.

I usually don't write these sort of things but I hope it was a decent read for you.