Tech Feature: HPSL Shading Language

Overview

HPL3 is our first engine to support both PC and consoles. To make it easy to support multiple platforms and multiple shading languages we have decided to use our own shading language called HPSL. Shader code written in HPSL goes through a shader parser to translate it to the language used by the hardware.

The shader written in HPSL is loaded into the engine at runtime, the code is then run through a preprocess parser that strips away any code that is not needed by the effect or material. After that the stripped code is translated to the language used by the hardware (GLSL #330 on PC and PSSL on the PS4) and then compiled.

HPSL uses the same syntax as the scripting or engine code. HPSL is based on GLSL #330 but some of the declarations are closer to HLSL.

// Example code

@ifdef UseTexture

uniform cTexture2D aColorMap : 0;

@endif

void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;

@ifdef UseTexture

                    vColor *= sample(aColorMap, px_vTexCoord0.xy);

          @endif

          out_vColor = vColor;

}

//Preproccess step

void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;

          out_vColor = vColor;

}

// Translation step

#version 330

#extension GL_ARB_explicit_attrib_location : enable

in vec4 px_vColor;

in vec4 px_vTexCoord0;

layout(location = 0) out vec4 out_vColor;

void main()

{

          vec4 px_vPosition = gl_FragCoord;

          bool px_bFrontFacing = gl_FrontFacing;

          int px_lPrimitiveID = gl_PrimitiveID;

          vec4 vColor = px_vColor;

          out_vColor = vColor;

}

Preprocessing

All the shader code used in SOMA is handwritten. In order to keep all the relevant code at the same place and to be able to quickly optimize shaders HPL3 uses a preprocessing step. This has been used for our previous games as well. A preprocessor goes thorugh the code and removes large chunks that are not needed or used by the effect or material. The lighting shader used in SOMA contains code used by all the different light types. Changing a preprocess variable can change a light from a point light to a spotlight or can be used to enable shadow mapping. The preprocessor strips blocks of code that are not used, this increases performance since code that has no visual effects is removed completely. Another feature of the preprocess parser is the ability to change the value of a constant variable, this can be used to change the quality of an effect.

// SSAO code

for(float d = 0.0; d < $kNumSamples; d+=4.0)

{

          // perform SSAO…

}

The preprocessor makes it easy to do complex materials with multiple textures and shading properties while only performing the heavy computations for the materials that need it.

Translation

After the preprocess strips the code it is ready to get translated. In the first step all the variable types and special functions are converted to the new language. Then the main entry function is created and all the input and output is bound to the correct semantics. In the last step the translated code is scanned for texture and buffers that get bound to the correct slot. 

Compilation

The translated code is then compiled. If a compilation error occurred the translated code is printed to the log file along with the error message and corresponding row for easy debugging.

Summary

In order to deliver the same visual experience to all platforms and to make development faster we decided on using our own shading language. The code is translated to the language used by the hardware and compiled at runtime. Supporting other shading languages in the future will be very easy since we only need to add another converter. 

HPSL translates to GLSL #330 which requires OpenGL 3.3 (DirectX 10 feature set). This means that SOMA will require a DirectX 10 or newer graphic card.

Modders will still be able to write shader code directly in GLSL if they chose to.

HPSL Reference

Syntax

HPSL uses the same syntax used by the scripting language.

 

Variable Type	Description
int	32 bit signed integer
uint	32 bit unsigned integer
bool	Stores true or false
float	32 bit float
double	64 bit float
cVectorXf	Vector of floats
cVectorXl	Vector of signed integers
cVectorXu	Vector of unsigned intergers
cMatrixXf	Square float matrix
cMatrixXxXf	Non-square matrix (Ex cMatrix2x4f)
cBuffer	Container of multiple variables that get set by the CPU

Texture Type	Description
cTexture1D	Single dimension texture
cTexture2D	Standard 2D texture
cTexture3D	Volume texture
cTextureCube	Cubemap texture
cTextureBuffer	A large single dimension texture used to store variables
cTexture2DMS	A 2D render target with MSAA support
cTextureXCmp	A shadow map texture used for comparison operations
cTextureXArray	Array of cTextureX textures

A texture contains both the image and information about what happens when it is sampled. If you are used to OpenGL/GLSL then this is nothing new. DirectX uses a different system for storing this information. It uses a texture for storing the data and a separate sampler_state that controls filtering and clamping. Using the combined format makes it easy to convert to either GLSL or HLSL.

Textures need to be bound to a slot at compilation time. Binding is done by using the “:” semantic after the texture name.

//bind diffuse map to slot 0

uniform cTexture2D aDiffuseMap : 0;

Variable Type Modifier	Description
uniform	A variable or texture that is set by the CPU
in	Read only input to a function
out	Output of a function
inout	Read and write input and output to a function
const	A constant value that must be initialized in the declaration and can’t be changed

Entry Point and Semantics

The entry point of a shader program is the “void main” function. Input and output of the shader is defined as arguments to this function. The input to the vertex shader comes from the mesh that is rendered. This might be information like the position, color and uv mapping of a vertex. What the vertex shader outputs is user defined, it can be any kind of information that the pixel shader needs. The output of the vertex shader is what gets sent to the pixel shader as input. The variables are interpolated between the vertices of the triangle. The input of the pixel shader and the output of the vertex shader must be the same or else the shaders won’t work together. Finally the output of the pixel shader is what is shown on the screen. The pixel shader can output to a of maximum 4 different render targets at the same time.

Some of the input and output are System defined semantics. System Semantics are set or used by the hardware. 

System Semantic	Description	Type	Shader Type
px_vPosition	Vertex position output. Pixel shader input as screen position. This is required by all shaders	cVector4f	Vertex (out), Pixel (in)
: X	Output color slot, where X must be in the range 0-3	cVector4	Pixel (out)
vtx_lVertexID	Index of the current vertex	int	Vertex (in)
vtx_lInstanceID	Index of the current instance	int	Vertex (in)
px_lPrimitiveID	Index of the triangle this pixel belongs to	int	Pixel (in)
px_bFrontFacing	Indicates if the pixel belongs to the front or back of the primitive	bool	Pixel (in)

Input to the vertex shader is user defined. HPL3 has a few user defined semantics that work with our mesh format.

Mesh Semantic	Description	Type
vtx_vPosition	Position of the vertex	cVector4f
vtx_vTexCoord0	Primary UV coord	cVector4f
vtx_vTexCoord1	Secondary UV coord	cVector4f
vtx_vNormal	World space normal	cVector3f
vtx_vTangent	World space tangent, w contains binormal direction	cVector4f
vtx_vColor	Color	cVector4f
vtx_vBoneIndices	Index of the bones used to modify this vertex	cVector4l
vtx_vBoneWeight	Weight to multiply the bones with	cVector4f

It is possible to add more user defined semantics if needed

//vertex shader

uniform cMatrixf a_mtxModelViewProjection;

void main(in cVector4f vtx_vPosition,

               in cVector4f vtx_vColor,

               in cVector4f vtx_vTexCoord0,

               out cVector4f px_vColor,

               out cVector4f px_vTexCoord0,

              out cVector4f px_vPosition)

{                          

          px_vPosition = mul(a_mtxModelViewProjection, vtx_vPosition);

          px_vColor = vtx_vColor;

                             px_vTexCoord0 = vtx_vTexCoord0;

}

//pixel shader

uniform cTexture2D aColorMap : 0;

void main(in cVector4f px_vPosition,

               in cVector4f px_vColor,

               in cVector4f px_vTexCoord0,

               out cVector4f out_vColor : 0)

{

         out_vColor = px_vColor * sample(aColorMap, px_vTexCoord0.xy);

}

Functions

HPSL is based on OpenGL 3.3 and GLSL version 330 and supports almost all of the GLSL arithmetic functions.

There are some functions that are different from GLSL. This is to make it easier to support HLSL and PSSL.

Arithmetic Function	Description
mul(x, y)	Multiplies two matrices together (multiplying by using * not supported for matrices)
lerp(x, y, t)	Interpolates between two values

Texture sampling use functions specific to the HPSL language.

Texture Function	Description
sample(texture, uv) sample(texture, uv, offset)	Samples a texture at the specified uv coordinate. Can be used with an integet offset
sampleGather(texture, uv) sampleGather(texture, uv, offset)	Samples a texture but returns only the red component of each texel corner
sampleGrad(texture, uv, dx, dy) sampleGrad(texture, uv, dx, dy, offset)	Performs texture lookup with explicit gradients
sampleLod(texture, uv, lod) sampleLod(texture, uv, lod, offset)	Samples the texture at a specific mipmap level
sampleCmp(texture, uv, comp_value) sampleCmp(texture, uv, comp_value, offset)	Performs texture lookup and compares it with the comparison value and returns result
load(texture, position)	Gets the value of a texel at the integer position
getTextureSize(texture, lod)	Returns the width and height of the texture lod
getTextureLod(texture, uv)	Gets the lod that would get sampled if that uv coord is used
getTextureLevelCount	Gets the number of MipMap levels

It is also possible to use language specific code directly. Some languages and graphic cards might have functions that are more optimized for those systems and then it might be a good idea to write code specific for that language.

@ifdef Lang_GLSL

                  vec4 vModifier = vec4(lessThan(vValue, vLimit));

@else

                  cVector4f vModifier = step(vValue, vLimit);

@endif

GDC 14 Lecture Resources

I am making this blog as place with extra resources for a talk I will be giving at GDC on Tuesday next week (14.20 at the Narrative Summit). The talk is called "Making Storytelling a Fundamental Part of the Gameplay Experience" and will be about a new approach to narrative design called 4-Layers. I only have 25 minutes to give the talk and there is a lot of stuff I cannot explain at proper depth; this page is a way to make up for that. I will also post a link to the cleaned up script of the talk here when I get back.

If you are planning on attending this talk, these links should also serve as some warm up to some of the
concepts I will cover!

Here comes resources:

Terminology
This is a huge stumbling block when discussing narrative in games, as story means so many things to a lot of people. Here I have tried to give a basic overview on what I mean with the different words. (Also check comments for further discussions).
http://frictionalgames.blogspot.se/2013/05/nailing-down-terminology.html

Core Elements of Interactive Storytelling
This is more in-depth information on the core elements that I discuss in the lecture:
http://frictionalgames.blogspot.se/2013/08/5-core-elements-of-interactive.html

http://frictionalgames.blogspot.se/2013/07/thoughts-on-last-of-us.html
My post on Last Of Us go over some more examples of when a game manages to really shine with storytelling, as well as the moments when it does not.

http://unbirthgame.com/TheSelfPresenceStorytelling.pdf
This is a longer essay that goes over the idea that the core of a game is made up in the player's mind.

Puzzles
When designing puzzles, it is important to not just see them as some intrinsically interesting element, they must serve some purpose to the narrative. This means there is a lot of things you need to think about when designing one. Some of these things should also apply to non-puzzle gameplay.

http://frictionalgames.blogspot.se/2012/12/introduction-i-recently-started-to-play.html
Here I go over how the high level design of most adventure games is flawed from a narrative perspective.

http://frictionalgames.blogspot.se/2013/02/puzzles-what-are-good-for.html
When are puzzles useful?

http://frictionalgames.blogspot.se/2013/03/puzzles-and-causal-histories.html
Thinking of puzzles as a way for the designer to make the player do certain actions but still feel agency.

People of Frictional: Mikael Hedberg

Who am I?

I am Mikael Hedberg and I’m the writer. I write all sorts of text that shows up in the game. I give actors lines to say and then ask them to scream a few times, because their character will most likely die at some point.

Background

I’ve been a gamer for as long as I can remember. I played everything I could get my hands on, board games, RPGs, and of course computer games. I can’t remember a brand or anything, but the very first computer I owned was a green keyboard with rubber keys. It didn’t have any memory to actually save things, so you had to program every game you wanted to play. I had a few sheets of paper filled with code which you painstakingly had to type before you could play. And what riveting games I got to play. Oh, the thrills of guessing that number between 1 and 100 or dodging those falling letters. Those were the days.
I did eventually upgrade to an Amiga which was the first time I encountered real story in games. LucasArts and Sierra titles quickly became my addiction. I liked all of them, but more than any other I would say Gabriel Knight: Sins of the Fathers really made me go, “Oh, these can get pretty dark and tell real stories.”

I was kind of late to the computer RPG party, because playing a lot of tabletop RPGs made me think they were pretty bad in comparison. I did start liking them around the release of Fallout 2 and played a bunch of the RPGs released around that time. I still consider Arcanum: Of Steamworks and Magick Obscura to be one of the best computer games I’ve ever played. You see, the thing about being a tabletop RPG player is that I’ve played and hosted easily over a hundred different stories in this very free and responsive type of play. Going to computer games, it’s really difficult to get excited by the storytelling because it’s rarely that interesting. Not because it doesn’t tell the right stories, but because I never feel like I’m really doing anything. I feel like I’m slowly moving a movie forward. It’s like all I do is power the projector by pushing keys. People often get dumbstruck when I don’t go nuts over games like the Last of Us. They are great at story, they tell me. Yes, they really are, but we have always had good writing and conventional storytelling. Look at the Sierra games from the 90s. They are absolutely fantastic in that respect.

I do get more excited with games that either try to give me that sense of responsibility like the Fallout games or games that try to merge gameplay and story like Heavy Rain. If you mess up, you don’t just fail, you actually fail someone in the game, you change the story. That’s when story matters, that’s when it gets interesting to me.
I’ve always wanted to tell a really good story in a game where it makes you feel like I do when I play tabletop RPGs. A game where you are concerned about what you are doing, not because you might fail some gameplay, but because it may take the story somewhere you don’t want it to go. Where your actions affect the lives of the characters you meet in a meaningful way and their well-being is reflected back on you. Basically to be human, but in another world.

My "corner office"

I started out my career at a pretty big company in Stockholm. It was a very educational experience concerning how the business works. However, it was very frustrating trying to get anything done. In the end I felt like I put maybe 5% of my time into the writing and 95% into company politics, talking to people and trying to get everyone on board. The one thing I learned about being a game writer is: You are responsible for the story, but you don’t have mandate to actually control the story. Impossible situation, you say? I thought so, so I kind of ran away to Japan for a while.

One day as I sat there in my house on a snowy peak in Japan there came a knocking on my door. It was Jens, an old friend of mine from school.
“Gotham needs you,” he said.
“What?”
“Sorry, wrong house. Oh, hi Mike. You want to do some writing for Frictional Games?”
“No, man. I’m done with game writing. My life is all about crochet now. Wanna see?”
“Wow, that’s like really bad…”
“Well, I just started.”
“No, I mean, like really really bad…”
“All right, you made your point. I’ll take the writing gig.”
“What is this even supposed to be?!”
“I said I’d do it already!”

That’s how I ended up at Frictional Games. And there’s at least five words of truth to that story.

SIDE – Voice recording studio
Home away from home.

What do I do?

Explaining what I do and the amount of control I have over the narrative is very difficult, but I’ll give it try. Unlike what many seem to think I don’t set up the game like a screenwriter would set up a movie. I don’t precede the making of the game like a screenwriter, I write the game while in production. What happens in preproduction is more of setting up what the game will be rather than the story and the plot itself. Thomas, my boss, will explain what he wants thematically and roughly what events he wants the player to experience. It often sounds like story, but it really isn’t, even though it sometimes lends itself to something looking like a narrative. This preproduction talk mostly limits the story through theme and game structure. In the case of the Frictional Games titles I’ve worked on they all go with real-time linear progression of levels with a horror theme. This setup could make a thousand different stories if not more, but it also makes thousands of stories impossible to tell. After preproduction I should have an idea of what the game could be, but also what it absolutely can’t be.

Thomas then starts designing levels, providing a procedural journey through the levels with some suggestions for story content that could or sometime absolutely should be included. When I get to do the real work, actually writing the text you read or the dialog you hear in game, all the people in the company are already doing their thing, working from the level design. So, when I start writing there is already a level in production, gameplay designed, and a good sense of what the player will be doing. To make this work I can’t just start writing what I think would make a good story, but rather what would make the game work - and if there’s enough room, try to make it interesting beyond just being functional.

Step one in my work is always to have whatever the player is doing make some sense. Before I can get into shaping interesting characters or explain the plot, I need to hit that mark.

Take one of the very first puzzles in Amnesia. The door is covered in fleshy goo (the Shadow). The player is to create a concoction to dissolve it to continue.

The door itself and the goo in front of it takes care of itself. The player understands that this is not like other doors and something needs to be done to proceed. The first step is to introduce the solution, the recipe to dissolve the goo. Now this is pretty strange for someone to have lying around, so it needs to be justified somehow. I could start with saying Alexander’s plumbing is clogging up and that he purposefully created the recipe. Problem solved. Now that it is justified, I look at it. Can it be improved? Can I say something else about the world and the characters within it?

“This is my third attempt to produce artificial Vitae. The former compounds lacked the potency I need, but I sense I'm close. Calamine and Orpiment are a given and the Cuprite binds them well. This time I will attempt Aqua Regia instead of Aqua Fortis in hope it will produce a more even solution.The experiment was unsuccessful. The solution is highly acidic and proves impractical to put to any use except as a detergent. Organic tissue reacts especially violently to the solution and should be handled with the greatest care. I might be able to use the recipe, but I'm losing hope that I will find an alchemic solution to my predicament.”

Yes. Let’s make it one of Alexander’s experiments to create the vitae he wants so badly, but a failed one that has the effect the player wants. Now suddenly the note justifying the puzzle just told us about Alexander and further detailed his journey. Also the kind of throw away attitude Alexander has towards the concoction gives the player a sense of “Even though Alexander didn’t, I can put that to use”. It’s not just Daniel blindly following Alexander’s footsteps. Daniel actually finds a use to something that Alexander didn’t, which creates a nice distance between the two characters. So what started as a strictly functional solution ends up helping me talk about these two characters and enrich the story and reveal Alexander’s motives.

Justification followed by building story is the foundation of my work.

Pretty early on I know what I want the overall story to be and I map out what I need to tell that story in my head. To accommodate for changes and things that others, mostly Thomas want to include, I need to keep this plan pretty flexible. I try to keep this in mind when writing early texts as well, so I don’t paint myself into a corner. The further we get into the game, the more I dare to nail things down as the rest of the team has subconsciously started to take this as the obviously best version of the narrative.

Illustration time! Let’s say I have three pieces of information I need to get into the game so the story works the way I want it to. Let’s call them Transform, Conflict, and Payoff.

"Oh, boy! Don’t they look great together? 

And then we put the game and its structure on top of that:

"You bastards! Look what you did to my words!"

Then we make the best of the situation:

"I suppose Change is kind of like Transform, so if I just shuffle them around a little. There, they totally fit."

"— Mike, we cut another level."

"Muddafu—!"

Maybe not the best of illustrations, but it’s sort of what I do. I try keep an idea of the overall information I need to put into the game to tell the story I’m aiming for, and then keep it flexible enough so I can tell that story in a range of different ways. It’s not that I come up with thousands of versions, but instead remain opportunistic enough to fit the information I need where ever it will fit.

What’s probably most surprising to people is that I don’t get to boss people around and claim that the story demands this or that. Many would probably equate this to the story taking a backseat and is more of an afterthought, but that would be missing the point about the process. Yes, I don’t get to plan and direct the story, but I kind of get to cultivate it. Over time shaping a nice little bonsai tree out of the chaos that feeds it. And since we started on SOMA, Thomas has started to show more interest in storytelling which takes a much more transformative form as he has the power to actually tell the team to rework things. Thomas and I more openly discuss where we think the story should go this time around and what we would both like to see, so it’s easier getting a good framework down to work from. Funnily enough, I consider Thomas my most powerful ally as well as the most destructive force on the team, since he could suddenly decide to reshape a level to make it fit the narrative better or on the other hand decide to cut a critical scene due to gameplay flow. Ultimately I need to trust him to make the game he wants. Because if I cultivate a little story bonsai tree, I’m sure Thomas is worried about a whole garden of different fauna, like graphical coherency, technical efficiency and engaging gameplay.

As always I just need to roll with the punches.

LevelEditor Feature: Poser EditMode

So I'm back with another HPL LevelEditor feature update, the Poser EditMode.

Back in the Amnesia days, whenever we wanted to add details which implied a unique skeletal model with different poses, we had to go all the way back to the modelling tool, set them up there and save them as a different mesh files. So we pretty much ended up with lots of replicated and redundant data (the corpse piles come to mind as I'm writing this), plus the burden of having to prepare everything outside the editor.

So, how to fix this? This is where the poser mode comes in. In a nutshell, it takes a skeletal mesh and exposes its bones to be translated and rotated in whatever way you fancy. This is useful not only for organic and creature geometry, but to create details like cables, piping and anything you can add a skeleton to.

There's not much to be added, so see it in action in this little video.