Tech Feature: HPSL Shading Language

Overview

HPL3 is our first engine to support both PC and consoles. To make it easy to support multiple platforms and multiple shading languages we have decided to use our own shading language called HPSL. Shader code written in HPSL goes through a shader parser to translate it to the language used by the hardware.

The shader written in HPSL is loaded into the engine at runtime, the code is then run through a preprocess parser that strips away any code that is not needed by the effect or material. After that the stripped code is translated to the language used by the hardware (GLSL #330 on PC and PSSL on the PS4) and then compiled.

HPSL uses the same syntax as the scripting or engine code. HPSL is based on GLSL #330 but some of the declarations are closer to HLSL.

// Example code

@ifdef UseTexture

uniform cTexture2D aColorMap : 0;

@endif

void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;

@ifdef UseTexture

                    vColor *= sample(aColorMap, px_vTexCoord0.xy);

          @endif

          out_vColor = vColor;

}

//Preproccess step

void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;

          out_vColor = vColor;

}

// Translation step

#version 330

#extension GL_ARB_explicit_attrib_location : enable

in vec4 px_vColor;

in vec4 px_vTexCoord0;

layout(location = 0) out vec4 out_vColor;

void main()

{

          vec4 px_vPosition = gl_FragCoord;

          bool px_bFrontFacing = gl_FrontFacing;

          int px_lPrimitiveID = gl_PrimitiveID;

          vec4 vColor = px_vColor;

          out_vColor = vColor;

}

Preprocessing

All the shader code used in SOMA is handwritten. In order to keep all the relevant code at the same place and to be able to quickly optimize shaders HPL3 uses a preprocessing step. This has been used for our previous games as well. A preprocessor goes thorugh the code and removes large chunks that are not needed or used by the effect or material. The lighting shader used in SOMA contains code used by all the different light types. Changing a preprocess variable can change a light from a point light to a spotlight or can be used to enable shadow mapping. The preprocessor strips blocks of code that are not used, this increases performance since code that has no visual effects is removed completely. Another feature of the preprocess parser is the ability to change the value of a constant variable, this can be used to change the quality of an effect.

// SSAO code

for(float d = 0.0; d < $kNumSamples; d+=4.0)

{

          // perform SSAO…

}

The preprocessor makes it easy to do complex materials with multiple textures and shading properties while only performing the heavy computations for the materials that need it.

Translation

After the preprocess strips the code it is ready to get translated. In the first step all the variable types and special functions are converted to the new language. Then the main entry function is created and all the input and output is bound to the correct semantics. In the last step the translated code is scanned for texture and buffers that get bound to the correct slot. 

Compilation

The translated code is then compiled. If a compilation error occurred the translated code is printed to the log file along with the error message and corresponding row for easy debugging.

Summary

In order to deliver the same visual experience to all platforms and to make development faster we decided on using our own shading language. The code is translated to the language used by the hardware and compiled at runtime. Supporting other shading languages in the future will be very easy since we only need to add another converter. 

HPSL translates to GLSL #330 which requires OpenGL 3.3 (DirectX 10 feature set). This means that SOMA will require a DirectX 10 or newer graphic card.

Modders will still be able to write shader code directly in GLSL if they chose to.

HPSL Reference

Syntax

HPSL uses the same syntax used by the scripting language.

 

Variable Type	Description
int	32 bit signed integer
uint	32 bit unsigned integer
bool	Stores true or false
float	32 bit float
double	64 bit float
cVectorXf	Vector of floats
cVectorXl	Vector of signed integers
cVectorXu	Vector of unsigned intergers
cMatrixXf	Square float matrix
cMatrixXxXf	Non-square matrix (Ex cMatrix2x4f)
cBuffer	Container of multiple variables that get set by the CPU

Texture Type	Description
cTexture1D	Single dimension texture
cTexture2D	Standard 2D texture
cTexture3D	Volume texture
cTextureCube	Cubemap texture
cTextureBuffer	A large single dimension texture used to store variables
cTexture2DMS	A 2D render target with MSAA support
cTextureXCmp	A shadow map texture used for comparison operations
cTextureXArray	Array of cTextureX textures

A texture contains both the image and information about what happens when it is sampled. If you are used to OpenGL/GLSL then this is nothing new. DirectX uses a different system for storing this information. It uses a texture for storing the data and a separate sampler_state that controls filtering and clamping. Using the combined format makes it easy to convert to either GLSL or HLSL.

Textures need to be bound to a slot at compilation time. Binding is done by using the “:” semantic after the texture name.

//bind diffuse map to slot 0

uniform cTexture2D aDiffuseMap : 0;

Variable Type Modifier	Description
uniform	A variable or texture that is set by the CPU
in	Read only input to a function
out	Output of a function
inout	Read and write input and output to a function
const	A constant value that must be initialized in the declaration and can’t be changed

Entry Point and Semantics

The entry point of a shader program is the “void main” function. Input and output of the shader is defined as arguments to this function. The input to the vertex shader comes from the mesh that is rendered. This might be information like the position, color and uv mapping of a vertex. What the vertex shader outputs is user defined, it can be any kind of information that the pixel shader needs. The output of the vertex shader is what gets sent to the pixel shader as input. The variables are interpolated between the vertices of the triangle. The input of the pixel shader and the output of the vertex shader must be the same or else the shaders won’t work together. Finally the output of the pixel shader is what is shown on the screen. The pixel shader can output to a of maximum 4 different render targets at the same time.

Some of the input and output are System defined semantics. System Semantics are set or used by the hardware. 

System Semantic	Description	Type	Shader Type
px_vPosition	Vertex position output. Pixel shader input as screen position. This is required by all shaders	cVector4f	Vertex (out), Pixel (in)
: X	Output color slot, where X must be in the range 0-3	cVector4	Pixel (out)
vtx_lVertexID	Index of the current vertex	int	Vertex (in)
vtx_lInstanceID	Index of the current instance	int	Vertex (in)
px_lPrimitiveID	Index of the triangle this pixel belongs to	int	Pixel (in)
px_bFrontFacing	Indicates if the pixel belongs to the front or back of the primitive	bool	Pixel (in)

Input to the vertex shader is user defined. HPL3 has a few user defined semantics that work with our mesh format.

Mesh Semantic	Description	Type
vtx_vPosition	Position of the vertex	cVector4f
vtx_vTexCoord0	Primary UV coord	cVector4f
vtx_vTexCoord1	Secondary UV coord	cVector4f
vtx_vNormal	World space normal	cVector3f
vtx_vTangent	World space tangent, w contains binormal direction	cVector4f
vtx_vColor	Color	cVector4f
vtx_vBoneIndices	Index of the bones used to modify this vertex	cVector4l
vtx_vBoneWeight	Weight to multiply the bones with	cVector4f

It is possible to add more user defined semantics if needed

//vertex shader

uniform cMatrixf a_mtxModelViewProjection;

void main(in cVector4f vtx_vPosition,

               in cVector4f vtx_vColor,

               in cVector4f vtx_vTexCoord0,

               out cVector4f px_vColor,

               out cVector4f px_vTexCoord0,

              out cVector4f px_vPosition)

{                          

          px_vPosition = mul(a_mtxModelViewProjection, vtx_vPosition);

          px_vColor = vtx_vColor;

                             px_vTexCoord0 = vtx_vTexCoord0;

}

//pixel shader

uniform cTexture2D aColorMap : 0;

void main(in cVector4f px_vPosition,

               in cVector4f px_vColor,

               in cVector4f px_vTexCoord0,

               out cVector4f out_vColor : 0)

{

         out_vColor = px_vColor * sample(aColorMap, px_vTexCoord0.xy);

}

Functions

HPSL is based on OpenGL 3.3 and GLSL version 330 and supports almost all of the GLSL arithmetic functions.

There are some functions that are different from GLSL. This is to make it easier to support HLSL and PSSL.

Arithmetic Function	Description
mul(x, y)	Multiplies two matrices together (multiplying by using * not supported for matrices)
lerp(x, y, t)	Interpolates between two values

Texture sampling use functions specific to the HPSL language.

Texture Function	Description
sample(texture, uv) sample(texture, uv, offset)	Samples a texture at the specified uv coordinate. Can be used with an integet offset
sampleGather(texture, uv) sampleGather(texture, uv, offset)	Samples a texture but returns only the red component of each texel corner
sampleGrad(texture, uv, dx, dy) sampleGrad(texture, uv, dx, dy, offset)	Performs texture lookup with explicit gradients
sampleLod(texture, uv, lod) sampleLod(texture, uv, lod, offset)	Samples the texture at a specific mipmap level
sampleCmp(texture, uv, comp_value) sampleCmp(texture, uv, comp_value, offset)	Performs texture lookup and compares it with the comparison value and returns result
load(texture, position)	Gets the value of a texel at the integer position
getTextureSize(texture, lod)	Returns the width and height of the texture lod
getTextureLod(texture, uv)	Gets the lod that would get sampled if that uv coord is used
getTextureLevelCount	Gets the number of MipMap levels

It is also possible to use language specific code directly. Some languages and graphic cards might have functions that are more optimized for those systems and then it might be a good idea to write code specific for that language.

@ifdef Lang_GLSL

                  vec4 vModifier = vec4(lessThan(vValue, vLimit));

@else

                  cVector4f vModifier = step(vValue, vLimit);

@endif