What are CSS Shaders?

Adobe announced a proposal for CSS Shaders at Adobe Max two days ago. This was backed by both Opera and Apple, and the spec will be developed by the FX Task Force at the W3C. For those of you who are not aware, the FX task force is an elite band of ninjas’s chosen from the ranks of the CSS and SVG Working Groups to work on specs common to both technologies. Their most notable work so far is perhaps the Filters Effects 1.0 spec, but they’re also working on a common model for Transforms and Animations.

So what exactly is the CSS Shaders proposal, and how does it relate to the existing proposals and technologies? First I will introduce you to the required terminology (you can skip that section if you are already into computer graphics), then I will dig into the proposal and see what it can do. I’m basing this off the spec as I don’t have a build to play with.

Shaders terminology

A shader is a set of software instructions used to calculate rendering effects (usually) on a GPU. A graphics pipeline takes a representation of what will be rendering using 3D primitives (such as vectors) as an input, and through various stages in the pipeline transforms it, then renders it as a 2D rasterisation (pixels) onto a 2D plane (usually a screen). Traditionally, fixed-function pipelines were used, which support common pre-defined geometry and pixel shading functions (more on these below). With shaders, instead of using pre-defined functions, developers can program their own custom functions to do whatever transformations they want to specify. Due to the pipeline approach, shaders can be chained so that the output of one feeds into the input of the next one. This will be familiar to you if you’ve used SVG Filters or programs like Automator on Mac or Yahoo! Pipes.

Their are two main types of shaders in the CSS Shaders spec, which I will describe below.

Vertex shaders

A vertex shader describe the properties of a vertex (the points at the corners of geometric shapes, usually triangles). These include the position and colour of the vertex, and the texture mapping. If you can imagine a model is represented as a number of triangles, with the vertices at each corner, the shader will transform these vertices, such as change the position in 3D space, the colour, or the depth. This could for example transform a flat shape, such as a flat piece of paper, into something that looks like it has a 3D perspective, such as the curved page of a book. The spec includes examples such as a page curl, waving flag or wobble effect.

Fragment shaders

Fragment shaders (terminology used in OpenGL and also CSS Shaders) can also go by the name of pixel shaders (used in Direct3D). Fragment shaders describe the properties of pixel, such as the colour, opacity, and bit-depth. As you can imagine from vertex shaders, a fragment shader manipulates these pixels, such as changing the colour. They can be used for things such as lighting effects or colour manipulation such as changing an image to greyscale or sepia tone. You’ve probably seen a number of these in graphics packages such as Photoshop. CSS Filters includes a number of pre-defined ones, but fragment shaders will allow you to define your own.

The CSS Shader proposal

Now you have a rough idea of what shaders are, what exactly is the CSS proposal for shaders? At its basic level, the proposal is to bring these effects to HTML and SVG content via CSS. Fragment shaders extends the capability of Filters so that you can define your own pixel manipulations, and vertex shaders brings the new capability of distorting the 2D HTML and SVG elements. The vertices will be drawn as a grid of triangles over the 2D plane of the element(s) in question, and they can be manipulated as mentioned above to make a faux-3D effect on the screen. Shaders are defined to work well with CSS modules such as Transitions and Animations, so that you can animate the distortions, such as turning elements into a waving flag, or a page that curls. As you can imagine combining multiple shaders and animations can produce some very sophisticated effects.

Applying shaders with CSS

A shader is applied via CSS using the filter property from Filters 1.0, or it can be applied to SVG using the filter element. The filter property/element is usually used for pre-defined filters. To use a custom shader you use the custom() function (or feCustom as a child element of the filter when using SVG elements) as the value. They can be chained with other shaders and built-in filters. For fragment shaders the custom function needs a url that points to the shader and arbitrary optional parameters. The shader is written using a shaders language, which defines the parameters it expects, if any. Vertex shaders are similar but also can accept a vertex mesh as the second argument. This defines how many vertices are used and if it maps to the border box, padding-box, content-box or filter box. The filter property accepts the functions as a space separated list to allow for chaining. Custom filters are also defined as a space separated list inside the custom() function. Lets look at a quick example from the spec to make this clearer.

 .old-book-page {
        filter: grayscale(1.0) custom(url('book.vs') url('old-page-paper.fs'));
 }

The filter property in this example first uses the grayscale() built-in filter to turn the element black and white. It then feeds the output into the custom() function. This uses a vertex shader called book.vs which manipulates the flat element into an open book shape, then applies the old-page-paper.fs fragment shader to manipulate the pixels to give it an aged look. The shaders found at those URLs will define how these manipulations happen. They don’t define any custom parameters or a vertex mesh.

If the elements the filters applies to looks like the following:

After the filter is applied it will turn into the following:

Lets look an example where the vertex mesh is defined and custom parameters are used. Transitions are also used to animate the vertex shaders.

.waving      {
        filter: custom(url('wave.vs'), 20 20, phase 0, amplitude 50);
        transition-property: filter;
        transition-duration: 0.2s;
}

.waving:hover {
        filter: custom(url('wave.vs'), 20 20, phase 90, amplitude 20);
}

You may recognise the url function from the previous example. This links to the custom wave vertex shader. The 20 20 after defines the vertex mesh of 22 x 22 vertices. It may seem strange why it makes a 22 x 22 grid rather than 20 x 20, but I will describe later. The phase and amplitude parameters are defined by the shader. These two properties are transitioned over 0.2 seconds, which makes it wave like a flag. The example will look something like the following (minus any animations):

Defining a vertex mesh

In the above example we used a vertex mesh, which defines the vertices that can be manipulated by the vertex shader. This accepts three values: two integers and a box value. It is specified as the second argument (separated by a comma) in the custom function when using CSS, or using the vertexMesh attribute of the feCustom element when using the SVG syntax.

The first integer defines the number of additional vertex lines (you can think of this as number of points on the y axis) and the second is the number of additional vertex columns (you can think of this as the number of points on the x axis). There is however two extra vertices than the number defined. This is because the default value is 0 0 which is one line and one column. In this case there are four vertices; one at each corner (top left, bottom left, top right, bottom right). You can see this in the diagram below:

If you add 1 line and 1 column you get these four vertices at the corners plus the 1 you add in the centre of the row and column. Therefor the algorithm to calculate the number of vertices in each direction is n+2, and to calculate the number of lines or columns it is n+1. This may be confusing, so there is a note in the spec considering if the default should be 1 1 instead of 0 0. In the following diagram the vertex mesh is set to 5 4:

The last parameter is the box value. This is used to align the vertex mesh to the box specified. If it is absent it defaults to filter-box. This is the border box plus additional filter margins (so that any additional content that through vertex or pixel manipulation is displayed outside the margin box is still visible). The other possible values are border-box, padding-box and content-box, which you are probably already familiar with from the CSS box model. If you are using SVG then border-box and padding-box is the equivalent of content-box. You could imagine a shader specifying a mesh similar to the one in the diagram above, using an explicit box value could be something like the following:

filter: custom(url('whatever.vs'), 5 4 filter-box);

Defining the shader

This is the meat of the shader. It is trivial to add; you just use the url() function as the first argument of the custom() function in CSS, or in the vertexShader or fragmentShader attribute of the feCustom element for the SVG syntax. With this you point to the file where the shader is defined. Vertex shaders use the .vs extension and fragment shaders use the .fs extension. Writing the shader is where the hard part comes in. I expect most developers won’t write their own, and libraries will pop up that define the common ones that are needed.

The proposal recommends that shaders should is written using GLSL ES. This is the OpenGL ES Shading Language (PDF link). This was chosen as it is already used in WebGL. Having consistency between both makes sense to me. It is a full programming language in its own right, and quite close to the metal, so it is beyond the scope of this humble blog post to describe how to use it.

Passing parameters to the shader

While shaders can be interesting in their own right, they are much more flexible when you can pass in parameters to affect the transformations that take place. You specify these in the custom() function, or the params attribute of the feCustom element if using the SVG version. This takes in space separated key value pair. The first is the name of the parameter (this will be defined in the shader) and the second is the value you are passing to it. This can either be true, false, a number (as used in the example above), an array (array( ‘[wsp]*’)), a transform (either a CSS 3D transform or a matrix), or a texture (texture()). With texture you can pass it the result of a previous filter or shader using the result() function. You pass it the name specified in the result attribute of the filter in question.

Here is an example defining the filter, then a shader with a vertex mesh and a number of parameters:

filter: grayscale(0.5) custom(url('groovy.vs'), 5 4, awesome false, randomness 15, stuff array(1 23 43 32));

This will apply a grayscale filter. The texture which is outputted from this passed to the groovy vertex shader. This will create a vertex map of 6 lines (7 vertices) and 5 columns (6 vertices) attached to the filter box. It will then pass in the three parameters, do whatever the vertex filter defines to manipulate the texture then will output the result and rasterise it.

Defining the filter margins

There are filter margins outside the border box to give space for the various transformations. These can be set with 5 new CSS properties: filter-margin-top, filter-margin-bottom, filter-margin-left, and filter-margin-right work the same way as the the equivalent margin properties. The filter-margin property is the shorthand that works the same way as the margin property. If you are using the SVG syntax you can use the x, y, width and attributes on the filter element instead. If both are defined then the SVG attributes takes precedence.

Relation to other specifications

The CSS shaders spec relates most closely to the Filters 1.0 and WebGL specs. As explained earlier, the shaders spec uses the custom function from Filters 1.0. Fragment shaders are similar to filters, but allow you to define your own from scratch, rather than just being able to use the predefined filters in Filters 1.0.

WebGL is a web-based version of the OpenGL ES 2.0 specification. It is a powerful low level 3D graphics language, which provides a 3D context to the canvas element. It provides both fragment and vertex shaders using the same GLSL ES Shader language, plus all the other 3D features. The use cases are quite different as WebGL works on content in the canvas element while CSS Shaders works on any HTML or SVG content. WebGL is a 3D drawing API similar to 2D canvas, while HTML and SVG is retained mode, using the DOM.

The same trade offs, advantages and use cases with SVG Vs Canvas are likely with CSS shaders + HTML/SVG Vs WebGL. You’re likely to use WebGL if you want highly interactive 3D games where you don’t care about keeping access to what was rendered on screen and you’re making 3D models, while you’re more likely to use HTML/SVG + CSS shaders if you want to retain the DOM, such as keeping textual content as actual text, and just want to add some visual effects.

The shaders spec is also defined to work with CSS transitions and Animations. The filter property is specified to be animatable. For interpolation to happen the filter values have to have the same number of filter functions and appear in the same order .

4 months ago  /  79 notes  /