In graphics, backface culling is the process by which the faces of a mesh that are facing away from a particular direction (e.g. camera) are culled in some form so that we avoid or reduce the GPU cost of processing them.

Diagram showing:

- detailed culling, i.e. test each face individually and see if it's culled

- coarse culling, i.e. work out a conservative cone of directions that can be culled and cull faces based on that.

Red (cull) / Green (visible) circles on bounding box boundaries show extremes. Circle in the middle shows the combined (conservative) cull cone.

Coarse culling done by looking at bounding box and assuming any face direction can appear anywhere within that box. That's conservative (i.e. actual model will probably have a different distribution of faces) but allows for quick rejection.

Can split mesh into clusters, based on average face direction. Store average face direction + cone per cluster.

]]>The Function Graph object in the editor allows you to plot cartesian and polar graphs of arbitrary functions. Since the functions are authored in Javascript, it’s actually quite easy to add logic and more or less arbitrary code to them, i.e. we’re not restricted to stricly maths-like functions. One of the advantages of this is that it was quite easy to expose ray hit testing results as inputs to the functions. This has allowed for some quite interesting graphs!

For the Function Graph object the user supplied function is evaluated once for each value on the X axis. The framework passes the the x value of the evaluation as the ‘x’ parameter of the fuction. Using the same approach we can pass all sorts of data to the user function, for example the results of a ray cast.

At each evaluation position we fire off a ray into the scene. For polar graphs the ray starts at the center of the graph and is shot along the evaluation direction. For cartesian graphs the ray starts at the evaluation point on the X axis and it’s shot along the direction of the Y Axis. The results of the raycasting are made available via the ‘`hitDistance`

‘ argument that is now passed to the user function. If the ray doesn’t hit any scene geometry then `hitDistance`

is set to `undefined`

.

From a rendering perspective, the obvious use case for this functionality is to create a visibility function. Let’s see an example of this. Use Mouse Drag to move the function graph around.

The code for this function was simply this:

return hitDistance ? 0 : 5;

If the `hitDistance`

parameter is set to something (in this example it doesn’t matter how far away the collision was) we return zero, otherwise we return 5. In other words, we’re making a graph that has non-zero values whererever it’s unobstructed.

Of course it makes more sense to present this as a polar graph! See how the graph changes as you move the origin around the scene (use mouse drag to move).

With ray casting results available to us, we’re very close to being able to do some nice things! The previous graph has a non-zero value where the ‘view’ of the world in that direction is unobstructed. We could use this as the visibility term of the Lighting Equation:

$$L_o = \int_\Omega \mathit{f}_r(i,o) L(i) V(i)(\overline{n \cdot i}) d\omega$$

where:

- \(L_o\) The light being reflected along direction \(o\)
- \(\int_\Omega\) An integral over the hemisphere
- \(\mathit{f}_r(i,o)\) The BRDF at point of interest evaluated for incoming direction \(i\) on the hemisphere and for outgoing direction \(o\).
- \(L(i)\) The amount of lighting arriving at the point of interest along direction \(i\).
- \(V(i)\) The visibility factor for direction \(i\), i.e. if our view of the light coming in from direction \(i\) obstructed?
- \((\overline{n \cdot i})\) dot product between surface normal n and light direction \(i\), clamped to zero.

Let’s simplify this equation a little by assuming a pure Lambert BRDF (i.e. \(\mathit{f}_r(i,o) = \frac{\rho}{\pi}\)) and a constant lighting envinronment, \(L(i) = 1\). This leaves us with just the following:

$$L_o = \frac{\rho}{\pi}\int_\Omega V(i)(\overline{n \cdot i}) d\omega$$

which is effectively a convolution of the visibility function with clamped cosine lobe.

As it happens, the Function Graph objects in the editor now support convolution integrals, which is quite handy because it means we can visualise this simplified lighting equation!

We’ll have the base function be just the visibility term:

return hitDistance ? 0 : 5;

and we’ll add a convolution function to represent the integration with the clamped cosine lobe, \((\overline{n \cdot i})\):

return Math.max(0, Math.cos(dx));

Hey presto, we have a moveable preview of the visibility integral of the lighting equation! You can toggle between the base and the convolved function, which is the same as showing radiance vs irradiance.

Like most of the diagrams in this blog it has no real value in itself. It’s intended to form the basis for experimentation and to perhaps help new graphics programmers get a better understanding and better intuition on how lighting behaves.

Having said that, I recently spent quite a bit of time at work discussing how visibility varies across a scene and how we might best represent and store that information. It would have been useful to be able to quickly knock up a diagram like this:

All the diagrams on this blog are editable. You can just right click and select ‘Edit Local Copy’. To create the above visibility grid diagram all I had to do was edit a copy of the single graph version and the copy & paste it along a grid. The grid itself was trivial to make; it’s just a rectangle object with its rows & columns set to 4 and the option to show the cell centers ticked. I could then snap each copy of the graph to a cell center.

]]>I recently added support for convolution integrals on the Function Graph object in the editor. I thought I’d take the opportunity to write a quick introduction on the topic and present a few interactive diagrams. As with previous posts, this is not meant to be a complete teaching resource on convolution integrals. There are many excellent resources out there already! It’s just another take on them with some, hopefully interesting, diagrams.

Note, if you already know all about convolution integrals and just want to see some nice interactive diagrams, just scroll down to here!

Wikipedia has a good page on convolution integrals which covers some of their mathematical background. I personally found that I understand them much better in the context of 2D images. Let’s work backwards a little and assume you want to blur a 2D image. As a graphics programmer you might write a pixel shader that samples a few texels around the position you’re writing to, average them, and output the resulting color. You’d then run this shader over ever pixel of your target image. Intuitively, you’d understand that the wider your sampling pattern, the blurrier the resulting image would be.

Let’s try and turn this into pseudo-code.

For each output pixel: Sample a bunch of texels from image. Average them. Write to output.

Those ‘bunch of texels’ aren’t just anywhere in the source image though. They are in the vicinity of the pixel we’re rendering. Let’s amend the pseudo-code.

For each output pixel p: Sample all the texels that are in the vicinity of p. Average them. Write to output.

What if we were to clarify what ‘in the vicinity of p’ meant? Let’s express it as those texels that are within a certain distance threshold from p.

For each output pixel p: Sample all the texels whose distance from p is less than threshold. Average them. Write to output.

Ok, how do we find all the texels that are within a certain distance of p? Let’s pretend that performance isn’t an issue and that reading every texel on the source image is free. We might then write code like this:

For each output pixel p: average = 0; For each input texel t: if (distance(t,p)&amp;lt;=threshold) average += image(t); Average them. Write to output.

Based on the logic above we make a binary decision to either include or exclude a texel in our calculation. If it’s within our distance threshold we include it, other we don’t. What if we were to express that as a multiplier? We’ll multiply by 1 if we want to include the texel, and by 0 if we want to exclude it.

For each output pixel p: average = 0; For each input texel t: if (distance(t,p)&amp;lt;=threshold) average += image(t) * 1; else average += image(t) * 0; Average them. Write to output.

Let’s start calling this mutliplier a ‘weight’. Our algorithm works out the weight of each texel, i.e. how much it contributes towards that average.

For each output pixel p: average = 0; For each input texel t: weight = 0; if (distance(t,p)&amp;lt;=threshold) weight = 1; else weight = 0; average += image(t) * weight; Average them. Write to output.

Now, let’s take a look at that ‘Average them’ statement. The blur operation takes in contributions from many texels and produces a single color. As long as the contibution from each (participating) texel is the same, then combining them together simply means taking their average. Since participating texels have a weight of 1 and non-participating a weight of zero, then simply adding up the weights will give us the number of participating texels.

For each output pixel p: average = 0; totalWeight = 0; For each input texel t: weight = 0; if (distance(t,p)&amp;lt;=threshold) weight = 1; else weight = 0; average += image(t) * weight; totalWeight += weight; average /= totalWeight; Write to output.

Expressing things via weight and totalWeight allows for an important mental leap. Not all contributions have to be the same. Each texel may contribute a different amount towards the final pixel. As long as we add up all the weights and divide the sum of the weighted texels by the sum of the contributions then we’ll a (mathematically) correct answer. This is known as weighted average.

Let’s rearrange the above pseudo-code a little to decouple the averaging process to the weight calculation.

function TexelWeight(delta) if (delta&amp;lt;=threshold) return 1; else return 0; For each output pixel p: average = 0; totalWeight = 0; For each input texel t: weight = TexelWeight(distance(t,p)); average += image(t) * weight; totalWeight += weight; average /= totalWeight; Write to output.

The averaging process hold true regardless of the implementation of the `TexelWeight`

function. We just need that function to return the contribution weight, given the distance of the texel being considered away from the pixel being rendered.

The input to the weight function is important. Rather than passing t and p, we’re passing the (signed) distance between them. We’ll visualize this in a bit, but this effectively means that the weight function moves/slides across the domain of the input function. Passing a distance of zero means we’re asking for the contribution of the texel at the same location as the pixel we’re writing to. Passing distances away from zero means we’re asking for the contibution factors of the texels away from the pixel being shaded.

If we think of `TexelWeight`

as more of mathematical rather than programming function, we should be able to plot its graph.

We can do this in the editor. Just add a Function Graph object and set the function code to this:

(or just right click on the diagram area and select Edit Local Copy!)

return Math.abs(x)&amp;lt;1 ? 1 : 0;

This type of filtering function is called a box filter and it’s characterised by the fact that all contributing samples have the same weight, i.e. it’s a straight average. Note that in the above formulation we normalize the sum by dividing by the sum of the weights. You’ll often see filtering functions (or ‘kernels’) where the weights have already been normalised so that they add up to 1.

The Box, Tent, and Gaussian filters are three of the most commonly used filters.

Notice how all these filtering functions are centered around the x=0 axis. That’s because the filtering function is applied over the entire range where the base function is defined, ‘grabbing’ nearby values and doing a weighted average to produce a set of output values.

Let’s see this in action! On the diagram below we have the base function at the top, the filtering function in the middle and the convolution integral at the bottom row. The convolution process ‘fetches’ values from the base function, multiplies each value by the value of the filter function and finally adds all these results together. This process happens once for each point on the resulting function. Notice how the filter function ‘slides’ across the range of the base function, combining multiple input values into a single output.

I’ll close things off with some more examples. In the diagram below you can see how a random noise input function is smoothed out by the various filtering kernels. You can try out different filters, but also vary the width of each. The wider the kernel, the smoother the resulting graph becomes.

All this of course applies to polar graphs as well where things are more interesting and more relevant to graphics programming.

For a more practical example of how convolution integrals can be used in computer graphics check out the post on cubemap filtering!

]]>You can have a look at the new editor here!

So, out with the old, in with the new! I’ve taken the slightly unusual approach of having a horizontal Property Panel. Most scene objects have few properties and they’re easy to fit on a single row. I also made the property widgets pop out on mouse hover. Again, slightly unorthodox, but most of the workflow is directly on the canvas. The value of the property is always displayed but there is no need to take screen real estate on a slider widget until the user needs to edit the property.

Combining the line & fill properties (color, alpha, width, etc) into a single, feature rich widget also helped to save space, but also to provide a more consistent interface across the various objects.

As far as the menu is concerned, I abandoned the original ‘mobile like’ button and went for a traditional top menu bar. It’s given the editor a more ‘desktop app’ feel that I was hoping to avoid, but it’s a convenient and familiar way of exposing a lot of functionality.

My main aim was to free up screen space and keep the canvas as the primary focus. Being restricted to a single monitor/single window can be a bit challenging, but I’m quite happy with the results (for now anyway!). I did look into using existing libraries to do UI and windowing management for me. I won’t lie, a good chunk of the reasons why I didn’t use any was because I wanted to just write it myself. This is a side/hobby project where I can self-indulge after all! There are some good libraries out there (if anything there are too many) but most gave me the impression that I would spend more time learning how to use them than getting the UI up & running. (again, this is a self indulgence project, not pro coding!)

I also spent a bit of time making the editor a bit more user friendly and intuitive. The purpose of the tool is to allow the user to create diagrams using an approach similar to pen & paper, using a ruler. It borrows a lot concepts from CAD packages, but also other general purpose vector packages. While a lot of functionality was already there, I was relying a lot on text hints to inform the user of what they can do.

I’ve made a few changes to improve the experience (though there is still a lot to do). You now get a visual buttons telling you what the Ctrl, Alt and Shift keys do in each mode:

Where possible, I’ve tried to provide visual, in-diagram hints of what the behaviour is going to be, for example when moving with axis lock (Ctrl) enabled:

or provide info while modifying an object:

I also managed to find some time to implement a couple of new objects.

The first one is a Function Graph. The user supplies the function in Javascript and it plots it as either a Cartesian (XY plot) or a Polar plot.

It’s still early days on this object, so it’s nowhere near the slick implementation of www.desmos.com, but I have high hopes for it! Being able to plot both Cartesian and Polar graphs was really important to me as they both come up quite a lot in graphics theory. More specifically, I find that some concepts I can understand better as an XY plot, whereas others are more intuitive as radial graphs.

I’ve taken the same approach as on the Bar Chart object, where the user is expected to type in Javascript code for the function. This obviously very programmer oriented, but that fits in well with my intended audience. The editor is there as a supporting feature of the blog and is very much aimed at graphics programmers and technical artists. Providing a more user friendly interface for the functions would take up a significant amount of time and, I think, would actually make it less intuitive for technical people.

The other object I implemented is a bouncing ball! Why? Because it’s fun! There a few different ways I want to develop this. Some educational, for example provide a ‘Trajectory’ object similar to the BRDF Ray, where it will show you how a projectile would bounce around the rest of scene, under the influence of gravity and other forces. Others more fun, for example I wouldn’t mind implementing an “I’m bored” mode where you play ‘keepie uppie‘ with the scene geometry!

As always there are tons of stuff to do and very little time to do them in! One of the major drawbacks at the moment is that the scene objects don’t store or use any form of transform matrix. This makes some forms of manipulation difficult or impossible. There is a ‘Transform’ tool at the moment but, to be honest, it’s pretty awful! Having a general purpose transform matrix that lets me apply scale and rotation would make things much simpler and intuitive.

While working on the editor I’ve also been making some notes on new blog post ideas. There a quite a few areas that I’d like to write about but time is sparse!

]]>Growing up as the son of a Civil Engineer, I was fortunate to have access to a proper HP pen plotter. That thing was AMAZING!

A surge of nostalgia and a drawer full of various bit of electronics, Arduino and motor parts and I was quickly putting together a pen plotter design.

There are many DIY plotter builds out there, ranging from those that reuse part of old disk drives to some really professional looking designs. I’m quite keen on building my own. It will be a fun, learning experience and I also have some particular requirements:

– It needs to fit fully inside an A4 page. That’s so that I can plot on any notepad I have, including those with spiral binding.

– It needs to have multiple pens! My diagrams use colors. I need to reproduce those colors!

The first requirement isn’t too bad. Most of the parts are quite small so I’m sure I can fit everything inside an A4. The print are will probably be quite small. I was hoping for A5 (half an A4), but i think it will be smaller.

The multiple pen caddy is a nice problem to solve! I had already started designing a 3 pen version when I realized that it’s not enough. I was thinking of red, green and blue, but I’ll also need black. Moving to a 4 pen design make things a bit more difficult, things are taking up more space, but it’s still doable. The bigger problem is how to switch from one pen to another.

I’m planning on using a spinning disk to lift/lower each pen in turn. As the disk turns it will lower one pen while lifting the next. At any one time only one pen will be down. The disk will be rotated using a servo motor, which allows precise control of its angular position. This would also let me do a half rotation, which would effectively lift all the pens off the paper to allow for repositioning.

Alas, moving to a 4 pen design creates a design flaw. With 3 pens you can move from any one pen to any other without ‘passing by’ another pen first. So, let’s say you have red, green and blue pens, you can move from the red to the green without ever lowering the blue one.

With 4 pens you can’t do that. To move from pen 1 to pen 3 you have to lower pen 2 first! This will end up putting a dot on the paper when there shouldn’t be any!

I’m now working on adding a proper z axis motor, to lift the whole caddy up/down. This means that I won’t have to program in a half turn to keep all pens off the paper and it will allow switching from one pen to another without any extra ink marks.

I’ve already gone through a few iterations of the design, but it’s not finished yet. Kudos to Fusion 360 for being such brilliant software to use!

Initial design with a massive caddy. This was too big, it would reduce the print are down to a post stamp!

A smaller design, but still not quite happy with it.

This is the current favourite!

More pictures and build progress notes to follow!

]]>Last week was British Science Week and like every year my wife and I both volunteer to run an activity at the local Primary school. The theme this year was ‘Exploration & Discovery’ and I thought it would be fun to introduce Year 4 children (ages 8-9) to the basic principles of GPS. The focus was on trilateration, i.e. draw three circles, see where they intersect.

The activity itself was quite simple: “A few explorers have found themselves lost somewhere around the UK. Their GPS devices are malfunctioning and do not report their exact position. They only show their distance from a few satellites. Can you use trilateration to help locate them?”.

Preparing the material for the activity seemed quite simple at first, just print off a map, mark the location of the satellites, pick the location of the explorers and measure some distances. In practice though, there were some complications.

My original idea was to have the explorers lost in different cities around the world. For that I would need to get hold of a world map…but what projection method do I use? As you may be aware there are many different ways to project a sphere onto a 2D plane, each with different properties. Some are designed to preserve areas, some to preserve distances, and so on. Fun fact, did you know that Google actually came up with their own projection method to use on Google Maps? It’s called Web Mercator and it’s a variant of the Mercator projection method. Unfortunately one of the properties of this projection is that converting straight line distance measurements (e.g. what you’d measure using a ruler) to true, i.e. great circle, distance is not trivial because the distortion changes with latitude.

One alternative would be to use a Gnomonic projection where great circles are represented as straight lines. That would work, but it makes getting hold of a map a bit more difficult.

In the end I decided to restrict myself to just UK (in fact just England and a bit of Scotland). The small extents of the area make the projection distortions largely irrelevant and makes city and town names easier to read when printed off on A3 paper. I *could* have just printed off any old world map and let the kids draw cirlces on that and it wouldn’t have impacted on the learning outcome of the activity, but I felt I should *try* and present correct information.

With the map area selected it was then time to pick the locations of the explorers and from that determine suitable satellite positions. To make the activity a bit more fun I wanted to pick cities and towns the first letters of which would spell out a word meaningful the kids of that particular school. This required me to pick 9 locations, each starting with a particular letter of the alphabet. To avoid confusion I had to avoid locations with more than one word (e.g. Isle of Man) and prefer somewhat isolated locations to allow for errors when the kids draw the circles.

This gave me the following locations:

With the locations set all I needed to do was figure out the satellite positions. This would be really easy, just pick any 3 spots on the map and then make a list of distances to each of the explorer locations. Alas, this would give me distances with decimal points. Location i from satellite j might 11.2 cm away. In a calm & quiet environment, on a one-on-one setting, providing help and supervision I’m sure I can get an 8 year old to draw a circle 11.2cm in radius. A group of 10+ of them though, probably not!

Let’s make things easier and require that all locations are an integer number of cm away from at least 3 satellites. Drawing a circle at whole numbers shouldn’t be too difficult, provided you have the right instruments.

How do you get an 8yo to draw a circle? Assume they don’t have a drawing compass (either for health & safety reasons or because there aren’t enough to go around).

My first thought was a piece of string. Tie a small loop around one end, that’s where the pencil goes. They measure the length using a ruler, hold the other end at the center with their thumb and draw the circle. Simple! Couple of problems…for starters with normal (rather than mechanical) pencils the string kind of bunches up under the pencil lead and you end up scraping it along the paper without the lead making contact.

You can move the loop higher up the pencil so that the string doesn’t bunch up under it….but then there is no guarantee that they’ll keep the pencil vertical, it will most likely swivel around the height at which the string is sitting on, causing the radius to fluctuate.

You could put an attachment of sorts at the end of the string to stop it bunching up while also keeping the string near the paper, but there is anoter, more fundamental problem. A string only enforces half of the circle constraint; that no point will be further away that the radius. As the kids are tracing the circle through they don’t maintain the tension on the string and they end up drawing something like an inwards spiral! We need something rigid, so that both halves of the contraint are honoured.

I did a few experiments with 3D printed bits and bamboo skewers (which by the way don’t have uniform thickness) and looked online for cheap rulers (I needed 30 of them), but in the end I went with a simple low-tech solution. I drew out a 22cm ruler at 1cm increments and had printed onto the thickest card my local print shop could do. I then punched holes at each 1cm mark. To draw a circle the kids would put one pencil through the 0cm hole (the center), another through the X cm hole and then just trace through. The card had just about enough rigidity to stop them moving inwards, though if I had to do it again I’d glue each ruler onto a second layer of card. It was a simple, cheap, disposable solution with the main downside being that it involved a fair bit of manual labour on my part (23 holes x 30 rulers).

So, 9 locations, 3 satellites per location, at integer distances. That’s a lot of satellites! I *could* create 9 sets of maps, each with just 3 satellites, but that would make the logistics of printing and running the activity more difficult. More importantly, that would not be *the way of the programmer*, certainly not one with a few evenings to spare before the activity day! Let’s think this through…how can we work out the least number of satellites that each have an integer distance away from as many locations as possible?

My first instict was to fire up Visual Studio and start writing from C++ code to do all the number crunching. Then I remembered that I’m a graphics programmer and that I have a programmer friendly, JavaScript based, 2D visualisation framework all setup!

For each location, we can visualise the valid satellite positions by drawing concentric circles, 1cm apart. Take any pair of locations and where their circles meet, that’s a satellite location that is an integer distance away from both of them. Drawing a 1mm radius dot on each intersection allows us to factor in a bit of tolerance. Placing a satellite anywhere inside the dot will give us *near* integer distance to the locations.

At this point I started thinking I might be able to get away with a purely visual solution. Just draw a black disc at each intersection point, but use a low alpha value, something like 0.1. Each disc represents a point that is integer distance away from 2 locations. Where more than one disc overlap, ever partially, then we have a point that is an integer distance away from 2, 3 or 4 locations. The more overlapping discs the darker that pixel would be. Let’s see what that looks like :

Ok, that’s pretty good. We can see that there are some fairly dark spots, indicating good candidates for satellite locations. It was encouraging to see that there are indeed more than a few suitable points as I was worried that I wouldn’t get enough points unless I increase the tolerance by quite a bit.

I could have stopped here. I could save out the canvas, put it in Photoshop and do some image space manipulation. A bit brightness & contrast adjustment would isolate the most suitable candidates. It would however be a very manual process. To get the satellite positions I’d need to note down the pixel coordinates in the image and convert those back to diagram coordinates. I’d also not have any correlation between the positions and which locations they are an integer distance away from. I could easily find that out by measuring, but the whole process felt a bit manual.

I decided to press on and look for a more analytical solution. The image based approach had served its purpose, showing me that there is a viable solution. It also gave me the idea to just brute force this. Given a position on the map, calculate its distance to each of the explorer locations. If the distance falls inside the min/max values of the ruler and it’s within a certain tolerance of an integer value (1mm) then we have a candidate. Loop around all the explorer locations and keep track of how many meet the distance criteria.

This gave some good answers! Searching the entire map at 1mm interval shows that there are lots of positions that are an integer distance away from at least 5 of the explorer locations. Increasing the limit to 6 locations however only gives 3 satellites, which isn’t enough. We need 3 satellites per location. Reducing the search spacing to 0.1mm (making even more of a brute force!) reveals that there are in fact 7 satellite locations that each is an integer distance away from 6 of the locations. Furthermore, a visual inspection reveals that each location is covered by at least 3 satellites (and most by 4) so it’s a viable overall solution as well! Perfect!

Well almost…brute forcing this at 0.1mm intervals in JavaScript is quite slow. The A3 page that I’d be using has dimensions of 29.7cm x 42cm, giving a whopping 12.4 mil checks. Technically it doesn’t matter, it was meant to be more or less a one-off activity, so waiting a couple of seconds to get the satellite locations wasn’t that big a bother. But it did make the map navigation quite sluggish and it felt like I could do a bit better.

Taking a step back, given two explorer locations we can pick any two integer radius values and draw circles around them. The circles will generally intersect at two points. We know those two points are an integer distance away from the two locations we picked. How about the rest of the locations? Maybe they happen to be an integer distance away from some other locations too.

So, we can improve the brute force check with an algorithm like this:

For every pair of locations: +- For every pair of integer radius values in [minRadius, maxRadius] range: +- If the sum of the radii is less than the distance between the location ->skip. +- For each of the 2 intersection points: +- Loop through all other locations: +- If the distance of the candidate point to the location is near integer -> keep the point

This gives the following:

That’s pretty good! We get good answers at a fraction of the cost. You may notice that we have less satellites with 6 locations each than the super-fine (0.1mm) brute force approach. That’s because the above algorithm picks the first point with zero tolerance (it’s the intersection of the two circles) and I think that’s missing out some near by good candidates.

At this point I decided to call it a day. I could improve the algorithm to do a bit of a search around those first couple of points, or better yet come up with a more direct algorithm altogether. However I was running out of time as I still needed to do all the printing, cutting, hole punching, etc as well as perpare a PowerPoint presentation!

I’m happy to say the activity went very well! All the children seemed to enjoy the presentation and they all managed to trilaterate at least 2 explorers in the 15 minute practical I had with each group. Hopefully the whole experience served as extra encouragement towards S.T.E.M. subjects.

]]>There are still some improvements to come. Now that the UI is looking reasonable and has a few usage tips, I’m going to focus on making some usability changes on the drawing itself. There is a lot of functionality (typically accessed via keyboard modifiers) that isn’t very obvious. The overlay rendering (snaps, visual hints, etc) aren’t very nice either (again, programmer art!). Some of these I will address by having nicer, more intuitive graphics, and others by providing example and tutorial scenes.

]]>Rendering engines often use cubemaps to store some form of lighting. There are a myriad of techniques that use them ranging from static to dynamic, direct and indirect, diffuse and specular lighting and so on. In this post we’ll try to illustrate some of the common steps in capturing, filtering and using cubemaps to achieve some form of lighting.

Let’s start with the cubemap capturing process, using the a simple environment (full of awesome programmer art) below. I’ve added a few random colorful objects just to make things a bit more interesting!

Next, let’s define our cubemap’s boundary as well as the capture point. It’s quite common to have the capture point being at the center of the cubemap, but it’s not always necessary to do so. In some cases the center of the cubemap might lie inside some scene geometry so moving it a bit further away would help. You can use the Shift + mouse drag to move the capture point.

With the cubemap in place we can start thinking about how to populate it. We need to decide what values to store on each texel and how we’ll calculate them. The choice of content will have implications the cubemap texture format, bit depth and compression method, but for now we’ll focus on illustrating the capturing process.

The simplest capture we can make is to simply record the color value of the scene at each texel location. We can visualise this by firing a ray from the capture point to the center of each texel. Have a go at doing this by moving the mouse cursor through the diagram below.

In a real rendering engine we probably wouldn’t be firing rays from the capture point, but rather setup 6 cameras (for a 3D cubemap), each pointing at the center of each cube face, with a 90° field of view and a square aspect ratio. We’d then render the scene from each camera and store the results (e.g. render to texture) on each cube face.

But what is it we’re capturing? In the case of the diagram above when the ray hits an object we just copy that object’s color into the cubemap texel that corresponds to that direction. In that case we have assumed that the entire object has a constant color, i.e. it’s the same across the entire surface and doesn’t vary with the incident vector. This would be the case if all the objects in the scene were purely emissive.

A more realistic model would be one where the color we capture is based on the lighting that affects the object as well the material properties and shading model (BRDF) of the surface at the point where the ray hits.

Take a look at the example below. The scene is illuminated by a single point light and we’re interested in a single point of the ceiling, P. The surface properties at P indicate that the any light arriving there is not reflected equally in every direction (as would be the case for a purerly Lambertian surface). Instead there is a predominant reflection vector where the relfection intensity is highest and it falls off on vectors further away from it. If point P was captured by the two probes shown below, they would each get a different color value since the amount of light reflected along each of the probe rays is different.

One implication of this is that the lighting stored in the cubemap is only correct at the capture point. The further away we are from the sampling point, the more incorrect the stored lighting becomes.

So what does that mean when capturing a cubemap using a render-to-texture camera? A first attempt would probably be to use the normal rendering pipeline to render the scene from the probe’s point of view. This means that the usual lighting shaders would be used, which, presumably, would do both diffuse and specular illumination and therefore produce results similar to the ‘raytraced’ examples above. That’s not necessarily a problem but rather something to be aware of.

What’s more important to be aware of is any image post processing the normal rendering pipeline performs. A typical linear lighting pipeline would include some tonemapping and gamma correction that transforms a linear HDR buffer to an sRGB LDR one. That’s fine when it comes to showing images on screen but probably not when doing a cubemap capture. We’d generally want to keep the lighting values in linear color space and in high precision and only do any transforms necessary to take them to whichever storage format we wish to use.

At this point it should also be obvious that the process described above captures *indirect* lighting. Lights aren’t rendered directly into the cubemap, but rather the light that bounces off the surfaces of the scene is. We’re capturing both diffusely and specularly bounced lighting. That has nothing to do with how we’re going to use the cubemap (e.g. to provide indirect diffuse or specular illumination), it’s just to clarify that both types of bounced lighting are captured. The contribution from emissive surfaces would also be handled in the same way.

So now we have a cubemap that has been fully populated to contain the incoming radiance along the direction vector corresponding to each texel. In its current form it cannot really be used for any form of indirect illumination, at least not in a PBR pipeline. We need to do some preprocessing first. More specifically we need to apply a bit of intelligent blurring!

Earlier we saw how the BRDF of a shading point tells us how much of the lighting coming from a particular direction is reflected in any other direction. In a sense it tells us how light going *into* the surface is *scaterred* outwards. We now want to do the reverse, given a particular *outgoing* direction, we’d like to know how much light we can *gather* from each direction along the hemisphere.

In the scene below the shading point is illuminated by 3 discrete lights. The BRDF will give us the contribution of each light along the orange vector pointing towards the camera. Summing these up (i.e. integrating over the hemisphere) will gives us the total amount of light in the direction of the camera vector.

We’ll use the same principle to gather the contributions from each of the radiance cubemap texels to calculate the total illumination (irradiance) along a given viewing direction. The diagram below illustrates this process. The mouse cursor controls the view direction and the colored lines are the contribution of each radiance texel based on the BRDF. The sum of these contributions represents the irradiance on that surface and we store it on a texel of a separate, irradiance, cubemap.

In a sense this is the same as having a chrome sphere positioned at the capture point and having the camera looking straight down on it along the direction of the mouse vector. The roughness of the sphere will have a significant effect on the reflections, the rougher the surface, the blurrier the reflections.

You can use the controls below to change the BRDF properties. Click the Populate Irradiance button to fully populate the irradiance cubemap, which will show a disc that simulates sampling the entire map. Have a play with the roughness slider using the Phong BRDF to see how the reflections change. Also note how when using the Lambert BRDF the roughness doesn’t make a difference.

It’s worth noting that the above *gather* operation was done by pretending there was a flat white surface element oriented along the mouse vector and we have a camera looking straight down on it, i.e. the normal, N and the view vector, V are aligned. The importance of this will be explained further down.

We’re almost there now! We have an irradiance cubemap that was generated using a particular BRDF. If have an object located at the capture point and that has the exact same BRDF as we used to generate the cubemap and it’s facing along the camera view vector, then we’re all set! However, chances are at least one of those things won’t be true. The object, or rather the shading point, probably isn’t at the capture point, its BRDF isn’t exactly the same as the one we did the integration with and its orientation probably isn’t along the camera vector. We need to either account or accept all these differences.

We already came across the issue of sampling a cubemap away from the capture point when discussing the capturing process. Sampling away from the capture point suffers from incorrect specular radiance being used, incorrect occlusion and incorrect parallax. Of those, incorrect occlusion is probably the most noticable (reflections being picked up when they should have been occluded). There is no straighforward solution involving just the cubemap alone. Often additional structures are used to capture the occlusion information and/or use alternate cubemaps.

Incorrect parallax on the other hand is fairly straightforward to fix in some of the simpler cases and involves simply some bounding box information. We’ll cover this in a future post.

The incorrect specular radiance is something we can just probably live with.

The other issues occur when the BRDF of the surface we need to shade isn’t the same as the BRDF we used for the radiance integration and when the surface orientation is not aligned to the view vector. There are couple of very good SIGGRAPH papers that describe a split-sum approximation the lighting integral for image based lighting (Karis13, Lazarov13, Lagarde14). As part of that approximation we integrate the radiance for a particular roughness and a look up table provides a runtime scale and bias factors based on roughness and viewing angle. In terms of precomputation all we need to do is perform the cubemap integration for a range of roughness values.

One approach would be generate a unique cubemap per discrete roughness value that we need to support (roughness is typically expressed as a scalar in the [0,1] range). However it’s worth noting that with increasing roughness the cubemap becomes more and more blurry and we therefore need less texels to capture it accurately. One conventient approach then is to store the irradiace of each roughness at the mip of a mipmapped cubemap. Mip 0 (the highest resolution) stores the lowest rouhgness (i.e. the least blurry) and subsequent ones store increasingly higher roughness values. This also has the advantage that we can leverage trilinear filtering to interpolate between mip maps giving us a continuous range of roughness values.

While the emphasis has been on indirect specular illumination using the cubemap, all the principles described here are equally applicable to diffuse illumination. Computing the irradiance cubemap using a Lambert BRDF (basically a cosine lobe) would result in a cubemap suitable for that. As described in this great paper by Ramamoorthi however, a Spherical Harmonic is a much more compact and efficient way of achieving the same result. One potential exception to this is using the last mip (lowest resolution) of the specular cubemap mip chain to store the diffuse irradiance. Then a single cubemap can be used for both diffuse and specular indirect illumination.

There is still a lot to cover of course but hopefully these diagrams have given you a better understanding of what’s involved in capturing and using a cubemap for indirect environment lighting. Feel free to post questions and topic requests in the comments below!

]]>A Signed Distance Field is a mathematical construct where the distance to a closed surface is computed along a set of positions, with the sign of the distance used to indicate whether the position is inside or outside the surface. The positions are typically chosen to be on a regular grid and they work well in both 2D and 3D. They were made popular in computer graphics by this SIGGRAPH 2007 paper by Valve. If you haven’t already read it, it’s definitely worth a read!

In this post we’ll investigate using a simple 2D SDF to approximate a shape. It’s by no means the only use of SDF, but it’s one that’s easy to visualize and has practical use in computer graphics.

We’ll start with the surface shown below. It’s a closed surface so that we can tell whether a point is inside or outside its perimeter.

Next, we’ll overlay a regular grid over the surface. As you might expect, the size of the grid is important, the finer it is, the better the approximation will be.

For each grid cell, we can calculate the distance of the cell center to the nearest point of the surface. We’ll represent that with a circle. We’ll also indicate whether the cell center is inside the shape or not by coloring in the circle. Note that we only store one value per cell, calculated at its center, so while a cell may overlap the surface boundary, we only care about its center.

Go ahead and move the shape around (shift + mouse drag) to see how the cell distance values change.

Note that for reasons of clarity I’m only showing the circles that have a diameter of about one cell. The other cells are computed as well and their values will be used later on for the approximation.

Ok, so now we have a regular grid where we store the distance to the shape boundary as well as a bit that tells us whether we’re inside or outside the shape. How can we use that information to reconstruct the original shape?

Let’s have a look at the diagram without the original shape.

Imagine we throw a dart onto the grid. If the dart lands exactly at the center of a cell we’ll accurately know our distance to the shape outline and whether we’re inside it or not. That information will only be accurate if it lands exactly at the center because that’s where we evaluated the distance and sign. If the dart lands anywhere else (and chances are that it will) then we’ll need to make an approximation by interpolating between the 4 nearest cells.

Let’s see what that looks like! Move the mouse cursor over the image.

We’re nearly there now. We have a way of estimating the distance to the surface at any point inside the grid. All we need to do now is visualize all these estimates somehow.

We could for example paint anywhere where the distance is ≤ 0 to shade the interior of the shape. In the diagram below we’re drawing an approximation of the outline of the surface by painting anywhere where the absolute distance is within a threshold. The threshold value controls the ‘thickness’ of the approximate outline.

Have a play with the controls to see what effect it has on the approximation. Use shift + mouse drag to move the shape within the grid.

So what’s the point of all this?

Let’s compare a low resolution SDF of a surface to a same resolution texture that stores the surface color.

You can see that despite being the same, low, resolution, the SDF representation can provide a very high resolution outline of the shape compared to the very pixelated result of the texture approach. Before jumping to conclusions, we need to understand that we’re comparing apples to oranges! The SDF is a high level representation of the shape surface. The image is a capture of the surface color (and alpha). We may use both representations to achieve visually comparable results, but they’re fundamentally different things. Both representations need to be evaluated per pixel.

In the case of the texture this evaluation gives us the interpolated color values, producing the gradients at the edges of the pixels. The image can store different/arbitrary colors per texel, that don’t have to conform to any particular function, but the look of texture is (generally) fixed (e.g. we cannot easily add an outline). Color and opacity can be stored and handled independently.

In the case of the SDF the interpolation is giving us a smooth approximation of the distance to the surface of the shape. We’re free to translate that into a pixel color value using a function selected at runtime. The same SDF can be used with multiple functions to produce different results. However, unlike in the texture case, the entire SDF has to go through the same translation (i.e. it’s not trivial to produce a different color for an arbitrary part of the surface). This translation is a high resolution operation, so we don’t get any pixelation artifacts. Instead, the effects of the low resolution are seen on the accuracy of the approximation. You can see that the sharp corners of the original shape have disappeared and been replaced with rounded points instead.

Looking at the SDF we’ve created so far it should feel quite intuitive that we would store it in a low resolution texture. Ignoring format considerations for now, we’d store the distance and the sign at each texel. At runtime we can evaluate the entire SDF by sampling that texture in a pixel shader and use that to shade a quad. This is more or less what described in the Valve paper mentioned at the beginning. The quad will be rendered at a high resolution (e.g. screen resolution) and the interpolation (done for ‘free’ by the GPU) will give us the lovely crisp outlines we saw earlier.

However this is not always what we want (or are able) to do. While 2D SDF are easy to visualize, there is nothing inherently 2D about SDF in general. A 3D SDF can be used to represent a volumetric surface. How do we visualize that? We *could* render it as a point cloud, evaluating the SDF at each point, but that would get tricky and expensive. What about cases where we don’t care to visualize the SDF? What if it represents a surface we want to do collision detection against?

As it turns out Ray Marching is a good answer to both of these questions and when it comes to ray marching SDF there are a couple of cool tricks we can use. I’ll save the diagrams for this for part 2 of this post, but in the meantime have a read through this blog post and maybe have a look at Claybook to see how cool things can be made using SDF!

]]>