Rendering Lines and Bezier Curves in Three.js and WebGL

It’s relatively simple to draw lines and curves with CanvasRenderingContext2D. Not so with WebGL. In this post, I’ll explore some ways to draw lines and (quadratic bezier) curves in three.js with webgl (and on the gpu).

It was probably Firefox 3 that led me to explore the use of the Canvas element and the 2d context. Then it was Firefox 4 that led me to explore the webgl context, followed by discovering three.js. That was probably how I got involved with 3d graphics. The point is that there’s some connection between the 2nd and 3rd dimensions. Enough with my history and let’s start exploring some vector graphics.

Straight lines
A straight line connects 2 points.

2D Canvas – Here’s how we draw a line from (x1, y1) to (x2, y2) in canvas 2d.

ctx = canvas.getContext(‘2d’);
ctx.lineWidth = 2;
ctx.strokeStyle = ‘black’;
ctx.moveTo(x1, y1);
ctx.lineTo(x2, y2);
ctx.stroke(); 

Three.js/WebGL – Now here’s how we do it in three.js/WebGL.

geometry = new THREE.Geometry();
geometry.vertices.push(new Vector2(x1, y1, 0));
geometry.vertices.push(new Vector2(x2, y2, 0));
material = new THREE.LineBasicMaterial( { color: 0xffffff, linewidth: 2 } );
line = new THREE.Line(geometry, material);
scene.add(line); 

Drawing straight lines in webgl requires a little more code (even with three.js), but not really too much additional code for now.

Quadratic Bézier
A quadratic bézier curve connects points (x0, y0) to (x2, y2) interpolating through control point (x1, y1).

2D Canvas – Just one function name change from a straight line.

ctx = canvas.getContext(‘2d’);
ctx.lineWidth = 2;
ctx.strokeStyle = ‘black’;
ctx.moveTo(x0, y0);
ctx.quadraticCurveTo(x1, y1, x2, y2);
ctx.stroke(); 

Three.js/WebGL – Now it gets a little different.

SUBDIVISIONS = 20;
geometry = new THREE.Geometry();
curve = new THREE.QuadraticBezierCurve3();
curve.v0 = new THREE.Vector3(x0, y0, 0);
curve.v1 = new THREE.Vector3(x1, y1, 0);
curve.v2 = new THREE.Vector3(x2, y2, 0);
for (j = 0; j < SUBDIVISIONS; j++) {
   geometry.vertices.push( curve.getPoint(j / SUBDIVISIONS) )
}

material = new THREE.LineBasicMaterial( { color: 0xffffff, linewidth: 2 } );
line = new THREE.Line(geometry, material);
scene.add(line);

Unlike canvas 2d approach, we can't yet simply change one function call, because three.js / webgl doesn't support bezier curves as a primitive geometry. We need to subdivide the curve and draw them as line segments. With sufficient line segments, they would almost represent a nice curve.

Here are a couple of drawbacks though.

1. There needs to be sufficient subdivisions or the curve segments might look like straight lines instead
2. More objects and draw calls are created in the process.
3. ANGLE does not render lineWidth > 1 length by WebGL specification.

For #1, number of divisions can be estimated so the resulting curve still look pleasant at a particular zoom level. For #2, one can switch to BufferGeometry to reduce overheads.
For #3, its the least easily fixed. I encountered this with my bezier lights experiment last year.

WebGL LineWidth Limitations

Screenshot 2014-09-07 16.12.06

My "fix" then was to alter the experiment a little.

Screenshot 2014-09-07 16.13.37

Of course that doesn't exactly fix the lineWidth issue and someone tweeted about the approach they used for drawing lines in WebGL.

So this was last year. This year I revisited this thinking about how to render lines in WebGL for a visualization project I'm working on.

Screenshot 2014-09-07 16.19.20
(the screenshot here shows employs the easier canvas 2d approach)

The handy ParticleGeometry class

Earlier I created a class called ParticleGeometry for rendering the leaves in a cherry blossom experiment.

ParticleGeometry (probably not the best name again) is a wrapper around BufferGeometry (for performance reasons). It was created to be render rotatable sprites/particles in a more optimized fashion without being too difficult to use. 2 triangles are used for each sprite/texture which are rotated in the vert shaders (instead of in js) with values passed in via attributes. I find this approach to be more efficient than typical approaches in three.js (eg. using multiple plane meshes or using SpritePlugin). Later I discovered I could easily modify this class to work with drawing lines and curves.
ParticleGeometry for Lines

Let's start with initalizing a ParticleGeometry. LINES is the number of lines or sprites we allocate for our line pool. Internally, it would create 2x amount of triangles and other attributes required.


lineGeometry = new THREE.ParticleGeometry( LINES, size );

Straight Lines in WebGL.
the first modification to ParticleGeometry is a setLine method.

THREE.ParticleGeometry.prototype.setLine = function(line, x1, y1, x2, y2, LINE_WIDTH) {

Based on the line width, this method updates the 2 triangles for the referenced line from the starting to the ending point. With this, we solve the problem of rendering lines with line widths greater than 1. In fact, we could have custom individual line widths, which is probably more difficult to do with the default LineMaterial.

Demo: testing lines

Benchmarking Canvas 2D performances

Before we continue, I wanted to see how fast canvas 2d rendering lines and curves.

2000 lines - 60fps
5000 lines - 38fps
10000 lines - 20fps.

500 quadratic curves - 60fps
1000 quadratic curves - 40fps
2000 quadratic curves - 25fps
5000 quadratic curves - 12fps

These numbers look pretty decent to me. However these are stroked with lineWidth = 1. When I increase lineWidth to 2, the frame rates drop.

2000 lines - 38fps
500 quadratic curves - 6fps

In contrast with ParticleGeometry.setLine(), I get 30fps for 10000 lines. A tad faster, but not really faster than canvas 2d. The biggest difference come when I increased lineWidths to 5, I could still get 25fps, and 18fps at lineWidth 10.

The script used for running these numbers can be found in this gist.

Rendering Bezier curves with WebGL
So if we are able to draw straight lines with variable widths, how do we tackle bezier curves? I started experimenting with a couple of concepts.

Concept 1 - Place a 2d canvas as an overlay above the webgl context and use the 2d context api. Depending on your needs, this might not be a bad idea given canvas 2d performance isn't bad at all. Just an additional bit of effect projecting the 3d points back into 2d points for rendering.

Iteration 2 - extend ParticleGeometry.setLine() technique with Bezier curves. Now based on the performance numbers, if we have a pool of 4000 lines to work with at 60fps, we are could calculate how many line segments we want per curve. Let's say we are satisfied with 20 segments per curve, we could draw 200 curves. This could work, but based on our canvas 2d numbers, this doesn't look very promising.

Iteration 3 - Blinn-Loop approach.
A pretty well known technique developed by Charles Loop and Jim Blinn in Microsoft used for rendering vector art on the gpu. Their paper is called Resolution Independent Curve Rendering using Programmable Graphics Hardware.

I've used a similar approach earlier with this three.js vector font experiment.


In this approach, each triangle is used to fill a quadratic curve segment. However to stroke a bezier curve, we need to draw a line instead of filling an area. So I modify the frag shader glsl code with a threshold.

float inCurve(vec2 uv) {
float d = uv.x * uv.x - uv.y;
if (d >- 0.05 && d < 0.) return -1.;
return 1.;
}

Demo: stroke bezier with modified inCurve function

performance: 5000 curves - 30fps, 10000 curves - 20fps, 1000 curves - 60fps. So far, this looks like best performance for bezier curve so far. However, controlling the lineWidth is difficult in this approach.

So let turn on GL_OES_standard_derivatives to be able to estimate stroke bezier with modified sdCurve function

Now we have slightly better control of the lineWidth, however there's still an issue. Because we are only using 1 triangle, rendering becomes a little problematic when the stroke width is thicker than the height of the triangle.

4rd Iteration. Render bezier curve stroke in the fragment shader using a distance function.

A couple of days ago I watched the talk about GLyphy. In short GLyphy allows you to render vector fonts by representing the vectors on the texture to be rendered on the GPU. What was interesting was the way it had to convert bezier curves to arc segments so it'd be easier to compute in the shaders. It is also interesting how the original concept came from research for rendering vector graphics.

Similarly Taylor Holliday ported the hlsl code found in this paper, to produce this glsl shader code for rendering bezier curve by distance approximation and I was excited to be able to use this.

Screenshot 2014-09-08 08.01.27

The next challenge for me is how to employ this technique with my ParticleGeometry class. More specifically is how to position triangles so they could be used to draw the bezier lines. I wrote added a method just to do this.


THREE.ParticleGeometry.prototype.setBezierGrid = function(line, v0, v1, v2, LINE_WIDTH)

Basically I use 2 triangles to cover a grid over where the bezier stroke would be drawn. Based on the line width, I giving sufficient padding on the left, right, top and bottom so that the strokes do not get clipped. I calculate the transformed control point to the shaders via attributes, and this works pretty well!

Demo: stroke bezier with fast distance estimation

There are however some caveats with this technique:
1. It breaks for thicker line widths. However, it should be sufficient for most small line widths.
2. It breaks when the control point is colinear with the start and end points. For that iq has a fix which draws a straight line in these scenarios.
3. It breaks when the control point almost colinear with the start and end points. This disturbs me a little, but could be "fixed" by drawing a straight line with a bigger threshold.

5rd Iteration - Solve for actual distance. To fix the minor issue with the 4th iteration, I found an exact distant to bezier curve solver here.

The nice thing is that I just need to swap one glsl function call from the 4th iteration and we are done.

Demo: stroke bezier with distance solver

Now for some numbers:

4th iteration
1000 curves (~line width 1-3) = 45fps
2000 curves (~line width 1-3) = 30fps

5th iteration
1000 curves (line width 1) = 45fps
1000 curves (line width 3) = 30fps
2000 curves (line width 1) - 23fps

Here we see that 5th approach is slightly slower but not too much. The values are pretty similar to canvas2d bezier speeds at lineWidth 1, but the webgl performance is more scalable for high line widths.

Results
From thin straight lines
Screenshot 2014-09-08 09.42.15

Thick bezier lines
Screenshot 2014-09-08 09.44.38
in WebGL. (alright, aesthetically isn't better, but it's a WIP).

Conclusion

Hopefully I have shown you in this post how you could render lines and bezier curves in both canvas 2d and webgl.

I have shown how the Canvas 2D API is relative easy, and the performance of 2d canvas is nothing shaddy. For most use cases, why not use it?

I have also shown that despite challenges with working with webgl at a lower level, it is still possible to render lines and curves at great quality and speed.

The additional effort might be able to get you some performance gains for large quantity of curves with thickness. There are also some other reasons why you may opt to draw bezier curves the webgl way.

- make use of post processing
- integrate gpgpu
- keep things in the webgl workflow
- control other little details (eg. shading)

I'll leave you to consider these tradeoffs when deciding what to use.

Lastly, I'll leave you with some inspirations on some cool demos that can be created with bezier curves.

- Bateria by grgrdvrt
- Fat cat by Roxik
- Fluid Jelly by Fabien Bizot
- Muscular-Hydrostats by soulwire.