Author Archives: Zz85

About Zz85

I floating here amist my dreams. Whacha reading is my sleep talking.

Three.js Bokeh Shader

Posted on by

(TL;DR? – check out the new three.js webgl Bokeh Shader here)

Bokeh is a word used by photographers to describe the aesthetic out of focus or blur properties of a lens or a photo. Depth-of-field (DOF) is the distance objects seems to be sharp in a photo. So while “dof” is more measurable and “bokeh” subjective, one might say there’s more bokeh in picture with a shallower dof, because the background and foreground (if there’s a subject) is usually de-emphasized by being blurred when thrown out of focus.

Bokeh seems to be derived from the Japanese word “boke” 暈け – apart from the meaning blur, it might also mean senile, stupid, unaware, or clueless. This is interesting because in Singlish (Singapore’s flavor of English), it has that same negative meaning when referring “blur” to a person (it probably comes from the literal meaning of the opposite of sharp”. And now you might now know non graphical meaning of the word blur in my twitter id (BlurSpline).

IMG_8537
Here’s a photo of the Kinetic Rain I took at Changi Airport Terminal 1. Especially if you like kinetic structures, you should check out the official videos here and here (in which there’s much use of bokeh too)

I remember the time I knew little about 3d programming when I first tried three.js 2 years ago. I wondered whether camera.near and camera.far were the ways of defining when objects in the scene gets blurred when they are at distances at the far or near points.

Turns out of course that I was really wrong, since these values are used for clipping – improving performance by not rendering objects out of the view port. I had that naive thinking three.js work like real life cameras that I was able to create cinematic like scenes. Some helpful one on three.js IRC channel then pointed me to the post-processing DOF example done by alteredqualia who ported the original bokeh shader written by Martins Upitis.

So fast forward to the present, we have seen that shader used in ROME, and Martins Upitis has updated his Bokeh Shader to make it more realistic, and I attempted to port it back to three.js/webgl.


With focus debug turned on


Testing it in a scene


The example added to three.js with glitters.

So to copy what martinsh say the new shader does, it has the flexibility to
• variable sample count to increase quality/performance
• option to blur depth buffer to reduce hard edges
• option to dither the samples with noise or pattern
• bokeh chromatic aberration/fringing
• bokeh bias to bring out bokeh edges
• image thresholding to bring out highlights when image is out of focus
• pentagonal bokeh shape (experimental)
• bokeh vignetting at screen edges

The new three.js example also demonstrates how object picking can be used and interpolated for the focal distance too. More detailed comments about the parameters were also written on github.

Of course the shader is not perfect, as DOF is something not that simple (there are quite a few in depth Graphics Gems articles on it). Much of it is post-processing smoke and mirrors, the way is usually done in rasterization, compared to Path tracing or so. Yet I think its great addition to have in WebGL, just as we have seen DOF used in the Crytech, Nvidia demos or in other high end games. (There was a also a cool video of a minecraft mod using that DOF shader – but now seemed removed as I recently looked for it).

I would love to see the tasteful use of bokeh sometime, not just because it feels cinematic or been been widely used in photography, i think its also more natural given that’s how our eyes work with our brains (more details here).

Finally it seems that the deadline for the current js1k contest is just hours away – this means I gotta head off to do some cranking and crunching, maybe more on that in a later post! :D

Traveling with Animal Friends Around the World

Posted on by

Its a new year! Oh what blatant lie but still, its my first post for 2013!

You know, the feeling I’m feeling is that that feeling when there’s so much to do, even more that you wish to do, but so little that you really do.

Since resuming blogging has this great inertial, I decided to share some of travel adventures around the world (LA, Japan, Taiwan) with my animal friends (guess who they are!), before writing on more technical stuff.

I guess it all started on a trip to about 4 years ago. Then I was working in the San Francisco bay are then, and with a cheap pair of air tickets, I flew down to Los Angeles for a quick get away alone, until I met Piglet.

LA’s metro

Downtown LA


Walt Disney Concert Hall


Aquarium of the Pacific


Piglet and Jellyfish


Then it was the period of H1N1 Swine influenza, and so the saying that “pigs fly” became true.

Next stop, Japan with Doraemon, the Japanese Robotic cat from the future.


Doraemon on the plane


Doraemon in Tokyo’s JR train


Doraemon and the crowded Kaminarimon in Asakusa


Doraemon in Akihabara Electric Town


Doraemon watches sunrise at Atami


Doraemon and the Shinkansen


Doraemon and a big Rubik’s cube in Osaka


Doraemon in Kaleidoscope in Osaka’s Science Museum


Doraemon goes Kyoto cycling

Doraemon at the Ryōan-ji, a Zen temple famous for its dry garden. Here Doraemon is enlightened that you need to mediate about nothing to be enlightened about nothing.


At the beautiful Hirosawa lake


Doraemon in Arashiyama bamboo forest


The skies tear as Doraemon waves goodbye to Japan

Now for the next destination, Taiwan.

Angry bird at the Sun Moon lake.


At the beautiful mountain side of Cingjing.


At the sheep farm


At Taroko Gorge (thanks to my friend yeda for the correction from “Taroko” :)


Throwing Angry bird in a cable car up Maokong, Taipei.


Angry bird hikes up Jinguashi.


Angry Bird meets his enemies, at the local pig farm.


Angry bird launches into space, returning to my home in Singapore.

Where to, who shall I go with next? Thanks for reading, and hope you enjoyed this little adventure.

THE END

p.s. a technical bit – this is the first time I experimented writing this post with zenpen. Its a pretty cool tool (Since I usually post my photos on facebook, I just drag them in). I then used $(‘section’).innerHTML to extract the html code, clean some attributes before pasting into my blog.

JSCampAsia

Posted on by

Its been an awesome 2 days at JsCampAsia – great talks, great people, fun events.


Group picture from above!

It was about 2 years ago when I first watch Ryan Dahl’s node.js presentation at jsconf online. I subsequently found other quality jsconf talks so when I first learnt that JsCampAsia was going to be held in Singapore, I decided it was an event I should attend.

When Thomas Gorissen the JSCamp organizer asked if would like to conduct a workshop on three.js, I was really excited and agreed. So I learnt Ricardo/@mrdoob had mentioned me earlier at JSConf.EU, and it was a pity he couldn’t come as he was attending dotJs in Paris.


Mozilla’s Michal Budzynski asking Google’s Eric Bidelman a question. JSCamp’s Thomas on the right

Its my first time giving a talk at such scale, so its really nice to have some attendees came to me telling me that they have found have enjoyed the workshop or have found three.js interesting. (I’m sure no one was nasty enough to tell me if I was bad, so I’ll probably do some self criticism when I grab a copy of my video)


With one of the JSCamps’s helpers

Apart from the excitement of speaking, it just pleasurable to listen to the talks to the speakers coming from around the world – some really interesting, some informative, some engaging, enjoyable – and some I’ve missed out, I’ll be will catching them once the videos get released.


Of the best loved talks about replacing html & css with js by Jed Schmidt

So the last thing awesome about JSCamp is about meeting people, although some really briefly. To the audiences who said hi, thanks or even asking questions, I’m thankful and appreciate that. In turn, I probably have asked other speakers some difficult or silly questions.

On returning home (at the northern end of this small island), my body decided it deserved a day of hibernation.

Overall, JSCampAsia was a great conference, and to be a small part of it is my honor and pleasure. Hope to see you all again!

Some links
Official Blog – http://blog.jscamp.asia/
Collection of Links – https://gist.github.com/4167535

Handout for 1st session – http://zop.im/start-threejs
Slides for 1st session – http://zop.im/start-threejs-slides
Sample code – http://zop.im/start-threejs-code
Slides for 2nd session – http://zop.im/start-threejs2

Some photos


Speakers lunch


A picture of the conference hall


Jan from Amsterdam who spoke on dependency injection in “The Architect Way” on our way back.


With my colleagues from zopim who attended this conference too


Only bad thing about seating in the front is the “giraffe neck”.


Panel discussion – pretty sure @divya isn’t looking at her mobile phone this time.


This is me on my 2nd session showing a visualization of a photo with three.js


Everyone getting ready for the group photo!


Everyone with my Jelly Bean’s panorama shot.


Shim Sangmin from Korea, creator of Collie . He has great slides on High Performance Mobile Web Game Development in HTML5. He bought lots of Kaya jam home too :)

Making of Boids and Buildings

Posted on by

Here’s my latest three.js webgl experiment called “Boids and Buildings”. Its an experimental 3d procedural city generator that will run in a form of a short demo. Turn up your volume and be prepared to overclock your CPU (& GPUs) a little.

Here’s the story – early last month I was in Japan for a little travelling. Perhaps one thing I found interesting was the architectural. The cities are large and are packed with buildings (Edo, now called Tokyo, was once the world’s largest city).


(the huge billboards are at the famous dotonbori in osaka, the bottom 2 photos were houses shot in kyoto).

Sometime ago, I saw mrdoob’s procedural city generator. We have thought about creating a 3d version of it, but only during the trip I decided I should try working on it. Some of the process was written in this Google+ post, but I’ll summarize it a little here.

Firstly, the city generator works by building a road which spins off more roads along the way. The roads stop when it intersects with another road or reaches the boundary of the land. Another way of looking at this is that the land is split into many “small lands” divided by the roads. My approach was that if I could extract the shape information of each divided land, I could run ExtrudeGeometry to create buildings to fill the shape of each land.

The original js implementation of road intersections was done looking up pixel data on the canvas. While I managed to write a version which detects faces base on pixels, detecting edges and vertices was more tedious than I thought, as if I could write or use another image processing library similar to potrace. Instead of doing that, I work on an alternative intersection detection based on mathematically calculating whether each all lines/road intersected. This is probably slower when it takes up too much memory, but points of intersections were retained.


(here this version denotes where the starting and ending points of each road with red and green dots).

However some serious information is still missing – which edges connects the points, and which edges belongs to each face/land. Instead of determining these information after processing everything, using half-edge data structures can elegantly compute them on the fly. Demoscene coders might have seen this technique for mesh generation.

Without going into an in-depth technical explanation, here’s an analogy to how half-edges is employed. Since every land is defined by the roads surrounding it, and you build fences around the perimeter of the land to enclose the area within. The enclosed area defines each face and each fence is like a half-edge. Since each road divides the land into 2 sides (left and right), 2 fences will be constructed and its denote the land belonging to both sides. If a road is build through an existing land, fences on the existing land has to be broken down to connect to each side of the new road fences. In code, each new road contains 2 half-edges, and connecting new roads requires creating new split edges and updating of linked half edge pointers.

With that a 2D version…
2d generator

… and a 3D version was done.
3D

(on some hindsight, it could be possible to use half-edges with the original pixel collision detection)

Now we’ve got 3d buildings, but it lacks the emotions and feelings that I was initially thinking of. I thought, perhaps I’ll need to simulate the view out of the train. But that might require me to simulate train rails, although the stored paths of roads could be used for the procedural motion. As I was already started to feel weary of this whole experiment, I had another idea – since there’s a examples of boids in three.js, why not try attaching a camera to the boid, not only you’ll get a bird’s eye view, but you’ll get camera animation for free! I did a quick mesh up and the effects seems to be rather pleasant.

Incidentally, mrdoob in his code named the roads Boids, and I thought the name “Boids and Buildings” would be a rather fitting theme.

Let me start jumping around on bits and pieces I can think of to write of.

Splash screen
I abstracted the map generator into a class call Land which takes in parameters, and it was reusable for the 2d and 3d version. In the splash screen, a css3d was used to translate and scale the map generator in the background.

Text fonts
I wanted to create some text animation in the splash page, with a little of the boid/map generation feeling. I use mrdoob’s line font used in Aaronetrope and experimented with various text animations.

Visual Direction
To simplify working with colors, and used random grey for buildings at the start and ended up with a “black&white” / greyscale direction. For post processing, I added just a film post-processing shader by alteredqualia found in three.js examples, with slightly high amount of noise.

Scene control and animation
Since most of the animation was procedural, so during development, much of it was by code. When it was time to have some timeline control, I used Director.js which I wrote for “itcameupon”. The Director class schedules time-based actions and tweens, and most of the random snippet of animation code was added to it. So more or less, the animation runs on a fix schedule except for randomized parts (eg. time when roads just building on lands).

Pseudorandomness
This experiment allowed me to deal with lot of randomness, but based on probability distribution, lots of randomness can actually give expect results. I used this fact to help in debugging too. For example, you would like quickly to dump values out to the control in the render loop but you’re afraid of crashing your devtools – you could do this (Math.random()<0.1) && console.log(bla); This means, take a sample of 10% of the values and spit out the results out to the console. If you are in Canary, you can even do (Math.random()<0.01) && console.clear(); to clear your debug messages ones in a while.

Buildings
Buildings height are randomized, but they follow certain rules to make it more realistic and cityscape like. If the area of land is big, the building height would be lower. If its a small area but not too small, then it could be a skyscraper.

Boid cams
The boid camera was simply to follow the first boid, based on its velocity, placed the camera position slightly higher and behind the bird. I wanted to try a spring-damper camera system but opt for this simpler implementation instead – move the camera, a factor k position closer to the target position every render. In simple code, targetX = (targetX – currentX) * k where k is a small factor eg. 0.1 – this creates a cheap damping/easing effect. This effect is apparent toward the end of the animation when the camera is slingshot to the birds as the camera mode changes to the boidcams.

Performance Hacks

Shape triangulation is probably one of the most expensive operations here. In “It came upon”, slow computers experienced a short sharp delay when text get triangulated the first time (before caching). Over here, its a greater problem due to potentially high number of triangulations. Apart from placing a limits to buildings, one way to prevent a big noticeable lag is to use web workers. However I opt to use the old trick of using setTimeouts instead. lets say there are 100 buildings to triangulate, instead of doing everything inside one event loop which would bring a big pause, I’ll do setTimeout(buildBuilding, Math.random() * 5000); – based on random probability, 100 buildings would be triangulated across 5 seconds, reducing the noticeable pauses. I supposed, this is somewhat like an incremental garbage collection technique newer javascript engines employ.

Another thing I did was to disable matrix calculations, using object.matrixAutoUpdate = false; once buildings are completed building and animating.

Music
Pretty awesome music from Walid Feghali. Nothing much else to add.

Audio
Added my Web Audio API experiment of creating wind sound to simulate wind sound during the boidcams. Wind can simply be created with random samples (white noise?), I added a lowpass filter and a delay. Aptitude of wind noise is generated by the vertical direction boids are flying at. Wanted to add a Low Frequency Oscillator to make more realistic sounding wind, but I haven’t figured that out.

Future Enhancements
Yes, there’s always inperfections. Edge handling could be better, same with triangulation. Boids could do building collision detection. Improve the wind algorithm. Create variety of buildings, or making some cool collapsing buildings (like in inception).

Conclusion
So the demo is online at http://jabtunes.com/labs/boidsnbuildings/, sources are unminified, you’re welcome to poke around.

Going back to the original idea, I still wonder if I managed to create the mood and feelings I originally thought in my head. Here are some photos I shot overseeing Osaka in Japan, maybe you can compared it with the demo and make a judgement – perhaps you might think the demo is closer to a Resident Evil scene instead. ;P

edit: after watching this short documentary on Hashima island (where a scene from skyfall was at), i think the demo could resemble hashima’s torn buildings too.




Nucleal, The Exploding Photo Experiment

Posted on by

Particles. Photos. Exploding motions. The outcome of experimentation of more particles done this year. Check out http://nucleal.com/


This might probably look better in motion

Without boring you with large amount of text, perhaps some pictures to help do some talking.

First you get to chose how much particles to run.


Most decent computers could do 262144 particles easily, even my 3 generation old 11″ macbook air can run 1 or 4 millions particles.

On the top bar, you get some options on how you may interact with the particles, or select which photo albums if connect to facebook.

At the bottom is a film strip which helps you view and select photos from the photo album.

Of course at the middle you view the photo particles. A trackball camera is used, so you could control the camera with the different buttons of your mouse, or press A, S or D while moving your mouse.

Instead of arranging the particles in a plane like a photo, you could assemble them as a sphere, cone, or even a “supershape”.

Static shapes by itself aren’t good, so physics forces could be applied the particle formation.

Instead of the default noise wave, you can use 3 invisible orbs to attract particles with intensity relating to its individual colors

Or do the opposite of attracting, repelling

My favorite physics motion is the “air brakes”.

This slows and stops the particles in their tracks, allow you to observe something like a “bullet time”.

While not every combinations always look good, it’s pretty fun to see how what the particles form after sometime, especially with air brakes between different combinations.

Oh btw noticed the colors are kind of greyscale? That’s the vintage photo effect I’m applying in real time to the photos.

And for the other photo effect I added, a cross-processing filter.

(this btw is my nephew, who allows me to spend less attention and time on twitter these days:)

So hopefully I’ve given you a satisfying walk-through of the Nucleal experiment, at least way simpler than the massive Higgs boson “god” particle experiment.

Of course some elements of this experiments are also not entirely new.

Before this myself have also enjoyed the particle experiments of
- Kris Temmerman’s blowing up images
- Ronny Welter’s video animation modification
- Paul Lewis’s Photo Particles

Difference is now that brilliant idea of using photos to spawn particles can reach a new level of interactivity and particles massiveness, all done in the browser.

While I’m happy with the results, this is just the tip of the iceberg. Since this being an experiment, there’s much room for improvements in both artistic and technical areas.

A big thank you again to those involved in three.js, for which this was built on, to those adventurous who have explored GPGPU/FBO particles before me, to those who blog and share their knowledge about GLSL and graphics for which much knowledge was absorb to assemble this together, and not the least to Yvo Schaap who support me and this project.

Thank you also to others who are encouraging and make the internet a better place.

p.s. this is largely a non-technical post right? Stay tune for perhaps a more in-depth technical post about this experiment :)

Virtual Rendering The Million Items Roundup

Posted on by

Some days I’m unproductive, some other days I explore, discover random stuff too quickly that I easily forget stuff I do. So in this post, I intent to round up the experiment started in my previous post of “Virtual Rendering 1,000,000 Items Efficiently”.


The Canvas Approach

In the last post, I showed how one could implement a virtual scroll bar and the display of 1,000,000 row items in a virtual scrollpane quickly. That was done by creating dom elements when they get scrolled in the view and removing them and they get scrolled out of view. In this post, I would talk about how the HTML5 Canvas element could be used instead for rendering the contents. This would not be about which approach is superior, as both have its pros and cons, but I’ll just note down aloud some thoughts I observed in Canvas implementation and the differences of both approaches.


Can you spot the visual differences?

Since this prototype hack was created quickly on Chrome, it has probably been tested in these browsers, still different versions, differenent computers and different OS typically give different results.

- most computers typically redraw the scrollpanes less than 20ms (and under 10ms most of the time), which I think is a satisfying result.
- both seems to be performing on par in the above comparison jsdo.it fiddle.
- The Canvas approach is typically much faster over the DOM approach when the scroll viewport is huge – eg. 1600×900 as the DOM approach seems to slow down more considerably in the bigger view port.
- There are certain things much simpler to implement in DOM vs in Canvas. For example, dashed borders could be written in a line of css, but the Canvas approach require some custom coding esp when dashed line drawing hasn’t been supported in the Canvas API.
- In implementing custom code to mimic the dashed borders, the Canvas rendering slows down to.
- At first, I was making these expensive calls, because paint calls were made after every dash stroke (eg. lineTo for a 3px line, stroke, move to a 3px gap)

	//After every row
	var ww = 0;
	while(ww < w ) {
		ctx.beginPath();
		ctx.moveTo(ww,~~item.y + 0.5);
		ctx.lineTo(ww+3,~~item.y + 0.5);
		ctx.closePath();
		ctx.stroke();
		ww+=6;
	}

- after I discovered it would be more efficient doing it in this manner.

	//After every row
	var ww = 0;
	ctx.beginPath();
	while(ww < w ) {
		ctx.moveTo(ww,~~item.y + 0.5);
		ctx.lineTo(ww+3,~~item.y + 0.5);
		ww+=6;
	}
	ctx.closePath();
	ctx.stroke();

- Here stroke calls are batched, rendering speeds up, but falls behind the browser native css rendering. An exception is that Chrome on windows seems to match both the CSS and custom rendering speeds pretty close.
- In canvas, an advantage might be that drawing custom objects would be one step easier, compared to creating new dom objects and placing custom objects in there.
- Some rendering done in canvas could be quicker than using complex css rules which the browser might requires reflow calculations.
- Using a canvas approach, one might find sub-pixels or anti-aliasing issues. For drawing a 1 pixel line cleanly on Canvas might have to resort to tricks like ~~(y_position) + 0.5;

Its interesting making things work in both approach, but as seen above, there are too many variables and this is not intended to be performance comparison test, but a documentation of observations. Depending on one’s use case, either approach might fit the developer better.

Working with Canvas brings many possibilities but brings many challenges at the same time. While there were many interesting projects that used Canvas for rendering, some didn’t have good endings.

Examples
1. Google Wave – I believed they used canvas for rendering the custom elements, scrollbars, cursors, etc . Those who have used Google wave might have already waved it goodbye.
2. Mozilla Thunderhead – This was the custom library used to developed Mozilla Bespin, a realtime collaborative editing and development platform. Many of its UI components were Canvas drawn – esp for handling its text editor, showing multiple cursors while multiple users were editing collaboratively. Sadly, the developers left mozilla, Bespin became Skywriter, and now uses the ACE library (more DOM based) which is also used in the cloud9 IDE.
3. Blossom – An RIA library which attempts to use HTML5 canvas for rendering. This is another library spin off from the Sproutecore library, and while this library seems to have some potential, it also seems to be at its infancy.
4. xgui.js – Certainly it doesn’t mean a gloomy end for custom Canvas UI components, and I think xgui.js, a handly controls gui , is a good example that useful and innovative libraries can be built on the 2D Canvas.

So, whether is DOM, CSS, Canvas, there are practical use cases and interesting possibilities. I’ll pause this experimentation at this point, but when I have the need for some huge data and virtual scrolling, I’ll be back.

Virtual Rendering 1,000,000 Items Efficiently

Posted on by

You probably haven’t heard from me for a while, and probably some reasons, and one could well be attributed to my dark periods of un-productivity.


Link to the Demo. Recommended that you’re run it with Chrome.

But if you are hearing from me now, that’s because there’s some progress I would like to share on a small experiment, which is an attempt to render large amount of data using html, javascript in the browser efficiently.

Here’s the problem: Let’s say you have a million rows of data, and to simply create a million Divs and placing them to your html document’s body is a good way to freeze or crash your browser, that’s after chewing up large amount of RAM and CPU. That doesn’t only to browser applications, because many text editors are pretty incapable of opening large files.

One example of how I encountered this problem was running three.js inspector with a particle scene, and I realized attempting that the couple thousand or even hundred elements representing them was locking up the browser. One motivation for this experiment was also to try creating new UI components for a more efficient three.js scene inspector.

Solution: Placing objects in memory are way faster than placed in the dom or rendered. In the case of huge list in a scroll panel, the trick is to hide all objects and render only the items in view. Though this isn’t super simple, it isn’t totally new, and a couple of good libraries have already utilized such techniques. CodeMirror, the extensible and powerful browser based code editor, uses handles large amount of code in this manner. Slickgrid, a powerful open source grid/spreadsheet library created by a Google employee, is built on this technique to allow large rows of data. Sites that implement the “infinity” scrolling interface (eg. Pinterest, Facebook timeline) utilizes a similar technique to reduce huge memory footprint (twitter, I’m looking at you).

Approach: So why did I try to do this myself? Similar to reasons why one would write a library is to have more understanding and control over your own code, while experimenting on stuff you otherwise wouldn’t have ever try. In this experiment, I got to attempt skinning my own scrollbar, as well as to experiment and benchmark a DOM versus a Canvas implementation of Virtual Rendering. In this post, we would look at the DOM approach, which is similar to how CodeMirror and Slickgrid do it.

If you take a look at the source, there 4 simple classes.

1. SimpleEvent
A minimalistic approach to do the observer patten in the style of Signals.

For example usage,

2. ScrollBar
This is a UI component that displays a scrolltrack and slider, that fires some scroll events when scrollbar is clicked or dragged via .onScroll. Its 2 public interfaces are .setLength() for defining the size of the slider block in percentage, while .setPosition() moves the slider according to the document viewport’s position by a fraction.

(See bottom for more UI/UX notes on the scrollbar)

3. RowItem
The row item is responsible for storing its own data and rendering its dom representation when it comes into view. Here, it simply stores a string for its text data.

For visual representation, its needs to store its x, y, width and height to be activated and positioned correctly for rendering.

4. ScrollPane
This is class which integrates all the above into this Virtual Rendering component. Firstly, it needs to be represented as a div element to be inserted to the dom. ScrollPane tracks the items, and keeps the total virtual area it contains. Based on its dimension, it calls ScrollPane to update its visual elements. The ScrollPane listens to its component for mousewheel events and listens to the ScrollBar for scroll events.

On any request to update its viewport, ScrollPane iterates over its item, quickly find which RowItem are in view and calls draw(). RowItem would create dom objects on demand, and position it. Upon another update, dom objects found visible in the previous render would be reused and those elements scrolled out of view would be remove from dom and destroyed to free memory.

Results: While this has only been tested and ran in Chrome, I’m pretty satisfied with the results of this hack. At a million objects (I’ve tested up to 6 million objects), scrolling is pretty responsive, initially loading it takes ~2 seconds to load, each repaint takes ~25ms, and the heap memory footprint stays below 100Mb. Apart from requiring more refactoring, i think its has quite minimal javascript and css.

Applications: Apart of my planned usage for a revamp three.js inspector, there are perhaps a few mini usages for this widget. For example, the developer’s console can choke up when displaying huge amount of data (now chrome devtools splits them up into small sub chunks, so it works but not the best way imho), and such could be a solution to this problem.

So hopefully this little experiment would find itself some nice usage. Until you hear from me again, continue reading for more side thoughts when I was playing around on the scrollbar. And the link to the demo if you didn’t see it earlier.

UI/UX notes on the scrollbar.
While working on the Scrollbar implementation, I observed that clicking in the track outside the slider has a same effect of “page up” and “page down” both in mac and windows. Instead of moving the slider to the clicked position (as I thought was the correct behavior initially), it fires onScroll event to a controller listener. While it seems that Mac Lion “un-obstructive” scrollbars are “in”, evidenced by sites like Facebook that imitate that, I’m however not a fan of it. Somehow I find those scrollbars ugly for applications that have a non-white background, so I’ve tried to style my scrollbars in the dark gradient colors after Sublime Editor instead.

Another challenge is that the slider would not be able to size proportionally as it would result in thinner than a pixel height for huge list. Therefore the scrollbar has an minimal size which makes it more practical for dragging. There’s probably more tweaks that can be done here, and there are alternative UI designs that eliminates the scrollbar in large lists (eg. your iphone contact list).

The Ascii Effect

Posted on by

Brief: Sometime ago, I decided to try what happens if the three.js renders ascii. Originally called the AsciiRenderer, it has been renamed to AsciiEffect and included with three.js examples.

**since this screenshot is displayed in ascii format**


                       *++==---::::::.                      
                    %**+++====--..                          
                  @%**++++-----.....                        
                 @@###++++------.....                       
                @@%###+++++-----:.....                      
               @@@####+++++-=====......     .               
               @@%%%%%*****=======......  ...               
               @@@%%%%******======-....::....               
               @@@%%%%******======+::::::....               
               @@@%%%%***###*++++++=:::::..--               
                @@@@@@@######+++++++::::=---                
                 @@@@@@#######++++++=====--                 
                  %%@@@@###%%@******+====*                  
                    %@@@@@@@@@@******###                    
                       %%%@@@@@%%%%%#                       
                     ..................                     
                    ....................                    
                  ........................                  
                 ..........................                 
                ............................                
               ..............................               
              ................................              
             ..................................             
                                                            
58 FPS (58-58)

Demo link: http://jabtunes.com/labs/3d/canvas_ascii_effects.html

I can’t really remember what triggered the thought process, but I thinking of that while being in the shower. Isn’t bathrooms quite a source of inspiration, like the Archimedes’ “Eureka!” story? While I’m not really an ASCII fanatic, I think I can appreciate some creative use of them. Some sites dedicated to these Ascii art are pretty interesting, such as 16 colors which utilises the javascript library escapes.js.

Some other ASCII effects I have found interesting includes a fluid solver, javascript raytracers, and a morphing animations! So much for the ASCII influence, that’s there even a text based category in the demoscene.

Moving to some implementation details:

1. JS Ascii Libraries
Originally I ambitiously wanted to complete the proof of concept in 15 minutes, so the first thing was to look existing ascii libraries in JS out there. Turns out there were a few libraries out there such as this and this, but I decided on nihilogic’s jsascii library which seems to provide a little more options despite being written a longer time before.

2. Extracting Image Data
Since jsascii and the similar libraries uses images as an input, the first integration approach extract the base64 encoded image data via the Canvas’s toDataURL() from CanvasRenderer, and import the image to the library and pull the Ascii output. Doesn’t take too long to realize its pretty silly wasting cycles when we could process the image data directly without encoding and decoding the image data by using canvas.getImageData().

So I started modifying jascii library to tap straight into the domElement of CanvasRenderer. I worked but performance was still horrible, so I being more tweakings, particularly the text resolution. By having a lower resolution, a larger font size would be use, resulting greater performance but less clarity. After making these adjustments, I found there certain ratios where there was good performance with reasonable clarity. In fact, the framerates were also on par or sometimes faster than rendering the canvas to screen. It ran well on Chrome, and ran even faster in Firefox.

3. Swapping WebGLRender with CanvasRenderer
The next little experiment was to use WebGLRenderer instead of CanvasRender for rasterizing the scene. Turns out that rendering ASCII with WebGLRenderer was slower than the CanvasRenderer. Alteredq suggest that it was probably more expensive to pull data out of the webgl context. Perhaps gl.readPixels might be a more efficient way of extracting data, but I haven’t tried that yet.

4. More Settings
Here’s another area to explore for the adventurous. JSAscii supports various options like enabling colors, inverting them, using blocks, and its pretty interesting to see some of these effects although some of these options have a big penalty on performance.

At the least, the reader might want to have some fun and try custom character sets. The ASCII library works by converting a pixel block into text by measuring its brightness and using an ascii character which represents that brightness. One could also try looking up and utilize unicode characters. In my experiments, I realized the use of even really simple pallets, for example one made of different sizes of dots, are interesting and effective too. (for example: start with something simple like ” .*%”)

5. Ascii Renderer -> Effects
In r49, the previously Anaglyph, Crosseyed, ParallaxBarrier Renderers were refactored to Effects, and are located in /examples/js/effects. AsciiRenderer follows the style and gets refactored to AsciiEffect. In r50dev, mrdoob works on several new software renderers, I did a couple of tests of the AsciiEffect with the new rasterizers, and they work too. Now in theory, with some modifications to the software renderer and ascii effect, one can effectively render a three.js scene in old non-canvas supported browsers, such as IE6, using ASCII output.

Finally, to throw out more ideas, one could create ASCII stereogram, combining the styles of Crosseyed and Ascii effect. ASCII stereograms may look like they would give a big headache, but they’re pretty cool!

Summing up, I hope the additional of the Ascii Effect provides a really simple way to do interesting 3d scenes and animation in Ascii, and probably a good way to relax when you get tired of writing glsl shaders.

Who knows if I’ll be writing up more text experiments again, but till then, it’d be interesting what else you may come up with!

ps. on a somewhat unrelated theme, I recently came across the demoscene production “the Butterfly Effect” by ASD and turns out that its a demo I really like…

Cool Street View Projects

Posted on by

Haven’t tried Google Street Views? You should! That’s a practical way to teleport and also a nice way to travel around the world virtually for free.

[asides: if you're already a street view fan, did you tried that 8-bit version? and if you've tried that, what about google's version of photosynth?]

Let me highlight a couple of innovative projects that came to my attention utilizing street views.

1. cinemascapes - street view photography

beautiful images (with post processing of course) captured using street views around the world. take a look.

[asides:
9 eyes has a similar approach, with a different style. street view fun is kind of the paparazzi version. so do you really think your iphone is the best street photography camera?
]

2. Address Is Approximate – stop-motion video

Creative and beatiful stop motion film using street views. Watch it!! (some may be reminded of the cinematography in Limitless)

[asides:
"Address is Approximate" was brilliantly executed but there's no harm to trying it yourself. like this]

3.Chemin Vert – interactive 360 video

Look around while being on a train travelling at 1500km/h, across five continents and four seasons. You need webgl + and a decent pc + gpu for this. Try the “HI-FI” version if you have lots of patience waiting for the video to load, and for your browser to crash. And the vimeo version if you like the little planets projections.

And if you’re are like me, watching the (un)projected video source is great enough. hifi version

[asides: though, the first time i saw this technique being employed was in thewildernessdowntown, which i think mrdoob and thespite worked on, see next point]

4. thespite’s Experiments – interactive meshups

Cool experiments by thespite shows that he has been doing such street view meshups with three.js for some while now. What’s better now is that he has just released his Street Views panorama library on github! Fork it!

[asides: thespite's a nice guy who gave me the permission to use GSVPano.js even before releasing it]

5. Honorable mention – stereographic street views

Stereographic Streetviews is an open source project which produces stereographic (aka “little planets”) views. Uses the power of graphic cards for processing, and allows you to easily customize the shader for additional post-processing.

[asides: some of the post-processing filters are quite cool. and some have others have also created videos with this]

So why I’m writing this? Course I think they are brilliant ideas which I could have thought of but would never get to execute them (like a meshup of GSVPano with renderflies).

Anything else I’m missing? Otherwise, hope to see great new street view projects and I’ll try working on mine. Have a good time on the streets!

Spline Extrusions, Tubes and Knots of Sorts

Posted on by

The recent development in three.js up to the recent 49th release has been really crazy – just look at the changelog! Some work that made it in were 3d spline extrusions, which you could follow the development reading the thread in issue #905. @WestLangley‘s involvement was also a great help!

3d spline extrusion examples

[TL;DR If this post seems too boring, or just too long (as i'm trying to release some air out of my head), feel free to skip everything but try the example. You might also like to turn on camera spline animation, create your own torus knot, or create a custom tube by writing a formula there]

View tube/spline extrusion example.

Related work (spline curves, shapes extrusion geometry, text bending) were previously mentioned, but extrusion geometry did not handle 3d spline extrusion well, so the discussion brought out new features to be implemented after being left in TODO for a couple of months. The work resulted with couples of jsfiddles, created new classes like THREE.ArrowHelper, THREE.TubeGeometry, improvements to some old classes and created a couple of new examples. Read on if you like to know what went through the thought/development process back in time.

Extruding a spline != Extruding on the Z-axis

I decided to tackle this issue one night, since I barely recall the internals of ExtrudeGeometry after leaving of quite awhile, so I tried to recreate a simple version. The first approach to take the points of the spline – which spline.getPoint(), then recreate the cross-section of the shape. Looks almost like it’d work, until you the spline move vertically up the y axis and the geometry falls apart. Ohoh. WestLangley reminds me that the geometry is not orthogonal – which refers to angles kind of perpendicular or 90 degrees from the tangent. His suggestion to modify the TorusKnotGeometry (done by @oosmoxiecode) to create a TubeGeometry is a wise choice.

Not a very correct path extrusion

Normal of a 2D line != Normal of a 3D line

Ok, I recalled having to deal with a path normals in the text bending implementation. That was done by getting the tangent of the path, and invert the vector like -y / x. I thought surely there’s a formula to do the same in 3D. Looking up the wikipedia topics of normals, there were formulas for normal to a plane, normals to a face, but no, there aren’t any formulas for a vector in space. And so I begun to understand and as @profound7 reminded me that there are just infinite amount of normals to a line in space.

There are however a set of formulas in tackling this issue, as @miningold brought up, the the Frenet–Serret formulas, and that became another topic in math for me to study. From the simple understanding I have now, the Frenet–Serret or TNB frames are like your thumb and 2 fingers in either the left hand (or right hand) rule. Each direction represents the tangent, normal and binormal which are 90 degrees / orthogonally apart.

Sounds simple? Could be, but the ill-rewards of not being a diligent student in school is having to now stare painfully at this paper “Tubes and the TNB frame”. Perhaps after damaged brain cells and hairs interpreting those seemingly foreign mathematica symbols, I thought I could try explain these formulas in my own understanding.

Frenet–Serret

Frenet–Serret Formulas for dummies like zz85

Spline = an imaginary line in space which a point will travel from one end to another end over time.

Tangent = Is the change of position at a particular point of the spline. This the first derivative of position over time. Now what we need to store is simply its unit vector, by dividing its length, so its magnitude is equals to 1, since we only need its direction, not magnitude.

Normal = Change of tangent unit at a particular point / time of the spline. I was wondering for a while why this is not the 2nd derivative of the position over time – and I realized that that’s because T is a unit vector rather its change in position.

Binormal = By doing a cross multiplication of T and N, you will get a binormal perpendicular to both vectors.

Frenet–Serret frames != Magic bullet for spline extrusion

Yeah, with Frenet–Serret formulas, that means I can just plug in a formula to get the normals, and it would be simple to implement the tube geometry? Not so soon again, because I soon hit the situation that miningold was facing earlier – there were some really ugly wraps in geometry. This call for some visual debugging of the geometries normals – WestLangley was doing something similar and so we thought these helpers would be useful. Those got refactored later to THREE.ArrowHelper.

Even with Frenet–Serret formulas, fail.

What happened? Because of inflections in portions of a cat-mull rom spline, the normals may flip around really unexpectedly rapidly. On the positive side, that seems like that’s a well known problem if you search on the net. From this link from CMU provides a pretty simple and usable solution, taking the binormal of the previous segment to compute the normals and binormals for the current ones, which is the moving frame approach. With that, we could already start to implement some fanciful geometry.

Heart Tube

Moving Frames != Ending Frames

The next problem was then brought up, that is normals of the ending and starting normals of closed tubes do not match. That could be quite obvious if the radius segments were few, and joining seam would have an ugly closure. I found links to some papers on RMF Rotation Minimizing Frames (RMF) – a method which double reflects, I guess for comparing rotation changes before and after segments, in order to process the frames correctly. While I didn’t manage to understand all these, WestLangley wrote the implementation for the Parallel Transport Frame approach, making sure the tube is slowly twisting so that the starting and ending binormals. Yeah!

To test out the new TubeGeometry, I started looking for some spline formulas. I also started looking into knots because @mrdoob proposed the possibility to implementing the TorusKnot with TubeGeometry. With a little practice, you would find a defining a curve/spline object using the method THREE.Curve.create() really easy, and I also started appreciating some beauty of mathematics and formulas, in that you could create beautiful, and even complex geometries with really simple code.

Decorated Knots

If you look into curve extras source, there’s a tiny compilation of links to some pretty good resources on curves and knots. However, one resource that particularly interest me was a university paper on decorated knots. By applying certain pattern of formulas to knots, you would get interesting and beautiful geometries. I have used some of the formulas in the spline extrusion example and you could easily use those formulas too.

Decorated Knots

Real-time Knots
So if you haven’t realized, three.js is versatile and powerful enough to be a 3d/math graphing software. Many of the existing sites on knots and curves would require you to download some graphing/knotting software, or at best uses some Java applets. So no longer do you need download any stuff or plugins, you can run that in your browser. In my example, you can create your own torus-knot by defining your p,q,r parameters, or even write an entire new curve by formulas. Well, it seems that Google’s have also thought along the same lines, having enabled a webgl 3d graph plotter in their search engine (Rose Example). Probably a good time now that SwiftShader (webgl software fallback) is getting activated in Chrome too.

Real-time Knots

Camera Movements
On turning on the spline animation, you would kind of follow the direction of the spline from a distance above it. The camera orientation is stabilized using the binormals of the same frames generated for the TubeGeometry. When I started experimenting with it, the simplest way would just to rotate the camera in the direction of the spline tangents. It probably works, but with a side effect, the control of the camera’s spin may be lost. Perhaps not too bad an idea, if you want to create a dizzy effect or re-enact Joseph Kittinger’s not so successful jump (before his world famous record breaking jump). And the “Look Ahead” option kind of gives a different feel by fixing the camera view on a point, a distance away on the spline. Scaling up the geometry while on the spline camera works too.

Camera Movements

What’s next?
ExtrudeGeometry has extended a similar kind of 3d extrusion used in TubeGeometry. It can probably be more refined for its purpose. There has also been some work on ParametricGeometry, which could be used for an internal TubeGeometry refactoring. I’m planning to work on a little three.js DIY rollercoaster experiment which would probably be a good use case for testing and refining all the work here – and a 3D spline editor to come with that (since there’s already some form of 2D spline editor)

Not enough curves?
Oh, and if you are really interested in mathematics, curves and splines, I stumbled upon this really interesting lecture on this topic, touching on the practicality of curves in buildings, bridges and even roller coasters. (the video lecture is not in a very high quality, but you can also download the lecture slides and audio). Have fun!

(ps. today’s three.js 2nd anniversary according to mrdoob. initially thought i could write another post to commemorate this day, oh wells… happy three.js day!)