Archive for the 'Interesting Stuff' Category

Rain, Water, Ripples with HTML Canvas, Javascript, JQuery

One of the exciting reasons to work with canvas is that you are not only able to create 2d graphics easily with javascript, one can easily do do pixel manipulation on images.

Think about it, in case you don’t think its cool right now, is that you only need notepad, a recent web browser and a few lines of code to do cool stuff work with 2d graphics without compilation. Take an example, you can create a greyscale image from a color photo into without photoshop or other software as see in pixastic.

In this post, I will outline how easy it is to code with javascript and canvas to simulate water ripples, along with some issues encountered. After I discovered this old water algorithm (during the DOS era I suspect) on creating waves, I thought why not try this with HTML Canvas, and so I did.

Ripples with Javascript

STEP 1. Creating a canvas element.
This can created easily with javascript, or just add some html code into the body

<canvas id="jripples"></canvas>

STEP 2. Load an image with JS when document is loaded for which we can use the jquery $(document).ready

$(document).ready(function(){
init(); // Run stuff when document is ready.
});

Here’s the code to load an image


var img = new Image();
img.src = "hello.jpg" // path and filename of image
img.onload = function() {} // run the next portion of code when the image loads

STEP 3. Draw the image onto the canvas


canvas = document.getElementById('jripples');
$('#jripples').width(img.width); // We set the canvas width and height with jquery
$('#jripples').height(img.height);
canvas.style.height = canvas.height = img.height ; // Set the canvas size to the image
canvas.style.width = canvas.width =img.width;

ctx = canvas.getContext("2d"); // Get the 2d drawing context
ctx.clearRect(0,0,canvas.width,canvas.height); // Clear the canvas
ctx.drawImage(img, 0,0,canvas.width,canvas.height); // Draw the image

Just a caveat though, to get or manipulate pixels from an image, the image needs to be loaded from the domain or would result in a browser security violation.

STEP 4. Build the data structures behind the wave
orginalData = ctx.getImageData(0,0,canvas.width,canvas.height).data; // The array of pixels for the image
myImageData = ctx.getImageData(0,0,canvas.width,canvas.height); // Modifications of pixels
buffer1 = []; // Create new array for the image buffer
buffer2 = [];
// orginal data length is 4 times the pixel dimensions since they store RGBA (red,green, blue, alpha)
for (var i=0; i buffer1[i] = 0;
buffer2[i] = 0;
}

STEP 5. Creating ripples (the wave algorithm) function

function processWater(source, dest) {
for (var i=imagewidth; i< source.length-imagewidth; i++)
{
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

dest[i] = (
((source[i-1]+
source[i+1]+
source[i-imagewidth]+
source[i+imagewidth]) >>1) ) -dest[i];

dest[i] -= (dest[i] >> 5); // Damping - Quick divde by 32 (5 bits)

}
}

while the original article explains how this works, this article explains it in another simpler manner

Two height maps are used to store the current and previous states of the water.
Each frame you will toggle between state maps.
For each array element in the current state array:-
Look at the neighbouring pixels from the previous state, i.e. above, below, left and right. Take the sum and divide by 2. Because we are dividing by 2 a right-shift will work beautifully.
Now subtract the value in the current state map.
If we left it like that the ripples would never subside so we need to diminish the strength of the ripple every pass. The most realistic way of doing this is to reduce the resulting height by a fraction of itself. Once again we can use right-shift to optimise this.

Note that I ensured the loops avoid any edge bound pixel to prevent wrap arounds.

STEP 6. Render the new image with waves.
function texture(buffer) {
var xoffset, yoffset;
for (var i=imagewidth; i {
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

xoffset = buffer[i-1] - buffer[i+1];
yoffset = buffer[i-imagewidth] - buffer [i+imagewidth];

var offset = i+xoffset+yoffset*imagewidth;

if (offset>0 && offset

for (var x=0;x<3;x++) { //4 for alpha
myImageData.data[i*4+x] = orginalData[offset*4+x];
}

}

}
// Draw
ctx.putImageData(myImageData, 0, 0);

}

This estimates the pixel offset due to refraction of water (read more on Transparent Surface Ray-Tracing) based on height of the wave and copies the pixels from the orginal array into the new imagearray object. This imagedata is returned to the 2d object using putImageData to be drawn.

STEP 7. Call the rippling function in a timer
rippling = setInterval(ripple, 50); // This calls the ripple function every 20 times a second

STEP 8
We can calling another timer to add rain
raining = setInterval(function() { // random rain
var randomX = Math.round(Math.random() * canvas.width);
var randomY = Math.round(Math.random() * canvas.height);
buffer1[randomY*imagewidth+randomX] += Math.round( Math.random()*-500);
},300);

STEP 9. Add more raindrops/ripples reactivity to mouse clicks. We use jquery to add mouse listeners quickly.
$(canvas).mousedown(function(e) {
var x = e.pageX - $(this).offset().left;
var y = e.pageY - $(this).offset().top ;

buffer1[y*imagewidth+x] = -400;
clicked.dragged= true;

}).

STEP 10. Provides FPS(Frames per second)
setInterval(function() {
$('#jdebug').html(fpsCount + " fps");
fpsCount= 0; // recent
},1000);

This isn’t the most accurate way to calculate FPS, since the callback may not be accurate in itself. Rather, one should try getting the actual time subtracted from the last elapsed time to calculate the fps.

Enjoy. Here’s the link to javascript ripples demo
Remember that you may need to upload your html and images to a webserver to test (or change your browser security settings).

Other Considerations.
This run on all modern browsers (Firefox, Safari, Chrome, Opera) except IE due to the lack of support with Canvas. While the Excanvas provides some compatibility, it does not provides get/putImageData calls. Until IE9 releases, flashcanvas pro may solve this compatibility issue.

From my observations, the bottleneck seems to be when the browser render the image after the putImageData javascript. Right now firefox seems to be able to render 2x faster than chrome, and the self proclaimed fastest browser on earth, Opera, seems to renders about 3 times faster than chrome (to 100fps on small images).

While this wave algorithm may be efficient when there are many waves, this method may not be as efficient when there are few waves on large images/canvases. Even a image of 640 pixels can hang the browser (be warned).

One simple way to speed up the waves is scale down the image. If putImageData is the bottleneck, then we can hope that the browser would optimize these soon. (with Direct2D just implemented for firefox nighties, maybe soon will be soon!) Other interesting alternatives would be to implement WEBGL or O3D which uses the C++ stack or GPU to speed up the rendering, although these are not widely supported right now. The better idea, I imagine, is to implement another algorithm which renders the ripples from its wave source, which would probably run faster on large images with few waves.

Finally, creating ripples seems to be a classic challenge and if you were to do a simple search, would be able to find implementations for DOS graphics, OpenGL or other 3d Programming, Java graphics, and for the most recent implementations, Processing applet and Flash/AS3. One can even find this effect on desktops with dreamrender or beryl/compiz fuzion. Still this seems to be the first implementation with javascript/canvas other than one other on the net have seem to created a “ripple effect” with canvas, which is not really similar.

School work along other commitments is keeping me really busy, besides this is not even part of any computer graphics classes I required to take. So while there are perhaps more enhancements and optimizations that can be done, I’ll be leaving that up to you readers :)

JS 中文笔画输入法 (Javascript Chinese Stroke Input)

In case the title didn’t appear correctly, this is a post for another of my javascript experiments. This demonstrate chinese character recognition using stroke based or radicals input, instead of the usual pinyin(romanized mandarin system) method. Although this method is usually slower for typing, it allows beginners to find or input a word he does not recognize or a word whose pronunciation and pinyin is forgotten. I thought what better time could I release this than during the Lunar New Year.

Chinese Stroke Recognition

To try it, click on the screenshot above. Wait for files to be completed loaded before clicking and dragging to write a chinese character in correct stroke order (top left to bottom right, outer to enclosed), and the suggested words would appear in the right box. Right now the recognition algorithm is pretty rigid, you need to have the exact amount of strokes in an almost perfect sequence and writing to get the correct recognition. As mentioned, its pretty experimental, but it shows how fast javascript can be, character lookups are usually within 1 second, often below 100ms. The biggest bottleneck right now is parsing of the huge data files (usually the case for javascript engines these days), which may be solved with an ajax call to the server instead of caching all locally. Okay, I’m going into more implementation details so non-technical readers can do some skipping.

So basically this application is a modified and really simplified port of Kiang’s Hanzi Lookup Java Application to Javascript.
For example, I used my javascript port of shortstraw (shortstrawJs) for substroke recognition. This works well in many situations and badly for many (when strokes are short, etc). There ought to be better input filtering, and better matching against the database. Right now, all the substrokes are flatten, ignoring the exact strokes the user make. There’s currently isn’t fuzzy matching, or partial matches which gives lots of room for improvements. Of course, if you are able to, you can be grab the code by viewing source and trying making changes.

Yi
Some traditional words supported. This is the traditional character for my middle character of my chinese name.

If I have the time, there are a few features I like to have in the future.
- One is creating a bookmarklet, for which this input method would work with any site - like my virtual keyboard bookmarklet released previously. The advantages that bookmarklets have over browsers addon is that the browser like firefox does not need to be restarted, making it a very useful tool for non-home computers. (Gangwen, if you are still finding a online IME, you may like to try Sogou’s Cloud Pinyin/启用搜狗云输入法 bookmarklet).
- Another is tying this input to a chinese dictionary, making it a valuable tool for chinese beginners.
- I also hope to implement custom font rendering (For example using 漢鼎繁舒體) rendered on canvas for computers without these custom chinese fonts installed. Currently this is another of my javascript experiments - I have managed to render a couple of characters, but have yet to figure out the Adobe’s CID implementation and quirk character mapping that
ttf2pt1 gives me.
- Another idea is hooking this up to a DIY Multi-touch pad if I get to built one.

And so I wonder if there’s a need for this application.
- I heard that Macs already have this input method using their touchpads built into the OS. Windows 7 seems to have pretty good chinese recognition with their stylus input as seen from their blog
- Newer HTC phones running Android have a HTC custom native input method called Touch Input. See HTC_CIME HTC的官方手写输入法. It doesn’t come with my HTC Magic, and requires some rooting or hacking - something I don’t have time for now, otherwise I could try writing android applications too.

Some friends may think that I started this experimented because of my inability to read chinese and maybe its true, or indeed its true my chinese language (similar my other verbals and written languages) is pretty weak! So maybe someone might still find this useful. Again, the source code is available for anyone to modify it.

Lastly for those still celebrating the launar new year, happy new year!

Dao Fu

p.s. I created an easter egg in view of the chinese new year. Double clicking on a suggestion turn the word upside down. Some homes turn their words upside down to symbolize prosperity falling to their homes. This animation is done using Css3 transform properties (supported by the new firefox, opera and webkits- safari, chrome browsers).

Making of Focus

Watching “behind the scenes”, or perhaps directors notes on dvds, makes appreciating a film more fun (although some think reading spoilers more so). The reasons for me writing about the making serves 2 purposes, one to document some of the technicalities used, and second to reflect what went wrong and could be improved. Or perhaps bring more depth to the story.

One of the questions most asked about this video is whether I did the blurring in post production. The answer is no and a reason is that it would create too much work. The video with a wide aperture (f/4), which means the lens is allowing more light in, creating “circle of confusion”, blurring objects not in focus. In general that’s called a shallow depth of field, and helps create the “bokeh effect”.

Frame of Focus
Framing.

In this still from the MJPEG, the focus is on the mirror frame, making the mirror image like a painting. Another question asked is whether someone shot the video. If you watch the last few seconds of the video, you would have an idea but this motion still tells the story.

Behind the scenes of Focus
Crew and equipment

Lastly, the usb cable connects the canon camera 40D to the PC, and recorded via EOS Camera Movie Record.

EOS Camera Movie Record
The software/hack which make the video possible.

What could have done better? The plot, being plain boring but it is the feeling when I got watching arts film in the past. The jerkiness, which leads me to find out about but not use the hardware solutions (eg. steadicam) and software approaches (eg. deshaker).

And without saying, one might question while watching, whether I’m in focus.

Simple Countdown Javascript App

I found this draft saved during January 14, 2007 at 8:28 pm, make a little changes such that I can publish it now (jump to last paragraph to avoid history).

That was my first “web coding” of the year after a time of isolation from web development during my national service.

In fact, that piece of js/web app I coded wasnt for the internet, nor on the intranet, but on a desktop of a workstation used in my unit. Its purpose was to display the time before I had left in the army.

There were other spreadsheet and applications some other soldiers did which counted the number of days to their ORD (Operational Ready Date), but I chose to create and personalise mine which

1. Tells me the absolute number of days, instead of working days
2. Updates and display the time change every milliseconds
3. Display visualisations of time spent instead of just numbers
4. Integrate it seamlessly on the workstation without doing any unauthorized stuff.

The result? Progress bars and numbers updating itself on the desktop automatically using- yup, active desktop. (We were using military computers, not developer’s machines, yes notepad was used). Colleagues thought it was pretty cool, while some of wanted it for themselves, others questions my reason for creating this as it seem that I couldn’t wait to finish my term in army. It wasnt surprising, as I knew some who counted the days they had to serve left in NS, and that me counting down with hours, minutes to fraction of a second seemed more extreme. The reply I had was that I lose track of time easily, and I wasn’t even counting, that my counter would not only tell me how much time I went though but to treasure the time I had left.

Noc Ord Counter Desktop

Previous Next Close
Noc Ord Counter Desktop

Fast forward to the present day, I thought I should treasure the time over here in noc silicon valley that I should use that I used over a year ago. I did have a non electronic copy of the source code I wrote and decide to modify it for my usage here. As seen on the screenshot, I use
+ a progress bar to tell the % of working hours left currently each day,
+ a progress on the percentage of time spent/left in NOC
+ a photo to appear as desktop wallpaper
+ configuration of active desktop
+ no, I wasn’t play games during office hours.

This running copy can be found over here, and you can follow the link and look at its source. Do take note of various limitations
- bad coding? it was coded as quickly as the idea came
- has bad browser compatibility. it was built for ie+active desktop
- resources hog. perhaps due to rapid reload

Another quick hack to the code I had was using gauge charts using google’s visualisations api.

Treasure every amount we have.

Geotagging Photos - 3 Simple Methods

What’s Geotagging?
Geotag. “Geo” + “Tags”. Tagging with geographical information or GPS coordinates.

The meaning is so simple, there isn’t a need to look up its meaning. Geotagging is one of the interesting stuff that makes me think about it, its brings more fun after having a GPS receiver, and the possibilities is really up to one’s creativity.

Lets take for example if write a traveller’s blog, you could look for posts categorised by time, by categories and by subject tags. Imagine having each post is tagged with its location, not only you know the location reading a post, you could easily navigate and search for posts by location and visually on a map. (Geojoey’s a free blog service for geotagged enabled blogs.)

Basically anything you wish to keep geographical information or data to be organise by location could be “geotag”, secret places (search geocaching), treasures, objects, animals, friends etc. It will be cool to have microblogging/status updates services support geotagging. Twitter tackles “what are you doing” but perhaps geotagging answers “where are you”.

Why Geotag Photos?
There’s should be some good reason why Google’s and Yahoo’s services supports this other than competition. When organising photos, you group them by usually by time, albums, folders etc. As the photos collection grow quickly, we need more to search and locate and there’s where descriptions, keywords tags comes into place. Perhaps with sophisticated software, we could find them by colours, mood, faces (I hope).

With the addition geotags, we know easily and exactly where a photo is taken, we could easily find photos taken in or around an area, we could layout photos over the world map and visualise where we had been before. (See SuperGeoTagged for flickr photos on google earth/maps)

3 Methods.
Likely, you need to know only 1. Make a choice.

Method 1: Automated Package.
You take a photo with your camera, and the location gets tagged to the photo automatically. Sounds simple but how? This usually requires some equipment. Either your camera has built in GPS, or you attach it to a GPS addon which is usually costly. A more practical situation is using mobile phones (and software) which takes advantage of its built in (or connected) GPS and camera. (Here’s a product for nikon cameras. Zurfer is a mobile software/service for automating and uploading geotagged photos from cameras phones)

Method 2: Map Assisted Geotagging
A little manual work but nevertheless the most affordable method. Select photos, then pinpoint it on the map on the computer to tag them. Since yahoo, google and microsoft already have their own map services, they already have services and software which allows users to do so easily. At the same time, there are software and online services and tutorials for providing such geotagging mashups. (examples: flickr, locr)

Method 3: Syncing GPS Logs
With a little work, this provides the most flexibility at affordable costs. Have a device to record GPS logs while you shoot away on a separate camera. Since GPS has multiple purposes, it would be a waste just to use it for tagging photos. With a GPS receiver, you could use it for navigation, finding directions, displaying routes and travel statistics, and while doing all these, record your tracks. With these track logs, a software would be able to extract the geographical data and tag locations to photos taken based on time contained in photographs.

My Choice.
Method 3. Since I already own a cheap bluetooth GPS receiver, I run the cool free trekbuddy on my mobile phones which tells me my location, speed, distance at the same time recording my travel tracks. The logs transferred to the computer would be synced with the photos, and coordinates are written to the photos’ EXIF and automatically displayed on maps with comptible services. The freeware I used is Geosetter, which is very good and powerful.

Photos on a boat

The benefits of a GPS Log can been seen here using Geosetter. Not only it shows you the path you took, it can shows you at exactly which spot you took the photos. On the boat, while Trekbuddy was tracklogging, its CMS can provide accurate ETA if you set target waypoint at the jetty. Click here for another screenshot.

More Examples.
Google supports geotagging in various services/products. Both Panoramio their community photo site, Google Maps (enable show photos under More) can view tagged photos, and Google Earth, Google Web Albums supports tagging. Microsoft’s softwares Virtual Earth and ProPhoto supports geotagging.
For method 3, you could purchase a gps data logger, but if you have already own a gps mobile phone, all the more its similar. A simple, free software like GPS Track could do the gps logging easily for you. Trekbuddy also provides inbuilt geotag for camera phones when u record a waypoint. (Method 1) Other free GPS software for mobile phones of interest are VlkGps and Mobile Trail Explorer.

More Links and Resources
Google’s Picasa Web Album - allows geotagging of albums
Yahoo’s Flickr - their organz feature allow you to geotag photos
Google’s Panaramio- service mainly to upload and specify coordinates. Photos on this service are shown in Google earth
Locr - A photo sharing site mainly with geotagging capabilities and location sharing (using maps by google, yahoo, microsoft)
Zooomr - another photo sharing community with Geotagging support.
Everytrail - For recording of tracks mainly but supports syncing for photos uploaded
Trekbuddy - Free powerful GPS j2me software for mobile phones
Geocoded Photo A similar article on wikipedia.