Think about it, in case you don’t think its cool right now, is that you only need notepad, a recent web browser and a few lines of code to do cool stuff work with 2d graphics without compilation. Take an example, you can create a greyscale image from a color photo into without photoshop or other software as see in pixastic.

In this post, I will outline how easy it is to code with javascript and canvas to simulate water ripples, along with some issues encountered. After I discovered this old water algorithm (during the DOS era I suspect) on creating waves, I thought why not try this with HTML Canvas, and so I did.

STEP 1. Creating a canvas element.
This can created easily with javascript, or just add some html code into the body

<canvas id="jripples"></canvas>

STEP 2. Load an image with JS when document is loaded for which we can use the jquery $(document).ready

$(document).ready(function(){
init(); // Run stuff when document is ready.
});

Here’s the code to load an image


var img = new Image();
img.src = "hello.jpg" // path and filename of image
img.onload = function() {} // run the next portion of code when the image loads


STEP 3. Draw the image onto the canvas


canvas = document.getElementById('jripples');
$('#jripples').width(img.width); // We set the canvas width and height with jquery
$('#jripples').height(img.height);
canvas.style.height = canvas.height = img.height ; // Set the canvas size to the image
canvas.style.width = canvas.width =img.width;

ctx = canvas.getContext("2d"); // Get the 2d drawing context
ctx.clearRect(0,0,canvas.width,canvas.height); // Clear the canvas
ctx.drawImage(img, 0,0,canvas.width,canvas.height); // Draw the image

Just a caveat though, to get or manipulate pixels from an image, the image needs to be loaded from the domain or would result in a browser security violation.

STEP 4. Build the data structures behind the wave
orginalData = ctx.getImageData(0,0,canvas.width,canvas.height).data; // The array of pixels for the image
myImageData = ctx.getImageData(0,0,canvas.width,canvas.height); // Modifications of pixels
buffer1 = []; // Create new array for the image buffer
buffer2 = [];
// orginal data length is 4 times the pixel dimensions since they store RGBA (red,green, blue, alpha)
for (var i=0; i buffer1[i] = 0;
buffer2[i] = 0;
}

STEP 5. Creating ripples (the wave algorithm) function

function processWater(source, dest) {
for (var i=imagewidth; i< source.length-imagewidth; i++)
{
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

}
}

while the original article explains how this works, this article explains it in another simpler manner

Two height maps are used to store the current and previous states of the water.
Each frame you will toggle between state maps.
For each array element in the current state array:-
Look at the neighbouring pixels from the previous state, i.e. above, below, left and right. Take the sum and divide by 2. Because we are dividing by 2 a right-shift will work beautifully.
Now subtract the value in the current state map.
If we left it like that the ripples would never subside so we need to diminish the strength of the ripple every pass. The most realistic way of doing this is to reduce the resulting height by a fraction of itself. Once again we can use right-shift to optimise this.

Note that I ensured the loops avoid any edge bound pixel to prevent wrap arounds.

STEP 6. Render the new image with waves.
function texture(buffer) {
var xoffset, yoffset;
for (var i=imagewidth; i {
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

}

This estimates the pixel offset due to refraction of water (read more on Transparent Surface Ray-Tracing) based on height of the wave and copies the pixels from the orginal array into the new imagearray object. This imagedata is returned to the 2d object using putImageData to be drawn.

STEP 7. Call the rippling function in a timer
rippling = setInterval(ripple, 50); // This calls the ripple function every 20 times a second

STEP 8
We can calling another timer to add rain
raining = setInterval(function() { // random rain
var randomX = Math.round(Math.random() * canvas.width);
var randomY = Math.round(Math.random() * canvas.height);
buffer1[randomY*imagewidth+randomX] += Math.round( Math.random()*-500);
},300);

STEP 9. Add more raindrops/ripples reactivity to mouse clicks. We use jquery to add mouse listeners quickly.
$(canvas).mousedown(function(e) {
var x = e.pageX - $(this).offset().left;
var y = e.pageY - $(this).offset().top ;

}).

STEP 10. Provides FPS(Frames per second)
setInterval(function() {
$('#jdebug').html(fpsCount + " fps");
fpsCount= 0; // recent
},1000);

This isn’t the most accurate way to calculate FPS, since the callback may not be accurate in itself. Rather, one should try getting the actual time subtracted from the last elapsed time to calculate the fps.

Enjoy. Here’s the link to javascript ripples demo
Remember that you may need to upload your html and images to a webserver to test (or change your browser security settings).

Other Considerations.
This run on all modern browsers (Firefox, Safari, Chrome, Opera) except IE due to the lack of support with Canvas. While the Excanvas provides some compatibility, it does not provides get/putImageData calls. Until IE9 releases, flashcanvas pro may solve this compatibility issue.

From my observations, the bottleneck seems to be when the browser render the image after the putImageData javascript. Right now firefox seems to be able to render 2x faster than chrome, and the self proclaimed fastest browser on earth, Opera, seems to renders about 3 times faster than chrome (to 100fps on small images).

While this wave algorithm may be efficient when there are many waves, this method may not be as efficient when there are few waves on large images/canvases. Even a image of 640 pixels can hang the browser (be warned).

One simple way to speed up the waves is scale down the image. If putImageData is the bottleneck, then we can hope that the browser would optimize these soon. (with Direct2D just implemented for firefox nighties, maybe soon will be soon!) Other interesting alternatives would be to implement WEBGL or O3D which uses the C++ stack or GPU to speed up the rendering, although these are not widely supported right now. The better idea, I imagine, is to implement another algorithm which renders the ripples from its wave source, which would probably run faster on large images with few waves.

Finally, creating ripples seems to be a classic challenge and if you were to do a simple search, would be able to find implementations for DOS graphics, OpenGL or other 3d Programming, Java graphics, and for the most recent implementations, Processing applet and Flash/AS3. One can even find this effect on desktops with dreamrender or beryl/compiz fuzion. Still this seems to be the first implementation with javascript/canvas other than one other on the net have seem to created a “ripple effect” with canvas, which is not really similar.

School work along other commitments is keeping me really busy, besides this is not even part of any computer graphics classes I required to take. So while there are perhaps more enhancements and optimizations that can be done, I’ll be leaving that up to you readers

The long story: I went up to the piano with one day without anything in mind but started playing some tunes when i least expected it. I quickly grab a pen and paper to jot down some notes in case I forgot them and recorded this song on my netbook, everything done almost in a sitting.

Of course writing it down and typesetting it actually takes a lot more time. typed the piece of music using the software MuseScore 1.9.6 beta, a great free music notation software. I later save it as a lilypond format which was more familiar with me for tweaking the nice finishing touches for music scores layouts with the help of jedit lilypondtool.

Like several of my songs, I think it sounds a little nostalgia (like “Who am I” which Reuben Kee arranged and used for his musical). Then again, I wasn’t really expecting to compose any thing on the piano that day which prompted me to title it “Huh?”.

I got a mixed bag of reactions to the piece. Some say its sounds classical, some says pop, some says a mixed. Some thinks it from Chopin Ballet, or Granados Spanish dance - Andaluza, or a little Japanese, Korean, Amelie soundtrack sounding, some feels its too happy, too sad, etc. You get the idea. I think much would be coincidental and subjective to the listeners.

Some suggestions by my friend Gerald - playing the harmony with broken chord pattern instead of the oom-pah bass will make it sound “more fluid”, and “less static”. He also suggest arranging the melody on the flute with a harp accompaniment in broken chords, in which after I just found out from my harp friends its difficult for lever harps to play keys with 5 flats.

Have fun trying out this piece out and let me know if you have other comments.

p.s. Since I’m on the topic of music, do let me know if you want to attend the nusso concert on 19 march friday evening 2010 at VCH. Details here and here.

update: the audio generated by musescore can be heard here. http://soundcloud.com/zz85nus/huh-1
or http://soundcloud.com/zz85nus/huh-recording for the original recording.

To try it, click on the screenshot above. Wait for files to be completed loaded before clicking and dragging to write a chinese character in correct stroke order (top left to bottom right, outer to enclosed), and the suggested words would appear in the right box. Right now the recognition algorithm is pretty rigid, you need to have the exact amount of strokes in an almost perfect sequence and writing to get the correct recognition. As mentioned, its pretty experimental, but it shows how fast javascript can be, character lookups are usually within 1 second, often below 100ms. The biggest bottleneck right now is parsing of the huge data files (usually the case for javascript engines these days), which may be solved with an ajax call to the server instead of caching all locally. Okay, I’m going into more implementation details so non-technical readers can do some skipping.

So basically this application is a modified and really simplified port of Kiang’s Hanzi Lookup Java Application to Javascript.
For example, I used my javascript port of shortstraw (shortstrawJs) for substroke recognition. This works well in many situations and badly for many (when strokes are short, etc). There ought to be better input filtering, and better matching against the database. Right now, all the substrokes are flatten, ignoring the exact strokes the user make. There’s currently isn’t fuzzy matching, or partial matches which gives lots of room for improvements. Of course, if you are able to, you can be grab the code by viewing source and trying making changes.

Some traditional words supported. This is the traditional character for my middle character of my chinese name.

If I have the time, there are a few features I like to have in the future.
- One is creating a bookmarklet, for which this input method would work with any site - like my virtual keyboard bookmarklet released previously. The advantages that bookmarklets have over browsers addon is that the browser like firefox does not need to be restarted, making it a very useful tool for non-home computers. (Gangwen, if you are still finding a online IME, you may like to try Sogou’s Cloud Pinyin/启用搜狗云输入法 bookmarklet).
- Another is tying this input to a chinese dictionary, making it a valuable tool for chinese beginners.
- I also hope to implement custom font rendering (For example using 漢鼎繁舒體) rendered on canvas for computers without these custom chinese fonts installed. Currently this is another of my javascript experiments - I have managed to render a couple of characters, but have yet to figure out the Adobe’s CID implementation and quirk character mapping that
ttf2pt1 gives me.
- Another idea is hooking this up to a DIY Multi-touch pad if I get to built one.

And so I wonder if there’s a need for this application.
- I heard that Macs already have this input method using their touchpads built into the OS. Windows 7 seems to have pretty good chinese recognition with their stylus input as seen from their blog
- Newer HTC phones running Android have a HTC custom native input method called Touch Input. See HTC_CIME HTC的官方手写输入法. It doesn’t come with my HTC Magic, and requires some rooting or hacking - something I don’t have time for now, otherwise I could try writing android applications too.

Some friends may think that I started this experimented because of my inability to read chinese and maybe its true, or indeed its true my chinese language (similar my other verbals and written languages) is pretty weak! So maybe someone might still find this useful. Again, the source code is available for anyone to modify it.

Lastly for those still celebrating the launar new year, happy new year!

p.s. I created an easter egg in view of the chinese new year. Double clicking on a suggestion turn the word upside down. Some homes turn their words upside down to symbolize prosperity falling to their homes. This animation is done using Css3 transform properties (supported by the new firefox, opera and webkits- safari, chrome browsers).

Basically what’s a virtual keyboard is that its simply an input without a physcial keyboard.
If you add this as a Keyboard Bookmarklet (Firefox recommended - drag this to your toolbar), you may use this keyboard on various websites buy clicking on your bookmark which launch as translucent, mac like, on screen keyboard.

If you are thinking of projection keyboards, sorry, it isn’t as cool as that, yet.

However, you can use this keyboard to enter text into textareas and input field by using your mouse, stylus or touchscreen. If you decide to use your physical keyboard, the virtual keyboard gets “lighted up”.

Some have asked me the purpose of this project anyway. Some sites like this would say there’s enhanced security particular from keyloggers. Of course, this isn’t really true since virtual keyboards may be backdoors themselves so please do not use this or any other bookmarklet, greasemonkey, addons etc you do not trust for sensitive information.

Yes, one reason is that I’m having playing around with html canvas again. Another is that I have always been interested to with multiple inputs/users. Creating custom interfaces with canvas could create the possibility of interfacing 2 users in a browser - imagine 2 users surfing the net together by sharing a wide browser screen on a pc. Fifa 95 (thats over a decade ago!) allowed 2 users to play with or against each other - one with the keyboard, another with the mouse, and this would be possible with a virtual keyboard (for the 2nd user). Whether this is another lame project, or whether there are possible great ideas, I’ll leave it up to your imagination.

Before ending with the tons of technical details, try out the demo and let me know of your comments, if any

More details of this release
+ Uses jQuery 1.4.1 (Loads dynamically in bookmarklet)
+ Virtual Keyboard - Drawned with Javascript and HTML Canvas Element
+ The Mac lookalike Design of the keyboard inspired from this NetTuts tutorial which used html, jquery and css instead.
+ Autofire keys & work around for text selection when autofiring in Chrome.
+ Onscreen Keypresses Feedback as you typed - Keybindings aided with plugin jHotKeys
+ Onscreen Key Highlighting Disabled for Password fields
+ Keyboard is draggable around the screen
+ Works with Textarea and Input fields.
+ Demonstrates a simple drawing framework with JSON
+ Uses TextCanvas for canvas rendering of text
+ Bookmarklet - see this interesting jQuery Bookmarklet boot code

See here for source code, and here for YUI compressed version.

Future improvements I would love to see.
+ 100% Bugs Free
+ IE compatibility
+ Docking
+ API
+ Stay tuned

ShortStraw presents a simple yet accurate algorithm described in a recent (2008) paper for detecting corners.

To summarize this paper briefly from my understanding, ShortStraw takes in points of a stroke, resamples them, uses distances of points to determinate whether the points are from straight lines or corners.

I learnt about this a method when after stumbling on Felix Raab’s blog post, who created flex example of the ShortStraw algorithm implemented in actionscript.

This makes implementing a javascript version much easier, since most of the code can be derived heavily from the AS3 implementation. So here’s an example and the source code to my shortstraw.js implementation.

As usual, the demo uses html 5 canvas for drawing (sorry IE users, but any other modern browser like firefox, chrome or safari would work), jquery for simple event handling, and of course the shortstraw javascript file for corner detection.

For the daring, you may want to look at the paper Revisiting ShortStraw which describes iStraw, a technique which attempts to address the shortcomings of shortstraw, mainly in the area of curved lines.

p.s. The screenshot is in contrast with Steve Hanov’s html 5 canvas Web draw, which creates “wavy lines” while Shortstraw helps to “straighten” hand drawn lines.

All the great things about it- simplicity, leanness, exactly what xml wasn’t but ought to be.

Soon I reasoned that the json format can be persisted in storage in its own form. One can build a simple, portable database using json records with minimal code.

“Why hasn’t anyone else built it?” I thought.

(sketch from couchdb’s site)

Well, now we have CouchDB, a document based storage project by apache and Cloudant, for your couchdb cloud needs.

Its kinda sad. We drove over 4 states (5 if we count montana) for over a day (another half to get to old faithful), setup my equipment, and when the recording of the eruption was almost over, I tripped over my own cables which caused this laptop to crash to the floor while recording the video to crash , with the battery pack disengaged.

Maybe its one of those times my heart dropped together with my items, but since the rugged laptop still manage to boot, and some hope lied within me that I would be able to recover the video one day. While I thought streaming broken video files were no big deal, it wasn’t so easy. My usual tools- media player classic, virtualdub, videolan etc were unable to play this file at all. It wasn’t until yesterday I attempted to rescue this file again.

1st I tried this open source “DivFix++ is #1 AVI Video Repair & Preview Utility” called DivFix++. While it managed to detect FourCC frames, the resulting video file was still unplayable. Okay let’s take a step back. Then because the Canon 40D I was using have officially has no video features, I used the open source EOS Video Record which allowed some form of video recording if the camera was tethered to a pc.

EOS Movie Record saves the resulting video in a MJPG format, so I tried figuring if JPGs could be extracted from the motion video file. I tried looking for MJPG2JPEG converters but none existing other than 2 examples of source codes here and here. I also started to examine and studying the file structures of AVIs, RIFF, JPEG etc, and with the help of a hex editor (the freeware HxD works great ), started digging around the file.

So when I’m peeking around the file at HEX Offset 04F0000 or human friendly number 75431936, the viewer showed full of zeros. Not a good news, because it meant there was not data beyond that point (buffer were not flush). From my estimation, based on file sizes, what was an approximately 2minute video is probably a sub 20s video. Which makes me feel like rewriting the subject as: “repairing a broken heart”, but read on…

I noticed DivFix++ almost generated a 75MB file which i meant it must have found the same errors I suspected and strip them off. So using the software did not entirely gone to waste. On examining the headers of the file, I found it empty which is a pretty good reason that nothing could play this file. I then remembered that EOS Movie Record writes the video headers only after recording. So following the avi riff structure, I copied in header values like frame rate, resolution etc, and eureka, the video became playable.

So, its not too sad and not all’s lost. The badly abused laptop (actually my mom’s and the 1st laptop I used much) still survived, I managed to recover 16s, and learn a couple of things about video file formats. And my photography apprentice of that day Ben Chua did took some photos with my spare camera.

p.s. You might be able to see the short segment of the recovered video below.

To find out, head over to http://howmean.com. This site belongs to the couple to be, Caleb and How Mean. And no, rather than its perceived adjective “how mean!”, I hope one experience some sweetness visiting the site.

There are several talented people behind the making of this site. Graphics and concept were designed by Grace Wong, and the web design was done by Meng Lung of Gold Coast Web Design. My little contributions would be some subjective augmentations to the site and one might make intelligent guesses to what I did by spotting non-traditional elements on the site. For those who aren’t guessing, here’s a list

+ Mouse over the dandelions near the footer, and one may see little particles floating away. On most modern browsers except IE, these particles are drawn with html 5 canvas elements. In unsupported browsers, 1 pixel rendering with css are used instead.
+ Meant to be an easter egg, rolling the mouse wheel renders lighter dandelions particles around the screen.
+ Ajax calls are used for navigation when possible to prevent disruption of the background music
+ To support browsers not running flash (the iPhones and Androids mainly!), flash objects degrade gracefully
+ To degrade the photo gallery for non flash clients, the xml used by the flash slideshow player is parsed and rendered with jquery.

+ A small widget (which dates back to my legendary ORD count down wallpaper) is used to display the time till or from the wedding date in the credit page.

Of course all these would not be present if not for the weds-to-be.

So this’s my 1st post of the new year, and its a great reminder to reflect whether each passing day gets sweeter. A blessed new year for you readers!

I decided to implement a Panning Navigator in Canvas while experimenting with various UI navigation working on my notation project (like the navigator in Sibelius). This panning navigator is what one might usually see in Adobe Photoshop, Illustrator, etc, and it seems few have implemented this, perhaps a handful implementations with flash, but none I came across with javascript. As for its name, I was not even sure what’s the name but seem this site seem to suggest this ui pattern as a panning navigator for dealing with large canvases with a draggable viewport.

So without further to do, check out the demo here. http://jabtunes.com/notation/panning_navigator.html

Here are some of the features
+ Drag the viewport in the navigator to view around
+ Mouse wheel can control the zoooming
+ Fill, Fit to screen for quickly zooming out, 1:1 for actual size and 3:1 for zooming in.
+ Fullscreen photos and navigator response to browser resizes
+ Features some photos from my early photography
+ Canvas used for drawing the navigator and creating the “thumbnailed” image.
+ Css for aligning navigator, resizing and repositioning image
+ Seems useful with large photos and panoramas
+ Should run iphone and android browsers although experience may be improved
+ Oh also released as do what every you like to do with the source license, but remember to let me know your comments.
(Warning: Some images are huge, please be patient while waiting)

Comments are welcomed. Maybe, when its polished up, bug free and extensible, I might release this as a jquery plugin.

Goodbye 2009 and an early happy new year to all!

p.s. One might check out the jQuery supersized plugin for where the fullscreen photos were partly inspired by.

p.p.s. For source code: use your browser “View Source”.

At times when I’m doing something experimental I think, maybe someone would find such information useful if I place it on my blog- since I learn a great deal from many other’s developers’ blogs.

Just to touch quickly, I’m working on a personal project to build an online music notation software which works in a modern browser (think Noteworthy Composer in the cloud with Html 5). I soon find the canvas element really powerful (supported in all major browser - firefox, safari, chrome, opera except ie), but a really raw api. Yet possibilities are already very attractive, and closes the gap between flash and web design (as mentioned in my last geeknote on facebook).

Anyway, more the project next time, but what I would like to cover here is dealing using scrollbars to deal with an oversized canvas. If you ever did any Java AWT/Swing programming, you might know this can be done simply with placing your components into a JScrollPane. So here’s my current quick implementation of a scrollpane with CSS.

First the html elements.
<div id="omw_scrollpane">
<canvas id="omw_canvas" width="800" height="320"></canvas>
</div>

Basically, this wraps the canvas element into a div container we call scrollpane, responsible for providing the scrollbars.

Then the css.

#omw_scrollpane
{
width:400px;
overflow:auto;
border: solid 1px white;
background-color:white;
}

This simply constrains the enclosing div to a view port, and the overflow properties automatically adds scrollbars when the canvas element is larger than the specified width. The canvas would continue to work without recalculating coordinates.

Real simple hack? Yes and no. This may works for simple application, but perhaps more complex situations may require custom scrollbars (which brings me back to the flash pre-mx days where scrollbars need custom coding). Its seems to me that having a canvas of over 80,000 pixels width seems to hang chrome and opera 9 browsers.

The only other example I could find on the net which require scrollbars with their canvas application is actually the amazing Mozilla Bespin project (collaborative “code in the cloud” with your browser). Right now their implementation uses a custom scrollbar which docks/hides itself when not being used. One might wish to investigate the ui framework mozilla built ontop of canvas call Mozilla Thunderhead used in the bespin project. Seen by some to be a competitor to flex or other RIAs, you may wish to learn more following one of their developer’s blog.