Category Archives: Ideas

Rain, Water, Ripples with HTML Canvas, Javascript, JQuery

One of the exciting reasons to work with canvas is that you are not only able to create 2d graphics easily with javascript, one can easily do do pixel manipulation on images.

Think about it, in case you don’t think its cool right now, is that you only need notepad, a recent web browser and a few lines of code to do cool stuff work with 2d graphics without compilation. Take an example, you can create a greyscale image from a color photo into without photoshop or other software as see in pixastic.

In this post, I will outline how easy it is to code with javascript and canvas to simulate water ripples, along with some issues encountered. After I discovered this old water algorithm (during the DOS era I suspect) on creating waves, I thought why not try this with HTML Canvas, and so I did.

Ripples with Javascript

STEP 1. Creating a canvas element.
This can created easily with javascript, or just add some html code into the body

<canvas id="jripples"></canvas>

STEP 2. Load an image with JS when document is loaded for which we can use the jquery $(document).ready

$(document).ready(function(){
init(); // Run stuff when document is ready.
});

Here’s the code to load an image


var img = new Image();
img.src = "hello.jpg" // path and filename of image
img.onload = function() {} // run the next portion of code when the image loads

STEP 3. Draw the image onto the canvas


canvas = document.getElementById('jripples');
$('#jripples').width(img.width); // We set the canvas width and height with jquery
$('#jripples').height(img.height);
canvas.style.height = canvas.height = img.height ; // Set the canvas size to the image
canvas.style.width = canvas.width =img.width;

ctx = canvas.getContext("2d"); // Get the 2d drawing context
ctx.clearRect(0,0,canvas.width,canvas.height); // Clear the canvas
ctx.drawImage(img, 0,0,canvas.width,canvas.height); // Draw the image

Just a caveat though, to get or manipulate pixels from an image, the image needs to be loaded from the domain or would result in a browser security violation.

STEP 4. Build the data structures behind the wave
orginalData = ctx.getImageData(0,0,canvas.width,canvas.height).data; // The array of pixels for the image
myImageData = ctx.getImageData(0,0,canvas.width,canvas.height); // Modifications of pixels
buffer1 = []; // Create new array for the image buffer
buffer2 = [];
// orginal data length is 4 times the pixel dimensions since they store RGBA (red,green, blue, alpha)
for (var i=0; i buffer1[i] = 0;
buffer2[i] = 0;
}

STEP 5. Creating ripples (the wave algorithm) function

function processWater(source, dest) {
for (var i=imagewidth; i< source.length-imagewidth; i++)
{
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

dest[i] = (
((source[i-1]+
source[i+1]+
source[i-imagewidth]+
source[i+imagewidth]) >>1) ) -dest[i];

dest[i] -= (dest[i] >> 5); // Damping - Quick divde by 32 (5 bits)

}
}

while the original article explains how this works, this article explains it in another simpler manner

Two height maps are used to store the current and previous states of the water.
Each frame you will toggle between state maps.
For each array element in the current state array:-
Look at the neighbouring pixels from the previous state, i.e. above, below, left and right. Take the sum and divide by 2. Because we are dividing by 2 a right-shift will work beautifully.
Now subtract the value in the current state map.
If we left it like that the ripples would never subside so we need to diminish the strength of the ripple every pass. The most realistic way of doing this is to reduce the resulting height by a fraction of itself. Once again we can use right-shift to optimise this.

Note that I ensured the loops avoid any edge bound pixel to prevent wrap arounds.

STEP 6. Render the new image with waves.
function texture(buffer) {
var xoffset, yoffset;
for (var i=imagewidth; i {
// check for bounds
var xi = i % imagewidth;
if ((xi==0) || (xi==imagewidth-1)) continue;

xoffset = buffer[i-1] - buffer[i+1];
yoffset = buffer[i-imagewidth] - buffer [i+imagewidth];

var offset = i+xoffset+yoffset*imagewidth;

if (offset>0 && offset

for (var x=0;x<3;x++) { //4 for alpha
myImageData.data[i*4+x] = orginalData[offset*4+x];
}

}

}
// Draw
ctx.putImageData(myImageData, 0, 0);

}

This estimates the pixel offset due to refraction of water (read more on Transparent Surface Ray-Tracing) based on height of the wave and copies the pixels from the orginal array into the new imagearray object. This imagedata is returned to the 2d object using putImageData to be drawn.

STEP 7. Call the rippling function in a timer
rippling = setInterval(ripple, 50); // This calls the ripple function every 20 times a second

STEP 8
We can calling another timer to add rain
raining = setInterval(function() { // random rain
var randomX = Math.round(Math.random() * canvas.width);
var randomY = Math.round(Math.random() * canvas.height);
buffer1[randomY*imagewidth+randomX] += Math.round( Math.random()*-500);
},300);

STEP 9. Add more raindrops/ripples reactivity to mouse clicks. We use jquery to add mouse listeners quickly.
$(canvas).mousedown(function(e) {
var x = e.pageX - $(this).offset().left;
var y = e.pageY - $(this).offset().top ;

buffer1[y*imagewidth+x] = -400;
clicked.dragged= true;

}).

STEP 10. Provides FPS(Frames per second)
setInterval(function() {
$('#jdebug').html(fpsCount + " fps");
fpsCount= 0; // recent
},1000);

This isn't the most accurate way to calculate FPS, since the callback may not be accurate in itself. Rather, one should try getting the actual time subtracted from the last elapsed time to calculate the fps.

Enjoy. Here's the link to javascript ripples demo
Remember that you may need to upload your html and images to a webserver to test (or change your browser security settings).

Other Considerations.
This run on all modern browsers (Firefox, Safari, Chrome, Opera) except IE due to the lack of support with Canvas. While the Excanvas provides some compatibility, it does not provides get/putImageData calls. Until IE9 releases, flashcanvas pro may solve this compatibility issue.

From my observations, the bottleneck seems to be when the browser render the image after the putImageData javascript. Right now firefox seems to be able to render 2x faster than chrome, and the self proclaimed fastest browser on earth, Opera, seems to renders about 3 times faster than chrome (to 100fps on small images).

While this wave algorithm may be efficient when there are many waves, this method may not be as efficient when there are few waves on large images/canvases. Even a image of 640 pixels can hang the browser (be warned).

One simple way to speed up the waves is scale down the image. If putImageData is the bottleneck, then we can hope that the browser would optimize these soon. (with Direct2D just implemented for firefox nighties, maybe soon will be soon!) Other interesting alternatives would be to implement WEBGL or O3D which uses the C++ stack or GPU to speed up the rendering, although these are not widely supported right now. The better idea, I imagine, is to implement another algorithm which renders the ripples from its wave source, which would probably run faster on large images with few waves.

Finally, creating ripples seems to be a classic challenge and if you were to do a simple search, would be able to find implementations for DOS graphics, OpenGL or other 3d Programming, Java graphics, and for the most recent implementations, Processing applet and Flash/AS3. One can even find this effect on desktops with dreamrender or beryl/compiz fuzion. Still this seems to be the first implementation with javascript/canvas other than one other on the net have seem to created a "ripple effect" with canvas, which is not really similar.

School work along other commitments is keeping me really busy, besides this is not even part of any computer graphics classes I required to take. So while there are perhaps more enhancements and optimizations that can be done, I'll be leaving that up to you readers :)

JS 中文笔画输入法 (Javascript Chinese Stroke Input)

In case the title didn’t appear correctly, this is a post for another of my javascript experiments. This demonstrate chinese character recognition using stroke based or radicals input, instead of the usual pinyin(romanized mandarin system) method. Although this method is usually slower for typing, it allows beginners to find or input a word he does not recognize or a word whose pronunciation and pinyin is forgotten. I thought what better time could I release this than during the Lunar New Year.

Chinese Stroke Recognition

To try it, click on the screenshot above. Wait for files to be completed loaded before clicking and dragging to write a chinese character in correct stroke order (top left to bottom right, outer to enclosed), and the suggested words would appear in the right box. Right now the recognition algorithm is pretty rigid, you need to have the exact amount of strokes in an almost perfect sequence and writing to get the correct recognition. As mentioned, its pretty experimental, but it shows how fast javascript can be, character lookups are usually within 1 second, often below 100ms. The biggest bottleneck right now is parsing of the huge data files (usually the case for javascript engines these days), which may be solved with an ajax call to the server instead of caching all locally. Okay, I’m going into more implementation details so non-technical readers can do some skipping.

So basically this application is a modified and really simplified port of Kiang’s Hanzi Lookup Java Application to Javascript.
For example, I used my javascript port of shortstraw (shortstrawJs) for substroke recognition. This works well in many situations and badly for many (when strokes are short, etc). There ought to be better input filtering, and better matching against the database. Right now, all the substrokes are flatten, ignoring the exact strokes the user make. There’s currently isn’t fuzzy matching, or partial matches which gives lots of room for improvements. Of course, if you are able to, you can be grab the code by viewing source and trying making changes.

Yi
Some traditional words supported. This is the traditional character for my middle character of my chinese name.

If I have the time, there are a few features I like to have in the future.
- One is creating a bookmarklet, for which this input method would work with any site – like my virtual keyboard bookmarklet released previously. The advantages that bookmarklets have over browsers addon is that the browser like firefox does not need to be restarted, making it a very useful tool for non-home computers. (Gangwen, if you are still finding a online IME, you may like to try Sogou’s Cloud Pinyin/启用搜狗云输入法 bookmarklet).
- Another is tying this input to a chinese dictionary, making it a valuable tool for chinese beginners.
- I also hope to implement custom font rendering (For example using 漢鼎繁舒體) rendered on canvas for computers without these custom chinese fonts installed. Currently this is another of my javascript experiments – I have managed to render a couple of characters, but have yet to figure out the Adobe’s CID implementation and quirk character mapping that
ttf2pt1 gives me.
- Another idea is hooking this up to a DIY Multi-touch pad if I get to built one.

And so I wonder if there’s a need for this application.
- I heard that Macs already have this input method using their touchpads built into the OS. Windows 7 seems to have pretty good chinese recognition with their stylus input as seen from their blog
- Newer HTC phones running Android have a HTC custom native input method called Touch Input. See HTC_CIME HTC的官方手写输入法. It doesn’t come with my HTC Magic, and requires some rooting or hacking – something I don’t have time for now, otherwise I could try writing android applications too.

Some friends may think that I started this experimented because of my inability to read chinese and maybe its true, or indeed its true my chinese language (similar my other verbals and written languages) is pretty weak! So maybe someone might still find this useful. Again, the source code is available for anyone to modify it.

Lastly for those still celebrating the launar new year, happy new year!

Dao Fu

p.s. I created an easter egg in view of the chinese new year. Double clicking on a suggestion turn the word upside down. Some homes turn their words upside down to symbolize prosperity falling to their homes. This animation is done using Css3 transform properties (supported by the new firefox, opera and webkits- safari, chrome browsers).

[geeknotes] Panning Navigator with jQuery, Html 5 Canvas

Seems that I happen to like coding several stuff when it comes to the end of the year. As such, I’ll decide to delicate this piece of script/code/site to my all my fellow photo enthusiasts and geeky friends as a late Christmas but an early new year’s gift.


Panning Navigator - jQuery + Canvas by Zz85

I decided to implement a Panning Navigator in Canvas while experimenting with various UI navigation working on my notation project (like the navigator in Sibelius). This panning navigator is what one might usually see in Adobe Photoshop, Illustrator, etc, and it seems few have implemented this, perhaps a handful implementations with flash, but none I came across with javascript. As for its name, I was not even sure what’s the name but seem this site seem to suggest this ui pattern as a panning navigator for dealing with large canvases with a draggable viewport.

So without further to do, check out the demo here. http://jabtunes.com/notation/panning_navigator.html

Here are some of the features
+ Drag the viewport in the navigator to view around
+ Mouse wheel can control the zoooming
+ Fill, Fit to screen for quickly zooming out, 1:1 for actual size and 3:1 for zooming in.
+ Fullscreen photos and navigator response to browser resizes
+ Features some photos from my early photography
+ Canvas used for drawing the navigator and creating the “thumbnailed” image.
+ Css for aligning navigator, resizing and repositioning image
+ Seems useful with large photos and panoramas
+ Should run iphone and android browsers although experience may be improved
+ Oh also released as do what every you like to do with the source license, but remember to let me know your comments.
(Warning: Some images are huge, please be patient while waiting)

Comments are welcomed. Maybe, when its polished up, bug free and extensible, I might release this as a jquery plugin.

Goodbye 2009 and an early happy new year to all!

p.s. One might check out the jQuery supersized plugin for where the fullscreen photos were partly inspired by.

p.p.s. For source code: use your browser “View Source”.

Twitter Timeline Updated

In my previous post Timeless Belt Of Time: How I Integrated Twitter in Timeline, I posted the code for laying out twitter posts on the timeline.

Previously that was done by calling the json Twitter API. It work well for the current 20 status but the archive call (80 posts) didn’t work. I filed a bug report at Twitter, but when I last check again 3 days ago, it was still not working.

To implement a workaround for archives, I modified my server side php code to get the XML format. SimpleXML was used for parsing the xml, then since my server didn’t support json, I use the json library from pear php.

Here’s the updated code for download.

Recently, I have a regain interest in adobe products, namely flash lite and flex. My first flash lite app is a Metronome, and maybe I might post it if I polish it up.

Facebook Musical Wall Application

In less than a week, together with my teammates released a wall-like facebook application for our 2nd assignment.

My Facebook Music Application

The difference? Instead of the usual messages and links, videos & photo attachments, users get to record or create their music by clicking on a virtual 3d piano before sending to their friends.

Application Description Link: http://www.facebook.com/apps/application.php?id=9575698857
Application Link: http://apps.facebook.com/musicalwall/

Main Page

Previous Next Close
Main Page
Piano Studio
Previous Next Close
Piano Studio
My Music
Previous Next Close
My Music
My Wall
Previous Next Close
My Wall
Gallery
Previous Next Close
Gallery

Current features include:
+Playing on a virtual piano
+Playback and save the music played
+Send it to a friend’s wall
+Share the music with other users
+Download music as a midi file
+Convert to RTTTL ringtone format
+View and playback music on your own wall

Not forgetting my capable teammates. “)
Teammates

And for those interested: Here’s a link to the javascript version of the flash piano I experimented with.

Piano

Happy Lunar New Year!

Juicing Up Photo Viewing Experience: Part I

on this blog, on your wordpress site, or anywhere else. This series is a result of my recent craze with scripting/programming and photo ideas.

Screenshot of Fotobook Highslide

Previous Next Close
Fotobook Highslide - Mash up of WordPress, Facebook Photos & Highslide JS

1. Photo Gallery
I tried a couple of wordpress plugins in the past: Gallery2, Simple Extended Gallery, Flickr….

They were perhaps good, but perhaps complex, lacking and I didn’t seem to like it. Then I created PhotoZip and even though Photozip plugin had its short comings (eg. perhaps a little slow due to batch processing), I’m glad Tim have the interest to improve it so much better than I would.

It is only recently I found out about Aaron Harp’s Fotobook, an integration/mashups of Facebook’s photos into your blog. So I tried it, and it works.

The integration is good (since I use facebook too), the layout neat, the configuration all done with much ease. It seems like it was something I tried looking so hard before, now that I found it, I love it. My photo gallery can be viewed by clicking on the Photos tab.

2. Highslide
Tim’s improved PhotoZip supports LightBox, so does Fotobook. No doubt it gives great effect and I like it too when used appropriately. Even my cousin’s portfolio uses it to enhance viewing experiences.

However, a reason why I don’t use it? Perhaps it was due to some negativity when I experienced it on some sites. But when I come across the Highslide javascript library, I thought I would use it for the following reason
1. At the first impression it reminds of LightBox- to expand an image in the same window.
2. However, unlike Lightbox, it is less obstructive. The background is not grayed out, and window can still scroll, and the expanded image can be dragged around.
3. It seems more reactive.

Not only it good to find the right tool for the right job, it good to find the best package available. So I’m using Highslide WordPress plugin. Since fotobook was written in a way it was simple to customise it, I created a style/theme similar to Aaron’s lightbox style, except it works with highslide. Check out one of my gallery for example.

(Tip: You can use arrow buttons on your keyboard to navigate through the photos. You can also click/expand more than one photo at a time too.)

Click here to download Highlight Style for Fotobook.

Instructions:
Ensure you have Fotobook and Highslide plugin working.
Copy the highslide folder into fotobook styles folder
(eg. /wordpress/wp-content/plugins/fotobook/styles)
Select the highslide style under fotobook options.
View your photo gallery, test and enjoy!

Online Trends

Some of my random thoughts of how things changes and turn out

  • Library > Google
  • Encyclopedia > Wikipedia
  • Dictionary > Spellcheck
  • Books > e-Books
  • Telephone > Skype
  • BBS > Newgroups, Mailing Lists > Forums
  • Notice Boards > Portals > Blogs > Feeds
  • Yearbook > e-Circles > Friendster > Facebook
  • Maps / Street Directory > Google Earth / Maps
  • Bookmarks > del.icio.us > Digg
  • Icq > Msn > Web messesaging
  • Irc > ShoutBoxes > Twitter

become twitter
cartoon from gapingvoid.com

Facebook Ideas

I’m thinking of creating a Facebook Application.

Some ideas that I have.

  • Universal Search
  • Bookmarks
  • Blog Post
  • MSN
  • What you are doing
  • Where you are
  • Online Time Tracker
  • Online Votes
  • Memorable History

Do let me know if you have any ideas. If the idea is really worth the effort, I may do it if not it will be one of the many gabra apps on Facebook.

Article Repository

I’m considering the different php solutions for building a Christian article database.

1. Solutions out there?
2. Blog
3. From scratch

PDF

Platform running on is: XAMPP

1. Existing solutions
My impression tells me they are usually either too general or specialized for my needs.
General in a CMS-typed like Mambo and Specialised stuff are dull university projects which do not usually have too nice an interface.

Next option please.

2. Blog-based solution
As usual, our friend WORDPRESS is recalled. 2.1.2 the latest release for now.

Its seems a good solution as it has already some of the features needed – Writing posts easily, Categories, Search.

Others need some research, customisation and building. * means recommended links

a) Keywords tagging + search

http://vapourtrails.ca/wp-keywords

b) Providing Word and PDF formats for download.
i) HTML to Word
* http://conort.googlepages.com/generate-word-from-php

ii) HTML to PDF
* http://www.rustyparts.com/pdf.php

http://www.digitaljunkies.ca/dompdf/about.php

http://www.easysw.com/htmldoc/pdf-o-matic.php?source

iii) Plugins
* http://www.zirona.com/software/wp-to-pdf/
* http://wp2pdf.sourceforge.net/

http://www.pdf24.org/blog/two-wordpress-pdf-plugins/

3. From scratch.
Why hurt myself when I have better alternatives like above?

**Lost in thoughts as usual**

LilyPondTool updates

I’ve distracted long enough to be lost from the lilypond and lilytool world.

Was surfing round the lilypond mailing list web archives I found several updates interesting enough to share.

Bert, the creator of LilyPondTool, created a new webbie for LilypondTool at http://lilypondtool.organum.hu/

Bert now has his own blog over at http://briffid.blogspot.com/

LilyPond 2.10.19 and 2.11.19 are available for download. From what I can see, the installation is much simple on linux than before with its installer script, and I see ample windows builds (Wait till I buy vista before windows again). As usual new features keep coming in.

LilyPondTool 2.10.4 released Seems it has many new features including

LilyPondTool 2.10 demo is released demonstrating some of the most exciting features -instant errorchecking, instant help, instant point-and-click). See more cool demos

There’s this new webpage called A-Play was first seen here then mentioned as a shell for lilypond and discussed more over here Now, I don’t really get the entire idea of how to use it, but I already had the idea of how it should be used before I ever seen of this.

(This paragraph contain random thoughts)
In no way to suggest bad for lilytool in anyway, but I feel the web platform would be the way really for lilypond to reach out to the masses. I mean, what can be harder than opening a web browser compared to installing >10mb of stuff. Normal users would most likely seen a web browser 10x more than file editor, which cuts down lots more learning curve for interface alone. Graphical interface could be easily implemented in html too. Expert users can use a textarea with autosave feature enabled, providing a portability around the world. Syntax highlighting, indentation can be done with a iframe. Shortcuts can be added to keyevents. Snippets can be drag and drop or provided via menu or toolbars. Combined with good server side support to store, render and process, and server/browser communication using AJAX to provide seamless integration between user and machine, it seems only good can come out from this. Well of course this is another of my wild imaginations, although I think its not fairly difficult to implement, not many people who can implement will do it, but perhaps an employee or someone in his 20% time might do it for fun.

Lastly I glad that even LilyPond’s “boss” Hans-Wen has openly supported LilyTool as a “official recommended tool”. Kudos to Bert! Well, as this point of time I can’t wait to open my development tools like Eclipse and start coding goodies for LilyPondTool, but not yet. Got to install a running copy of jEdit with its plugin first. Then settle lots of other stuff I needa do, followed by keep a cool head to not to rush, keep the ideas and program well.

Slow and steady wins the race.