Despite all the interest, I had a slow start with Emscripten – which was the topic in my last post. Overcoming some of the difficulties was I able to enjoy some fun and success with it. Still challenges lurked along the journey, and at end of the previous post I had 2 problems to solve:
a) getting link-time optimizations on glsl-optimizer
b) difficultly of porting a wavetable midi synthesizer to the web browser
One day @thespite mentioned to incorporate my emscripten port of GLSL-optimizer into his cool WebGL Shader Editor extension for Chrome, and that prompted me to revisit my project and Emscripten. So I continued the journey with a couple more tales to tell but as these stories become slightly unwieldy, I decided to cover the second section in another post.
Have you tried playing old games on emulators? A decade or more ago, I used to follow development on emulators (eg. playstation) to check out what additional new game compatibility they have made with new releases. Somehow the idea of being able to emulate and play many games from a different platform amazed me.
This draws a parallel to application of Emscripten. Huge codebase originally written in C or C++ could be “emulated” for the web. It excites me to see stuff like server libraries, desktop applications, graphics platform, codecs, even programming languages ported to JS that runs either on node.js or in the browser. 2 examples of projects I thought was really interesting is emscripten-QT and pypy.js! This wiki page has a great list of projects to check out!
While Emscripten has been known to work of huge amount of existing code, it might also mean that it could potentially generate huge amount of code too. Well, like many other cross-compilers or machine generated code, this isn’t new.
Which is why emscripten isn’t always the magic pill, there are other alternatives like hand porting or using other tools – bonsai-c, cheerp (though from some of my initial tests cheerp seems to be generating a bigger code ).
GLSL Optimizer Releases
Of course the file size wasn’t satisfactory. One thing I wasn’t doing was to run it at -O1, -O2 or -O3 optimizations. Running with those flags can activate link-time optimizations and runs the resulting code through JS minification which would improve overall file size and performance. Strangely, using these optimization flags resulted in infinite loops during runtime.
— Graphics Noob (@BlurSpline) February 19, 2015
Why weren’t the builds optimized
One factor I suspected why optimizations were failing was because I was using Embind for bridging JS and C world. I observed that for some strange reasons I was able to run at -O1 instead of -O0 when I had embind disabled, so I decided to revert to non-embind bindings and use function return values to pass back successful and failure values back to JS land.
But that seem only to be able to get me as far as -O1 optimizations. The resulting JS weren’t even minified so I wanted to check if I could do closure minification without O2. I ended up filing an issue in github because the flags didn’t allow me to do that with emscripten. So even though that might be a bug, minification without link-time optimizations would also have limited effectiveness (besides, running huge code through the minifier tends to end up with crashes).
Alon Zakai aka kripken asked why the compiler optimizations and pointed me to some compiler settings I could use to trace memory segmentation faults. Those settings turns out to be really useful to debug emscripten code in general.
Tweaking around the flags, I still couldn’t solve the problem. I started thinking that there’s a possibility it was a bug emscripten. I lay this matter to rest till some time later, there were new versions of emscripten and I decided to give it another shot. I upgraded to 1.30.0, no luck. I decided to git clone master version and try again, still it did not work run time problems.
Since I was on the master branch, I decided to check out the latest developments. Emterpretor was in. Asyncify was out. Interesting, but what was that?
I ran Emscripten + Emterpretor anyways and O2 still fails. On the bright side, through the network pane I found the resulting code to be much more compact. Wow! 8.7MB→1.8MB (1.7MB→720KB gzipped) So what happened?
— Graphics Noob (@BlurSpline) June 29, 2015
While uploading the tinier builds to github, I realized something. Github pages weren’t enabling gzip compression with emscripten.js.mem. As a hack, I renamed that to mem.js, make emscripten load the custom memory file sogithub-pages would pull a gzip served asset. Yah!
At this point, I’m tempted again to derail my topic to talk about the not-too-long ago announced WebAssembly/WebASM/WASM. It has some of the ideas of Emterpretor, AST byte code format solving faster load times and allowing optimizations in the JS engine. I think it’s an exciting topic. It seems like a natural progression for asm.js -> emterpretor -> WebASM which is at the same time backward compatible with polyfills. It’s even greater that all vendors agree on this standard. I believe it is also something that can make the people asm.js bothers happy. It is also awesome that emscripten can also support that with a flick of a switch. But let me go back on topic.
Up to now, I’ve been trying to get around fixing the optimization problem looking everywhere, except one place – the code base (one of my emscripten lessons I shared was to understand the original code base, apparently I didn’t heed my own advice). In an issue in three.js, Ben Adams mentioned my project in a thread that leads @tschw to the original optimizing issue I’ve open for glsl-optimizer. With his sorcery (which he denies), he was able to trace to an issue upstream from glsl-optimizer to the MESA glsl compiler that was causing how the optimizations not to work. Whoever tschw is, awesome work there!
Finally we can perform -O3 optimizations!!!!
— Graphics Noob (@BlurSpline) July 22, 2015
If we stop here for a moment, I think we have a happy ending. The GLSL-optimizer has also been added to the Shader Editor Extension.
So here we have a little milestone, but I believe there’s more work that can be done. thespite suggested for a tree-shaking pruning (aka dead-code elimination) feature that doesn’t alters the original GLSL variables. I think that’s a absolutely good feature to have, unfortunately it may be difficult to alter MESA GLSL parser or GLSL optimizer to do so. Others have mentioned other tools: peg.js, glsl-unit, glslprep.js, glsl-simulator and the StackGL set of tools: glsl-tokenizer, glsl-parser etc. These suggestions are great, someone just have to look into them and apply them. Maybe one day when I have too much time on my own, I might also write my own parser in JS to play around with the GLSL code. Well, maybe as always.
So that’s all for “Optimizing the GLSL-Optimizer” in this post, and I’ll try to write about “Setting WildMidi Wild on the Web” into the next post, till then feel free to drop me comments @blurspline 😀
Related Links / Readings
– https://brendaneich.com/2015/06/from-asm-js-to-webassembly/ From asm.js to Web Assembly
– The Emterpreter: Run code before it can be parsed
– https://twitter.com/BlurSpline/status/568271236632956929 Original announcement of glsl-optimizer on twitter
– Web Assembly (Google, Microsoft, Mozilla And Others Team up, Design FAQ, prototype)
– https://twitter.com/search?q=emscripten Twitter News on Emscripten