Thoughts on WebAssembly

A new web technology has been announced by the big browser developers (Microsoft, Google, Mozilla).

https://github.com/WebAssembly

These are my initial thoughts. I reserve the right to change my mind, and almost undoubtedly will, but I thought it would be interesting to set down what I think now, so I can compare it with how it will all turn out later.

What is it?

The name is a good clue. It is an attempt at creating a portable web assembly language. Wait, isn't that what asm.js claims to be? Answer: yes. 

According to its design documents WebAssembly wants to be a better version of asm.js. One that instead of being built on top of Javascript is actually built directly into the browser. The main advantage being performance (both execution speed and code size).

Is this really needed?

I'm sure this is going to be the main thrust of the debate as asm.js already exists. My own thoughts are that this has the potential to bring web applications on par with native applications in terms of performance and this in itself makes its adoption almost inevitable. But the thing is that asm.js was almost certainly going to achieve that anyway, so why bother? Answer: standards.

Asm.js is tied to Mozilla, and while it works on the other browsers (because it is just Javascript after all), it would be difficult for those other browsers to to optimize asm.js code to native code speeds. This is 'who controls the standard' problem is the same reason why NaCl/PNaCl is exclusively a Chrome thing.

By the Big 3 agreeing to cooperate on a standard it means that developers (application, tool, etc...) can focus all of their efforts and thus achieve progress faster.

How likely is this to succeed?

Right now I see its general adoption as quite likely. The fact that asm.js exists proves there is a real craving for a portable fast browser targeted assembly-like language. Because it isn't Javascript there are a lot of Javascript-language limitations that can be sidestepped rather than coded around (numbers and math calculations mostly but also potentially things like threading and memory constraints). Because the Big 3 browsers are all involved from the start, and all can benefit directly by making the browser a more popular option for a wider range applications, there is a huge incentive to make it work.

It all adds up to a lot of forces pushing in the same direction. The only real way for it to fail is technically.  However, there are so many people who want and would benefit by this, it is hard to imagine that the brain-power needed to solve the (relatively small) technical challenges won't be there. For the most part the technical solutions already exist, it is just a matter of picking and choosing what will work from what doesn't. The devil will be in the details, but there is going to be a lot of angles whipping the devil into submission.

What does it mean for the Web, the Universe, and Everything?

It means that the web comes a step closer towards being the world computing platform. There is going to be less and less desire for so-called 'native applications'. Why write an application for only this platform or that platform, when I can write an application for all platforms simultaneously? 

This is quite attractive to developers (one of the big reasons why Java has the popularity that it does even though it never fully achieved this). This is also attractive to users who no longer will be plagued by the chores of installing and maintaining applications. In the end almost everyone benefits by this sort of standardization which drives it forward into a sort of crystalized inevitability.

As I look at the computer I am using to write this post (which itself is being written inside of a webapp and so already no longer a native app) I see that my 'main' native applications that I use are: IDEs, virtual machines, note taking, and games. All of the other applications I use are really just OS support type apps (backups, machine monitoring, etc...), and of course my web browser(s).

The IDE and virtual machine are used in tandem and at this point I consider them part of a set. There are already primative web-based IDEs and cloud computing is everywhere. I can well imagine a not too distant future where my main IDE becomes web based, with the IDE-webapp tying directly into various cloud-vm infrastructures for testing/deployment/etc (In fact it already kind of exists).

The fact that I still prefer a native app for note taking exemplifies that web apps still aren't treated as 'first class' citizens from a conceptual point of view of the OS. Once I can easily bind a hot-key to a URL the need for a native app disappears. This is a UI problem, and one I feel confident will be resolved within the next 5-10 years as web apps continue their relentless domination of the desktop.

Games. Games will almost certainly be the final hold-outs, but paradoxically they are also the main drivers for WebAssembly and a web-centric future. Many of the simpler games I play should already be web-apps (card games), and the larger games (FPS, 4x grand strategy, etc) are just waiting for the performance and standardization that WebAssembly promises to be achieved.

In short, the gap from native to web apps is already quite small and shrinking. WebAssembly could be the final bridge that allows the world to complete the transition and relegate native apps to 'background plumbing'. It will be interesting to watch (and hopefully participate in) the migration.

My Own Private Blockchain

A year or so ago I got interested in creating my own blockchain in an attempt to fully grok how blockchains work, and to play around with some ideas surrounding how a blockchain can be used.

Bitcoinj of course exists and anyone thinking about creating a real 'Java Bitcoin fork' should almost certainly look there first. My humble blockchain is essentially an academic exercise where I'm learning and trying out wild and crazy ideas. It is definitely not the sort of thing one would use if security and scalability were immediate concerns.

The code for the project can be found here:

https://github.com/matthewjosephtaylor/java-blockchain

The basic basics are up and running. It can create a block, and validate that the proof of work for that block is valid.

Merkle Miracle

The most interesting parts of the code so far are the bits dealing with the Merkle tree. Merkle trees are magic. Solving the Byzantine General's problem was what made the blockchain possible, but Merkle trees are what make them practical.

Imagine that one has a group of files, and that one wants to compactly verify the integrity of the entire set while also being able to verify the integrity of each individual file in a random access sort of way. With one relatively small hash code (256 bits typically and usually called the 'Merkle root') this can be done with a Merkle tree. That one hash can easily and space efficiently verify terabytes of data and millions of files. When one learns what this simple hash structure can do it is hard not to be amazed by it. (Harder still not to try to 'Merkle all the things!' :) ) I heartily encourage you to read up on how they work and take a look at the code for yourself.

/src/test/java/com/thaler/miner/MerkleTests.java

/src/main/java/com/thaler/miner/merkle

Proof of Work

It has been quite the journey exploring all of the various proofs of work, and attempting to understand their various pros and cons. As I write this in 2015 both proof of work (POW) and proof of stake (POS) are attempting to battle it out. I side more with POW as the 'absolute best' answer, but POS might turn out to be 'good enough' to win in the end.

There is also considerable debate about whether CPU mining, GPU mining, or ASIC mining are best for POW.

In my blockchain I am currently using Scrypt.  Scrypt is a venerable CPU mining algorithm that attempts to make it difficult for GPU/ASIC miners to compete. There are several newly developed 'memory-hard' algorithms specifically designed to be much better at this task. Among them Lyra2 to me seems at the moment to be the top competitor, but as of today the competition is still ongoing.

My biggest takeaway from this is that it is useful to be able to change the POW algorithm midstream. I don't know if CPU,GPU, ASIC, or Quantum miners will turn out to be best. It is possible that POS will beat them all on pure economics. My 'crazy idea' is that the blockchain should have some sort of voting mechanism built in so that miners can vote in a new POW as the computing landscape changes. I can even imagine that the verification algorithm itself could be written in an interpreted language like Javascript, and embedded in the blockchain. That way embedded systems (like hardware wallets) could still migrate and recognize the changes without new code having to be expressly written for them.

/src/test/java/com/thaler/miner/MineBlockTests.java

/src/main/java/com/thaler/miner/util/CryptoUtil.java

Numbers and Bits

I will mention no names but I once worked for a company developing a large ERP system.  I learned a lot about accounting, but my greatest takeaway from my time there was that money and floating point decimals don't mix. Having to explain to non technical business people how IEEE 754 floating point was the root cause of why relatively simple calculations were coming out wrong is not an experience I wish to repeat (much less the time to refactor the giant mountain code to expel those damned floats...shudder).

So for that reason, in my blockchain I have banished the idea of decimal representation and used the more elegant solution of fractions. No longer will one be vexed by how to split a 1 three ways! The answer is simply 1/3. No longer will naive programmers be lured into thinking float can represent a monetary sum! Finally the tyranny of decimalization is over! :)

The problem of course is that Java doesn't have a good facility for dealing with fractions. There are fraction libraries (I recommend commons-math) but they weren't quite good enough to deal with converting bits into fractions in the way I wanted. So in the time honored tradition of developers everywhere I said 'to heck with it' and I created my own numerical objects (grumbling the whole time that a library should exist for exactly what I wanted to do damn it!).

/src/test/java/com/thaler/miner/BitManipulationUnitTests.java

/src/main/java/com/thaler/miner/ddb

Next Steps

There is still much left to be done. In particular the problem of how to express a transaction is still open. How expressive should the transaction language be? I'm still puzzling it out. Also paired with this is how transactions should be represented (do they need a binary format, will JSON suffice?). As I continue along the path of adding little bits and bobs to the code I plan to update here to explain my reasoning and reflect on what I've learned.

Until next time....

Still Alive

I haven't added anything to this blog in a while but I haven't abandoned it. :)

I had been posting a few musings at http://www.matthewjosephtaylor.com/ but I think I'm going to attempt to restart this blog here as a more 'long form' repository for my writing.

For shame, even on my other blog that I started with the intention of quick posts, I haven't posted anything in over a year. What can I say? I've been relaxing and I honestly think it has done me a world of good.

Now I think it is time to get back on the saddle and re-engage with the world.

One of the ways I plan to do that is to get a bit more serious about posting to this blog, publishing some of my pet projects on github and in general share what I can, be it my opinions, my code, or anything else I can think of.

No promises on how often I plan on posting. As always it will be whenever the mood strikes me.  I do have a few pet projects I've worked on that I think deserve the light of day however, so my plan is to start there. My first few posts will likely be done in conjunction with sharing those projects on github. I've also had some time to think about the world in general so I expect I will attempt to share my point of view on other things as well.

Stay tuned...more to come!