My Own Private Blockchain

A year or so ago I got interested in creating my own blockchain in an attempt to fully grok how blockchains work, and to play around with some ideas surrounding how a blockchain can be used.

Bitcoinj of course exists and anyone thinking about creating a real 'Java Bitcoin fork' should almost certainly look there first. My humble blockchain is essentially an academic exercise where I'm learning and trying out wild and crazy ideas. It is definitely not the sort of thing one would use if security and scalability were immediate concerns.

The code for the project can be found here:

https://github.com/matthewjosephtaylor/java-blockchain

The basic basics are up and running. It can create a block, and validate that the proof of work for that block is valid.

Merkle Miracle

The most interesting parts of the code so far are the bits dealing with the Merkle tree. Merkle trees are magic. Solving the Byzantine General's problem was what made the blockchain possible, but Merkle trees are what make them practical.

Imagine that one has a group of files, and that one wants to compactly verify the integrity of the entire set while also being able to verify the integrity of each individual file in a random access sort of way. With one relatively small hash code (256 bits typically and usually called the 'Merkle root') this can be done with a Merkle tree. That one hash can easily and space efficiently verify terabytes of data and millions of files. When one learns what this simple hash structure can do it is hard not to be amazed by it. (Harder still not to try to 'Merkle all the things!' :) ) I heartily encourage you to read up on how they work and take a look at the code for yourself.

/src/test/java/com/thaler/miner/MerkleTests.java

/src/main/java/com/thaler/miner/merkle

Proof of Work

It has been quite the journey exploring all of the various proofs of work, and attempting to understand their various pros and cons. As I write this in 2015 both proof of work (POW) and proof of stake (POS) are attempting to battle it out. I side more with POW as the 'absolute best' answer, but POS might turn out to be 'good enough' to win in the end.

There is also considerable debate about whether CPU mining, GPU mining, or ASIC mining are best for POW.

In my blockchain I am currently using Scrypt.  Scrypt is a venerable CPU mining algorithm that attempts to make it difficult for GPU/ASIC miners to compete. There are several newly developed 'memory-hard' algorithms specifically designed to be much better at this task. Among them Lyra2 to me seems at the moment to be the top competitor, but as of today the competition is still ongoing.

My biggest takeaway from this is that it is useful to be able to change the POW algorithm midstream. I don't know if CPU,GPU, ASIC, or Quantum miners will turn out to be best. It is possible that POS will beat them all on pure economics. My 'crazy idea' is that the blockchain should have some sort of voting mechanism built in so that miners can vote in a new POW as the computing landscape changes. I can even imagine that the verification algorithm itself could be written in an interpreted language like Javascript, and embedded in the blockchain. That way embedded systems (like hardware wallets) could still migrate and recognize the changes without new code having to be expressly written for them.

/src/test/java/com/thaler/miner/MineBlockTests.java

/src/main/java/com/thaler/miner/util/CryptoUtil.java

Numbers and Bits

I will mention no names but I once worked for a company developing a large ERP system.  I learned a lot about accounting, but my greatest takeaway from my time there was that money and floating point decimals don't mix. Having to explain to non technical business people how IEEE 754 floating point was the root cause of why relatively simple calculations were coming out wrong is not an experience I wish to repeat (much less the time to refactor the giant mountain code to expel those damned floats...shudder).

So for that reason, in my blockchain I have banished the idea of decimal representation and used the more elegant solution of fractions. No longer will one be vexed by how to split a 1 three ways! The answer is simply 1/3. No longer will naive programmers be lured into thinking float can represent a monetary sum! Finally the tyranny of decimalization is over! :)

The problem of course is that Java doesn't have a good facility for dealing with fractions. There are fraction libraries (I recommend commons-math) but they weren't quite good enough to deal with converting bits into fractions in the way I wanted. So in the time honored tradition of developers everywhere I said 'to heck with it' and I created my own numerical objects (grumbling the whole time that a library should exist for exactly what I wanted to do damn it!).

/src/test/java/com/thaler/miner/BitManipulationUnitTests.java

/src/main/java/com/thaler/miner/ddb

Next Steps

There is still much left to be done. In particular the problem of how to express a transaction is still open. How expressive should the transaction language be? I'm still puzzling it out. Also paired with this is how transactions should be represented (do they need a binary format, will JSON suffice?). As I continue along the path of adding little bits and bobs to the code I plan to update here to explain my reasoning and reflect on what I've learned.

Until next time....