Koinos Blockchain Framework

Introduction

One of the biggest problems holding back blockchain adoption is how difficult they are to upgrade. In this paper we propose a novel blockchain architecture that is optimized for upgradeability. An incidental outcome of this approach is the creation of a truly general purpose blockchain that can be easily adapted to any use case (public, private, hybrid) using any programming language that compiles to WASM and little blockchain-specific expertise.

The purpose of Koinos is to make blockchain technology (and therefore smart contracts) accessible to all. In order to accomplish this, what’s needed is a general purpose blockchain (“layer 1”) capable of rapidly adapting to the ever-changing needs of developers and their end users. It must also be fee-less which is covered in our mana whitepaper.
This paper outlines the blockchain framework (sometimes called a “layer 0”) needed to support this use case, which winds up being the first truly general purpose blockchain framework. The design of the system is modeled off of a traditional operating system. We conceptualize a blockchain as a kernel of natively implemented system calls that contain only the cryptographic functions necessary for constructing a technically true blockchain along with logic for dispatching (“dispatch logic) a node between natively implemented system calls and system calls implemented as smart contract modules running in the virtual machine. This combination of native implementations and “system smart contracts” form a high-performance, vertically scalable, and upgradeable blockchain *framework* that allows for any behavior to be added to the blockchain without requiring a hard fork.

Blockchains as decentralized computers

Blockchains are a new method for performing computation that is still in its infancy but has massive disruptive potential as a result of its ability to store digital ownership. This novel database structure, first pioneered in Bitcoin, enables developers to leverage decentralized networks of computers to perform computations in a trustless manner and incorporate digital ownership into their applications.

They are a kind of “public cloud computer” and the unique advantage of such computers is that they are so inherently trustworthy (“trustless”) that they are capable of storing digital assets that can’t be double-spent (digital ownership). A major problem with existing blockchains, however, is how difficult they are to upgrade. Typically, the complex behaviors of the blockchain like the consensus, governance, and resource systems, are implemented natively to maximize performance and efficiency. The problem is that because they are implemented natively, making any changes to these behaviors requires a system reboot (or “hard fork”).

Previous Work

When Ethereum first launched the native implementation approach made sense because it was the way Bitcoin was designed and because their desire to allow developers to run arbitrary code (smart contracts) in a virtual machine would make maximizing system performance even more critical.

To accomplish their goal of creating the world’s first general purpose blockchain, the Ethereum developers had to develop their own programming language (Solidity) and VM (the Ethereum Virtual Machine), but most of Ethereum’s system would remain natively implemented to maximize performance. The end result was smart contracts that could not be executed anywhere near the speed of native implementations and a system that generally can only be upgraded through a hard fork.

WASM

Two years after Ethereum was released, WebAssembly was released by a W3C workgroup backed by Google, Mozilla, Microsoft and Apple all of whom leverage WASM to deliver powerful applications like games using a virtual machine (the browser) without forcing the user to download a separate application (i.e. native implementation).

WASM solved this problem by allowing developers to convert the programs they write in any language into something already very close to machine code so that it can be quickly loaded into the browser and executed. This gave web app developers access to more performant programming languages than JavaScript (like C++) while giving the browser a file that could be executed more efficiently, thereby benefiting both the developers and the browser companies (Apple, Microsoft, Google, Mozilla). This approach has seen spectacular success which has led to constantly increasing language support and performance improvements that are pushing the performance of WASM ever closer to native implementations.

EOSIO

The EOSIO blockchain framework was one of the first, if not the very first, to take advantage of WASM by implementing WASM smart contracts and developing their own blockchain-specific WASM virtual machine (EOSVM).

EOSIO introduced several innovative concepts like in-band upgradeable smart contracts and a few “system smart contracts” which allowed them to change some system parameters without a hard fork. However, there are still a great many scenarios where hard forks are necessary, because most of the “business logic” is still implemented natively in C++.

In order to leverage WASM for blockchain one must take advantage of their “host functions” to specify additional native features that WASM doesn’t have “out of the box.” The way EOSIO works is by having the user load a WASM module with blockchain-specific features added as host functions (accessing databases, etc.), but which still exist as native implementations in C++ and ultimately means they can only be changed by executing a hard fork.

In short, EOSIO did not solve the problem of hard forks by leveraging WASM to separate the higher level rules that determine how data should be created, stored, and upgraded (i.e. the business logic) from the lower-level details of the system infrastructure which is why EOS, and EOSIO-based chains, are still bottlenecked in terms of their rate of improvement by the need to hard fork in necessary system upgrades.

Polkadot

Polkadot does leverage WASM to allow “forkless upgrades,” however, it remains inaccessible to most developers and does not provide the degree of modularity and flexibility of Koinos because it still does not separate out the business logic from the native implementation through a design analogous to an operating system. Instead, Polkadot divides the node into a “Runtime” and a “Host” where the Runtime can be upgraded without a hard fork, but the host cannot.

The components of a Polkadot Host are:

• Networking components
• State storage, storage trie, database layer
• Consensus
• Wasm interpreter and virtual machine
• Low level primitives for a blockchain
• *

Only once a compiled Polkadot runtime containing the necessary logic is uploaded can a host build a blockchain. In other words, the Polkadot Host is not a true general purpose blockchain because 1. It includes specific business logic (like consensus and finality) and 2. It cannot produce blocks. Of course, the Runtime can’t function without the host therefore it too is not a true general purpose blockchain. Our approach, which we call “modular upgradeability” is fundamentally different and results in a truly general purpose blockchain.

Microservice Architecture

To maximize modularity and upgradeability we start from a microservice architecture. Within this schema “the blockchain” is conceptualized as just one of potentially many microservices. A microservice within this schema is no different than in any other, meaning that it is entirely up to the user (a node operator) whether to upgrade, customize, or create a microservice. All the node operator, and the network at large, “cares about” is whether valid blocks are being added to the blockchain.

We identify five microservices as the initial set; block store, p2p, mempool, block producer and chain (the blockchain). This list is not necessarily exhaustive. Individual microservices, as always, should be optimized for a single responsibility. Simply by adopting a microservice architecture we immediately acquire a high degree of modularity and upgradeability.

Chain Microservice

We propose the chain microservice be implemented as a minimalist blockchain that only includes the low level primitives needed for a blockchain and “default” native implementations. The default implementations are implemented such that the node can construct a blockchain and transmit blocks over a p2p network, unlike Polkadot’s implementation, but that is all it can do. It has absolutely none of the “real-world” rules that describe how information should be created, stored, or altered within the blockchain.

In other words, the chain service has no business logic but instead contains primarily dispatch logic. It contains only the logic necessary to construct a technically true “blockchain” (include and order transactions) plus logic that allows the system to call into smart contract implementations at the moment of code execution (“runtime”).

Native Implementations

We propose the following list of native implementations;

• prints
• exit_contract
• verify_block_sig
• verify_merkle_root
• apply_block
• apply_transaction
• apply_reserved_operation
• apply_upload_contract_operation
• apply_execute_contract_operation
• apply_set_system_call_operation
• db_put_object
• db_get_object
• db_get_next_object
• db_get_prev_object
• execute_contract
• get_contract_args_size
• get_contract_args
• set_contract_return
• get_head_info
• hash

Because all business logic is to be implemented as in-band upgradeable smart contracts, obviously the chain microservice requires native implementations for supporting smart contracts which is why the list includes those required for the input and output of smart contracts such as logging (prints) and database management (db_put_object and db_get_object).

Ironically, even the developer of an application-specific blockchain would implement their business logic in smart contracts! For example, one could use JavaScript to write a proof of work contract and a token contract that effectively describe the behaviors of Bitcoin, upload them as system contracts to the Koinos Blockchain Framework thereby creating a high performance, highly upgradeable implementation of Bitcoin! This has been effectively demonstrated with the Koinos testnet.

The astute observer might recognize some native implementations that are not necessarily critical for the construction of a technically true blockchain. The objective when choosing these implementations is to restrict ourselves to only those that will constantly be in use by the blockchain. While not technically necessary, they are devoid of business logic and implementing these as smart contracts would result in unnecessary computational overhead.

System Calls

The mental model motivating this design is therefore of “the blockchain” as a kernel of “system calls.” In traditional computing, a system call is a way for a computer program to interface with the operating system kernel; the core of the OS which has complete control over the entire system. They are the OS industry-standard-solution for safely regulating access to the most important part of the operating system.

The kernel is the default set of system calls necessary to create a minimally viable operating system along with additional infrastructure that allows the operating system to override the “native implementations” in the kernel. This design allows the kernel to be optimized for security, efficiency, and flexibility instead of specific behaviors (business logic) which are instead handled by the higher level operating system. This is exactly the type of system we seek to emulate in a blockchain context.

But the OS analogy does more than simply provide a more dynamic guide to the system design of a blockchain, it provides us with robust, battle-tested, and secure technical solutions with which to implement our system. The system call structure itself was borrowed directly from how computer operating systems deal with system calls, for example, through the use of hardware interrupts.

Native Implementations

“Native implementations” are the default system calls necessary to create a minimally viable blockchain. They form a blockchain “kernel” designed to empower developers to launch a blockchain (i.e. an operating system for decentralized applications) as quickly and easily as possible while maintaining the ability to rapidly improve that system over time.

Whereas EOS implemented its system as a large set of WASM host functions, Koinos effectively has only one host function; invoke_system_call. This allows for the passing of a system call ID along with arguments in the form of a character array which are then parsed by the system and used to identify the desired function which could be implemented either natively or as a WASM smart contract that overrides the natively implemented system call (system call override).

When a computer attempts to run a blockchain with this design, it will execute native implementations until it hits an override at which point it is dispatched to a smart contract containing the alternate business logic. That business logic will be executed in lieu of the native implementation and the computer will be dispatched to the next module of business logic whether that is contained within a native implementation or another “system” smart contract.

Dispatch Logic Only

Changing this infrastructure would still require a hard fork but it is a simple infrastructure that has no business logic in it whatsoever. Instead it contains primarily the cryptographic primitives necessary to construct a blockchain and dispatch logic, i.e. the logic needed to dispatch the machine between higher level business logic modules.

This combination of native implementations and “system smart contracts” form a high-performance, vertically scalable, and upgradeable blockchain *framework* that abstracts nearly all complexity (i.e. business logic) out of the base layer (“the blockchain”) and into WASM smart contracts running in the Koinos virtual machine, all of which can be upgraded in-band (without the need for hard forks).

Multi-Layered Live Testnet

With the separation of business logic from the native implementation complete, stakeholders can now observe a desirable behavior as it operates on the live network, first as a user-owned smart contract (“user contract”), then as a system contract (if granted permission via governance), before deciding whether the system would benefit from a native C++ implementation (hardfork) turning the mainnet into an effectively multi-layered testnet.

Continuous Upgrades

The issue is not that hard forks are inherently evil, it is that they should not be *required* in order to upgrade the protocol. Koinos should be able to continue getting better without hard forks, and retain the option for hard works exclusively for optimizations or bug fixes but not *behaviors* which can be achieved just as well by smart contracts overriding native implementations. This system allows for an incredible amount of flexibility in upgrade paths so that adding behavior does not require an out-of-band upgrade.

Therefore hard forks should only contain changes that are optimizations or bug fixes, not behaviors (i.e. no business logic), and thus have very little contention for acceptance. These changes can still be bundled to reduce the frequency of forks, but because so much less will *require* a hard fork, the overall quality of a given hard fork will be much higher.

Conclusion

In this paper we have outlined the design for the world’s first truly general purpose blockchain framework. This framework allows for the implementations of all blockchain business logic as smart contract modules with system level permissions. The result is a functional blockchain and framework in which nearly everything can be upgraded without a hard fork. Even critical components like the consensus algorithm, block production, resource management, digital signature algorithms, account systems, and account authorities can be upgraded without a hard fork. This allows governance alone to determine whether an upgrade should be made, and individual upgrades can be pushed to the network much like an operating system patch but with with an on-chain record of the entire upgrade path

This novel architecture has profound implications on how engineers should even approach blockchain development, allowing them to employ agile practices at the system level for the first time. Because no technical decision is ever final on Koinos, they can more easily defer commitment and wait until much later in the development process to determine what smart contracts to include in their systems, even waiting until after the launch of their mainnet!

The modular upgradeability of Koinos means that no technical decision is ever “final” and blockchain implementations can continue to improve over time while maintaining an on-chain record of all the governance decisions and resultant upgrades. This makes time the strongest ally of Koinos-based chains, unlike those based on other architectures for whom time is their worst enemy.