How blockchain and serverless processing are impacting event-driven architectures

As application patterns evolve to event-driven architectures, it’s more likely that serverless and blockchain will be used together

Taking a look at the similarities and differences helps to gain a better understanding of both serverless and blockchain. The deeper analysis also informs how each of the technologies might impact this next wave of computing cycles.

Over the past few months, I’ve spent a lot of time diving into the world of blockchain. Much like serverless computing, there is a lot of pieces to consume and absorb. It took a fair amount of effort to work through the basic workflows, understand the different platforms, and relate together the various components.

I started on my blockchain discovery journey a few months ago after a conversation with Lawrence Hecht from The New Stack. During our call, we discussed how a high percentage of Alexa Voice Service/Echo applications are using AWS Lambda as their primary processing engine. In other words, Alexa plus Lambda equals application.

His hypothesis — one that I wholeheartedly subscribe to — is that serverless processing in combination with emerging API-driven platforms will be leading the way in serverless adoption.

Alexa + Lambda = Application

Certainly serverless processing will touch on almost all areas of IT computing. But this theory implies that the most rapid growth and adoption will be in combination with new platforms. In other words, new platforms plus serverless processing equals higher rates of growth.

Since these applications are starting from scratch, there is no legacy code or existing architectures to worry about. Serverless processing supports rapid development and quick access to scale. As a result, it’s a smart choice for people wanting to develop applications quickly and scale them without much additional effort.

Another driver is that new applications are often event-driven — or at least have large event-processing needs. This characteristic lends itself to microservices and functions as a service (FaaS) architectures. The new tools that serverless processing provides are perfect for handling event-driven computing patterns.

New Platform + Serverless = Many Applications

For this reason, I believe that the combination of blockchain plus serverless will have a sum far larger than their parts. The combination will be a predominant method for building and supporting blockchain-related applications — especially when it comes to private blockchain networks.

While exploring the world of blockchain, I found that most of the published material was either too high-level, too effusive, or dived too deep into cryptography and consensus algorithms. What was missing was a clear explanation — directed at architects and developers — that addressed the practical questions about building a blockchain-based applications for business use.

Digital currency is based on blockchain — and the currency exists because of blockchain technology. Digital currency transactions are essentially notarized by the processing and storage nature of the blockchain. In other words, the currency’s value is protected by the chained hashed blocks and the distribution of the networks.

It is difficult, if not impossible, for digital currency to exist outside of a distributed blockchain network. For that to happen, digital currency would have to put faith into a single trusted entity — and hope that it wouldn’t game the system, get hacked, nor inflate the monetary supply. These are options that very few adherents of digital currency are willing to trust.

Blockchain networks are powered by digital currency. Organizations that operate nodes within the network are rewarded via the use of digital currency within the system. For example, payment for processing transactions in the Bitcoin network are paid in Bitcoin. Processing on the Ethereum platform is paid via Ether — the coin used in Ethereum.

Did you know that non-currency assets can also be traded and the transactions preserved within a blockchain?

Since blockchain technology can also be used for applications separate from digital currency, there is significant activity and investment in the space. Companies are using blockchain technology to create transactional-based solutions for trading and transferring physical and digital assets in a manner that is secure and readily verifiable.

With traditional marketplaces and trading platforms, transactions are stored in a centralized ledger owned by a single entity. Blockchain platforms allow these transactions to be digitally signed and preserved as an immutable record. It also stores the transactions in a distributed ledger across multiple independent and replicated nodes.

At its core, a fully formed blockchain network becomes a mechanism for designing the rules of a transactional relationship. The blockchain network acts as programmed adjudication or final settlement, reducing the need to appeal to human institutions. As a result, the blockchain become a programmable social contract which allows for trusted, validated, and documented interactions between parties at a very low cost.

Blockchain networks are comprised of near identical nodes operating in a distributed but independent manner. The network of nodes is used to validate sets of transactions and encapsulate them within blocks in a chain. At the core of the blockchain platforms is a distributed transaction processing engine that validates and cryptographically seals transactions.

These transactions are maintained in a distributed ledger that is replicated, shared, and synchronized within any participating nodes. Blockchains use cryptographic technology to create a permanent record of transactions. A set of transactions are cryptographically stored within a “block”. Successive blocks are added in a chain — secured and preserved in order using hashing algorithms.

The ledgers for public blockchains, such as Bitcoin and Ethereum, are stored in thousands of nodes across the world. Private blockchain networks, on the other hand, may only have a few nodes. Typically, any full participant within a blockchain network would want to maintain an active and operating node — ensuring the validity of the ledger independently from anyone else.

A consistent record of truth is made possible by the cryptographic and shared nature of the ledger. This is critical for addressing several of the problems blockchains are trying to solve:

1. eliminating the need for central source of truth
2. eliminating contention when parties dispute the status of transaction(s)

Blockchain is a consistent record of truth that is shared among participants in the network. It becomes the ultimate arbitrator — eliminating any version of “he said, she said” disputes.

As noted earlier, there are a number of blockchain platforms. While Bitcoin is the most popular cryptocurrency, Ethereum has one of the most popular blockchain platforms — especially for purposes that go beyond just digital currency store. Hyperledger is a Linux Foundation project and has received significant support from large finance and technology institutions. It is also popular as measured by the number of projects using it and the level of community support.

Blockchain networks may be public, private, or hybrid. This means that a public transaction would be encrypted into the public ledger, while private transactions would be stored in a private ledger. Private transactions could also be stored in the public ledger, hence, the hybrid designation.

To support hybrid use cases, the Enterprise Ethereum Alliance is working hard to keep the public Ethereum network and private platform compatible. According to a source monitoring the effort, a big topic for the group is private transactions on a permissioned, or private, blockchain. Chase Bank’s fork of the Ethereum Go client (Quorum) has added private transactions — where only the sender and receiver know the details of the transaction. Compatibility with the interactions of the public chain, though, is still a driving tenet.

Digital currencies are tied to particular blockchains. Transactions involving a particular currency are represented, denoted, and enshrined in the distributed blockchain for that currency.

Bitcoin transactions are handled on the Bitcoin platform and stored in a Bitcoin ledger. Ethereum transactions are be handled on the Ethereum platform and stored in a Ethereum distributed ledger. Hyperledger transactions are handled on the Hyperledger platform and stored in a Hyperledger distributed ledger. And so on.

Some blockchains feature the concept of a digital tokens as a secondary asset or currency. These assets are priced in the base digital currency. The digital tokens can most often can be used for services on a particular application or sub-platform within that blockchain platform. A look at the listings in TokenMarket will show the digital assets that are available under the Ethereum blockchain.

For example, envision an Uber digital token that could be used as currency for any Uber block-chain enabled service. The service could simply draw from any Uber digital tokens in your Uber account. The tokens could be tied to a digital currency such as Ether, or just be built on the platform and gain or lose value within the platform — as well as via speculative interest.

Digital tokens are being released in ICOs — initial coin offerings. In some ways, it’s similar to pre-registration stock offerings in the 1800s and early 1900s. Any entity can create a token and offer it for sale in ICO. The transaction would be performed in the blockchain digital currency.

A report referenced in Blockchain News indicated that one quarter or $250M of the $1B of investment raised by blockchain companies was the result of ICOs.

Digital assets simply refer to the digital or physical items at the heart of a trade. Example of assets that might be purchased and transferred include a house, a car, shares in a company, or a painting. These transactions would be registered within a blockchain, and the asset becomes the item referenced in the exchange. A shipment could also be considered an asset at the core of a set of blockchain transactions — a blockchain use case that is already in operation.

All blockchain platforms contain a processing component that is a critical part of transaction assurances. The blockchain networks are set up for “miners” that competitively “forge” each successive blockchain.

The terms mining and forging are used to describe the process of validating and preserving transactions in blocks, as well as receiving new digital currency tokens in return for the work. The process of mining introduces new currency into the system at a reasonably fixed rate of frequency.

Miners are compensated for being the first to solve a block solution. This means being the first to calculate a hash for the block that is less than an arbitrary threshold set by arbitrary mechanisms of the network. Blockchain platforms are self-arbitrating with respect to setting and adjusting the thresholds — this allows the aggregated set of miners to mine blocks within specific and regular time windows.

All miners have access to each transaction. As part of forging a block, or a set of transactions, they process the code for each transaction and attempt to arrive at the hash solution.

In the article, “Bitcoin Mining explained”, the author provides a detailed explanation of the process. It describes how each block’s hash is produced using the hash of the block before it, becoming a digital version of a wax seal. The process confirms that this block — and every block after it — is legitimate, because if you tampered with it, everyone would know.

This distributed nature of blockchain processing means that each active node can theoretically executes each transaction within the system. For example, the transfer of a public digital currency from one person to another might get processed by every node in the entire network.

In the case of Bitcoin or Ether, this could be mean over 10,000 nodes are executing the code for a given transaction. This same processing replication takes place for each type of transactions — whether it’s as simple as transfer of a digital asset or a transaction with extremely complex processing logic.

The code for each transaction is referred to as a smart contract, and the processing is referred as on-chain processing.

In the case of Ethereum, the smart contracts are written in Solidity — a high-level contract-oriented language whose syntax is similar to that of JavaScript and designed to target the Ethereum Virtual Machine (EVM).

Functions are uploaded as part of the process for creating an asset type in the blockchain, The uniformity of the language and the processing model ensures a high degree of idempotency — meaning the outcome of executing a smart contract should be the same on each node within the system.

In the case of Hyperledger Fabric, smart contract processing is performed in Go. The language is OCaml is being used for a new blockchain platform expected to be released in June 2017. According to CryptoAcademy, “OCaml [is] a powerful functional programming language offering speed, an unambiguous syntax and semantic, and an ecosystem making Tezos a good candidate for formal proofs of correctness.”

Within public blockchain networks, there is a charge for processing transactions — a cost borne by the party submitting the transaction. In the case of Ethereum, the processing charge is called the “gas” and it’s submitted as part of the transaction in the form of ether, the base currency in Ethereum.

Private blockchain networks, however, can be more flexible with respect to transaction costing methods. The operating costs could be borne by the network participants with little or no cost, or could be accounted in a manner as determined by the network participants.

Various blockchain platforms are working on improved forms for arriving at processing consensus. The current form is called Proof of Work. A proposed approach within the Ethereum network is called Proof of Stake. You can read about the proposal and debate here, here, and here.

Blockchains are likely to make an impact in the way we trade assets, store assets, conduct food safety regulation, record patient information in healthcare, perform supply chain management, vote in elections, and much more. Join us in this interesting Blockchain certification course and dive into the world of blockchain.

The blockchain processing model differs significantly from serverless processing. Not only do most serverless platforms support multiple languages, but the goal for serverless processing is one-time processing.

Processing the same transaction on a concurrent basis is antithetical to the serverless processing philosophy. Within blockchain platforms, however, this concurrent identical processing model is a key feature — ensuring and maintaining the blockchain as a validated and trusted source of transaction history and current state.

Given the closed nature of blockchain processing, there is no need — nor any entry way — for serverless processing of on-chain transactions for execution of smart contracts. However, there is a tremendous amount of need for processing off-chain transactions — especially in terms of setting up transactions, helping to perfect transactions, and addressing post-transaction workflows.

Why is there such a need for off-chain transactions? The reasons are because 1) on-chain processing capabilities are severely limited and 2) on-chain data storage is severely limited. As a result, off-chain data processing will need to take place for transactions that are complex and/or data-heavy.

With respect to the first issue of limited processing capabilities, on-chain transaction logic needs to be kept to a minimum in order to arrive at effective transaction throughputs. Cost mechanisms for processing transactions — ones that provide equitable rewards for processing transactions and operating the ledger — also impose costs for transactions.

Without the “gas” charged for transaction processing, transaction parties would get free rides on the network. In addition, they could easily overwhelm the processing capabilities of the network — blockchain DDOS attacks anyone?

To arrive at the optimal balance of performance, cost, and consistency for each transaction, transaction logic for blockchain applications need to adequately address both on-chain processing and off-chain processing. This also applies to addressing on-chain data and off-chain data. As a result, effective blockchain design means using the blockchain network for only for the most minimal amount of data processing and data storage necessary to perfect and preserve the transaction.

The separation of on-chain versus off-chain is illustrated by a recent post that describes using the Ethereum network as a mechanism for validating a game of chess.

The writer, Paul Grau, describes how it’s not practical to submit every move as a transaction into the chain. Each transaction would not only take a significant amount of time to settle, but also be potentially cost prohibitive because transaction cost. For example, a move by a player might take seconds whereas committing the move onto a blockchain might take minutes.

In assessing the problem, the team realized each move did not need to be a part of the chain. Independent arbitration — blockchain processing — is only needed to establish the start of a game as well as resolve a dispute in the game. Once a game is established, each player submits their move along with their perceived state of the game to the opposing player as a signed transaction off-chain. If the opposing player signs and submits a subsequent move, the prior move is deemed accepted.

A transaction is submitted to the blockchain only in situation where there is a dispute — a player believes there is a checkmate, stalemate, or timeout condition.

In such a situation, the processing of the smart contract would determine if such as condition was true, thereby dictating the outcome or next step — continued play, declared winner, or stalemate. It’s inevitable that questions will arise as to how complex the smart chain logic should and can be. In the case of chess, it’s proposed that a reasonably quick algorithm can be created to assess the condition within a smart contract. More complicated games such as Go, however, are another matter.

The split between on-chain processing and off-chain processing means that off-chain processing needs must be able to set up transactions and manage any post processing needs. Remember, blockchain provides a trusted record of transactions — but parties that are making use of that transaction don’t need to do so in an on-chain manner.

For example, a voting application that uses blockchain to verify eligibility of voters can access blockchain records for confirmation. Aside from registering a vote, any of its processing does not need to happen within a blockchain platform. Likewise, using a blockchain ledger to preserve the accident and repair history of an automobile will have incidents committed to the blockchain, but any actions performed on those incidents do not need to be done in an on-chain manner.

The limitations for storing data on-chain also has implications that directly relate to serverless processing. Any data that is supporting a transaction will need to be digitally preserved and linked as part of the transaction — like the actual contract that is recorded in the blockchain as being signed. Services are already being developed to perform this capability within several public blockchains. Tierion, for example, is performing this for Ethereum-based applications. For private blockchain networks, potential candidates for serverless processing include preparing the data, validating it, and accessing it post-processing.

The difference between what is processed on-chain versus off-chain largely comes down to trust levels. On-chain processing is designed to be trustless — meaning the parties do not have to trust each other in order to perfect a transaction. Off-chain processing performed by one party in a serverless environment is situated for cases where 1) there is no transaction effected, 2) two or more parties trust each other to forego any sort of consensus algorithm, or 3) there is a consensus algorithm in place to verify the results of the off-chain processing.

Blockchain and serverless processing are two independent technical innovations which are markedly different, but they share a number of things in common. While serverless is intended to be stateless, blockchain provides a publicly and independently verifiable way to maintain transactional states.

As application patterns quickly evolve to event-driven architectures, the need for independently verifiable transactional states will increase — and the more likely the serverless and blockchain will be used together. The use case for this combination is especially true in private and/or permissible blockchains where the trust level is higher, and the allowances for use of outside components and services more tolerable.

The next article will explore blockchain platforms and components in more depth, as well as dive into some of the issues blockchain platforms are trying to address. In addition, we’ll cover a few areas where serverless platforms can build hooks to better support the development and operation of blockchain applications.

Want to expand your blockchain and serverless skills? A Cloud Guru's training library has wide range of courses, learning paths and labs!

• Learn why blockchain technology has the potential to transform every industry
• ACID and AWS DynamoDB transactions: The ins and outs of all-or-nothing
• Using Google’s Site Reliability Engineering Philosophy for IT Operations
• From Serverless to Blockchain
• The challenges of developing event-driven serverless systems at massive scale

Thanks for reading! If you like what you read, hit the ❤ button below so that others may find this. You can follow me on Twitter.