What is blockchain sharding

What is blockchain sharding? An introduction to a blockchain scaling solution

As a blockchain developer, are you trying to find solutions to blockchain scaling and performance issues? You might be looking for information on blockchain sharding. In this article, I'll explain what blockchain sharding is.

Contents

The decentralized blockchain and its value

Promising blockchain technology has taken the world by storm because of its two central promises:

  1. Decentralization;
  2. Immutable record.

Decentralization enables the creation of completely new business models, e.g.

  1. The decentralized Bitcoin payments network is completely beyond the control of governments and central banks, and users can send Bitcoin payments over a peer-to-peer (P2P) network.
  2. Many blockchain and crypto projects have created their crypto tokens through Ethreum's blockchain platform and intend to disrupt the centralized economy. For e.g. Storj is a decentralized cloud storage network that could one day disrupt cloud computing giants such as Amazon, Google, Microsoft and IBM.

Immutable records on blockchain give users the peace of mind that their transaction records are tamper-proof, and this creates trust in the system. However, for the purpose of this article, I will focus on the decentralization aspect of this technology.

How is a decentralized blockchain implemented?

The main concept of a decentralized blockchain is a P2P network. “Knot”, d. H. Computers on this network have all of the information on the blockchain, so each node is a ledger of all transactions. This is why we also call blockchain “Distributed Ledger Technology”.

As you can see, there is no central administrator in this P2P network, so no one can censor or mediate. Hence, blockchain technology eliminates middlemen. This enables peer-to-peer transactions that enabled many new business models after the Ethereum project introduced the concept of the “smart contract”.

Also, consider the network's advantage over hackers. Hackers enjoy an advantage if they can exploit a "single point of failure". A central server is a popular target for hackers. However, there are many nodes in the blockchain and they all have the entire transaction book!

Even if hackers take over one node, there are always other nodes, and hackers cannot manage to hijack all of them! In this distributed network, hackers cannot perform a “51% attack”. A large part of the computing power in the network is captured during these attacks. How many computers will hackers overwhelm?

Thanks to the decentralization, blockchain is very secure. Cryptographic hash functions, the encryption of data with public and private keys and the consensus algorithm increase security.

No cyber attacker has ever hacked a public, permissionless, decentralized blockchain. The cryptocurrency hacking incidents you hear about are all cases of hackers attacking centralized servers of crypto exchanges.


Even the Ethereum DAO hack failed to target the blockchain network. Only a loophole was used in the Ethereum DAO smart contract code that runs over the Ethereum blockchain. Read more about it in the “Beginner's Guide: What is Ethereum Classic?”.

The cost of the decentralized blockchain network

Before I can explain what blockchain sharding is, I need to explain the context in which the idea even popped up in the minds of blockchain developers. You have seen the benefits of the decentralized blockchain network, but it also has a cost.

The most famous blockchain networks, e.g. Bitcoin and Ethereum use a consensus algorithm called “Proof of Work” (POW). It is required that all nodes participate in the transaction validation process. Read more about it in "PoW Vs. PoS: A Comparison Between Two Blockchain Consensus Algorithms".

This requires that each node process all transaction validation requests, so each node must store all transactions. Bitcoin, Ethereum and similar popular blockchain networks are growing with more users and transactions every day. This means that nodes have to store an increasing number of transactions.

When a new user runs a full Bitcoin node, the “Initial Block Download” (IBD) can take several days! Read this Bitcoin StackExchange discussion thread to see how time consuming this process is.

In addition, all transaction validation processes on these blockchain networks are sequential, i.e. H. Transaction validation for multiple blocks cannot continue at the same time. Since every node has to take part in the validation, the blockchain network is only as fast as the slowest node!

While this requirement on each node storing all transactions secures the public blockchain networks, it also makes those networks less scalable. Blockchain developers started thinking of alternatives because of this problem.

The concept of blockchain sharding emerged from database sharding

The concept of sharding comes from database management technology, and the word "shard" means "a small part of a whole". It is the partitioning of a large database into smaller parts that can be stored on different server instances.

There are indexing mechanisms for shards. Depending on the database query, the system retrieves the data from the corresponding shard. This makes databases more powerful and scalable. Read more about database sharding in this TechTarget Definition of Sharding.

So what is blockchain sharding? The blockchain database follows the database sharding concept exactly and is divided into horizontal partitions. One group of nodes manages one such partition, while another group of nodes manages another shard.

This means that all nodes no longer have to store the entire blockchain database. With this arrangement, even slower nodes can now work faster because they don't have to load the entire ledger. This improves the scalability of the networks.

Sharding requires a different blockchain consensus mechanism

By now you can see that when implementing blockchain sharding, nodes can no longer see the entire blockchain database. Then how does the POW consensus algorithm work? All nodes need to take part in the transaction validation, and now nodes can't even see the entire blockchain ledger!

Blockchain sharding requires a different blockchain consensus algorithm called Proof of Stake (PoS). With this algorithm, some nodes issue their own crypto tokens and take responsibility for transaction validation.

The more tokens a node sets and the longer the duration of use, the higher the probability that this node will be responsible for transaction validation. We call them “Stakers” ..

Since the implementation of sharding excludes the POW algorithm for transaction validation, the network must identify stakers for each shard that validates transactions. Therefore, a blockchain network must use the PoS algorithm to implement sharding.

Disadvantages of blockchain sharding

A discussion of what blockchain sharding is will be incomplete without discussing the cons. Note that the database sharding concept is not that easy!

You need very good database experts on your project team who can plan a very good indexing strategy for your database shards. Although in a different way, you need to plan the sharding of your blockchain ledger very well.

Occasionally, you can also hear that sharding can improve the scalability of the blockchain network, but at the expense of security. However, you need to consider what blockchain sharding is - it's just a partitioning technique. Partitioning a database alone cannot compromise the security of a database.

It's actually the PoS algorithm that provides less decentralized security, not sharding. If a hacker is buying a lot of crypto tokens and wagering them, they are likely to be a very preferred staker. He can then manipulate transactions.

However, the natural economic dynamism offers insurance against it. Any person who buys too many crypto tokens will get a lot of attention and drive the price up. In addition to the spotlight, the hacker must increasingly spend more money to eventually manipulate transactions.

In addition, the proposed "Casper" protocol for Ethereen provides for the planned transition to the PoS algorithm in order to assign stakers in a random manner. This reduces the likelihood of a malicious staker tampering with transactions. The Casper Protocol also suggests locking the bet amount and confiscating it for malicious stakers who won't have a chance to bet in the future.

Blockchain sharding is a relatively new concept. The SHARD Coin project uses it. We need to see how the technology evolves and whether it offers sustainable added value for the scalability and performance of blockchain.

Sorry! The author has not filled his profile.
Thanks! You've already liked this