What Steemit need to do next? SHARDING Solution for horizontal scalability of STEEM!

When it comes to the scalability of data storage, there are two solutions.

Vertical scalability

Increase the capacity of server components (in terms of processors, RAM, storage capacity, etc.) to handle data growth.

Solution currently used by the STEEM blockchain and requiring a high-powered machine to make the blockchain run correctly. And it's not Mira (Multi Index RocksDB Adapter) even if it's a good thing for the STEEM blockchain who will bring an adapted solution, it will only serve to delay the deadline where the maintenance of the blockchain will become too expensive and will be reserved for an elite. It's not for nothing that the biggest interruptions of the STEEM blockchain lately is when a full node is down. And for a good reason, there are only 8 active! (the list of the 8 in the excellent daily Full API Node Update from @holger80)

Horizontal scalability

Increase the capacity by adding resources (servers) in order to distribute the data growth in a load balanced way. A solution avoiding elitism since it allows to stay on reasonable server configurations. Bringing flexibility, since it only needs to add new machines and not to upgrade the existing one (no interruption of service or risk related to a hardware upgrade that goes wrong).

In the case of databases this results in the partitioning of data into smaller parts (shards) which are distributed on different servers. Necessary solution to overcome the exponential increase in the size of data to be stored.

What is sharding?

Sharding is a form of database partitioning, it can split a very large database into smaller, more manageable segments. It can also enable geolocation of data, thus shortening the path between the user and the data of interest (for example a post in French that would be on a server in France).

Ok, but in case of a blockchain?

It also works even if the difficulty is greater because of the need to keep a protection against Byzantine attacks and to remain totally decentralized.

As a proof, Ethereum is working on its 2 next evolutions which are Plasma (network of sidechains) and Sharding (Partition the existing blockchain into smaller pieces).

In fact, it's not the only one! You can take a look at Cardano too and many others.

The most interesting case

To make this a little more interesting, let's look at the case of Elrond "The Internet Scale Blockchain" (open source project).

14th June 2019: Testnet Online
Server: AWS T2.Medium machines (dual-core processor with 4GB of RAM)
Number of Server: 126

Few days later, 100 million transactions with more than 100,000 blocks per shard (1 block/6 seconds)

• With only 5 servers: 12,500 Transaction Per Second (theoretical, mesured 11,966 TPS)
• With 20 servers: 65,000 Transaction Per Second (TPS)

In comparison, for a STEEM witness node you need at a minimum:

• 4 core Xeon
• 64 GB RAM
• 500 GB SSD

And for a full node:

• 8 core Xeon CPU
• 256 GB RAM
• 1 TB NVME

Conclusion

STEEM has a number of advantages, Communities and Smart Media Tokens from @steemit, Steem Smart Contracts created by @harpagon, Steem-engine from @aggroed and @yabapmatt in partnership with @harpagon, a lot of great frontend made by the Community without forgetting a very wide range of different types of operations possible on the blockchain. But it also has a big weakness, its scalability.

If we want STEEM to stay in the race, the next big project must be the implementation of horizontal scalability.

How about you? What do you think? What should be the next big challenge for STEEM?

If you liked this article
Feel free to let me know by
[ Upvote | Resteem | Comment ]
THANKS :)
@french-tech