Celestia's Shwap Upgrade Enhances Data Availability with 12x Faster Sampling
Iris Coleman Oct 15, 2024 11:58
Celestia's new Shwap upgrade boosts data availability sampling speed by 12x, reduces storage needs by 16.5x, and paves the way for larger block sizes in blockchain technology.
Celestia Mainnet Beta has successfully completed its initial upgrade, introducing a new set of features to its consensus network. The upgrade, known as Shwap, is now active on the Arabica and Mocha testnets with version 0.18.2 of celestia-node, significantly enhancing the data availability (DA) network by making DA sampling 12 times faster and reducing storage requirements by 16.5 times. This development aims to support larger blocks and smaller nodes, according to the Celestia Blog.
Key Features of Shwap
Shwap is a crucial step towards Celestia's roadmap target of scaling to 1-gigabyte blocks, as outlined in CIP-19. It introduces a new messaging protocol and storage system, enabling significant scaling of DA throughput while maintaining low requirements for light nodes. This upgrade accelerates existing crypto applications and enables new, unstoppable applications across various virtual machines (VMs). Additionally, it facilitates running light nodes within browsers or wallets, paving the way for verifiable web applications.
Following further testing, Shwap is expected to be integrated into the Mainnet Beta by November. The exact timing of these network upgrades will be determined by the Celestia community.
Current State of the DA Network
The DA network operates using two protocols: one for circulating chain headers via go-header and another for managing block data as a data square of shares or samples. These shares are sampled by light nodes, and the existing protocol, developed during the Devnet era, relies heavily on hash-addressability, which has proven unscalable for larger block sizes. Shwap addresses these limitations by introducing a new approach to sampling and storage.
Shwap's Innovations
The Shwap upgrade introduces a new data square storage system and a composable networking framework, achieving O(1) data availability sampling (DAS). This eliminates the need for storing extensive Merkle proofs and global indices, resulting in a 16.5x improvement in storage efficiency. It also introduces an immutable file format that supports optional parity data, enhancing data retrieval speeds.
Networking Enhancements
Shwap enhances Celestia's DA network with a flexible messaging framework that supports various transport protocols like TCP/QUIC and application layer protocols such as HTTP/x. Key networking improvements include expressivity, protocol decoupling, and ShrEx integration, which collectively improve block data synchronization and reduce resource consumption. The integration with Bitswap further optimizes DAS, reducing the need for multiple round-trips to a single trip for data retrieval.
Impact on Light Nodes
For light nodes, Shwap's O(1) sampling reduces DAS time from 24 hours to just 2 hours and decreases storage requirements by 2.66 times. This makes light nodes more efficient and capable of maintaining network participation with minimal resources.
Future Prospects
While Shwap represents a significant milestone towards achieving 1GB block data squares, further optimizations are needed to reach this goal. The DA network, with its new foundation, is expected to focus on maintenance, security enhancements, and performance improvements, without major rewrites in storage and networking protocols.
Image source: Shutterstock.jpg)