We make a conjecture that most techniques in different tiers can be used jointly. In tier-based classification, we propose a novel hybrid technique framework with five tiers and conduct a comprehensive comparison between these techniques. In resource-based classification, we group these techniques according to the resource they optimize and introduce them in detail. Moreover, based on our observation, we propose two classifications: resource-based classification and tier-based classification. First, we generalize the state-of-the-art researches and introduce existing classifications. In this work, we seek to summarize them systematically and group them in a new form. However, all of these techniques were proposed individually previously. A variety of update schemes based on erasure coding have been proposed in the literature to optimize data update, such as computation optimization, network traffic overhead reduction, IO overhead reduction, and modern hardware acceleration. Compare to data repair, data update is much more common. However, it introduces two prominent issues: data repair and data update. Evaluation shows that compared to HDFS-EC, NetEC significantly improves reconstruction rate by 2.7x-9.0x and eliminates CPU overheads, with low switch memory usage.read more read lessĪbstract: Erasure coding is the leading technique to achieve resilient redundancy in cloud storage systems. We implement NetEC on hardware programmable switches. NetEC addresses three key challenges: computation offloading of complex EC operations, rate synchronization of multiple downloading streams, and deep payload inspection/assembly. NetEC prevents multiplexing network I/O through on-switch downloading stream aggregation, thus significantly improving reconstruction speed. In this paper, we present NetEC, a novel in-network accelerating system that fully offloads EC to programmable switching ASICs. However, downloading streams of EC multiplex the available network I/O on the receiving end, leading to a substantially low data reconstruction speed. EC can reconstruct missing data by downloading parity data from survived machines. Abstract: In distributed storage systems, erasure coding (EC) is a crucial technology to enable high fault tolerance with lower storage overheads than data replication.

Erasure novel