Author: Peter Tschipper 2015-12-02 20:16:19
Published on: 2015-12-02T20:16:19+00:00
Building a custom, Bitcoin-aware compressor may provide higher compression ratios than the Lempel-Ziv family of compressors. While LZ compressors assign unique numbers to every new subsequence they encounter, they are nowhere as compact as they could be for mostly binary data. A Bitcoin-specific compressor can learn the sequence that may be repeated later on and replace it with a short number referring back to the long sequence. However, Bitcoin-specific compressors will be closely tied to the contents of messages, which might make it difficult to change the wire format later on. Additionally, all outputs are verified separately later on during processing, so compression/decompression errors can potentially be detected. The nature of LZ compressors leads to much higher compression ratios being possible by building a custom, Bitcoin-aware compressor. Compression can buy additional throughput at zero cost, modulo code complexity. Given the amount of acrimonious debate over the block size, it seems criminal to leave potentially free improvements on the table. Building a custom compressor could be done through a programming challenge/contest, which would bring in new programmers into the ecosystem. Many would love to discover the limits of compressibility for Bitcoin bits on a wire. The results would be interesting even if the final compression engine is not enabled by default, or not even merged.
Updated on: 2023-06-11T01:30:54.001494+00:00