Published on: 2018-05-24T04:02:17+00:00
The email thread revolves around the discussion of compression ratios for TXO (transaction output) bitfields and their feasibility in committing to block headers. The main objective is to compress the bitfields when sending proofs of spentness/unspentness to light clients, especially considering bandwidth limitations. One proposed solution is constructing a Merkle Mountain Range over the bitfield chunks to efficiently prove spentness to clients. The top graph presented in the email shows the compression ratios achievable on a TXO bitfield split into 4 KiB chunks using gzip (level=9) and lz4. The data was collected at block height 523,303. It was observed that the compression ratio is lower for older chunks and even worse for more recent blocks. Over the entire history, gzip achieves a compression ratio of 34.4%, lz4 achieves 54.8%, and bz2 achieves 37.6%.Additionally, the bottom graph illustrates the sizes of per-block bitfields, which are more suitable for constructions where an output's position is determined by its block hash plus relative index. In terms of overall compression ratios, gzip achieves 50%, lz4 achieves 70%, and bz2 achieves 61.5% for these separately stored bitfields.During the discussion, Bram Cohen suggests that using run length encoding on the number of repeated bits and compressing it with elias omega encoding could potentially yield better results than standard compression algorithms as it natively handles bitfields.Overall, the research aims to explore the possibility of committing to TXO bitfields with block headers to enable UTXO inclusion/exclusion proofs for light clients. The analysis of compression ratios reveals the efficiency of different algorithms, highlighting the importance of choosing the appropriate compression method based on the nature of the data.
Updated on: 2023-08-01T23:18:28.758991+00:00