Published on: 2017-11-27T20:49:07+00:00
In a recent email conversation on the Bitcoin-dev mailing list, Jeff Johnson presented a potential solution to increase the amount of data in a block without increasing its size on disk. He conducted tests using different compression algorithms on a raw Bitcoin block and achieved a compression ratio of approximately 50%. The compression methods he tested included Gzip, LZMA, ZStandard, and LZ4.Among these methods, LZ4 demonstrated the fastest decompression speed at 3.3GB per second, which is almost half the speed of memcpy. Considering that blocks will be decompressed significantly more times than they will be compressed, decompression time is of greater importance.If this solution were to be implemented in the Bitcoin protocol, there would need to be a mechanism to specify the compression type using a set of bits. This would allow for the potential addition of future compression algorithms. Miners would have the flexibility to decide how many transactions they want to include in a block while keeping the compressed size within the limit.To accommodate the compression, the Bitcoin client code would require modifications to handle the appropriate compression bits and adjust the limits of signature data accordingly. It is worth noting that Schnorr signatures, which are expected to be introduced as a compression gain of 25%, could potentially achieve even further compression when combined with the compression methods mentioned by Johnson.The issue of bandwidth usage during historical block transmission was also addressed. If bandwidth is a concern, on-the-fly Gzip compression similar to Apache's mod_deflate could be applied. Furthermore, if disk space is an issue, it is suggested that the database layer should handle the compression.Although previous discussions have revolved around similar approaches, it remains uncertain whether compression should be integrated into the Bitcoin protocol directly or implemented through different layers. Regardless, Jeff Johnson's findings on a recent Bitcoin block can be accessed through the link provided in the original message.In summary, Jeff Johnson has proposed a solution to increase block data without increasing its size on disk. By utilizing various compression algorithms, he achieved a 50% compression ratio on a raw Bitcoin block. The fastest decompression speed was observed with LZ4, and the practical choice for compression was ZStandard. Implementation of this solution would require miners to specify the compression type, while the Bitcoin client code would need to be modified accordingly. There is potential for further compression gains when combining Schnorr signatures with the proposed compression methods. Bandwidth concerns could be addressed through on-the-fly Gzip compression, while disk space issues may be handled by the database layer.
Updated on: 2023-08-01T22:10:14.417453+00:00