[BIP Draft] Datastream compression of Blocks and Transactions



Summary:

The email discusses the idea of using dictionaries that already contain data to compress messages being transmitted. Compressed messages are the same as uncompressed messages, but with the addition of indexing data and compressed data. The receiver must indicate that compressed data is preferred and the height of the latest valid block it holds, and the sender must express the ability to send compressed data. Compressed data can use the first 16,777,215 blocks or the last 4,244,635,647 ending with the block at the tip of the receiver's chain as a dictionary, or it can specify a run of zero bytes. Decompression identifies a block height and adds size bytes from that block to the decompressed data. Step 3.3 of decompression causes an error but could be used to indicate a parameterized dictionary entry. Matt Corallo expressed concerns about adding compression libraries that are directly accessible to the network on financial software, while the author argued that LZO has no current security issues and is configurable by each node operator. The author provided a table showing that as block sizes increase, the performance improvement by compressing data also increases. Compression helps in the transmission of objects, but not in post-processing and validating the block. The only issue that required testing was how much time the compression of data would add to sending the data.


Updated on: 2023-06-11T01:32:21.691622+00:00