Compact Block Relay BIP

Summary:

There was some confusion over an email that was posted to the bitcoin-dev mailing list regarding a BIP-formatted design spec for compact block relay. The email contained suggestions from Tom about improvements to the protocol flow and data structures used. The email acknowledged the work of Matt Corallo but suggested that there could be more improvements made.In the section titled "Intended Protocol Flow," Tom suggested that a new block should be announced to all nodes equally, and individual nodes can respond with a request of either a 'compact' or normal block. He also suggested how detection of remote nodes that support compact blocks can be done. In the section titled "Short transaction IDs," Tom suggested that using just the first 8 bytes would suffice instead of calculating short transaction IDs. Finally, in the section titled "Acknowledgements," he suggested that Matt should acknowledge more people or remove the paragraph.The Bitcoin Improvement Proposal (BIP) describes a new protocol for relaying blocks that reduces the amount of data being transmitted. The proposal introduces a new inv type called MSG_CMPCT_BLOCK and several new protocol messages including sendcmpct, cmpctblock, getblocktxn, and blocktxn. Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. These short transaction IDs are calculated by single-SHA256 hashing the block header with the nonce appended in little-endian, XORing each 8-byte chunk of the double-SHA256 transaction hash with each corresponding 8-byte chunk of the hash from the previous step, and adding each of the XORed 8-byte chunks together iteratively to find the short transaction ID.Nodes are recommended to send a sendcmpct message to up to three peers selected based on their past performance in providing blocks quickly. All nodes should send a sendcmpct message to all appropriate peers to reduce their outbound bandwidth usage. Nodes with limited inbound bandwidth should request blocks using MSG_CMPCT_BLOCK/getblocktxn requests when possible. Nodes sending cmpctblock messages should limit prefilledtxn to only around 10KB of transactions and only include the coinbase transaction when in doubt.The MSG_CMPCT_BLOCK section does not require that nodes respond to MSG_CMPCT_BLOCK getdata requests for blocks which they did not recently announce. Thus, nodes must not request blocks using MSG_CMPCT_BLOCK getdatas unless it is in response to an inv/headers block announcement, and must not request blocks using MSG_CMPCT_BLOCK getdatas in response to headers messages which were, themselves, responses to getheaders requests. While the current version sends transactions with the same encodings as is used in tx messages and elsewhere in the protocol, the version field in sendcmpct is intended to allow this to change in the future. For this reason, it is recommended that the code used to decode PrefilledTransaction and BlockTransactions messages be prepared to take a different transaction encoding if and when the version field in sendcmpct changes in a future BIP.The Bitcoin network has announced a new policy that aims to relay blocks between nodes in 0.5*RTT instead of 1.5*RTT at least 75% of the time. The protocol will use New VarInts, which saves an extra byte per transaction-difference, and short transaction ID calculation, which takes minimal processing time during block compaction to avoid introducing serious DoS vulnerabilities. The XOR-and-add method is used for calculating short transaction IDs primarily because it is fast and can limit the ability of an attacker who does not know the block hash or nonce to cause collisions in short transaction IDs.Backward compatibility is also ensured with older clients remaining fully compatible and interoperable after this change. The implementation can be found on Github and thanks were given to Gregory Maxwell for the initial suggestion as well as a lot of back-and-forth design and significant testing. This document is placed in the public domain.

Updated on: 2023-06-11T04:51:14.031737+00:00