Published on: 2017-03-08T03:07:31+00:00
In a discussion on the Bitcoin-dev mailing list, a user raised concerns about the expense and scalability of having a commitment to a "balance" of an "address." The user argued that this approach is not particularly useful for clients. However, it was also discussed that creating infrequent utxo commitments could allow new clients to download just the contents of the utxo set without retrieving the entire blockchain history.The proposal for a commitment-suitable UTXO set "Balances" file data structure was introduced in the bitcoin-dev mailing list. This proposed structure would allow all nodes in the network to verify their UTXO set's data integrity and enable pruned nodes to start synchronizing at a Balances file's block height instead of the genesis block. To make this work, Bitcoin would need a new policy where a UTXO commitment is made every "Balances/UTXO Commitment Period" (BCP) blocks. The file/commitment should be made in a background thread, starting at least BCP/2 blocks after the last block containing a utxo commitment.The balances file summary includes a header with file type, file version, size of balances, and root hash. The balances are listed sorted by txid, with each balance containing the length, txid, and variable-length CCoins. The proposal suggests dividing the 2GB of balance data into 65536 pieces, with each piece being 32*1024 bytes. The transaction index is an array of txix and piece offset, while the Merkle tree hashes consist of leaf hashes and node hashes.One member of the Bitcoin-dev mailing list proposed using Merkle tree hashes to store output transactions and output numbers, but another member suggested using a patricia trie instead to allow for incremental updates. The proposed Merkle set format would work well for the intended usage and could be synced with the work difficulty reset interval. However, a shorter block period would require nodes to keep more copies of the balances file. On the other hand, a patricia-based format would allow for more frequent UTXO commitments with minimal cost of updating and handle reorgs by undoing the last few transactions in the set and rolling forward.The proposed Commitment-suitable UTXO set "Balances" file data structure allows pruned nodes to satisfy SPV (Simplified Payment Verification) nodes and start synchronizing at a Balances file's block height instead of the genesis block. All nodes in the network can verify their UTXO set's data integrity through this structure. A new policy is needed where a UTXO commitment is made every "Balances/UTXO Commitment Period" (BCP) blocks. The commitment is made on the state of the UTXO at BCP blocks ago, and the file/commitment is made in a background thread, starting at least BCP/2 blocks after the last block containing a utxo commitment.The Balances file summary consists of a header, balances, balance index, and Merkle tree hashes. Balances are listed by txid, and a piece represents a portion of the balance data. The proposed piece size is 32*1024 bytes. The transaction index includes the first N bytes of a txid and piece offset. The Merkle tree hashes consist of leaf hashes, node hashes, or copying up if only one child. The design suggests that the piece size should be small enough to avoid wasting effort when receiving invalid pieces and should also be small if this data structure is used instead of block history for SPV proof. The child count = 2 Merkle tree structure is necessary only if this data structure is used to satisfy SPV clients. Otherwise, the root hash could have the leaf hashes as its direct children. The balance index serves to support SPV clients.Other design notes suggest making the "Balances/UTXO Commitment Period" 5000 blocks, allowing more frequent checks on UTXO set integrity and enabling new pruning nodes to start syncing closer to the tip. The suggested design change to the chainstate "CCoinsViewDB" utxo database proposes modifying the "CCoins" data structure to keep track of spends that shouldn't be included in the commitment. Vtipspends could hold {vout_ix, blockhash} instead of just vout_ix. To check whether a txo is spent, a parameter specifying the "fork tip hash" or "fork chain" would be required. Feedback is welcome on these proposals.
Updated on: 2023-08-01T19:44:42.094778+00:00