Rolling UTXO set hashes [combined summary]



Individual post summaries: Click here to read the original discussion on the bitcoin-dev mailing list

Published on: 2017-05-23T20:43:45+00:00


Summary:

A discussion on the bitcoin-dev mailing list focused on the efficient construction of UTXO commitments. The group considered the ability of lightweight nodes to validate the hash without accessing the entire UTXO set. One proposed method involved using SHA512 with a truncated output as a key to seed a ChaCha20 PRNG. Another approach suggested a simple txid||outnumber hash that could be validated by lightweight nodes.The debate also touched on the serialization of transaction outputs (txout). One approach was minimal and had no normative data structures, while another reused existing primitives. Gregory Maxwell preferred the second approach as it allowed for simple validation without knowledge of the UTXO set. However, he mentioned that in a SegWit world, including additional information in the serialization would result in a larger UTXO set required for validation.Peter Todd argued that previous UTXO commitment schemes did not require miners to participate in generating a commitment. However, network-wide choices were still necessary for these approaches to be useful. In a partial deployment, producing proofs of unspentness became more expensive than running an old or new full node. Todd suggested investigating solutions to efficiently compare UTXO sets separately from reducing full node costs.The advantages of committing to UTXO data were discussed, particularly for lite nodes. Lite nodes could download UTXO sets from full nodes and verify them by validating block headers starting from the genesis block. Compact membership proofs were deemed necessary for most arguments. Additionally, there was growing interest in full block lite clients for privacy reasons, which complemented membership proofless usage.Pieter Wuille proposed gaining experience with operating without committing to UTXO data before implementing it. He emphasized the need for careful consideration due to the consensus change involved. Wuille also highlighted the importance of adding the merkle tree root of transactions to the commitment for audibility and engineering reasons. Otherwise, DOS attacks could occur. There was also a suggestion to commit to the UTXO of the previous block and the merkle tree root of transactions in the current block to reduce SPV mining.The email mentioned various use cases for a rolling set hash. It could replace Bitcoin Core's gettxoutsetinfo RPC's hash computation, making the process instant and more usable. The rolling set hash could also assist in fast sync methods with known good blocks/UTXO sets and database consistency checking. Another proposal involved a "chain-flip" hard fork where the UTXO hash would be in the block header instead of the merkle tree root of transactions.Pieter Wuille initiated a discussion on efficient UTXO set hash computation. Two approaches, MuHash and ECMH, were presented for efficient addition and deletion of set elements. Preliminary benchmarks showed that both approaches were fast enough for full nodes to maintain a rolling cryptographic checksum of the UTXO set at all times. The rolling set hash could replace the current gettxoutsetinfo RPC's hash computation and assist in fast sync methods and database consistency checking.Overall, the discussion focused on finding efficient ways to construct UTXO commitments, improve transaction outputs serialization, and explore the advantages and implementation considerations of committing to UTXO data. Various proposals and approaches were discussed, along with their potential use cases and implications.


Updated on: 2023-08-01T20:37:59.843555+00:00