A design for Probabilistic Partial Pruning [combined summary]

• A design for Probabilistic Partial Pruning Keagan McClelland 2021-03-01 20:55:10+00:00
• A design for Probabilistic Partial Pruning eric at voskuil.org 2021-03-01 09:37:06+00:00
• A design for Probabilistic Partial Pruning Craig Raw 2021-03-01 06:22:01+00:00
• A design for Probabilistic Partial Pruning Leo Wandersleb 2021-02-28 03:41:06+00:00
• A design for Probabilistic Partial Pruning David A. Harding 2021-02-27 23:37:52+00:00
• A design for Probabilistic Partial Pruning Yuval Kogman 2021-02-27 22:13:29+00:00
• A design for Probabilistic Partial Pruning Yuval Kogman 2021-02-27 22:09:48+00:00
• A design for Probabilistic Partial Pruning David A. Harding 2021-02-27 19:19:34+00:00
• A design for Probabilistic Partial Pruning Igor Cota 2021-02-27 07:10:39+00:00
• A design for Probabilistic Partial Pruning Keagan McClelland 2021-02-26 18:40:35+00:00

Published on: 2021-03-01T20:55:10+00:00

Summary:

The bitcoin-dev mailing list has been discussing a proposal to address the issue of ongoing storage costs for full nodes and pruned nodes. Currently, users have two options: either prune everything except the most recent blocks or keep everything since genesis. The proposed solution is to randomly prune some of the blocks from history based on a unique node seed and threshold expressed as a percentage of blocks. This would increase data redundancy and share the load across nodes more uniformly, reducing the pressure on full archive nodes during the initial block download (IBD) process.The discussion also explores the use of fountain codes to increase data redundancy and improve the distribution of blocks. Fountain codes create an infinite stream of codewords that can be XORed to recover the full message. This approach, known as SeF, retains a fixed number of codewords from the encoding concatenated blocks and serves them to compatible clients. It has been found to be more robust than retaining a random sample of blocks.The proposal suggests advertising the node's seed and threshold as a peer service once the IBD is complete. This would allow other nodes to determine which blocks each peer has. However, concerns about fingerprinting weaknesses have been raised, and it is noted that popular node software is unlikely to adopt this idea unless it solves an urgent problem.Another suggestion is to decide on a range of blocks to keep beforehand, rather than making the decision block-by-block. This could be computationally lighter and better serve other nodes due to the sequential nature of IBD. A paper proposing a distributed hash table (DHT) scheme by Ryosuke Abe is recommended for further reading.Overall, the proposal aims to provide a solution to the ongoing storage costs for full nodes and pruned nodes by randomly pruning blocks from history based on a unique node seed and threshold. Feedback is being sought on the protocol design and the barriers to implementing this functionality into Core.

Updated on: 2023-08-02T03:15:34.692962+00:00