Author: Olaoluwa Osuntokun 2017-06-09 03:59:17
Published on: 2017-06-09T03:59:17+00:00
The Bitcoin Improvement Proposal (BIP) has proposed a new light-client mode using client-side filtering based off of Golomb-Rice coded sets. This is intended to support a more lightweight back-end that enables the usage of Lightning on mobile phones and other devices. Full-nodes maintain an additional index of the chain, which they serve as a compact filter to light clients who request them. Light clients then fetch these filters, query locally and possibly fetch the block if a relevant item matches. The proposal includes a concrete specification with reference implementations for client-side filtering reversal of BIP-37.The proposal specifies a fixed false-positive rate of P = 1/2^20 as opposed to allowing clients to dynamically tune their false positive rate. The document also includes test vectors for a series of fp rates (1/2 to 1/2^32) which include data such as the block height, block hash, raw block itself, previous basic+extended filter header and more. The latest version includes fixes for typos, incorrect components of the specification and some sections have been clarified. Parameter selection is an area where feedback is sought. The team has done analysis and optimization attempting to optimize the sum of filter_download_bandwidth and expected_block_false_positive_bandwidth. Alex has made a JS calculator that allows users to explore the affect of tweaking the false positive rate in addition to the number of items. Finally, the proposal includes statistics for Feb 2016 until about a month ago (since height 400k iirc) on total size, total average, total median, total max, regular size, regular avg, regular median, regular max, extended size, extended avg, extended median and extended max.Laolu Osuntokun, an engineer at Lightning Labs, has shared some statistics on the size of Bitcoin's new privacy layer, Neutrino. The privacy layer uses a probabilistic method called Golomb-coded sets (GCS) to reduce the amount of data that needs to be transmitted when syncing a Bitcoin wallet with the network. Laolu used a calculator to estimate the expected bandwidth utilization using the CDF of the Geometric Distribution. The results show that the total size of the index for mainnet comes out to 6.9 GB. Laolu also uploaded a CSV file of raw data for each block (mainnet + testnet).Karl Johan Alm (kallewoof) has done an extensive analysis of his own, focusing on a distinct encoding type. He found that his filters use a binomial encoding directly on the filter contents, while Lightning Labs creates a Golomb-Coded set with the contents being hashes (they use siphash) of the filter items.In order to consider the average+median filter sizes in a world worth larger blocks, Laolu also ran the index for testnet. The maximum filter sizes are a bit larger due to some of the habitual blocks he created last year when testing segwit (transactions with 30k inputs, 30k outputs, etc). The results show that the total size of the index for testnet comes out to 2.75 GB.The privacy layer is implemented in Lightning Lab's Neutrino client [1]. The implementation uses a forked version of btcd [2], which includes support for GCS filters. The GCS code is included in a separate repository [3]. Lightning Labs is also behind the development of the Lightning Network daemon, lnd [4]. They hope to receive feedback from users and developers regarding their findings.
Updated on: 2023-05-20T02:42:46.451156+00:00