Gossip Propagation, Anti-spam, and Set Reconciliation

Summary:

Alex Myers has been investigating set reconciliation as a means to reduce bandwidth and redundancy of gossip message propagation in Lightning Network. The idea is that each node will build a sketch representing its own gossip set, and Alice’s node will encode and transmit this sketch to Bob’s node, where it will be merged with his own sketch. These differences should ideally be exactly the latest missing gossip of both nodes. However, the sets must otherwise match very closely and each node and lightning implementation may have its own rules for gossip acceptance and propagation. Depending on their gossip partners, not all gossip may propagate to the entire network.Michael Folkson, who also attended Alex's presentation at Bitcoin++, asked him questions about how challenges of using Minisketch for Erlay in Bitcoin Core differ from the challenges of using Minisketch for Lightning gossip (node_announcement, channel_announcement, channel_update messages). Michael also asked why hash collisions are a concern for Lightning gossip and not for Erlay, is it not a DoS vector for both? Alex explained that introducing a short hash function that produces a 64 bit fingerprint creates a concern with collisions between hash functions, which could potentially be taken advantage of and grind out a hash that would resolve to the same fingerprint.Additionally, Michael was surprised to hear that channel_update made up 97% of gossip messages and asked if it is not recommended to make too many changes to your channel since it will likely result in failed routed payments and being dropped as a routing node for future payments. Alex replied that by default, every two weeks, each channel needs refreshed gossip to inform the network that the channel is still active and its parameters haven't changed, resulting in 90% channel updates. He agreed that there would be a benefit to nodes taking a more active role in tracking calls to broadcast gossip.The Lightning Network has implemented rate limiting for incoming channel updates and node announcements, with a default rate limit of 1 per day and a burst of 4. Analysis of publicly broadcasting half-channels over a 14-day period found that 18% of them violated the spam-limiting rules at least once, with many appearing to flap due to Tor or other intermittent connections. Relaxation of the rate limit resolved most cases, but a smaller subset of channels broadcasted frequent updates with minor adjustments, likely due to automated channel management by power users.The upcoming Core-Lightning release will update rate-limiting to achieve higher acceptance of incoming gossip, prompting discussion of a commonly adopted rate-limit standard and internal checks of RPC input. The aim is to develop a set reconciliation gossip type using a common, simple heuristic to accept or reject gossip messages. Finally, Alex mentioned that he is leaning towards per-peer sketches with inventory sets (like Erlay) rather than global sketches since it is moving in a direction where your peer connections are more stable as you are storing data on what your peer's understanding of the network is. There could even be centralized APIs which allow you to compare your current understanding of the network to the centralized service's understanding. Relevant links include minisketch, sample output, and the routing gossip RFC.

Updated on: 2023-06-03T08:22:05.878623+00:00