Building a node crawler to map network



Summary:

Rick has developed minna patches for nio based on bitcoinj and is working on a DNS seed service as well as some weather reports. He is happy to start a branch when the committers are ready. On the other hand, Steve started building a node crawler to map out the bitcoin network and provide useful statistics using a mutilated bitcoinj to connect. He is hoping to get input into what would be useful as well as strategy for getting the maximum possible connections without distorted data. He is storing version message and the results of getaddr for each node that he can connect to. Steve says that the vast majority of nodes do not accept incoming connections due to the lack of upnp, so it seems the active crawl approach is not ideal for the purpose. A honeypot approach would probably be better if there was some way to make a node 'attractive' to other nodes to connect to. If there is a way to influence other nodes to connect to the crawler node that solves the problem. If there isn't, then perhaps another approach is to build an easy-to-deploy crawler node that many volunteers could run and that could then upload collected data to a central repository. Steve is also asking questions regarding the getaddr message and how most nodes return about 1000 addresses in response to this message. He wonders where this list comes from, whether it's a mixture of addresses obtained from other nodes somehow sorted by timestamp, and whether it includes some nodes discovered by IRC/DNS or if those are only used to find the first nodes to connect to. Finally, he is looking for input to build something that's useful for the network.


Updated on: 2023-06-04T18:57:27.738366+00:00