Published on: 2011-09-08T10:29:50+00:00
To avoid confusion and repetition, it is recommended to keep the discussion on a single mailing list. Duplicate threads in different places can lead to redundancy and people making the same points repeatedly. In this specific case, the discussion was initially intended for the c developers' list but was also posted in another forum to gather more responses. However, to streamline the discussion, it has been decided to move it to the bitcoinj list. The link to the other list has been provided for those interested in following the conversation.The proposed solution in the mailing list discussion suggests addressing version skew by disconnecting older clients and not sending received transactions/blocks to the bitcoind. Instead, the bitcoind will hear about them from its own p2p connections, allowing for the reception of all valid transactions/blocks that can be relayed, cached, or have inbound duplicates dropped. To handle inv/getblocks/getheaders requests, the proxy can parse and handle them so that clients connecting and catching up with the chain don't place any load on the bitcoind. If a client requests data that the proxy doesn't have in RAM, it can fetch it from the underlying bitcoind. The author proposes starting with this simple approach first and then improving upon it in v2 if scalability improvements are demonstrated in v1.Another solution suggested is to serialize all request/response exchanges. When a request comes in from a remote node, the proxy acquires a lock on the proxy-localdaemon channel and sends the request. The channel remains locked until a response is received or a timeout occurs, in which case the remote node will not receive a response. Once a response is received, the channel is unlocked and the response is sent to the client. Messages that do not expect a response, such as relaying a transaction broadcast from a remote node, can be pushed down a locked channel to improve performance as they will not interfere with sequencing. However, there may be idle time for the channel while waiting for a response.In addition to the above solution, the suggestion is made to use all 125 available bitcoind connections in a channel pool. Acquiring a lock on a channel involves checking for an unlocked channel first and then waiting in a queue for one to become available.The author plans to create a proof of concept for the multiplexing proxy discussed by Gavin and Mike. Initially, the plan is to make it a dumb proxy between one local bitcoind and one remote node. However, dealing with request-response exchanges from multiple remote nodes poses a challenge. Three approaches are suggested to tackle this problem. The first approach involves generating a unique key from both the request and response to recycle responses for similar requests from different clients. However, this approach may require large keys for guaranteed uniqueness and carries the risk of sending back the wrong response.The second approach proposes modifying bitcoind to accept sequence numbers for request/response type messages, similar to the 'id' field in json-rpc. This approach is more reliable but invasive to the bitcoin protocol and loses the inherent de-duplication of requests present in the first approach.The third approach suggests making the proxy intelligent enough to handle these requests itself. The proxy maintains its own local cache of headers and handles getheaders messages by checking if it has all the requested headers. If not, it requests the missing ones from the local daemon, adds them to its cache, and builds a headers response itself. This approach achieves the best combination of request/response matching reliability and de-duplication of work but requires more protocol awareness and creates maintenance dependency for future protocol changes.The author leans towards the first approach for an easier initial implementation with the intention of eventually moving to the third option. Before studying the protocol, the author believed that the third approach would be impossible, but the depth of protocol awareness for the first and third approaches is not significantly different. Option two allows for a thinner proxy but sacrifices potential efficiencies. Ultimately, the proxy should progress to option three in the future, but the routing problem needs to be solved from the beginning. The author welcomes suggestions for a simpler approach to the request/response routing problem.
Updated on: 2023-08-01T02:24:54.317798+00:00