LevelDB benchmarking [combined summary]



Individual post summaries: Click here to read the original discussion on the bitcoin-dev mailing list

Published on: 2012-07-21T18:49:54+00:00


Summary:

Stefan has completed a task related to Bitcoin's project and raised a pull request for it. The pull request can be found on Github with the URL: https://github.com/bitcoin/bitcoin/pull/1619. It is mentioned that the author of the mikehearn/bitcoin repository has added more commits to the codebase, but it is not yet ready for a pull request. Auto-migration and migration progress UI are present, but testing with empty wallets still needs to be done. The LevelDB source is checked into the main tree, and bitcoin-qt.pro has been updated to use it. However, there are some remaining tasks, such as more testing with actual wallets and updating non-Qt makefiles. The author asks for help with testing and makefile work.In a recent discussion on the #bitcoin-dev platform, it was discovered that the CTxDB::ReadOwnerTxes code has never been used in the Bitcoin codebase, even in version 0.1.5. As a result, the author plans to delete the code when they next work on the patch.Another conversation on #bitcoin-dev suggests that Satoshi's code uses ordered access/iteration in at least one place where it looks up the "owner transactions" of a tx. However, it is unclear what that code is used for and whether it is the best solution for the problem.A Bitcoin forum post discusses the issue of block propagation times. It is mentioned that the regular network lags behind p2pool in propagating new blocks by 0-60 seconds. Optimizing IO load and threading transmission verification are suggested as important improvements. Lukes preview functionality is also seen as a useful tool in this regard.In a message, someone expresses gratitude for the information that BitcoinJS uses LevelDB but points out that LevelDB was originally designed for servers and comes from BigTable. The speaker anticipates that most Bitcoin nodes on servers will continue to be HDD based for the foreseeable future. Additionally, the speaker mentions that Satoshis code uses ordered access/iteration in at least one place, but questions whether it is the best way to solve the problem.LevelDB is discussed as an ordered data structure that is good for desktop software with multiple use cases. It was written to power IndexedDB in Chrome and uses LSM trees as a database type. However, it is mentioned that LevelDB may not be ideal for Bitcoin due to its lack of ordered access and predictable characteristics. Nonetheless, it seems to work well for the Bitcoin use case. The speaker suggests lowering the file descriptor limit or combining tables into one to address any potential issues.The problem of slow block verification and propagation is discussed, and making existing nodes faster is proposed as a solution. This does not replace moving users to SPV clients. The robustness of LevelDB is uncertain, but it has an API call to repair the database. However, the lack of infrastructure poses a concern regarding reworking bitcointools to read the LevelDB Blockchain.In an email exchange, Gavin Andresen seeks decisions from colleagues on switching to LevelDB as a replacement for bdb. They discuss the benefits and drawbacks of LevelDB, including its minimalist build system and compatibility with different operating systems. They suggest including the source code in the main Bitcoin tree and linking it statically for simplicity. They also discuss possible tweaks to the database schema during migration.Mike Hearn suggests moving away from BDB and replacing it with LevelDB in an email conversation. Gavin Andresen expresses concerns about LevelDB's obscurity, newness, and compatibility guarantees. He suggests including the source code in the main Bitcoin tree and linking it statically. Other tweaks to the DB format are also discussed.Gregory Maxwell suggests running ECDSA caching on multiple cores for linear speedup. He realizes that he may have been wrong about being IO bound in a benchmark. Gavin is expected to make decisions on whether to proceed with the work, how to deal with LevelDB's minimalist build system, and how to handle any necessary tweaks to the db format.The author changes the transaction database to Google LevelDB library and performs tests comparing it with Regular BDB. The results show that LevelDB is faster, reducing the running time from 95 minutes to 50. Signature checking remains the bottleneck.


Updated on: 2023-08-01T03:41:23.398194+00:00