Published on: 2012-10-10T00:03:52+00:00
Jeff Garzik suggests that in order to improve software testing for Bitcoin, data-driven tests should be written that are compatible with clients other than the reference client. He also recommends embedding tests in the testnet3 chain for better testing. The discussion then turns to the issue of confirmation bias in software testing, which leads to over-testing success cases and under-testing failure cases. It is emphasized that failure cases are critical for testing implementation compatibility and preventing network splits.To ensure comprehensive testing, it is essential to have a diverse collection of testers who can think outside the box and test corner cases. Developers should use multiple approaches rather than relying on a single one. Combining different mechanisms of software testing enhances the detection of issues. For example, using valgrind significantly improves almost all testing by lowering the threshold of issue detection. Automatable testing such as coded data-driven, unit, and fuzz testing works well with diverse compilation.However, it is noted that valgrind is not available on Windows. Dr. Memory may be an alternative, or ASAN could be ported to GCC to enable mingw ASAN builds. Arklan expresses gratitude to Jeff Garzik for sharing his response on software testing efforts for Bitcoin. Jeff suggests various types of testing that can be helpful for the community, including "it works" testing, major features testing, stress and fuzz testing, regression testing, unit function testing, full peer automated testing, and data-driven tests. He encourages users to download and run the latest version of Bitcoin software, report any problems or success, and contribute to testing and test-case writing at any level they are comfortable with.Arklan plans to set up a virtual machine to start testing himself and asks Steve for updates on the test cases he set up. The context concludes with two quotes by Arklan about facing fear and leaving the world with a bang.In summary, Jeff Garzik suggests using data-driven tests that are compatible with clients other than the reference client and embedding tests in the testnet3 chain for better testing. Confirmation bias in software testing is discussed, highlighting the importance of testing failure cases for implementation compatibility and network split prevention. It is advised to have a diverse collection of testers who can think outside the box and test corner cases. Using multiple approaches to software testing enhances issue detection, such as incorporating valgrind or alternatives like Dr. Memory or porting ASAN to GCC for mingw ASAN builds.Arklan expresses gratitude for Jeff Garzik's suggestions and plans to set up a virtual machine for testing. The community is encouraged to download and run the latest version of Bitcoin software, report any issues or success, and contribute to testing and test-case writing at any level they are comfortable with. The context ends with two quotes by Arklan about facing fear and leaving a lasting impact.
Updated on: 2023-08-01T03:59:36.592911+00:00