Deciding how to incorporate stability testing into your overall test strategy can be a challenging task. In this article, we discuss how to incorporate stability testing with your feature validation, performance, and regression testing, and how to leverage CDRouter's parallel testing capabilities to streamline the process.
As we cover in our definitive guide, “How to Build an Automated Test Strategy,” there are four primary types of testing that are each applicable to different phases of product development.
As we’ve discussed in this article and video, stability testing helps identify any potential issues or limitations that may arise during the regular use of connected devices. So much of QA Testing uses a fresh reboot and does a test on the scale of an hour. Customers leave them running for weeks. The issues found by QA and customers in these two scenarios are completely different. The reality is that customers will perceive these issues as a poor device even though it is, in fact, just a bug that reveals itself after prolonged usage.
Stability testing enables developers to address problems early on and prevent them from affecting the end user's experience. In the long run, stability testing contributes to customer satisfaction and reduces the risk of device failure. You can find an example and sample test package for stability testing in CDRouter here.
Because stability testing can take up a lot of testing time, it can be difficult to make time for it in your overall test strategy. Instead of treating stability testing as an isolated process, consider integrating it with other types of testing by testing in parallel.
CDRouter's parallel testing capabilities allow you to run multiple test instances simultaneously, reducing the overall test execution time. This works great with a Continuous Integration/Continous delivery system that utilizes CDRouter’s automation API to start tests, track the results, and trigger further testing or action. Using parallel testing, a single CDRouter platform can be maximized to cover all four of the major testing types, as shown in this infographic:
In this example, four physical versions of the same product (Model A) run different types of tests in parallel. The first three are builds of Firmware #1, with DUT #1 possibly incorporating changes on a per-commit basis through a CI/CD pipeline. DUT #4 is also hardware model A, running Firmware #2 to compare it to results from Firmware #1.
Even when one CDRouter system is maximized with five devices running simultaneously, additional CDRouter systems can be used to test other products (Models B, C, D, etc.), all automated through the same CI/CD system, if desired.
Overall, CDRouter's parallel testing capabilities enable you to run testing on multiple devices concurrently, allowing you to maximize your test environment's efficiency. By using parallel testing, you can:
You’ve successfully incorporated stability testing into your testing cycles. What do you do when youuncover issues that arise from stability testing? You’ll want to isolate the issues and reproduce them efficiently and quickly so developers can fix them. Look for future articles on stability testing to address this critical next step!