Stability testing is an extremely valuable process you should add to your overall automated testing strategy. In this video, Matt summarizes what to do once you’ve got some stability test results: what to look for, how to find it, and what to do next to help improve your products.
In summary, Matt suggests the following tips once you have some stability test results available:
Use the Performance Visualization tool within CDRouter to review the performance over time. Spot any unexpected dips or drops can highlight potential problems that need further investigation.
If an anomaly is detected, the first action is to verify its consistency. Rerun the test package to see if the issue can be reproduced. Reproducibility is important in validating the stability issue and not dismissing it as a one-off event.
Check the functional results from each loop of the test to ensure consistency. This step involves looking at two aspects:
Often (but not always), functional failures and performance degradation are related. An increase in the duration of individual tests, or anomalies in log data, may hint at the source of the performance issue. With these correlations, you can drill down to find clues into the underlying cause of the problem and give you insight into what smaller duration test runs you can perform to isolate and resolve the issue.
If your test results don't reveal anything, it's an opportunity to expand your stability testing. This can include:
Use CDRouter's parallel testing to conduct multiple stability tests at once. This strategy can be useful for testing different device configurations simultaneously, offering a comprehensive view of the system's overall stability.
Stability testing is a critical part of the product development cycle that plays a pivotal role in ensuring the robustness and reliability of devices over extended periods of use. Stability testing using CDRouter ensures consistent product performance but also helps to uncover potential issues that might otherwise go unnoticed. The power of this testing lies in the ability to stress the device with real-world network environments and interactions quickly and repeatably, finding issues that will affect the quality of your product and the end-user experience long before they appear in the field.
Find the rest of our series on stability testing here: