Automating RDK-B Testing with Sophie Pool of DT

With the rapid expansion in both power and scale of broadband as a critical component of people's lives, the need for efficient, effective quality control and testing of Broadband CPE and Wi-Fi routers has never been more important.

In this special session with Sophie Pool, she explores Deutsche Telekom (DT)’s innovative and rigorous approach to automated testing that includes CDRouter as a core component. Their approach to Broadband CPE and Wi-Fi Router testing, particularly within their Reference Design Kit (RDK-B) program, offers a wealth of insights for the broader industry that are extremely valuable to project leads who have to wrangle the development and deployment of these complex, powerful devices and the end-user experience that they provide. Watch the video above, with our helpful summary and recommendations below.

Many of these are echoed in our comprehensive guide for developing an automated test strategy, which is an excellent companion to these recommendations!

Guiding principles of DT’s RDK QA testing

The DT RDK QA team focuses on delivering a test program based on three core principles. Together, they have a profound impact on the overall product quality and customer satisfaction.

Speed of deployment

Testing should be able to be done quickly and reliably, with the ultimate goal to deliver new builds and features to their CPE as fast as possible with fewer bugs. A key part of this is the choice to rely on an open-source CPE operating system like RDK-B, which helps ensure that features will work well on a standardized platform.

Speed of testing also means automating more for faster feedback and iterations. The faster the testing team can obtain feedback from their tests, the quicker they can make the necessary adjustments. This not only helps in refining the product but also in preempting potential issues before they escalate.

Scale of testing

DT’s team is always looking to expand the amount of testing they are able to do. That includes both increasing the number and kinds of products able to be tested simultaneously, and increasing the scope of tests that are run on each build.

Simplicity of the process

To make it easier to stay consistent in testing and be able to quickly isolate problems, the DT team makes simplicity one of their core goals, including:

  • Streamlined processes: Sophie highlights the importance of having testing processes that are straightforward. This ensures that the testing team can focus on their primary task without being bogged down by cumbersome procedures.
  • Ease of replication: A simple, well-documented testing process ensures that it can be replicated across different teams and geographies, and that testing can be repeated consistently from one run to another.
  • Intuitive tools and platforms: The selection of tools like CDRouter, which offers an easy setup and extensive training support, underscores the emphasis on ease of use for their testing team. When the tools are intuitive, the team can direct their energies towards innovation and problem-solving rather than grappling with complicated custom interfaces.

How DT uses CDRouter as part of their test strategy

CDRouter is an integral tool in Deutsche Telekom's toolkit, working within their automation frameworks to accomplish repeatable, comprehensive testing largely during the integration phase of development. DT uses CDRouter in the following ways:

Build validation

  • Nightly, weekly, and sprint validation: DT uses CDRouter to ensure that every software build, whether developed nightly, weekly, or at the end of a sprint, undergoes rigorous testing. This cadence ensures that any introduced changes or updates don't inadvertently introduce new issues.
  • Adjusting test packages to specific cases: The DT team utilizes CDRouter’s ability to build and adjust test packages to tailor them to the nightly, weekly, or sprint builds mentioned above. For example, nightly tests may have specific functional tests, while weekly builds are a good time to test performance.
  • Configurable testing parameters: A significant advantage with CDRouter is the ease with which configuration files can be altered. DT leverages this by creating multiple configuration profiles tailored for specific devices. This customization allows for a more granular and device-specific testing approach, ensuring each device's unique parameters are duly considered. 

On-demand troubleshooting

When facing specific issues, either during build validation or from external sources, CDRouter assists in debugging and troubleshooting these problems. The team at DT selects from CDRouter’s pre-existing list of test cases to understand the problem more thoroughly and get closer to the root cause of the issue.

With this deeper insight, the team then feeds back this information to development and customer issue tracking systems to get things on track to resolution in future builds.

Trend Analysis

DT emphasizes the importance of trend analysis in their work. The primary goal is to ensure that from one software build to the next, the quality is either maintained or improved. CDRouter's dashboard aids in this process by showcasing result trends and highlighting inconsistencies between builds. For example, if there's a noticeable difference between two builds without any discernible software change, it prompts a deeper investigation.

The RDK “Automatics” open-source test framework is also currently working with the CDRouter API. The DT team plans to use this to compare CDRouter trend results with other frameworks. While their various frameworks target distinct aspects, there is some overlap. This comparison capacity offers a deeper understanding of results across different platforms.

Lastly, for long-term validation such as stability testing, the trend analysis becomes crucial. Given the longer time frames and larger sets of results, a dashboard is invaluable for discerning differences and patterns across multiple test runs, ensuring that customers consistently receive quality products.

DT’s RDK QA testing is all about best practices

Deutsche Telekom’s holistic approach to integrating test automation within its RDK program provides some very valuable best practices for all QA and development teams. Incorporating CDRouter’s test cases and robust API into the overall strategy - and feeding that information back to the RDK community - is improving the industry as a whole around open-source CPE product development.

CDRouter is a core part of that strategy. To learn more about developing a complete automated test strategy, download our comprehensive guide.