4 min read
We get a lot of feedback from our users discovering new and interesting results when combining functional testing with throughput, latency, and loss testing. Having a good understanding of how performance tests work and the caveats around their results can help you determine how your functional tests are impacting performance, and visa-versa.
What is “line rate”?
CDRouter Performance is designed to measure “application level” throughput. That is, it measures the amount of actually data transfered over TCP or UDP, representing the successful delivery of data that a user is interested in when using a web application or streaming. It does not measure “interface level” traffic - a metric reserved for lower-level, interface specific performance tools. For this reason, it is important to be careful when comparing the results of one test tool to another. They may not be the same!
Suffice to say, application level “line rate” is different from interface level “line rate”. In order to think in terms of the former, we must take into account the data used by the data link, network and transport layer headers in a UDP or TCP packet before calculating what the expected line rate should be. Comparing the data used by protocol encapsulations versus the interface “line rate” can be thought of as the “efficiency” of the protocol.
For example, consider a standard TCP packet over IPv4. If we assume the maximum size of an Ethernet Frame is 1518 bytes, and another 20 bytes to be included for the preamble, frame-delimiter, and inter-frame gap, for a total transmitted frame of 1538 bytes. Since an Ethernet header is 18 bytes, the IPv4 header is 20 bytes, and the TCP header is 20 bytes, this leaves us with an effective payload of:
1518 Ethernet Frame - 18 bytes (Ethernet) - 20 bytes (IPv4) - 20 bytes (TCP) = 1460 byte payload
To figure out how efficient this is, just divide payload size by the total transmitted frame size:
1460 byte payload / 1538 transmitted frame size = .949
Or 94.9% efficiency. This means that when CDRouter Performance is measureing application level throughput, it will never be more than 94.9% of line rate for TCP over IPv4. You can see similar calculations for other transport and network protocols here.
CDRouter Performance can be set for a target bandwidth or can be configured to try to achieve the maximum data rate for the link you are testing. Always remember, however, that your results will be bound by the efficiency of the protocol you are testing throughput over.
One of the other questions we get a lot are whether or not the DUT’s Wifi performance results are typical or the result of a problem.
The short answer is that it is very difficult to tell. Wifi is very hard to control in a lab environment unless you have a dedicated setup with a waveguide or an Anechoic chamber. Moreover, differing Wifi types between an Access Point and a client will limit you to the bandwidth of the lowest common Wifi mode. Channel width will determine the maximum throughput of your Wifi DUT; if it is set for auto, it will attempt to be conservative to play nice with neighboring Wifi channels.
Needless to say, your results will vary! Our support team can help you nail down that CDRouter Performance itself is working accurately, but measuring Wifi is best done in either a control environment or as a heuristic for deterimining whether the user of the DUT will have the experience they expect. Which boils down to…
Testing throughput, latency, and loss on broadband CPE, particularly Wifi devices, is a difficult challenge, but CDRouter Performance can help you get the results you need, and simulate a real environment where functional behavior is happening alongside application data. That said, when determining your performance metrics, it’s best to consider the applications your DUT will be handling for its users, and setting your bandwidth goals to those applications.
Instead of just trying to get raw line-rate-or-bust data, consider: what bandwidth will be necessary for web traffic? For video? For voice? And most importantly, how will the functional behavior of my DUT affect or be affected by those applications?
