USP/TR-369 - The User Services Platform
USP is a revolutionary standard that builds on the lessons learned from TR-069. In addition to being a faster and more flexible at meeting the existing use cases of TR-069, USP is built to enable:
- Managed Wi-Fi - USP is ideal for building the managed Wi-Fi systems that are in high demand by today’s service providers, with flexible transport mechanisms, a multi-controller architecture, and standardized Wi-Fi management objects.
- Bulk statistics collection - USP lets parties develop applications that gather the telemetry necessary to mass data processing, AI, and machine learning.
- Standardized IoT onboarding - USP unifies the fragmented world of IoT and smart home devices by proxying non-IP protocols like ZigBee, Zwave, etc.; onboarding and managing device lifecycles; and standardizing IoT interfaces and service objects in the Device:2 data model (TR-181).
- Customer self-care applications - USP Controllers can be built for end-user devices to allow for customer self-care without dependence on cloud resources.
- Application enabled smart gateways - USP extends and enhances the software module management features of TR-069 to allow “dockerized” device management and the lifecycle management of third-party applications like security, Wi-Fi telemetry, and parental controls.
- Secure communication - All USP applications can benefit from end-to-end application-layer TLS connections specified in the standard and robust trust establishment and access control for controllers.
A deep exploration of these use cases can be found in the Broadband Forum white paper Realizing the Promise of the Connected Home with USP.
Resources for development
Testing for USP/TR-369 implementations
The CDRouter USP add-on is the official test platform the Broadband Forum’s BBF.369 certification program for USP Agents. In addition, it has exhaustive validation for your implementation of the Device:2 data model. QA Cafe was involved in the development of USP early on and has built a solid base of expertise to assist you in your own development.
USP Primer
Watch this quick 13-minute primer on USP, its use cases, and technology:
The Broadband Forum’s USP Workshop Series
The Broadband Forum also has a full series of workshop presentations on USP. You can register to access them here.
Standards and data model resources
The specification for architecture, discovery, end-to-end message encoding, transport, and types, plus security and access control are defined in Broadband Forum TR-369 User Services Platform (USP). This can be found on the web at https://usp.technology.
The data model for describing the service elements exposed by USP Agents are defined in the Device:2 Root Data Model (published as TR-181 Issue 2). The models for CWMP and USP pull from the same common core with some minor changes for protocol-specific management objects. These models can be found at https://usp-data-models.broadband-forum.org.
Open source USP Agent
The Broadband Forum maintains an open source USP Agent on which developers can base their implementations. The Agent is improved through frequent releases. It is a great starting point for implementers while building a complete solution and testing it with CDRouter.
It can be found on GitHub here: https://github.com/BroadbandForum/obuspa
Overall architecture
USP consists of a network of Controllers and Agents that manipulate “Service Elements”. Service Elements are one or more objects defined in the Agent’s supported data model, and exposed as the Agent’s instantiated data model. A Controller may be an Autoconfiguration Server (ACS) similar to TR-069, or controlled by an application service provider to control specific elements, or even a Controller on a smartphone or in the gateway to manipulate elements in the user’s network.
Protocol stack
USP Protocol
All USP messages are encapsulated in a USP record. These records are used to guarantee integrity of the USP messages and provide a layer of security when security can’t be accomplished at the message transport layer. USP Records allow an optional Session Context that allows for protected messages (payloads) and the segmentation and re-assembly of USP messages when large messages need to traverse intermediate proxies.
Message transport agnostic
TR-069 is bound heavily to HTTP. In the design of USP, there was a desire to ensure that the protocol be cleanly layered so that transport of USP messages could be accomplished over multiple “transports” (the term Message Transfer Protocol was adopted to differentiate between message transport and the actual OSI transport layer). Message Transfer Protocols (MTPs) defined for USP 1.2 include the Simple Text-Oriented Messaging Protocol (STOMP), Message Queuing Telemetry Transport (MQTT), WebSockets over HTTP, and internally over Unix Domain Sockets (UDS).
Protocol buffers encoding
Rather than sending XML documents over HTTP, USP makes use of Protocol Buffers to encode messages in transport. Protocol Buffers encoding is binary, and ends up saving significant space. It also has advantages over JSON as the encoded fields are defined in one or more schemas (.proto files) that each endpoint can use to encode or decode messages reliably and interoperably. In USP, there is a schema for the USP Record (usp-record.proto) whose payload field contains a USP Message (usp-msg.proto).
For example, this is the Get message from the usp-msg.proto schema:
message Get {
repeated string param_paths = 1;
}
message GetResp {
repeated RequestedPathResult req_path_results = 1;
message RequestedPathResult {
string requested_path = 1;
fixed32 err_code = 2;
string err_msg = 3;
repeated ResolvedPathResult resolved_path_results = 4;
}
message ResolvedPathResult {
string resolved_path = 1;
map<string, string> result_params = 2;
}
}
Evolution from TR-069
Backwards compatibility with Device:2
One of the development goals of USP was to serve as a natural evolution of TR-069, allowing TR-069 users to seamlessly migrate to USP without having to re-architect their OSS/BSS or device/driver integration. USP makes use of the Device:2 data model with some modifications to allow the definition of commands and events with the data model.
Reduced, flexible message set
USP attempts to handle both RESTful and non-RESTful situations with a reduced message set that is made extensible by moving many operations into the Device:2 data model. As such, it includes CRUD (Create, Read, Update, Delete) based operations, plus a Notify message for notifications to be sent from the Agent to a Controller, and a Operate message to call functions (commands) defined in the Agent’s supported data model. These messages include:
- Add - used to create new instances of multi-instance objects in the Agent’s instantiated data model.
- Set - used to alter the state of an object’s parameters in the Agent’s instantiated data model.
- Delete - used to remove instances of objects in the Agent’s instantiated data model.
- Get - used to retrieve the value of one or more parameters of objects in the Agent’s instantiated data model.
- GetInstances - used to retrieve the instances and instance identifiers of objects that can have multiple instances in the Agent’s instantiated data model.
- GetSupportedDM - used to retrieve information about the Agent’s supported data model, including the supported objects and their parameters, commands, and events.
- GetSupportedProtocol - used to retrieve the USP versions the Agent supports. In version 1.0, the only valid value is 1.0.
- Notify - used by the Agent to alert a Controller of events that were triggered based on one or more subscriptions that a Controller configures in the Agent. These include some standardized messages such as ValueChange, ObjectCreation, ObjectDeletion, OnBoardRequest, and OperationComplete, plus a generic Event notification to carry information about object specific events defined in the Agent’s supported data model. Events that were defined in CWMP but are now defined in the Device:2 data model include things like BOOT, TRANSFER_COMPELTE, etc.
- Operate - used to call functions (called commands) defined by objects in the Agent’s supported data model, including a way to provide arguments to those commands, and retrieve return arguments. The mechanism includes support for both synchronous and asynchronous operations; the former returns arguments as part of the OperateResponse from the Agent, and the latter may return arguments in an associated OperationComplete event. Examples of operations that were RPCs in CWMP, but commands in USP, include things like Reboot(), Download() (which operates on Device.Firmware instances), etc.
- Register - Added in USP 1.3, the register message allows USP Services to register with a USP Broker as part of the software modularization and USP-enabled applications features.
Lighter weight messages
We mentioned above that Protocol Buffers saves space on the wire by creating standardized fields that are binary encoded in transport. This reduces complexity significantly.
In addition, USP messages that use the Device:2 data model (defined originally for CWMP) makes use of several features that reduce the amount of object path information that needs to be sent by the Controller and returned by the Agent:
- Relative paths - USP uses the concept of “relative paths” when addressing objects or parameters in the Agent’s database. For example, let’s say a Controller made a Get request on the Device.Wifi.SSID.1. object, in order to retrieve all of the parameters of the SSID. In CWMP, this would require the CWMP Endpoint to return:
Device.Wifi.SSID.1.Enable = <value> Device.Wifi.SSID.1.Status = <value> Device.Wifi.SSID.1.Alias = <value> Device.Wifi.SSID.1.Name = <value> Device.Wifi.SSID.1.LastChange = <value> Device.Wifi.SSID.1.LowerLayers = <value> Device.Wifi.SSID.1.BSSID = <value> Device.Wifi.SSID.1.MACAddress = <value> Device.Wifi.SSID.1.SSID = <value> Device.Wifi.SSID.1.Upstream = <value>
And more, for every sub-object of the SSID object.
In USP, the Agent only needs to return:
Device.Wifi.SSID.1. Enable = <value> Status = <value> Alias = <value> Name = <value> LastChange = <value> LowerLayers = <value> BSSID = <value> MACAddress = <value> SSID = <value> Upstream = <value>
Etc., for the remaining parameters, sub-objects, and their parameters in the Agent’s database.
- Addressing by key - Secondly, USP provides new ways of addressing instances of objects. Rather than try to wrestle with instance numbers, USP allows the addressing of an object instance by the object’s unique key. For example, the Device.NAT.PortMapping table has a unique key made up of three parameters; RemoteHost, ExternalPort, and Protocol. A particular PortMapping could be addressed as:
Device.NAT.PortMapping.
[RemoteHost==""&&ExternalPort==0&&Protocol=="TCP"]
If the instance number was not known or to keep consistency with multiple managed devices.
- Search Expressions - Lastly, USP provides a complex yet powerful means of searching through a set of objects for only those objects who have parameters that match certain criteria. For example, you could have the Agent return IPv4 Addresses for all IP Interfaces with a Normal type and Static addressing type that have at least 1 Error Sent with the following search expression:
Device.IP.Interface.[Type=="Normal"&&Stats.ErrorsSent>0].IPv4Address.[AddressingType=="Static"].IPAddress
Search expressions can be much more complex than this, of course, and include different operators and other complex syntaxes defined in the specification.
Optional session context with fewer round-trips
In USP, sessions are optional and used for security and integrity purposes. Even when sessions are used, communication channels are “nailed up” so that there’s no need to establish a USP session for every message. Moreover, there is no need for Connection Requests to initiate communication sessions, since Controllers can send messages (which results in session initiation, if desired) directly to an Agent at any time.
Our own developer Tucker made a hilarious graphic that contrasts the “noisiness” of TR-069 versus USP:
TR-069 vs. USP conversation length
Multi-Controller architecture
Lastly, USP opens the doors to a vastly more flexible ecosystem by exploding the one-to-one relationship that existed between a CWMP Endpoint and an ACS in TR-069. A given USP Agent can be manipulated by multiple Controllers, which may be in a Service Provider’s operational network (like today’s ACS), a Controller operated by a 3rd party application vendor, a Controller on a gateway facing the LAN, or even on an end-user’s smartphone.
USP accomplishes this by assigning access control rights to each Controller. This is configurable in the data model using the Device.LocalAgent.ControllerTrust. object, which allows the definition of Roles that may or may not apply to different objects and parameters in the Agent’s database.
For example, a Service Provider may have access rights to upgrade the firmware on a device that they have deployed in the end-user’s network. When giving monitoring control to the end-user, who wants to see a report of connected hosts, the Agent may limit a Controller on the end-user’s smartphone so it can’t accidentally mess with the firmware or other critical configuration aspects.
Broadband Forum USP Agent Certification
Get your USP Agent certified now with the official Broadband Forum test platform.
