System of Trust Pilots

In late 2020, we conducted an initial set of pilots that assessed:

1. A set of companies for general concerns.
2. A specific company as a supplier of critical infrastructure systems.
3. A software product for use by a specific community within the federal government.
4. An industrial base assessment for an organization dependent upon a specific technology and the industry capable of supplying it.

Highlights of the results are included below.

Early Pilots Show Promising Results

The preliminary results for Pilot 1 are illustrated below in Figure 1, an unweighted bar chart depicting the overall risk scores for the 11 companies reviewed in the pilot, and in Figure 2, which presents radar plots of five data-driven scores from the supplier risk categories (leveraging 52 questions in those areas) for three of the 11 companies of interest.

Figure 3 offers a deeper look into the risk scores that generated the radar plot for one of the companies involved in the pilot (Company 10).

All of the pilots use data sources that the SoT leveraged to generate the analytical assessments, which clearly show a larger risk profile for Company 10 compared to the others. This pilot provided a proof of concept that offers early evidence of this tool’s utility, with deeper and broader analysis to follow as the SoT is completed. The other three pilots had similar insights.

Figure 1. Risk scorecard based on the preliminary System of Trust scoring methodology for 5 top-level categories and 52 risk measure questions for 26 risk factors

Figure 2. Radar plots of 5 data-driven scores for 3 of the 11 companies reviewed in Pilot 1

Figure 3. Specific risk scores for one company involved in Pilot 1, in the form of a radar plot

In the next phase of the SoT effort, we will use the full array of data sources envisioned and tailor weighting and score contributions to fine-tune the emphasis on specific sub-risk areas used in any given assessment. Although the pilots only used a subset of the public, private, and restricted access data sources the SoT is anticipated to leverage, we are cataloging and capturing the numerous sources of potential utility in conducting such analyses.