The Responsible Artificial Intelligence (RAI) Defense AI Guide on Risk (DAGR) is intended to provide DoD AI stakeholders with a voluntary guide to promote holistic risk captures and improved trustworthiness, effectiveness, responsibility, risk mitigation, and operations.
The DAGR’s primary purpose is to highlight applicable RAI concepts, holistically guide risk evaluation, provide abstracted risk models to manage risk, and an approach to quantify the holistic risk of AI capabilities while promoting responsibility, trust, and the United States National Security Strategy. Further, DAGR aligns to the DoD AI Ethical Principles, NIST AI Risk Management Framework (AI RMF), best practices, other governing Federal and DoD guidance (including other DoD risk management guidance), and the considerations from allies and partners of the United States.
There are seven guiding dimensions of risk within the NIST AI RMF. According to the AI RMF, units should strive to ensure AI platforms and applications are (1) valid and reliable, (2) safe, (3) secure and resilient, (4) explainable and interpretable, (5) privacy-enhanced, (6) fair – with harmful bias managed, and (7) accountable and transparent. Further expansion of the seven guiding dimensions of AI risk can be found in the latest NIST AI RMF document. It is important to note that the series of questions provided by DAGR are not intended to serve as go/no-go criteria nor authority-to-operate (ATO), but rather, to serve as a guide to promote risk thought and assessment with AI capabilities.
Several approaches to examine risk and impacts are used within private and public domains, such as the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis or Political, Economic, Social, and Technological (PEST) analysis. In support of additional societal concerns, and federal regulations and guidance (such as Presidential Executive Orders and The Department of Defense elevating climate change as a national security priority), DAGR expands upon the PEST model to provide a novel approach, referred to as a STOPES analysis.
A STOPES analysis examines the Social, Technological, Operational, Political, Economic, and Sustainability (STOPES) factors.
Stopes refers to a mining concept of excavating a series of steps or layers, and the STOPES acronym is fitting to the concept of risk management within AI due to the significant implications that may arise from an AI capability and the requirement to analyze and explore layers of impacts and risk across multiple disciplines. As AI stakeholders refer to the guidance provided here, it is important to incorporate a STOPES analysis to fully realize the risks of AI capabilities and mitigate accordingly.
Risk is a dynamic concept, which is realized in context, therefore, it is prudent to evaluate risk not only throughout the operational window of an AI capability, but across the entire lifecycle. In addition to being dynamic, risk may be realized due to relationships with other AI capabilities or the environment it operates within. It is the responsibility of the decision maker, commander, and warfighter to appropriately assess risk throughout the lifecycle.
The DAGR deliberately utilizes the mnemonic to symbolize how – contrary to a common objection – RAI is not an impediment to warfighter effectiveness, but instead promotes increased capability to the warfighter and support for commander intent, all within acceptable risk parameters. Additionally, when performed appropriately, RAI earns the confidence of the American public, industry, academia, allies and partners, and the broader AI community to sustain our technological edge and capability from the boardroom to the battlefield.
The content of this document reflects recommended practices. This document is not intended to serve as or supersede existing regulations, laws, or other mandatory guidance.
According to the NIST AI RMF, use-case profiles are implementations of the AI RMF functions, categories, and subcategories for a specific setting or application based on requirements, risk tolerance, and resources of the framework user. For example, an AI RMF operational commander/decision maker profile or AI RMF cybersecurity profile may have different risks to highlight and address throughout the AI capability lifecycle.
The DAGR’s primary purpose is to highlight applicable RAI concepts, holistically guide risk evaluation, and provide an abstracted risk model to mitigate risk of AI capabilities while promoting responsibility and trust.
Future expansions of the DAGR may include risk highlights, questions, and mitigations for appropriate profiles. For example, an AI RMF operational commander/decision maker profile guide may be produced that delineates appropriate risk factors that must be evaluated at this level of responsibility, while also abstracting more technical and detailed factors that will be addressed by other profiles but must still be considered by the operational commander/decision maker. Another example may be an AI RMF data science profile that would conduct detailed risk evaluations with emphasis on data, such as computational biases, but the operational commander/decision maker profile must ensure it is addressed and mitigated appropriately without necessarily focusing on the technical details.
The DoD has made significant progress in establishing policy and strategic guidance for the adoption of AI technology since the 2018 National Defense Strategy (NDS) that highlighted the Secretary of Defense’s recognition of the importance of new and emerging technologies, to include AI. Since then, the DoD has released the “DoD AI Ethical Principles”, which highlights that the following principles must apply to all DoD AI capabilities, encompassing both combat and non-combat applications:
In addition to the established DoD AI Ethical Principles, the National Institute of Standards and Technology (NIST) released the AI RMF in early 2023. Within this framework, stakeholders should ensure that AI platforms and applications mitigate risk to an acceptable level across the seven guidelines. For the remainder of this document, AI systems, models, platforms, and applications will be referred to as AI capabilities from here on out. The seven guidelines are:
It is vital to the concept of RAI that both the DoD AI Ethical Principles and AI RMF be considered together to capture a greater risk picture.
When exploring and assessing AI risk, there are several risk relationship dynamics to consider prior to the deployment of an AI capability.
AI risk has a shifting dynamic, meaning that throughout the lifecycle of an AI capability, overall risk may shift and become more or less impactful.
AI risk may have a bidirectional dynamic with the environment. This implies that an AI capability may influence the environment it operates within, and the environment may influence the AI capability. Figure 1 represents this potential bidirectional dynamic between the AI capability and the environment.
Risks between AI capabilities may be interconnected because of a dependency between AI capabilities. This implies that residual risk for an AI capability may affect the risk of another AI capability, if they share a dependency. The risk dynamic between dependent AI capabilities does not necessarily have to be bi-directional, meaning that if a dependency between two capabilities exists, it does not have to be a bi-directional dependency. This risk dynamic may only be realized if there is a dependency between two capabilities. A dependency is when one capability requires another capability to perform it’s designated function.
During the lifecycle of an AI capability, it is critical for AI stakeholders to evaluate the shifting, interconnected, and bidirectional dynamics of risk across the social, technological, operational, political, economic, and sustainability (STOPES) factors. The below descriptions of the STOPES factors may not address every component or concern related to a factor, therefore, it is the responsibility of the decision maker to ensure all appropriate risk factors are considered. Within this section, the concepts of STOPES and high-level risk perspectives will be presented to guide further in-depth risk discussion and assessment.
Important note: Before reviewing the STOPES factors, it is necessary to highlight that supply chain risks may be evaluated across several of the STOPES factors. Common supply chain risks include poor contractor and/or supplier performance, labor shortages, funding, geopolitical, cyber, environmental and reputational risk, to highlight a few.
Reviewing the above list for supply chain risks as an example, it can be surmised that cyber supply chain risks related to software and hardware would reside within technological factors, but poor contractor and/or supplier performance, labor shortages, and funding would be categorized within economic factors of the STOPES analysis.
Social Factors: Factors related to community, social support, income, education, race and ethnicity, employment, and social perceptions.
Technological Factors: Factors related to the organizational impacts of a technological capability being inoperable, compromised or operating incorrectly, and appropriate supply chain risks related to technology and security.
Operational Factors: Factors that may result in adverse change in resources resulting from operational events such military (combat and non-combat) operations, operations inoperability or incorrectness of internal processes, systems, or controls, to also include external events, and appropriate supply chain risks related to operations. Operational factors also include reputation, legal, ethical, and human-machine interaction and the corresponding feedback loop of this interaction.
Political Factors: Factors related to government policy, changes in legislation, political climate, and international relations.
Economic Factors: Economic factors that may influence the organization, such as access to funding, acquisition processes and vehicles, labor costs and workforce skill, market conditions, and appropriate supply chain risks related to economics.
Sustainability Factors: Factors related to human-, environment-, social-, and economic-sustainability. The intersection and balance of environment, economy, and social equity support sustainability initiatives. It is important to note that the social sustainability and economic sustainability factors are specialized topics within the aforementioned economic and social categories and are recommended to be highlighted here. Factors include concepts related to climate change, the environment, energy usage, social responsibility, human security, and appropriate supply chain risks related to sustainability.
When evaluating risk across the STOPES factors, we recommend utilizing the DISARM hierarchy of AI risk. DISARM is an acronym for data, infrastructure, security, accountability, resources, and model operation. The DISARM hierarchy is intended to serve as an abstracted model to prioritize the evaluation and mitigation of risk of an AI capability. It is important to note that addressing the risk at a lower level of the hierarchy does not mean that a stakeholder can move up the hierarchy and ignore risk factors of lower levels at later intervals of the AI capabilities lifecycle.
Within the DISARM hierarchy of AI risk, AI stakeholders should strive to mitigate risks associated with data first since the collection, cleaning, and labeling of data are critical in reducing bias and promoting validity, reliability, effectiveness of AI capabilities, and mitigating risk at the higher levels of the hierarchy.
Continuing along the hierarchy, the AI stakeholder will continue evaluating risk and factors associated with infrastructure, security, accountability, resources, and model operations, in order. Without adequately mitigation risk at the lower levels, AI stakeholders cannot expect risk to be appropriately mitigated at the higher levels, and for the AI capability to operate as intended. Figure 3 depicts the DISARM hierarchy of AI risk.
Below is a description of each layer of the DISARM hierarchy of AI risk, but this is not intended to be an all-inclusive list.
According to the DoD Responsible AI Strategy and Implementation Pathway, the AI product lifecycle consists of the iterative activities of design, develop, deploy, and use. Equally iterative in nature is the need to evaluate and address risk throughout the AI lifecycle and across activities.
For simplicity, Figure 4 combines Deploy and Use within the same phase. The AI capability lifecycle consists of the following phases:
Although risk is already a complex and dynamic concept, which is further amplified by multifaceted operations and requirements of the Department, by leveraging the abstractions provided in the DAGR, the complexity of portraying risk in the context of AI capabilities can be simplified. Figure 4 describes the following interactions:
When evaluating risk for an AI capability, it is important to start the analysis from a foundational level and expanding the risk assessment based on the capability, operational need, and other external factors. The following initial guiding questions are not intended to be an all-inclusive review of risk against the DoD AI Ethical Principles and guiding principles of the NIST AI RMF through a STOPES analysis. But rather, to serve as a starting point to promote effective risk dialogue.
When evaluating AI capability risk, there are three general categories of evaluation: (1) independent AI capability (no dependencies), (2) unidirectional dependency between AI capabilities, and (3) bidirectional dependency between AI capabilities. Each of these processes will be explained below, but first, it is necessary to highlight how each described risk is evaluated for an individual AI capability.
When conducting a STOPES analysis, each risk must be qualitatively assessed with the following risk matrix and corresponding numerical values, as shown in Figure 5. Prior to beginning, it is necessary for the risk assessors to describe what consequences are considered extreme, major, modest, or minor. The probability of events are described as very unlikely (~0-20%), unlikely (~21-50%), likely (~51-80%), and very likely (~81-100%).
For each risk, the evaluator will assess the probability of occurrence against the consequence of the event and select the numeric value from the table. This step is to be repeated for every risk.
The next step is to select risk mitigation strategies and controls for each risk and re-evaluating the probability of event and consequence. The new residual risk score is produced by selecting the numeric value of the updated probability and consequence of event and is known as the residual risk.
Appropriate stakeholders must determine acceptable risk thresholds after mitigations have been selected. For example, stakeholders may determine if the acceptance of risk is a factor of ensuring each risk is below a predetermined threshold, and/or if the sum of residual risk for each STOPES factor is below a predetermined threshold. Another example may be that stakeholders have determined that an AI capability may be within an acceptable range of risk if every risk is below a value of 6, then there is zero risk of human and environmental harm, and the sum of each STOPES factor is below 30. Once again, it is the responsibility of the decision maker to explicitly delineate acceptable risk parameters.
If acceptable risk thresholds have not been achieved, then the fielding of an AI capability should be escalated appropriately.
Independent AI Capability (No Dependency) By performing the aforementioned steps of evaluating each risk from the probability of occurrence against the consequence, each risk is to be categorized within one of the social, technological, operational, political, economic, and sustainability (STOPES) factors. Once each risk and value has been annotated within the respective category, all scores for the category are added and annotate as the final sum for the category. An example of performing these steps is annotated in Figure 6. For example, from Figure 6, it should be noted that this AI capability has three technological risks with risk scores of 1, 3, and 3, with a total category sum of 7.
The next step is to implement one of the four risk mitigation strategies: accept, transfer, mitigate, and avoid. After a mitigation strategy has been selected for each risk, it is necessary to re-evaluate the risk score from risk matrix. Using Figure 6 as an example, assume a risk mitigation strategy was selected for Social Risk 2, which changed the probability of the event from likely to very unlikely, but the consequence of event remained as extreme. In this case, the risk value would decrease from 12 to 4. All residual risk scores for each factor are to be summed again.
Unidirectional Dependency of AI Capabilities For unidirectional dependencies, risk for each capability is calculated as an independent capability. The second step is to evaluate the strength of the dependency between the two AI capabilities, as a percentage between 0-100%. The third step is to take the factor sum of each category (after mitigation) from the Influencing Capability and multiplying this value by the relationship percentage. This calculated value is known as the Derivative. The final step is to take the Derivative for each of the factor categories and adding it to the corresponding Factor Sum of the Influenced Capability. This new value is known as the Dependency Sum.
Important note: It is important to examine whether or not a derivative should be accounted for when assessing the risk of the Influenced Capability.
Using an example, as shown in Figure 7, two AI capabilities share a unidirectional relationship, in which Capability 1 influences Capability 2. For this example, it is also assumed that each derivative is to be accounted for in Capability 2. Therefore, Capability 1 is known as the Influencing Capability and Capability 2 is known as the Influenced Capability.
Figure 8 demonstrates the Derivative from Capability 1, by multiplying each of the Residual Factor Sum values by 30% (0.30). For example, Capability 1 has a Residual Social Factor Sum of 7 and is multiplied by the relationship strength of 0.30, resulting in a Social Derivative of 2.1.
Each of the Derivative values from Capability 1 are added to the Factor Sum of Capability 2, resulting in the Dependency Sum, which is depicted in Figure 9. For example, Capability 1 produces an Economic Derivative of 0.3, which is added to the Economic Factors Sum of 4 from Capability 2, resulting in an Economic Dependency Sum of 4.3 for Capability 2.
The final Dependency Sum values for each STOPES factor is the final calculated risk for the capability.
Bidirectional Relationship of AI Capabilities For bidirectional relationships, risk for each capability is calculated as an independent capability. Then the same steps are performed as described in the Unidirectional Relationship of AI Capabilities section, but both directions must be addressed. This is because the strength of relationship between two capabilities may be different. For example, Figure 10 depicts that Capability 1 has a 30% influence over Capability 2, but Capability 2 has a 70% influence over Capability 1.
When evaluating risk for an AI capability, it is important to start the analysis from a foundational level and expanding risk assessment based on the capability, operational need, and other external factors. The contents of this document are not intended to be an all-inclusive review of risk against the guiding principles of the NIST AI RMF through a STOPES analysis, but rather, to serve as an initial guiding document to promote effective risk evaluation and dialogue of AI capabilities.
The proposed DAGR future works and roadmap will highlight the following: