1 hour ago
3
OPINION -- In an era where warfighters and decision-makers have on-demand access to vast data holdings and AI-generated insights, the future of intelligence analysis will be defined by the ability to apply modernized analytic tradecraft to transform data into decision-ready insight. Where others may optimize for speed, scale, and profit, the intelligence community must bring methodological rigor to create decision advantage.
The Promise and the Peril of Analytic Tradecraft
The scripture inscribed on the CIA’s Original Headquarters Building reads: “And Ye Shall Know the Truth and the Truth Shall Make You Free.” The words express an optimistic belief that objective truth can be identified and communicated. This premise, however, is increasingly under strain.
The Cold War-era architects of modern analytic tradecraft understood that intelligence analysts are no less susceptible to cognitive bias than anyone else. This led Sherman Kent, Richards Heuer, and others to develop a framework of analytic methods to ensure the objectivity of intelligence assessments. The methods encourage analysts to “show their work” and take active measures to mitigate cognitive bias, manage uncertainty, and boost decision-makers’ confidence in analytic conclusions.
The intelligence community’s adoption of a universal set of analytic tradecraft standards after the 9/11 terrorist attacks and the flawed assessment of Iraq’s WMD programs marked a high point in the use of analytic tradecraft.
Administrative formalization and the passage of time, however, have increased the risk of over-correction and ossification. Today’s analysts and managers must guard against the danger that tradecraft becomes a performative, backward-looking bureaucratic exercise disconnected from the national security mission.
The New Analytic Ecosystem
The U.S. Intelligence Community’s analytic tradecraft standards were designed for an information environment characterized by scarcity and secrecy, where long-form textual reports were the ultimate decision-support tool. We no longer operate in that world.
As of 2023, the National Geospatial-Intelligence Agency (NGA) was ingesting 70,000 new data points per second. At the same time, virtually anyone with a credit card can now access commercial GEOINT, commercial SIGINT, and millions of unique OSINT sources.
Amid this vast sea of data, political and military leaders now have access to a rapidly growing number of commercial analytic services. Dozens of defense technology companies are now testing these AI-enabled capabilities in Ukraine, Gaza, and Iran with an eye toward marketing battle-tested analytic tools to eager governments around the globe.
These changes in the information environment have given rise to a new analytic ecosystem in which the relative share of information the intelligence community provides to decision-makers is rapidly shrinking, and the community’s primacy as the go-to source for decision advantage is increasingly being challenged.
Operationalizing Modern Tradecraft
Amid revolutionary changes in the information environment and the rise of AI-enabled analytic capabilities, there is an urgent need to reaffirm the core principles of analytic tradecraft while simultaneously modernizing them to serve today’s analysts better.
1. Reaffirming mission relevance. Sherman Kent defined intelligence as knowledge for the purpose of action, and identified usefulness to the decision-maker as a key criterion for evaluating the quality of an analytic assessment. The OSS veteran observed that analysis that is inaccurate, incomplete, late to need, or lacks an obvious linkage to current national security decisions or future threats is—in a word—useless.
Intelligence Community Directive 203 lists decision-maker relevance as the fifth of nine analytic tradecraft standards on the sixth page of an eight-page policy document. It is not unreasonable that some may conclude that customer relevance is not a primary concern.
To be useful, analytic assessments should be decision-relevant, providing information and insights pertinent to U.S. national security and foreign policy, or highlighting emerging issues and threats that may require decision-maker attention. Assessments must be delivered in time to inform the decisions they are meant to support or to provide prompt warning of emerging threats. By necessity, an emphasis on decision-relevance and timeliness will create tension with analytic rigor and completeness. Skillful analysts and managers must actively manage these trade-offs to meet mission needs.
Usefulness is also a function of accuracy and focus. In today’s hyper-saturated information environment, analysts play a vital role in helping decision-makers determine which reporting and data can be independently corroborated. Analysts can also prevent information overload by prioritizing key reports and data essential to understanding an issue, while filtering out unnecessary details and unsupported judgments.
Maximizing the usefulness of analytic assessments for decision-makers requires a functioning relationship between intelligence agencies and the decision-makers they support. This necessitates active engagement and two-way dialogue that provide analysts with a deep and nuanced understanding of decision-makers' requirements and priorities. Without a functioning relationship, discerning decision relevance becomes an exercise in guesswork.
2. Reinforcing analytic objectivity. In a world where decision-makers have near-infinite choices in where they obtain information, the research methodologies, structured techniques, and standards that intelligence analysts use to ensure accuracy, rigor, and objectivity are key differentiators. These methods create decision advantage by helping reduce risk, manage uncertainty, and increase confidence in analytic conclusions.
Objectivity in AI-enabled analytic outputs must be defined technologically and methodologically, and analysts must make the case to decision-makers why they should have confidence in these tools.
The foundation of objectivity is transparency and methodological rigor. Existing tradecraft standards require analysts to provide detailed descriptions of the sources and methods used to form judgments. These descriptions include evaluations of their strengths, limitations, and potential biases. These standards must now be modernized and expanded to incorporate AI data inputs, prompt traceability, and model selection rationales. The standards should also be integrated directly into AI-enabled analytic tools. The recent Intelligence Community Directive on AI includes provisions intended to close this gap, but more work is necessary to fully integrate and align the community’s technology, data governance, and analytic tradecraft standards.
Analysis of alternative competing hypotheses, a long-standing method for detecting and mitigating cognitive bias, is both more important and more achievable with the proliferation of AI. Analysts are obligated to stress-test analytic conclusions against contradictory reports and competing hypotheses and report the results. While this has historically been a time- and labor-intensive task, AI-enabled tools and techniques can now test alternative hypotheses at scale. Establishing community-wide tradecraft standards for integrating these tools and techniques into analytic workflows will be instrumental in maintaining human accountability for analytic outputs.
3. Modernizing intelligence delivery. Current analytic tradecraft standards remain rooted in the analog text era. This technological latency is evident in the frustrations of decision-makers who use tablet computers to access analytic conclusions with the devices’ limited functionality, even though they have been in service for more than a decade. Along the same lines, the guidelines for formatting the endnotes in an analytic product rival the level of detail found in The Chicago Manual of Style, but are largely silent on how to maximize the usefulness and objectivity of visual and digital media. This can create operational risk when extremely high-quality visuals convey a level of certainty and confidence not supported by the underlying intelligence.
The intelligence community should modernize its analytic tradecraft, sourcing, and dissemination standards to better support the delivery of analytic conclusions via dynamic dashboards, visualizations, and structured analytic observations. Uncertainty and confidence must be encoded into these products, just as they are for text-based reports. This can be achieved by developing and consistently applying analytic tradecraft standards for new media that leverage interactive overlays, pop-ups, and uncertainty visualizations.
A Call to Action
The intelligence community now has the opportunity to proactively modernize its analytic tradecraft standards to sustain decision advantage. This window of opportunity will not remain open indefinitely.
The proliferation of AI-enabled analytic tools, in the absence of a shared set of standards and methodologies to mitigate cognitive bias, manage uncertainty, and ensure substantive accuracy, has introduced new risks into the national security decision-making process. The private-sector innovators leading the development of these new tools, and their commercial and foreign government clients, have different incentive structures and risk tolerances than the U.S. government, and they will not wait for the intelligence community to take action.
In today’s competitive analytic ecosystem, analytic tradecraft is a key differentiator. If the intelligence community does not act to reaffirm core tradecraft principles and modernize existing standards to take full advantage of AI-enabled tools, it risks being outpaced by private-sector intelligence providers and bypassed by national security decision-makers.
The Cipher Brief is committed to publishing a range of perspectives on national security issues submitted by deeply experienced national security professionals. Opinions expressed are those of the author and do not represent the views or opinions of The Cipher Brief.
Have a perspective to share based on your experience in the national security field? Send it to Editor@thecipherbrief.com for publication consideration.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
