“Do You Really Need More Information? The US Intelligence Community invests heavily in improved intelligence collection systems while managers of analysis lament the comparatively small sums devoted to enhancing analytical resources, improving analytical methods, or gaining better understanding of the cognitive processes involved in making analytical judgments.”
Psychology of Intelligence Analysis — Richards J. Heuer
Lt. General H.R. McMaster — the newly designated National Security Advisor to President Trump — is known to be a brave soldier and a man who never hesitated to ‘speak truth to power’. General McMaster is also a brilliant intellectual who knows a few things about ‘Intelligence’. Follow us on Twitter: @Intel_Today
Collection of more intelligence-related information does not necessarily translate into better intelligence. General H.R. McMaster recently wrote that:
“Because of limitations associated with human cognition, and because much of the information obtained in war is contradictory or false, more information will not equate to better understanding.”
Absolutely correct. Actually, the situation is even worse. Analysts are known to use much less of the available information than they think they do. And analysts can become over-confident when overwhelmed with information.
Psychology of Intelligence Analysis
Accurate intelligence judgments do not solely rely on the abundance and accuracy of the information. Indeed it has long been known that rigorous analysis of the information is at least as important as the gathered material in order to reach accurate intelligence estimates.
In the 1960s, CIA psychologist Richards Heuer investigated the correlations between the amount of information available to experts, the accuracy of judgments, and the experts’ confidence in the accuracy of these judgments. The results of these experiments are far-reaching.
In one of these experiments, experienced horserace handicappers were shown a long list of variables that included data related to the recent performances of the horses, the weight of the jockeys, the time since the last race, the weather conditions, etc. Each handicapper was asked to order these variables according to its perceived importance in the making of his prediction.
Next the handicappers were shown real data that had been renamed to ensure that they could not remember the events. Each of them was then given the five variables he had listed as the most useful. At that point they were asked to make a prediction as well as an estimate of the degree of accuracy of his prediction (from 0% to 100%). The same exercise was repeated after the handicappers were given 10, 20 and 40 variables.
The result of this particular experiment is abundantly clear. On one hand the accuracy of the prediction did not improve with additional information. As a matter of fact, several handicappers tended to be less accurate as more information became available to them.
On the other hand, their confidence in the accuracy of their predictions increased significantly as they were provided with more information. (See figure.) Note that only when provided with the lowest amount of information were the handicappers realistic about the probability of their predictions being correct.
Key findings from these experiments:
Once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of his or her estimates. Additional information does, however, lead the analyst to become more confident in the judgment, to the point of overconfidence.
Experienced analysts have an imperfect understanding of what information they actually use in making judgments. They are unaware of the extent to which their judgments are determined by a few dominant factors, rather than by the systematic integration of all available information. Analysts actually use much less of the available information than they think they do.
PS: The General also seems to understand the Constitution. In the same document, he wrote:
“The U.S. is a litigious environment within which the Army operates; intelligence leaders must understand the role legal limitations and authorities play in shaping intelligence support.”
Who is Lieutenant General H.R. McMaster
This experiment is described in a book published on the CIA website: Psychology of Intelligence Analysis — Richards J. Heuer Jr. Chapter V –Do You Really Need More Information?
Army Intelligence: A Look to the Future — Secrecy News