The Key Assumptions Check is a simple way to begin an intelligence analysis or research project. Despite its simplicity, it is also extremely powerful, because it allows us to question the underlying assumptions that we make influencing our decisions and thought processes.
Because of confirmation bias, we often exclude information from our worldview or from a decision that does not support a decision, while including information that does support her preconceptions.
The steps to performing a Key Assumptions Check are as follows:
- Come up with a “face valid” analysis. This is a final answer that you will work backwards to get to your assumptions
- List all the stated and unstated assumptions that must be true for this analysis to be valid. This may take some brainstorming; having other people to help you with this can be useful
- Consider alternatives. Ask what would need to be possible for your assumptions to be invalid. Does any of this make sense?
- Remove any assumptions that are not 100% necessarily for your analysis to be true.
This process will allow you to examine your spoken and unspoken assumptions. If you find that many of your assumptions are not relevant to your face-valid analysis, or the assumptions are in fact not supported by existing data, then you would be wise to change your analysis.
Sociometry is the study of social relationships, invented by the psychiatrist and sociologist Jacob Moreno. Similar to link analysis, sociometry involves using a tool called the sociogram, a graphical representation of social links.
Sociograms can be used to graph the communication between people over a long period of time (days, weeks or months), like a link diagram, or over as short a time as a single event. For instance, national security officials surveilling a meeting of hostile government officials may use a sociogram to determine associations between individuals based on who talks to who during a conference.
Creating a Link Analysis
Link analysis is a simple technique used in intelligence analysis to demonstrate the connections or relationships between people. It is commonly used in social science research, criminal investigations and national security.
The first step to link analysis is to collect the data, and decide on a coding scheme. An example coding scheme modified from one in Criminal Intelligence for Analysts is provided below:
Once you’ve collected the data, the next step is to build an association chart. This chart plots the interactions in a more preliminary format as a prerequisite to the link analysis chart. An example is below:
Drawing a preliminary chart yields the following:
Other techniques can be used to create more advanced link charts, as such:
For more additional reading, see the following resources:
There are a number of techniques for making predictions or analysis. It is particularly useful in governments, public policy, and other areas where facts may be disputed but generally based on a set of common foundational knowledge. Examples of this type include economics (where the same economic principles may be expressed differently by country), and psychology (where, for instance, suicide risk assessments may coe to different results but be based on the same risk factors.)
The Delphi technique is a simple but very powerful technique to leverage the power of experts to make decisions, particularly in areas where prediction or forecasting are important. It was named for the Delphi Oracle that figured in Greek mythology, a priestess famous for her prophecies.
How the Delphi Technique Works
The Delphi technique involves selecting a panel of experts to answer the question at hand. This is usually one with some debate; for instance, suicide risk assessment methods have been tested by presenting cases to experts who make their decisions in a method similar to the Delphi Technique.
Once you have a decision that needs to be made, a panel of experts, and a question to be answered, each expert receives the question and information requires independently. After answering, their judgements are displayed to all of the panelists.
In some variations, the experts are shown who among them made each contribution, but in others this information is kept anonymously. Then, the experts complete a second round; having seen the decisions (and usually, the rationalizations) that led to each of the expert’s decisions, they are free to revise their original forecast.
This process repeats until there is either complete agreement in the process (a rarity, indeed), or the experts are no longer willing to revise their predictions. This technique can also be used with non-experts, but is primarily designed to harness their specialized knowledge.
Examples of the Delphi Technique
One practical example of how the Delphi technique was applied was in validating the Suicide Intervention Response Inventory, a tool to evaluate the responses of helpline volunteers. The questions involved were given to a series of experts and their responses were used to formulate the “expert” answers. A person’s score then, is the degree of variance from their responses and the expert panel’s average response.
Edit Feb-2-15: The title of this post was updated to reflect the correct title of the book; the URL remains unchanged.
This title, “Intelligence Research and Analysis” by Jerome Clauser is an excellent overview of the tools used in intelligence analysis and basic research. Intelligence analysis is the processing of raw data that has been collected into actionable information or intelligence.
Contents of “Intelligence Research and Analysis”
The book starts with a definition of intelligence analysis and goes over strategic intelligence, the “highest level” or broadest type of intelligence, as opposed to tactical or operational intelligence which can cover more local or regional situations.
The next two chapters explains the type and purpose of research, as well as why properly conducted research is key to the production of intelligence. In addition to covering the core competencies required by analysts (some of which include reasoning, accuracy, open-mindedness, skepticism, patience and imagination), the fundamentals of conducting research are also discussed.
Sampling, defining and validating hypotheses, operationalizing variables and other topics are outlined, in order to give the reader some information on which to begin laying out their research project. While it is covered in some detail, I got the impression that if I hadn’t previously taken statistics courses I would have had trouble with the speed in which this material goes; this book functions more as a handbook than as an instructional text in its own right.
After discussing the basics of research, the book moves on to planning a research project. This information is of use to anyone planning research, whether involving intelligence or not. Data collection is briefly discussed, although texts specific to this topic will likely be necessary to anyone who doesn’t already have sources in mind.
The above concepts take up approximately half the book, before moving on to analytical concepts, beginning with Chapter 7. After discussing induction and deduction, it moves on to classification and coding – two important steps in identifying preliminary conclusions.
Chapters 9 to 11 cover what I feel is the most valuable information of the book, the elements of statistics and quantitative techniques for analysis. The topics covered include (links, where they exist point to their relevant articles on this site):
Detailed Topic List
Other topics which receive some coverage in these chapters (that I feel are less important) include:
- Terrain Profiling
- Delphi Technique
- Psychohistorical and Psycholingustic Analysis
- Morphological Analysis
The final section of “Intelligence Research and Analysis” covers preparing an intelligence report and provides an example of an actual intelligence study. These sections are likely less relevant to individuals who aren’t working or studying in the fields of national security but may still have value.
All in all, the book is a fantastic overview for anyone interested in learning more about intelligence analysis.