Cognition · Essay

Walking the Mindfield

A truly great intellect is one which takes a connected view of old and new, past and present, far and near… It possesses the knowledge not only of things, but also of their mutual and true relations.
— Fred Scott and Joseph Denney

This paper describes how systemic practitioners — those trained in system dynamics and/or systems thinking — avoid the reasoning pitfalls that affect most people. Systemic thinking provides advantages in two fundamental areas: in avoiding underestimation or misattribution of relationships when constructing an understanding of a situation, and in producing better action based on that understanding.

Its strength in these areas gives it a great advantage over the thinking commonly employed by those not trained systemically, and is, I believe, one of the main reasons for the superior results obtained by interventions based on system dynamics and systems thinking.

How we build a picture of the world

In order to act, humans must take in information from their environment. But the environment is too rich to process in full — there is simply too much of it. Reducing that flood requires two things to happen to the stream of information: it must be condensed, and it must be structured to reveal the linkages between different stimuli. Cognitive information-processing schemas — sets of propositions that structure information — do both.

They first condense experience from a string of unique stimuli into sets of similar experiences. This is essential; we could not afford to approach every door, for example, as if it were a completely unique structure and spend the time to test every possible way of opening it. As Taylor and Crocker (1981) point out:

Given some stimulus configuration in the environment, it would be prohibitively time-consuming to match it against all prior experiences… What schemas do is enable the perceiver to identify stimuli quickly, “chunk” an appropriate unit, fill in information missing from the stimulus configuration, and select a strategy for obtaining further information.

After processing information into manageable chunks and looking for similarities with past experience, we must construct an explanation for what we are perceiving. Here schemas serve as the tautologies that structure our experience by providing the links between the elements we experience. Taylor and Crocker continue:

When a stimulus configuration is encountered in the environment, it is matched against a schema, and the ordering and relations among the elements of the schema are imposed on the elements of the stimulus configuration… This lays the groundwork for subsequent inferences.

While the condensing and structuring functions of schemas are essential, their very importance means that errors in the schemas can have a large impact on the quality of our perceptions — and thus our decisions. It is in addressing the weaknesses that commonly appear in schemas that systemic thinking provides its advantage.

As Gregory Bateson (1979) notes, “an explanation is a mapping of the pieces of a description onto a tautology, and an explanation becomes acceptable to the degree that you are willing and able to accept the links of the tautology.” In other words, our mental models of the relationships between things determine what we accept as an explanation — and our satisfaction with the model determines our satisfaction with the explanation we build on it.

The Fundamental Attribution Error

The first area in which systemic thinkers avoid well-documented reasoning errors is in avoiding the underestimation or misattribution of relationships — an error that has come to be called the Fundamental Attribution Error. The tendency to neglect systemic influences on behavior is pervasive. Ross (1977) reports an experiment in which Bierbraur (1973) asked subjects to predict rates of disobedience in the classic Milgram (1963) obedience experiment. Bierbraur's subjects consistently underestimated the effect of systemic influences in producing obedience and overestimated the effect of unique personality factors — even after witnessing a faithful reenactment of the experiment.

Other experiments (e.g., Jones & Harris, 1967) find the error so strong that people underestimate systemic forces in explaining someone's behavior even when they know the person had no choice. Especially in cases like Milgram's, where small changes that reduced the authority principle produced marked changes in behavior, how do observers keep attributing that behavior to unique personal factors instead of systemic ones?

Dormitization

I suggest this is because cases like the Milgram experiment are only the most obvious instances of a very common cognitive phenomenon I call dormitization — the act of attributing the effect of a relationship to one of the constituents of the relationship. The name comes from a story in Molière's Le Malade Imaginaire, retold by Bateson (1979): at a medieval doctoral examination, the examiners ask why opium puts people to sleep. The candidate answers triumphantly, “Because, learned doctors, it contains a dormitive principle.”

The candidate attributes the result of a relationship — between the opium and the person — to one of its two elements. The problem is clear when we consider another case of the same mistake: early explanations for why things fall toward the earth. For many years before Newton, the attraction between bodies was explained by the objects “wanting” to be nearer each other. This, again, attributed the product of a relationship — the attraction between two bodies — to one of its constituents.

In the same way, dormitization explains why people fall victim to the Fundamental Attribution Error: they attribute the result of the relationship between an individual and his or her environment to the individual alone.

Why salience misleads us

Behind dormitization is a cognitive process for sifting information. One way we prioritize is by paying more attention to the most salient inputs — the ones that stick out. Experiments confirm that highly salient information is given more importance and remembered better. This is usually a useful filter, because salient information is often also the most important.

A problem arises when relationships determine the outcome — because relationships have no look, feel, or smell to make them salient, and so receive less attention in our explanations. By bringing salience to relationships — creating a cognitive space for them, studying and mapping them — system dynamics and systems thinking help their practitioners avoid dormitization and the Fundamental Attribution Error. System dynamics modeling adds a further advantage: it reduces the tendency to overlook the informational value of things that don't happen, by forcing the model builder to explicitly represent non-occurrences as well as occurrences.

From perception to action

This advantage grows in the second area — when it becomes time to act. To filter out unimportant information, schemas screen out inputs inconsistent with the schema. This produces the well-known self-reinforcing cycle in which what people believe influences what they see, and the filtered reality they then see reinforces what they believe.

The effect is that people caught in dormitization or the Fundamental Attribution Error see a self-confirming reality — one less affected by relationships and interactions. And because schemas influence not only perception but also processing and recall, people are more likely to remember and think about what fits their schemas. Since we tend to take the actions foremost in our minds, people whose perception has been shaped by dormitization are more likely to think of, and take, actions that underestimate relationships and interactions. Companies and managers may keep doing the same thing over and over, even when circumstances have changed, because it fits what they have done before and their schema-shaped idea of how their system works.

How we think affects what we see, how we interpret our experience, and how we act. System dynamics and systems thinking have shown their benefits in many areas, including shaping the way we think. Those who practice them not only gain powerful tools they can apply — they gain a powerful ally that shields them from common cognitive errors and the price those errors exact, in thought and in action.

References

Bateson, Gregory. Mind and Nature. NY: E. P. Dutton, 1979.

Bierbraur, G. Effect of set, perspective, and temporal factors in attribution. Unpublished doctoral dissertation, Stanford University, 1973.

Jones, E. E. & Harris, V. A. “The attribution of attitudes.” Journal of Experimental Social Psychology, 1967.

Milgram, S. “Behavioral Study of Obedience.” Journal of Abnormal and Social Psychology, 1963.

Sterman, J. “Learning in and About Complex Systems.” System Dynamics Review, Summer–Fall 1994.

Taylor, S. E. & Crocker, J., in E. Higgins et al., Social Cognition: The Ontario Symposium. Hillsdale: Erlbaum, 1981.

Daniel Aronson received his MBA from the MIT Sloan School and taught modern critical-thinking skills at the University of California, Berkeley. He hosts the Thinking Page. This article is Copyright 1996–2002, 2026 Daniel Aronson.

Keep going