Integrative Literature Review

See all articles, sample of a literature review, references for all articles attached. For the group of resources in each domain, (1)psychopathology(2)developmental psychology (3)cognitive psychology (4)psychopharmacology evaluate the reliability, validity, and generalizability of the research findings and provide a rationale for including the group within the domain. These rationales should include descriptions of how the research findings will function together in the Integrative Literature Review.Please use the format below for each of the four domains(1)psychopathology(2)developmental psychology (3)cognitive psychology (4)psychopharmacology. List the complete references for each of the six resources. Format the reference list in alphabetical order.Rationale:One to two paragraphs including the required information noted above For each references Sources?

Running head: INTEGRATED LITERATURE REVIEW

INTEGRATED REVIEW

Integrated Literature Review

Carolyn Bennett

PSY699: Master of Arts in Psychology Capstone

Instructor: Katrina Kuzyszyn-Jones

April 15, 2019

There are several domains in the practice of psychology that focus on different areas. This Integrated Literature Review are featuring these four domains, Cognitive, Developmental, and Clinical psychology, and

Psychopharmacology

. The consolidated composition study is dealt with in regions that clarify the investigation, study, and union from the examination articles picked in these fields.

These four domains will be reviewed in this paper. Psychology is too large to be understandable without these domains. Each domain can answer questions asked in one of the other domains. For instance, analysts concentrating Cognitive psychology research could use Clinical psychology science to see more top to bottom through chemical ratios how an individual considers or the thinking for what they are doing. All of the domains can use developmental psychology because the domains can grow all through the field. All the domains can interact with each other.

Psychopharmacology

Baumeister, A. A., Hawkins, M. F., & López-Muñoz, F. (2010). Toward standardized usage of the word serendipity in the historiography of psychopharmacology. Journal of The History of The Neurosciences, 19(3), 253-270. doi:10.1080/09647040903188205

In the article, it is attempted to take the contradictory views that are discussed in writings about the how serendipity plays a role in the discoveries that paved the road to modern psychopharmacology and try to resolve the differences by showing a definition of serendipity that is operational. There are eighteen discoveries that are explored in the use of the definition. The results are that the most main design found in the medications of early psychiatric is the observation of the serendipitous going in the direction to the demonstration of non-serendipitous of the clinical utility. This analysis also shows the examples of clean discoveries of non-serendipitous and serendipitous. These definitions seem the be valid and reliable.

Giles, L. L., & Martini, D. R. (2016). Challenges and Promises of Pediatric Psychopharmacology. Academic Pediatrics, (6), 508. doi: 10.1016/j.acap.2016.03.011

This article covers the evidence of the safety and effectiveness of all the pharmacologic drug classes in youth. There is much literature supporting the evidence of psychostimulants for the first type of treatment for ADHD. The treatment benefits are enhanced, and the medication adverse events are minimized by combining SSRI treatments with Cognitive Behavioral Therapy. Childhood schizophrenia treatment has been, for a long time, antipsychotics. This treatment has grown over the years. The side-effects have become more problematic in youths and pediatricians can understand better how to treat patients.

Goldberg, J. F., Freeman, M. P., Balon, R., Citrome, L., Thase, M. E., Kane, J. M., & Fava, M. (2015). THE AMERICAN SOCIETY OF CLINICAL PSYCHOPHARMACOLOGY SURVEY OF PSYCHOPHARMACOLOGISTS’ PRACTICE PATTERNS FOR THE TREATMENT OF MOOD DISORDERS. Depression and Anxiety, (8), 605. doi:10.1002/da.22378

This article helps to find which drugs are the best for which disorder so that the psychologists and doctor know what to prescribe. It shows the strategies that are used to influence the choices of medication for those with mood disorders.

Nemeroff, C. B. (2014). Psychopharmacology and the future of personalized treatment. Depression and Anxiety, 31(11), 906-908. doi:10.1002/da.22303

Investigating the extraordinary developments in treating many major disorders by being able to take single patients and match them to the best treatments are covered here. The ones that have been investigated the most are cardiovascular disease, some types of cancers and diseases that are infectious. These tests would let the psychiatrist be able to see if the ailment of the patient is really a disease and not a mental disorder.

Singh, A. N. (2015). Current research in psychopharmacology: Applications to clinical practice. International Medical Journal, 22(2), 62-64.

The “psychopharmacology” of today started accidentally with the discovery Hoffman and Stoll of Lysergic Acid Diethylamide when Moreau found that cannabis could be used in mental disorders in 1938. Today, the research for this has grown to Pharmacogenetic and Preventive, Adolescent, Adult, Geriatric, and Child domains to increase the areas for therapeutic treatment for mental disorders. Today, there are pressures getting greater to make sure that growing data gotten by cumulative psychopharmacological research interprets into clinical practice.

Steckler, T., Curran, H. V., de Wit, H., Howes, O., Hoyer, D., Lucki, I., & … Robbins, T. W. (2016). Editorial: Reporting guidelines for psychopharmacology. Psychopharmacology, 233(7), 1131-1134. doi:10.1007/s00213-016-4252-7

The right way to be able to reach all the correct assumptions, deciding what to do next experimentally an obtain the progress scientifically is by gathering data and reporting this data correctly. This can let the person reading understand the exact study, to look at the quality, and redo the study. This article covers what needs to be done about this n the field of psychopharmacology.

Clinical Psychology

Dimoff, J. D., Sayette, M. A., & Norcross, J. C. (2017). Addiction training in clinical psychology: Are we keeping up with the rising epidemic? American Psychologist, 72(7), 689-695. doi:10.1037/amp0000140

In this paper training of psychologists in the area of addiction is covered. This would help in all field of psychology because everyone is susceptible to getting addicted to any type of drug.

Dobson, K. S. (2016). Clinical psychology in Canada: Challenges and opportunities. Canadian Psychology, (3), 211.

Canadians history of clinical psychology is reviewed in this article. The discussions are about the progressions in three serious areas in it. These areas contain evidence-based practice, practice issues, and training. In this analysis, the parts of clinical psychology are looked over including movement towards practice based on evidence in health care. A succession of trials and changes in contemporary Canadian society that affect clinical psychology are identified in the final section of this article. The issues that are included are the development of the psychological workforce, medically assisted death, access to services, aboriginal services, prescriptive authority technological advances in treatments, and refugee services. The article ends with normal recommendations for the growth of clinical psychology in Canada.

Dotterer, H. L., Waller, R., Cope, L. M., Hicks, B. M., Nigg, J. T., Zucker, R. A., & Hyde, L. W. (2017). Concurrent and developmental correlates of psychopathic traits using a triarchic psychopathy model approach. Journal of Abnormal Psychology, 126(7), 859-876. doi:10.1037/abn0000302

Every psychiatric need to be trained in the field of Psychopathy. This talks about a varied group of bad dark behaviors and traits. These include irresponsibility, superficial charm, antisocial behavior, and callousness. The TriPM (triarchic psychopathy model) says the psychopathy is a mixture of three areas: meanness, lack of restraint, and self-assurance. Yet, there has been very little research that has examined the coexisting and developing correlations acquainted with these traits. Boldness was the only one that showed comparative constancy from developing predecessors in early childhood.

Mahoney, E. B., Perfect, M. M., & Edwinson, R. M. (2015). Comparing school and clinical psychology internship applicant characteristics. Psychology In The Schools, 52(10), 972-983. doi:10.1002/pits.21878

Here talks about a study examining the experiences of pre-internship and of clinical psychology and school trainees trying to get a child-focused, APA-accredited internship. The results show the students of the clinic gathered a good amount more intercession hours with adults than children. There were more combined valuation reports written by school psychology students

Perinelli, E., & Gremigni, P. (2016). Use of social desirability scales in clinical psychology: a systematic review. Journal of Clinical Psychology, (6), 534. doi:10.1002/jclp.22284

This report looked over the use of the desirability scales in lessons talking about the desire to be sociable clinical psychology. The results reviewed highlighting boundaries in the use of prestige scales in society in recent studies.

Proctor, R. W., & Urcuioli, P. J. (2016). Functional relations and cognitive psychology: Lessons from human performance and animal research. International Journal of Psychology, 51(1), 58-63. doi:10.1002/ijop.12182 Rees, C. S. (2016).

Requirements for communications between more than one branch of knowledge and to discover different explanations of “building bridges between functional and cognitive psychology” are being considered. This article explains how the connectivity between radical behaviorism and cognitive psychology would probably be unsuccessful, yet, on the off chance that the bridges are intended to bring the useful connections and psychological hypothesis together, no creation is fundamental reason the bridges are as of now there inside subjective brain science.

Developmental Psychology

Apud, I. (2016). PHARMACOLOGY OF CONSCIOUSNESS OR PHARMACOLOGY OF SPIRITUALITY? A HISTORICAL REVIEW OF PSYCHEDELIC CLINICAL STUDIES. Journal of Transpersonal Psychology, 48(2), 150-167.

Before psychopharmacology was developed the focus was on pharmacology of spirituality. In this article the past knowledge of the progress of this spirituality and where it came from is reviewed. It began in the 1950s and became as we know it today, the renaissance of psychedelic studies. This article will help the psychologists to look at mental illness in a spiritual way by understanding what role that developmental psychology plays in.

Farrell, A. H., Semplonius, T., Shapira, M., Zhou, X., & Laurence, S. (2016). Research Activity in Canadian Developmental Psychology Programs. Canadian Psychology, (2), 76.

This study was done to review the current study activity between 2009 to 2013 of the programs of developmental psychology programs in Canada. This article will enable the psychologist to see what the statuses for product research and the impact of it are in Canada.

Koops, W. (2015). No developmental psychology without recapitulation theory. European Journal of Developmental Psychology, 12(6), 630-639. doi:10.1080/ 17405629.2015.1078234

Here the explanation of where the modern version of Developmental Psychology came from and why it was developed. It explains the history before and after and why and who developed it. It should be read by the psychologist, so they would understand all the work that has been put into it.

Krojgaard, P. (2016). Keeping Track of Individuals: Insights from Developmental Psychology. Integrative Psychological and Behavioral Science, (2), 264. doi:10.1007/s12124-015-9340-4

This paper shows evidence of research showing a child’s sensitivity to an object’s history. First, they are sensitive to only one object of history. Second, their data seems to be related to the object’s appearance. In the end, the research on this has shown that a constant sense of me would probably be a necessary requirement to have memories that are episodic. This would enable the psychologist to be able to understand an infant better.

Noth, I. (2015). ‘Beyond Freud and Jung’: Sabina Spielrein’s contribution to child psychoanalysis and developmental psychology. Pastoral Psychology, 64(2), 279-286. doi:10.1007/s11089-014-0621-5

In here Sabrina Spielrein’s comparison of Jung and Freud and her research on the areas of developmental psychology and child analysis is shown. She made many contributions to the area and some of them were ahead of her time.

SLOBODCHIKOV, V. I., & ISA’EV, E. I. (2015). The Conceptual Foundations of Developmental Psychology. Journal of Russian & East European Psychology, 52(5/6), 45-136. doi:10.1080/10610405.2015.1199162

Here the focus is on the progress of reality of a person written about in two other books. The first part is focused on the study of paradigms and approaches that are scientific to the development of psychology. This article will help a psychologist to understand psychology development in a person’s life. The second part builds a copy of individual reality and the way it develops throughout a person’s life.

Cognitive Psychology

Alcorn, Mark B. (2013) Cognitive psychology Washburn, Allyson, Salem Press Encyclopedia of Health/http://eds.a.ebscohost.com.proxy-library.ashford.edu/eds/ detail/detail?vid=1 &sid=3fd54f16-b5cf-4acc-ad5b-04dcfb5a5217%40sessionmgr 4006&bdata= JnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=93871847&db=ers

In this article, you will read about Cognitive Psychology. This is the actual study of the way data is acquired, kept in the brain, getting the data again and then using it. It also studies how a person’s mind can make sense of the data that is in there and sees the data it is retaining and how it forms the patterns.

Cornoldi, C. (2013). Basic and applied cognitive research in a country discovering psychology. Applied Cognitive Psychology, 27(1), 137-138. doi:10.1002/acp.2840

These authors research inside the field of Cognitive Psychology is shown in this article. The things that are blocking are described in it and the prospects connected with residing where Psychology was not developed.

Noori, M. (2016). Cognitive reflection as a predictor of susceptibility to behavioral anomalies. Judgment and Decision Making, (1), 114.

This article is about a study on how cognitive reflection effects behavioral anomaly. To do this the cognitive reflection was measured by the cognitive reflection test. The study showed that persons with a cognitive reflection that was lower were likely to show the conservatism, illusion of control, base rate fallacy, overconfidence, and conjunction fallacy. These results do not show any association that cognitive reflection has with the status quo bias or self-serving bias. It has also been found that gender does have something to do with the relation of the self-serving bias and illusion of control This article would help a psychiatrist to be able to recognize any type of anomaly present in the patient.

Robert W. Proctor and Peter J. Urcuioli (2015) Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA / http://eds.a.ebscohost.com.proxy-library.ashford.edu/eds/pdfviewer/pdfviewer?vid=4&sid=5ce20c30-4a54-4226-855d-fb646cb9e6c6%40sessionmgr4009

This article shows how to discover the bridge between cognitive and functional psychology. The person must want to have a good relationship with someone for it to be successful. The two persons involved must be on agreeable terms for it to work.

Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, (1), 99.

Here all of the research covers whether questions on the CRT-2 are better than the ones on the CRT. Even though the main reason was to investigate the CRT-2, it was also seen the questions look appropriate as an additional source of different objects.

Ryder, A. G., Sun, J., Zhu, X., Yao, S., & Chentsova-Dutton, Y. E. (2012). Depression in China: integrating developmental psychopathology and cultural-clinical psychology. Journal of Clinical Child And Adolescent Psychology: The Official Journal For The Society Of Clinical Child And Adolescent Psychology, American Psychological Association, Division 53, 41(5), 682-694. doi:10.1080/15374416.2012.710163

Formative psychopathology examine has underscored pre-adult examples and intellectual models of causation; social clinical brain science and social psychiatry inquire about have stressed grown-up tests and the implications related with feelings, side effects, and disorders. The two ways to deal with the investigation of discouragement in China have yielded imperative discoveries yet have additionally featured issues that need to be tended to by joining other methodologies. Past dejection in China, the mental investigation of emotional wellness and culture more for the most part would profit by more prominent trade between formative psychopathology and social clinical brain research.

Psychology of Aesthetics, Creativity, and the Arts

Psychopathology in World-Class Artistic and Scientific

Creativity

Gregory J. Feist, Daniel Dostal, and Victor Kwan
Online First Publication, October 21, 2021. http://dx.doi.org/10.1037/aca0000440

CITATION
Feist, G. J., Dostal, D., & Kwan, V. (2021, October 21). Psychopathology in World-Class Artistic and Scientific Creativity.
Psychology of Aesthetics, Creativity, and the Arts. Advance online publication. http://dx.doi.org/10.1037/aca0000440

Psychopathology in World-Class Artistic and Scientific Creativity

Gregory J. Feist1, Daniel Dostal2, and Victor Kwan

1 Department of Psychology, San Jose State University

2 Department of Psychology, Faculty of Arts, Palacký University Olomouc

The role of psychopathology in creative achievement has long been a debated topic in both popular culture
and academic discourse. Yet the field is settling on various robust trends that show there is no one answer.
Conclusions vary by level and kind of creativity and level and kind of psychopathology. The current study
sought to replicate previous findings that linked lifetime rates of psychopathology to world-class levels of crea-
tivity. A total of 199 biographies of eminent professionals (creative artists, creative scientists, eminent athletes)
were rated by raters who were blind to the identity of the eminent person on 19 mental disorders using a 3-
point scale of not present (0), probable (1), and present (2). Athletes served as an eminent but not creative
comparison group to discern whether fame, independently of creativity, was associated with psychopathology.
Results showed that artists exhibited higher lifetime rates of psychopathology than scientists and athletes in the
more inclusive criterion for psychopathology (i.e., it was either probable or present), whereas both artists and
athletes exhibited higher rates than scientists in the stricter criterion for psychopathology (i.e., it was present).
Apart from anxiety disorder, athletes did not differ from the U.S. population in lifetime rates of psychopathol-
ogy, whereas artists differed from the population in terms of alcoholism, anxiety disorder, drug abuse, and
depression. These data generally corroborate and replicate previous biographical research on the link between
artistic creativity and life-time rates of psychopathology.

Keywords: creativity, mental illness, psychopathyology, artists, scientists

Supplemental materials: https://doi.org/10.1037/aca0000440.supp

The stereotype of the mad genius has been a popular notion for
quite some time. Brilliant yet mad artists, such as Vincent van
Gogh, innovators such as Howard Hughes, and mathematicians
such as Isaac Newton have inspired this view throughout history
(Brownstein & Solyom, 1986; Jeste et al., 2000; Perry, 1947). The
list of geniuses with mental illness could go on and on. Of course,
the list of creative geniuses not afflicted with mental illness would
no doubt be at least as long. The range of conclusions on the ques-
tions are highlighted in the following four quotes: “There is no

great genius without some mixture of madness” (Aristotle, as para-
phrased by Seneca, 2007; cf. Motto & Clark, 1992).

Thus the creative genius may be at once naïve and knowledgeable,
being at home equally to primitive symbolism and to rigorous logic.
He is both more primitive and more cultured, more destructive and
more constructive, occasionally crazier and yet adamantly saner, than
the average person. (Barron, 1963, p. 224)

“Psychopathology and creativity are closely related, sharing
many traits and antecedents, but they are not identical, and out-
right psychopathology is negatively associated with creativity”
(Simonton, 2006). “Despite centuries of professional attention, the
link between creativity and madness remains more stereotype than
science” (Schlesinger, 2017, p. 60).

Anecdotal evidence is just that—anecdotal. But is there truly a
legitimate empirical link between psychopathology and creative
genius? Over the last 10 to 15 years, the field has begun to settle
on various robust trends that show there is no one answer to the
question of creativity and mental health. Conclusions vary by level
and kind of creativity and level and kind of psychopathology
(Baas et al., 2016; Beaussart et al., 2017; Fisher, 2015; Glazer,
2009). As Simonton (2014, 2017a, 2017b, 2019) has recently dem-
onstrated, all of these positions may have validity, with both linear
and nonlinear relationships. As is true of all entrenched scientific
debates, there must be some truth to each side otherwise one side
would die off very quickly. The current study sought to replicate
and extend previous biographical findings that linked lifetime rates
of psychopathology to world-class levels of creativity by examin-
ing the moderating effects of the relationship.

Gregory J. Feist https://orcid.org/0000-0002-3123-1069
Daniel Dostal https://orcid.org/0000-0001-5489-7907
This research was in partial fulfillment of the Master’s Thesis for Victor

Kwan. Portions of these findings were presented in 2019 at the Southern
Oregon University Creativity Conference, Ashland, Oregon, and in 2016 to the
Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland.
The authors thank Abiola Awolowo, Brian Barbaro, Kimya Behrouzia,

Catherine Erickson, Evander Eroles, Janet Dai, Adrian Davis, Sheila
Greenlaw, Jennifer Kang, Illhame Khabar, Thomas Lu, Caitlyn Ma, Dat
Nguyen, Elizabeth Shallal, Kimia Sohrabi, Eldita Tarani, Ryan Willard,
and Laura Weber. Data collection was sponsored by a grant from the
Research Foundation, San Jose State University. We have no conflicts of
interest to disclose.
Correspondence concerning this article should be addressed to Gregory

J. Feist, Department of Psychology, San Jose State University, One
Washington Square, San Jose, CA 95192-0120, United States. Email: greg
.feist@sjsu.edu

1
Psychology of Aesthetics, Creativity, and the Arts

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by

th
e
A
m
er
ic
an

P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on

or
on
e
of

it
s
al
li
ed

pu
bl
is
he
rs
.

T
hi
s
ar
ti
cl
e
is
in
te
nd
ed

so
le
ly

fo
r
th
e
pe
rs
on
al
us
e
of

th
e
in
di
vi
du
al
us
er

an
d
is
no
t
to

be
di
ss
em

in
at
ed

br
oa
dl
y.

https://doi.org/10.1037/aca0000440.supp

https://orcid.org/0000-0002-3123-1069

https://orcid.org/0000-0001-5489-7907

mailto:greg.feist@sjsu.edu

https://doi.org/10.1037/aca0000440

Creativity

Creativity researchers have most regularly described creativity as
consisting of two qualities, namely originality/novelty and mean-
ingfulness/usefulness (Amabile, 1996; Feist, 2017; Runco & Jaeger,
2012; Sternberg, 1988). A creative endeavor must not only be dif-
ferent from what has been previously performed in a given domain
but also useful. In this case, the term “usefulness” can also mean
beautiful or provocative for artwork and literature. Some have
argued that the term “usefulness” could be replaced by the word
“meaningful” (Feist, 2017). With this change in terminology, the
need to qualify “useful” as also beautiful or provocative is no longer
necessary. Products of both art and science can be meaningful,
whereas a piece of artwork would not necessarily be useful. Mean-
ingful makes clear that the meaning is in the evaluator, and this can
be the general public, experts, peers, or historians, among others.
In the last 10 years, however, a few creativity researchers have

argued for three, rather than two, factor definitions of creativity.
Simonton (2013, 2016), for example, has proposed a logical quan-
titative model whereby a creative idea or solution is a multiplica-
tive function its originality, utility, and surprisingness. More
formally, and omitting the i subscript for each individual idea, cre-
ativity c = (1 – probability p) utility u (1 – prior knowledge of util-
ity v) and where p, u, v, and c each range from 0 to 1. The
compelling aspect of this formulation is its multiplicative function,
whereby any value of 0 for p, u, or v results in an idea that is not at
all creative.

Psychopathology and Creativity

Research and theory on psychopathology and creativity are
growing and contentious. A recent edited volume entitled Creativ-
ity and Mental Illness captures the history, current state of the
field, and the wide range of views on the topic (Kaufman, 2017).
Although the ancient Greeks (Aristotle in particular) were the first
people in the Western world to examine the nature of creativity
and its association with “melancholia,” it was not until the Roman-
tics in literature in the 1830s that the argument was made for any
connection between serious mental affliction and creative genius
(Becker, 2017).
The modern literature on the topic was jumpstarted in the 1980s and

1990s with the work of Andreasen, Jamison, Richards, and Ludwig. In
her early investigations, Andreasen (1978, 1987) reported qualified
relationships between creativity, especially literary creativity, and men-
tal illness, making use of historical, familial, and genetic studies. Jami-
son (1996) reported historical, biographical, and literary evidence for
the association between artistic creativity and mood disorders, most
specifically bipolar disorder. Jamison et al., (1980) also examined the
relationship between creativity and bipolar illness in noncreative peo-
ple and found that the hypomanic period led to heightened creativity.
Richards and colleagues (1988) also found that it was mild levels of
mania (hypomania) and bipolar (cyclothymia) that were most strongly
associated with creative thinking. Large-scale biographical examina-
tions by Ludwig and Post in the 1990s reported associations between
artistic creativity and lifetime rates of mental illness. Ludwig (1992,
1995) examined more than 1,000 eminent professionals, including, but
not limited to, artists, writers, scientists, and musicians and revealed
that extremely creative individuals, especially in the visual and literary
arts, exhibited elevated rates of various lifetime psychopathologies.

Post (1994) also drew a similar conclusion from biographical analysis
of more than 200 world-famous creative people. The sample in this
study was restricted to deceased subjects of biographies reviewed by
the New York Times. These biographies were then examined for signs
of psychopathology in each eminent professional and correlated with
each domain of expertise. The results showed that people who excelled
at creative endeavors such as poetry and fiction writing experienced
higher rates of psychopathology than scientists or politicians.

To be sure, partly owing to inconsistent empirical results, not all
scholars agree there is a connection between high levels of creativ-
ity and psychopathology (Sawyer, 2011; Schlesinger, 2009, 2017;
Thys et al., 2014). The most outspoken and harshest critique
comes from Schlesinger (2009, 2012, 2017). She essentially dis-
misses the entire field and literature on creativity and “madness”
as based on nothing more than poorly conducted and flawed
research (e.g., biographies are dismissed as little more than “gos-
sip”), even going so far as to call it a “hoax” (Schlesinger, 2012).
She takes particular issue with three of the key figures in the field,
Andreasen, Jamison, and Ludwig. Andreasen (1987) had too few
participants, relied on personal relationships with participants, and
overgeneralized results. Jamison’s (1996) “autopsy subjects” of
166 deceased artists, writers, and musicians is challenged. Schle-
singer critiques three of the 166 subjects (Michelangelo, Emerson,
and Cole Porter) and yet fails to provide a more general and struc-
tured critique of the participant selection. Her methodological
criticisms have some validity because no research is without limi-
tations and flaws, but Schlesinger does her arguments a disservice
when she makes many absolutist statements that no one really
claims, such as the supposedly common belief that “no one
receives the gift of genius without the curse of depression” (Schle-
singer, 2017, p. 60) or that much of the research gets propagated
without people reading the original articles.

Moderating Effects in the Relationship Between
Creativity and Psychopathology

The apparent contradictory set of results gains clarity when we
begin to look more closely at the reasons for the mixed, moderat-
ing, and complex findings. One study, for example, reported that
gender moderates the relationship between creativity and psycho-
pathology, with positive results only holding for men (Martín-Bru-
fau & Corbalán, 2016). More general findings highlight four
common moderating effects in the relationship between creativity
and psychopathology (Baas et al., 2016; Beaussart et al., 2017;
Feist, 2012; Fisher, 2015; Glazer, 2009; Silvia & Kaufman, 2010;
Simonton, 2017a, 2019):

• degree of creativity
• domain of creativity (art v. science)
• degree of psychopathology
• domain of psychopathology

Degree of Creativity

Creative achievement exists on a wide spectrum, from minor to
major contribution. For decades, researchers who study creativity
have realized there is a need for distinguishing levels of creativity.
To label someone creative is nearly meaningless if it does not dif-
ferentiate historical genius global level creativity from minor, indi-
vidual, everyday creativity. As Kaufman and Beghetto (2009)

2 FEIST, DOSTAL, AND KWAN

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

argued, there are at least four distinct levels of creativity that are
more or less developmental. First, there is mini-c creativity, that
is, personal level of creativity, where someone is creating ideas or
behaviors that are novel and meaningful to him or her only. If that
person goes on to produce novel products, ideas, or behaviors that
involve solving everyday problems creatively, we would call that
little-c creativity. This could involve activities such as making a
new meal, creatively making use of materials around the house to
build a play structure, or writing an essay for a class. Next, we
move to people who make a living doing creative work, such as
writers, musicians, scientists, architects, actors, and painters, but
whose acclaim and impact are regional rather than national or
international. This is Pro-C creativity, for “professional.” In other
words, the circle of people who find it meaningful or useful is rela-
tively narrow. It has not yet changed the course of the discipline or
started a new branch of the discipline. Lastly, there is Big-C crea-
tivity, or genius level or historically significant creativity. These
are people who change the course of their fields, may have biogra-
phies written about them, and often earn award and recognition
from their peers for doing the best work in their field, and some-
times are studied by historians. In short, these four levels of the
creative spectrum move from smallest to largest social/cultural
circle: self, family, region, and nation/world.
As we discussed at the outset, there are different degrees of cre-

ativity, different kinds of creativity, and how one defines and
measures it matters (Reiter-Palmon & Schoenbeck, 2020). One
reason for conflicting results in the literature on creativity and psy-
chopathology is the fact that degrees of creativity are conflated.
The size of the effect seems to be directly related to the degree or
level of creativity, with the effect being largest in Big-C creative
samples and smallest in little-c creative people. Moreover, method
of research is related to these two levels, with most little-c samples
being studied psychometrically or experimentally and Big-C or
Pro-C samples being studied historiometrically, biographically, or
epidemologically (Johnson et al., 2012; Paek et al., 2016; Rich-
ards, 1990; Silvia & Kimbrel, 2010; Taylor, 2017). For example,
in a large Swedish national-population sample that included tens
of thousands professional (Pro-C) creative artists and scientists,
Kyaga and colleagues (2011) reported significant associations
between creative professions and being treated for bipolar disorder
or having a sibling or parent treated for schizophrenia. Moreover,
in a study of an undergraduate (little-c creativity) sample, Silvia
and Kimbrel (2010) found that anxiety and depression could only
explain 3% of the variance in creative thinking. Finally, Johnson
and colleagues (2012) conducted an extensive qualitative review
of the literature on bipolar disorder and creativity and found stron-
ger effects with Big-C than little-c creativity.

Domain of Creativity (Art Versus Science)

One of the main empirical findings from the literature on creativ-
ity and psychopathology is the stronger effect size between artistic
than scientific creativity and psychopathology (Damian & Simon-
ton, 2015; Ludwig, 1995, 1998; Post, 1994). Trauma, stress, mood
disorders, and fear of death each seem to motivate artistic creativity
in a way they do not motivate scientific creativity or innovation in
business. For instance, Ludwig’s later analyses (Ludwig, 1998)
made clear that lifetime rates of psychopathology are mostly ele-
vated in the arts compared with the sciences, and in the expressive

arts in particular (writing (fiction and nonfiction), poetry, and visual
arts; see Figure 1). The performing artists (musicians and dancers)
had moderately elevated rates of psychopathology, whereas the for-
mal artists (architects) were not different from the general popula-
tion. He argued and reported, therefore, that it is the more
expressive, intuitive, and subjective creative professions where psy-
chopathology and creativity should be and was most likely to be
associated. The more formal, logical, precise, and objective profes-
sions should be and were less likely to see psychopathology. In
short, in the expressive arts, personal meaning, subjectivity, and
emotion play a motivational role in ways not common in the more
formal creative professions. Further analysis within the artistic
forms revealed consistently higher rates of psychopathology in the
emotive/expressionistic styles than in the formal/realism styles. To
be sure, these are correlational findings, so whether those with men-
tal and mood disorders are drawn to artistic careers or the other way
around has yet to be established.

Other scholars have reported similar patterns whereby scientists
suffer relatively low lifetime rates of psychopathology, whereas
the other professions, especially the arts, had elevated rates of
mental illness compared with base-rates in the general population
(Damian & Simonton, 2015; Post, 1994; Simonton, 2014). A bio-
graphical replication of the mental health status of 40 jazz musi-
cians from the 1940s and 1950s replicated Ludwig’s basic finding
(Wills, 2003). Wills found elevated rates on chemical dependency,
mood disorders, and anxiety disorders in jazz musicians. Heroin
addiction was also elevated in Will’s jazz sample, with 52% sam-
ple having heroin problems at some point during their lives.

The one exception to this general finding with scientists may be
autism-spectrum disorder (ASD), predominantly in the high-func-
tioning range (Baron-Cohen et al., 2007; Billington et al., 2007;
Focquaert et al., 2007; Thomson et al., 2015; Wei et al., 2017). We
should point out, however, that most of this research on ASD and
science and technology is with interest and careers in STEM (sci-
ence, technology, engineering, and math) and not necessarily highly
creative scientists and technologists. We are interested in examining
whether this relationship holds at high levels of scientific achieve-
ment and creativity.

Although Ludwig (1992) argued that psychopathology explained
very little variance in terms of scientific achievement, there are cer-
tain circumstances where psychopathology may exist in scientific
creativity. A good demonstration of this is the analysis by Ko and
Kim (2008) of 76 scientific geniuses from Simonton’s sample. Ko
and Kim predicted and found that the relationship between scien-
tific creativity and psychopathology would be moderated by the
kind of contribution the scientist made, namely whether it preserved
or rejected paradigms. Specifically, scientists without pathology
were more creative when they made paradigm-preserving than par-
adigm-rejecting contributions. Paradigm-preserving is a contribu-
tion that advances but does to change a field’s direction. Paradigm-
rejecting contributions do in fact change a field’s direction. Those
with psychopathology, especially psychotic disorders, were more
creative (based on biographical/encyclopedia index ratings) when
they made paradigm-rejecting rather than paradigm-preserving
contributions. In addition, this moderator analysis more than
doubled the variance explained (18% vs 8%) by psychopathol-
ogy in scientific creativity compared with Ludwig’s (1992) sam-
ple. An implication of these results is that psychopathology may

PSYCHOPATHOLOGY AND CREATIVITY 3

be detrimental to their creative contribution only to scientists
who make paradigm preserving contributions.
Taylor (2017) discussed important ways how art and science

may be different: art requires less formal education and has more
flexible work schedules (Simonton, 2010; Simonton & Song,
2009) and hence does not exclude people with mood disorder;
poets, writers, and so forth more likely to experience trauma, men-
tal health problems in childhood; Ludwig (1998) suggests “occu-
pational drift” owing to emotional expressiveness required for
different professions. In addition, we argue for internal versus
external experiences and meaning and understanding. That is, art
—especially the expressive arts of literature and visual arts—is of-
ten based in internal emotional/traumatic/stressful personal experi-
ences that lead to a need/motive to understand, give meaning to, or
to express these experiences in artistic form, whether it be music,
dance, poetry, visual art, or fiction writing (Akinola & Mendes,
2008; Gardner, 1973; Forgeard et al., 2017; Thomson, 2017). It
may not always be traumatic experiences, but at the least involves
a need or desire to express one’s perceptions of the world and their
place in it (Forgeard et al., 2017). Science, on the other hand, is of-
ten driven by a need to understand and figure out the external
world, especially in the physical sciences. Scientists most often
ask: “What is that? Why? How can we explain that?” That is a

search for meaning and understanding, to make sense of one’s
external rather than world.

Degree of Psychopathology

It has become clear with the accumulation of research that high
levels of mental illness are generally at odds with high levels of
creativity in any field, including art. Even when creativity and ill-
ness go together in certain people, it is mostly during periods of
relative calm and milder dysfunction that creative behavior may
coexist with pathology. In short, there is a nonlinear relationship
between illness and creativity, with mild to moderate levels of pa-
thology being most associated with creative achievement (Abra-
ham, 2017; Acar et al., 2018; Feist, 2012; Kinney & Richards,
2017; Simonton, 2017a; Swain & Swain, 2017; Wuthrich & Bates,
2001). Jamison’s well-known book Touched With Fire (Jamison,
1996), for instance, presented historical evidence for an associa-
tion between bipolar disorder and creativity, especially in literature
(i.e., Big-C creative people). However, she also made clear that it
was those with milder forms (cyclothymia) that were most crea-
tive. Other researchers have come to the same conclusion, namely
that the relationship between creativity and bipolar disorder, schiz-
ophrenia, and schizotypy is mostly curvilinear (Acar et al., 2018;
Cox & Leon, 1999; Gostoli et al., 2017; LeBoutillier et al., 2014;

Figure 1
Categories of the Arts and Sciences and Prevalence of Mental Illness

Note. Dark Gray: . 70%. Light Gray: . 60% , 70% White: , 60%. Adapted from Ludwig,
1998. See the online article for the color version of this figure.

4 FEIST, DOSTAL, AND KWAN

Ruiter & Johnson, 2015). For instance, Kinney and Richards
(2017) reported support for the nonlinear inverted-U hypothesis in
which creative thought and behavior were maximum at mild levels
of bipolar disorder (cyclothymia) and were relatively low at the
low and high ends of the disorder. Moreover, it was first degree
relatives of those suffering from bipolar who tended to exhibit
highest levels of creativity.
After reviewing literature on affective traits and creativity, Feist

proposed a quadratic model of mental health and creativity, that
makes clear the complex relationship (cf. Feist, 2012; see Figure 2).
The peaks of creativity tend to be with little and moderate levels of
psychopathology, with valleys of creativity coming when psycho-
pathology is low-medium and again high (cf. Feist, 2012).

Domain of Psychopathology

Not only degree but also domain of pathology matters. In gen-
eral, it is clear from research that particular forms of psychopathol-
ogy are more associated with high levels of creative achievement
than other forms. The milder forms of mood disorders, including
depression and bipolar disorder as well as milder forms of psy-
chotic disorders (schizotypy), appear to be among the more robust
correlates of creative achievement.

Mood Disorders

Of all psychological disorders, perhaps none is more often empiri-
cally connected to creativity than mood disorders, especially bipolar
depression (and its less severe offshoot, cyclothymia). The general
finding is there is an elevated rate of bipolar disorder exists among
creative people compared with general population (Andreasen, 1987;
Andreasen & Glick, 1988; Fodor & Laird, 2004; Furnham, Batey,
Anand, & Manfield, 2008; Gostoli et al., 2017; Jamison, 1996; Jami-
son et al., 1980; Johnson et al., 2012; Johnson, Murray, et al., 2015;
Johnson, Tharp, et al., 2015; Nowakowska et al., 2005; Ramey &
Weisberg, 2004; Richards, 1994; Taylor, 2017). Taylor’s (2017)

meta-analysis of studies published between 1987 and 2014 that
examined mood disorders (bipolar, cyclothymia, major depression)
in creative samples reported a Hedges g = .64 (95% CI [.45, .82]),
meaning creative people are nearly two thirds of a standard deviation
higher in mood disorder than noncreative people. When examining
simple correlational studies on creativity scales and mood disorder in
students and adults, however, Taylor (2017) reported a very small
effect (g = .09; 95% CI [.01, .17]). In short, the effect size was mod-
erated by level of creativity. Flaherty (2005) reviewed a large range
of neuroscientific evidence suggesting that frontal-temporal-limbic
brain activity as well as dopaminergic activation are implicated in the
relationship between creative drive and mood disorders. More specif-
ically, Flaherty proposed a two-dimensional model with frontal-tem-
poral activity being on the x axis and dopaminergic activity being on
the y axis. As abnormal temporal lobe activity and dopaminergic ac-
tivity both increase, mania, psychosis, and creative drive increase. As
abnormal frontal activity increases and dopaminergic activity
decreases, creative blocks become more likely.

The relationship holds in the other direction too. Other studies
have reported higher rates of creativity among bipolar patients
(Richards, 1994; Richards et al., 1988; Santosa et al., 2007; Simeo-
nova et al., 2005). For example, when compared with healthy con-
trols, patients with bipolar disorder scored higher on the Barron-
Welsh Art Scale (BWAS) measure of creativity (Santosa et al.,
2007; Simeonova et al., 2005). Of note, however, is Taylor’s
(2017) finding that people with mood disorder are not necessarily
more creative than those without mood disorder (g = .08; 95% CI
[�.00, .16]). Kaufmann and Kaufmann (2017) reviewed research
on the complex association between mood, mood disorders, and
creative thought and behavior. Both positive and negative affect
and mood can be associated with creativity.

An important qualification to the bipolar-creativity connection is
that it seems to be more mania than depression that is associated
with creative thought and behavior (Andreasen & Glick, 1988;
Jamison, 1996; Jamison et al., 1980). Given the quickness and

Figure 2
Nonlinear Model of Degree of Creativity and Psychopathology

PSYCHOPATHOLOGY AND CREATIVITY 5

fluency of ideas that occur during mania, its relationship with crea-
tivity is understandable (Jamison, 1996; Richards, 1994). Moreover,
numerous studies support the idea that milder hypomanic states are
most clearly correlated to creative thinking and achievement (Furn-
ham et al., 2008; Ruiter & Johnson, 2015; Schuldberg, 1990).
The relationship between unipolar depression and creativity is

less robust than bipolar. Paek and colleagues (2016) conducted a
meta-analysis that included 27 studies that reported results on
depression and creativity. These 27 studies consisted of 103 effect
sizes on over 14,000 participants. The mean effect size was essen-
tially zero (r = .04; 95% CI [�.08, .16]). Silvia and Kimbrel
(2010) reported the same very small effects between depression
and various forms of creativity in a college student sample. Some
research that broke nonclinical depression down into components
of rumination, self-reflective pondering, and brooding found that
rumination and self-reflective pondering but not brooding were
associated with creativity (Verhaeghen et al., 2005, 2014). Note,
however, that all of these studies were conducted with students
and were little-c creative samples.
Similar small effects between trait anxiety and creativity have

been reported (Silvia & Kimbrel, 2010). For example, Paek and col-
leagues (2016) also included in their meta-analysis research 32
studies that reported results on anxiety and creativity. These 32
studies consisted of 60 effect sizes on more than 15,000 partici-
pants. As was true with depression, the mean effect size between
anxiety and creativity was not significantly different from zero (r =
�.05; 95% CI [�.16, .06]).
Psychotic Disorders

The psychotic disorders—schizophrenia, schizotypy, schizoaf-
fective disorder, among others—also have a complex and not easy
to summarize relationship with creativity. With anecdotal excep-
tions such as John Nash, full blown schizophrenia is seldom linked
to creativity (cf. Nasar, 2011; Rothenberg, 1990). Kyaga and col-
leagues (2011) reported that people who had first degree relatives
suffering from schizophrenia and bipolar disorder were overrepre-
sented in creative professions. Moreover, Eysenck (1993, 1995)
proposed and found some support for the idea that the nonclinical
personality trait and psychoticism is associated with creative
thought and behavior. Psychoticism is a nonpathological rather
than clinical personality trait consisting of consistent social isola-
tion, aloofness, hostility, and unusual thoughts and behaviors.
Feist (1998) found support for this idea in a meta-analysis on the
personality correlates of creativity.
As numerous scholars have pointed out, however, psychoticism

is too broad and diverse a construct to consistently be related to cre-
ative thought and behavior (Batey & Furnham, 2008; Carson et al.,
2003; Mason et al., 1995; Nettle, 2006). They argue that psychoti-
cism’s specific and somewhat more clinical cousin, schizotypy, is
more robustly related to creativity. Schizotypy is a personality dis-
order in which subclinical symptoms of psychosis are exhibited,
such as unusual experiences, magical thought, eccentric behavior,
and cognitive disorganization (Claridge et al., 1996). In little-c crea-
tive and Big-C creative samples, schizotypy is associated with crea-
tive thought (Acar & Runco, 2012; Acar & Sen, 2013; Baas et al.,
2016; Batey & Furnham, 2008; Burch et al., 2006; LeBoutillier et
al., 2014; Nettle, 2006; Schuldberg, 1990). Baas and colleagues
(2016), for instance, argued for a moderation effect by type of

pathology in the association between creativity and mental illness.
More specifically, they proposed and found meta-analytic evidence
that approach-based pathologies (positive schizotypy and risk of
bipolar) were more strongly and positively associated with high lev-
els of creativity. Positive schizotypy consists of unusual experiences
and impulsive nonconformity, whereas negative schizotypy consists
of cognitive disorganization and withdrawn schizoid traits. In addi-
tion, avoidance-based pathologies (e.g., anxiety, negative schizo-
typy, and depressive mood) were associated with lower levels of
creativity. Similarly, Acar and Sen (2013) in a meta-analysis found
small negative effect sizes between creativity and negative schizo-
typy (r = �.09; 95% CI [�.12, �.06]; k = 76) and a small positive
association with positive schizotypy and creativity (r = .14; 95% CI
[.12, .17]; k = 121).

The Current Study

The primary purpose of the current study is to update and
attempt to replicate the results of Ludwig (1992, 1995, 1998) and
to test a more complex model of creativity and psychopathology.
Not only is the Ludwig sample itself over 25 years old, but the
subjects examined were required to be deceased, further distancing
them from their contemporaries. Therefore, an update and exten-
sion of the study is now in order. Additionally, the professional
categories proposed in Ludwig (1992, 1995, 1998) required
reworking in the current study. For instance, several of the profes-
sions listed under social sciences, such as historian and philoso-
pher, are not actually sciences at all and are frequently grouped
with humanities. The current study also improves on the previous
methodology, which was vulnerable to researcher bias owing to
the investigator’s awareness of the hypotheses (Ludwig, 1992).
Another goal of the current study is to see whether Ludwig’s find-
ings from 20 years ago and with a different sample still hold and
replicate in a more current sample. More importantly, Ludwig did
not compare eminent famous creative people against eminent fa-
mous noncreative people, and simply analyzed his data with chi-
squares for inequalities between groups. We decided, therefore, to
hold fame constant in our comparison group of famous athletes
(with published biographies) to determine whether fame more than
creativity could explain the presence or absence of mental illness
in our sample. In short, by having a control group that was eminent
and famous (i.e., biography-worthy) and yet not creative, we could
rule out pure fame and eminence as a confounding explanation in
any relationship between creativity and psychopathology.

Hypotheses

1. World-class creative artists will have elevated rates of any
lifetime mental illness relative to creative scientists and famous
athletes (controls).

2. The mental health difference between artists and scientists
will be most pronounced on mood/affective disorders (anxiety,
depression, bipolar) and chemical dependency, with artists
expected to have higher rates than scientists.

3. Creative scientists should show elevated rates of being on the
high functioning end of the autism spectrum relative to athlete
controls.

6 FEIST, DOSTAL, AND KWAN

Method

Subjects and Materials

The list of potential creative and eminent scientists, artists, and
athletes for inclusion in the study came from rankings in diction-
aries, encyclopedias, and best-of lists. The original list, after merg-
ing 163 source lists and removing duplicates, contained 17,689
distinct names. Each “best-of” source list was ranked by a member
of the research team (D.D.) on a 3-point scale for trustworthiness,
with a 1 being of questionable validity, 2 being more subjective,
and a 3 being very trustworthy. The primary criterion for a code of
3 was whether the list was created by experts in the field and/or
was of international award such as the Nobel Prize. A code of 2
was awarded if the list were created by professionals in the field
(e.g., a poll among more than a hundred contemporary leading
physicists conducted by Physics World magazine, Dunani &
Rodgers, 1999), whereas a code of 1 was awarded for lists made
by amateurs or based on unclear methodology. One such example
of an unclear methodology is the “Greatest Mathematicians of All
Time” list published on server thetoptens.com.1

Next, an index of eminence was calculated for each potential
subject within each domain as a sum of trustworthiness scores of
all lists in the domain containing the subject’s name. To prevent
overlap with Ludwig’s (1992) sample, subjects must have either
died after 1950 or been born before 1980, if they were still alive.
The 45 most eminent professionals in each domain were selected
as potential subjects in the sample. Individuals who tied for the
45th most eminent position were included in the sample. This pro-
cedure led to a total of 766 potential subjects. Professionals in
multiple domains were sorted in the category in which they ranked
in a higher position.
After the list of potential subjects was obtained, the next step

was to determine whether or not a viable and relevant biography
was written about that person. When available, e-versions (Kindle)
of biographies were purchased. If no e-version was available, hard
copies were purchased, had their bindings removed, and were digi-
tally scanned via optical character recognition (OCR) conversion.
To be selected for study, biographies had to be written for an adult
audience and include information on the creator’s personal life
and were not solely intellectual or work biographies. Moreover,
autobiographies, biographies written by close relatives, biographi-
cal chapters, letters, and memoires were excluded. Of the 766
potential subjects, 391 did not have appropriate biographies writ-
ten about them, leaving a potential sample of 375. If there were
more than one biography written about a person, we chose the one
that had the most life-history information. Owing to time and
resource constraints, 199 of the 375 biographies were purchased
for coding (18% female).2 Analyses revealed that the 199 biogra-
phies were representative of the larger 375 sample on proportion
of artists, scientists, and athletes as well as proportion of deceased
subjects. The proportion of women however increased from 13%
to 17.5% in the final sample of 199, v2(1) = 9.45, p = .002. Never-
theless, the proportion of women in each subgroup did not change,
v2(2) = 1.34, p = .510. In the final sample, there was a higher per-
centage of women in the arts than sciences or sports, 28%, 7% and
4% respectively, v2(2) = 16.11, p , .01. The overall sample was
83% White-Caucasian, 13% Black/African American, 2% Latinx,

and 1.5% Asian-Pacific Islander. In 2016 when the data were col-
lected, the U.S. demographics were 60% White-Caucasian, 18.5%
Latinx, 13% Black/African American, 6% Asian American (U.S.
Census Bureau, 2019).

Each subject was placed into either a scientific, artistic, or ath-
letic domain. Scientific domains were defined as technology/
invention, mathematics, physics, chemistry, biology/medicine,
psychology, and social sciences (anthropology and sociology).
Earth scientists (e.g., geologists, oceanographers, climatologists)
were excluded because of a lack of biographies. Artistic domains
were defined as visual arts, fiction writing, poetry, acting, musical
performance, and musical composition. Using these career group-
ings, the current sample consisted of 104 artists, 68 scientists, and
27 athletes (see Table 1). The entire sample had a median year of
birth of 1919 with an average birth year of 1921. The range of
birth years was 1873 to 1979. Of the 199 subjects in the final list,
46 (23%) were alive at end of data collection in 2016. For the 153
participants who had died, the average age of death was 72 (me-
dian = 75; mode = 81; range 25 to 98). For the 46 participants who
were alive, the average age was 71.70 (median = 73; mode = 75;
range 41 to 89). Ninety-three percent of the sample was married at
least once (mean age of first marriage = 27.13). There was no ca-
reer domain difference in mean age of first marriage. Artists
(59%) were more likely to have divorced than scientists (38%).

Procedure

Biography Selection and Preparation

After the subject-pool was narrowed down to subjects who had
usable biographies written about them, we purchased each biogra-
phy either in digital or bound format. If the book was in bound for-
mat, we then detached its binding and scanned the entire body of
the text (excluding front- and rear-matter) into readable ORC/digital
format. Next, we cleaned the digital books by removing all images,
headings, footers, foot notes and most tables and equations.

Pathology Selection

Before ratings of pathologies could be made, the research team
discussed and decided which specific diagnostic illnesses would
be coded. For this process we mostly followed Ludwig by obtain-
ing the original variable list. Ludwig’s team coded mental health
status of immediate family members as well as the creative person.
We coded only the creative person. Moreover, we also added a
few illness categories that we believed Ludwig missed and might
be relevant as exploratory analyses, such as Asperger’s syndrome
(high functioning autism) and synesthesia. The final list consisted
of 19 diagnostic illnesses listed in the Diagnostic and Statistical
Manual of Mental Disorders, 5th edition (DSM–5; American Psy-
chiatric Association, 2013; see Table 2).

Paragraph Selection

After each of the 199 biographies was scanned and cleaned, a
linguistic analysis program was created by the second author to

1
For complete Best of Lists, Awards, and Rankings, see https://doi.org/

.17605/OSF.IO/TFYDK.

2
Raw data are posted on Open Science Framework at https://doi.org/10

.17605/OSF.IO/TFYDK.

PSYCHOPATHOLOGY AND CREATIVITY 7

https://doi.org/10.17605/OSF.IO/TFYDK

automatically locate and highlight any of the 175 relevant key-
words related to 19 mental illness categories (see Appendix A in
the online supplemental materials). The initial list was based on
words used by Ludwig (1995) but was expanded through a discus-
sion between the investigators after a review of the DSM–5 and
Stein and colleagues (2010). From either the biographical or pub-
lic websites we obtained the following demographic variables:
profession/career, date of birth, date of death (if deceased), year of
mother’s death, year of father’s death, birth order, race/ethnicity,
gender, year of marriage (first), year of marriage (second), country
of birth.
Two trained graduate students further narrowed the biographical

texts to include only paragraphs relevant for assessment of mental
health of subject in question. For example, “depression” may have
been tagged by the program, but if it referred to the economic period
of the 1930s then that paragraph would be excluded from further rat-
ing. Similarly, if key terms were tagged but referred to someone
other than the target creator, those paragraphs were also de-selected
for further rating.

Pathology Ratings

In the fifth and final step, seven raters were selected and
trained to identify possible psychopathologies in each biography
excerpt. Raters could only begin once they met the .80 interrater
reliability with training data from Ludwig’s (1992, 1995). Poten-
tial raters were given paragraphs selected from a biography then
asked to code the given reading material for the psychopatholo-
gies described above. Ratings were compared against the origi-
nal coding data from Ludwig’s study. Interrater reliability was
measured using Gwet’s agreement coefficient AC1 (Gwet, 2008),
which was preferred over Cohen’s kappa coefficient as it pro-
vides unbiased estimate even in case of strongly uneven occur-
rences of categories (Gwet, 2002; Wongpakaran, Wongpakaran,
Wedding, & Gwet, 2013). To keep raters blind and free of any
previous bias, the name of the subject in question was replaced
with the word “Creator” in all biography excerpts. Subjects were
coded for lifetime prevalence of any of 19 psychopathologies.

Psychopathologies were rated on a 3-point scale of not present
(0), probable (1), and present (2) if they occurred at any point in
the creator’s lifetime. Rating present was used in cases where
DSM–5 criteria were clearly met or where the subject was diag-
nosed professionally during their lifetime. If not enough infor-
mation was given in the biography to provide a clear diagnosis
from DSM–5 criteria and yet there was some evidence that a dis-
order was suspected, then raters gave that a probable rating. In
short, a probable rating was provided whenever there was some
but not overwhelming evidence of a disorder. Present was pro-
vided when a professional diagnosis was made during the per-
son’s lifetime or when the biographical evidence was very clear.
During training on the Ludwig sample, if raters initially fell
below the .80 reliability criterion, research meetings were held
with other raters and the lead researcher (G.J.F.) to discuss dis-
crepancies and to reach consensus. For final nontraining ratings,
two independent and randomly assigned raters coded each biog-
raphy. Any disagreement was adjudicated by a third rater (G.J.
F.) to establish the final rating.

Results

Previous and Current Lifetime Rates of Disorders

For sake of comparison, in Table 3 we present the population
estimates of lifetime rates of psychological disorders published in
the literature. Two studies have reported large-scale national popu-
lation estimates of lifetime rates of any disorder. Kessler, Berglund,
and colleagues (2005) reported a rate of 46.4% and Lev-Ran and
colleagues (2013) reported a rate of 33.7% for any mood, anxiety,
personality or psychotic disorder. In this context, the lifetime rate in
our sample for creative artists and scientists was 49.7% (85 of 171)
and for athletes was 48.1% (13 of 27), v2(2) = 2.39, ns. For artists
only, the percentage of “present” cases was 61/103 (59.2%), and
for scientists it was 24/68 (35.3%). Over the course of their lifetime,
artists were more likely to have at least one form of psychopathol-
ogy than scientists, v22(1) = 9.38, p = .002. Artists were not more
likely to have a lifetime bout of psychopathology compared with
athletes (59% versus 48%, respectively) , v2(1) = 1.07, ns.

When the less exclusive “probable” cases were also included, the
frequency of psychopathology increased to 126 of 171 (73.7%) in
the creative groups compared with 16 of 27 (59.3%) athletes, v2(1) =
2.39, ns. The observed frequencies on “probable” lifetime psychopa-
thology for artists (83%), scientists (59%), and athletes (59%) were

Table 1
Specific Domains and group Sizes

Domain % White N of Men N of Women Total N

Artists 75 29 104
Visual arts 100.0 8 1 9
Fiction writing 85.7 26 9 35
Poetry writing 90.9 9 4 11
Acting 95.0 11 9 20
Music performance 46.2 18 8 26
Music composition 100.0 3 0 3

STEM 63 5 68
Technology/Invention 100.0 9 1 10
Mathematics 100.0 7 0 7
Physics 100.0 19 0 19
Biology/Medicine 100.0 6 1 7
Chemistry 100.0 4 1 5
Psychology 100.0 11 0 11
Social Sciences 100.0 7 2 9

Comparison group 26 1 27
Athletes 55.6 26 1 27

Total 164 35 199

Table 2
List of Rated Psychopathologies

Rated psychopathologies

Adjustment disorder Obsessive-compulsive disorder
Alcoholism Paraphilia
Anxiety disorder Personality disorder (of any kind)
Autism spectrum disorder Posttraumatic stress disorder
Conduct disorder Schizophrenia/Psychotic disorders
Depression/Depressive disorder Sleep disorders
Drug use/dependency Somatic disorder
Eating disorder Suicide/Suicide attempt
Gambling disorder Synesthesia
Kleptomania

8 FEIST, DOSTAL, AND KWAN

https://doi.org/10.1037/aca0000440.supp

different from chance, v2(2) = 14.68, p = .001. This effects stems
from artists having a higher lifetime rate than both scientists, v2(1) =
12.86, p , .001, and athletes, v2(1) = 7.45, p = .006.
Even though the overall frequency of any psychopathology was

extraordinarily high, relative frequencies of individual disorders
rarely exceeded a few percent. Occurrences higher than 5% were
only observed in depression/depressive disorders (26%), alcoholism
(16%), drug use/dependency (12%), and anxiety disorder (11%).
These compare with the rates of 28.8% for anxiety, 20.8% for
mood, and 14.6% for substance abuse in the general population
(Kessler et al., 2007; Kessler, Berglund, et al., 2005).
For a more sophisticated analyses of these trends, for each listed

psychopathology, a null hypothesis about the uniform distribution of
its occurrences in compared groups was tested. In each comparison, a
chi-squared test with a Monte Carlo simulated p value was used
because it has no assumptions about minimal expected frequencies
(estimates were done with 106 replicates; Hope, 1968). We have
included both present and probable occurrences of psychopathology
without differentiating between them into analysis. Observed frequen-
cies for each category and resulting p values are detailed in Table 4.
The results suggest that there are unequal frequencies between the do-
main of eminent individual and the occurrence of Alcoholism, Drug
use/dependency, Gambling disorder, Suicide/suicide attempt and also
Depression/depressive disorders, Anxiety disorder, Sleep disorder and
Autism spectrum disorder. Note that there were 21 significance tests
performed, which results in substantial increase in familywise first
type error rate. To keep familywise error rate under 5% Bonferroni
corrected p-values can be used. This correction suggests that the fre-
quencies in Autism spectrum disorder (pBonf. = .271), Suicide/suicide
attempt (pBonf. = . 282), and Gambling disorder (pBonf. = .169) are not
different from expected, so we should not consider those results as

conclusive. Finally, 52% of the artists and 24% scientists experienced
the loss of a parent in childhood, v2(1) = 15.04, p , .001.

Planned Analyses

Recall, the three main predictions were that compared with athletes
and scientists, artists as a whole would have elevated lifetime rates of
psychopathology, especially in the mood disorders and substance
abuse categories. We also predicted that scientists would have ele-
vated rates of high functioning autism compared with athlete controls.

To test these hypotheses, a Bayesian logistic regression3 was
used to model the relationship between mental disorder presence
and creativity domain. The dependent variable was an occurrence
of particular mental disorder. We have used two separate models
with differently defined dependent variables for each disorder. In
the first one, the present cases only were coded as 1. In the second
one, both present and probable cases were coded as 1. The values
of the dependent variable were predicted with categorical factor

Table 3
Published Population Estimates of Lifetime Rates of Psychological Disorders

Author(s), Date Disorder Percent of population

Baca-Garcia et al. (2010) Suicide attempted: Male 1.75%
Suicide attempted: Female 2.95%
Suicide ideation: Male 6.00%
Suicide ideation: Female 7.60%

Hudson et al. (2007) Eating disorder: Male 2.80%
Eating disorder: Female 5.90%

Lev-Ran et al. (2013) Any mood, anxiety, psychotic, personality disorder 33.7%
Kessler, Berglaund, et al. (2005) Anxiety disorder 28.80%

Mood disorder 20.80%
Impulse-control disorder 24.80%
Substance abuse 14.60%
Any disorder 46.40%

Merikangas et al. (2007) Bipolar I 1.00%
Bipolar II 1.10%
Subthreshold bipolar 2.40%

Nock and Kessler (2006) Suicide (ideation/attempt) 2.70%
Perälä et al. (2007) Psychotic disorder (any kind) 2.29%
Robins et al. (1984) Any disorder 28.8% to 38.0%

Anxiety disorder 10.4% to 25.1%
Substance abuse disorder 15.0% to 18.1%
Affect-mood disorder 6.1% to 9.5%
Psychotic disorder 1.1% to 2.0%
Personality disorder 2.1% to 3.3%
Eating disorder 0.0% to 0.1%

Note. Robins et al. (1984), consisted of three samples from New Haven, CT, Baltimore, MD, and St. Louis, MO, and interviews were conducted between
1980 and 1982. Bolded text highlights the overall “any disorder” category.

3
Computations were performed in statistical program R with the

rstanarm package (Stan Development Team, 2016). The student t-
distribution with 7 degrees of freedom and a scale parameter 2.5 was
chosen as a prior for all regression weights. The student distribution was
preferred from the normal distribution because its heavy-tailedness enables
substantial differences from expected value. The location was set to zero in
all parameters with the exception of the intercepts in which case logits of
population prevalences according to Kessler, Berglund, and colleagues
(2005) and Nock and Kessler (2006) were used. The parameter estimations
were computed with NUTS sampling method (Hoffman & Gelman, 2014)
on 8 MCMC chains each performing 6,000 iterations (2,000 in burning
phase). The e-values in favor of null hypothesis stating that odds ratio
equals one were computed using fbst package (Kelter, 2020). Above
mentioned priors were used as the reference functions. For details see
Pereira and Stern (2020).

PSYCHOPATHOLOGY AND CREATIVITY 9

domain with seven levels listed in Table 1. Dummy variable cod-
ing was used with the group of athletes used as a reference group.
Besides the categorical independent variable, the gender and two
quantitative covariates were included in the model to hold them
constant: year of birth of each subject and the length of their life.
If any biography was published during the life of the subject, the
age of the subject in the year of publication was used instead of
age of their death, as the biography author did not have informa-
tion about the subjects’ lives from that point. Both quantitative
covariates were centered and scaled to the z-score format.
The hypotheses about model parameters were tested with Full

Bayesian Significance test (FBST; Pereira & Stern, 1999). This pro-
cedure uses e value (evidence value) as a measure of statistical signif-
icance (Pereira & Stern, 2020). In its simplest form, e-value in favor
of the null hypothesis stating parameter equals zero is close to the
idea proposed by Thulin (2014): what is the maximal value of a
resulting in 1 � a posterior credible interval not containing zero.
Keeping in mind this analogy between p values/confidence intervals
and e values/credible intervals, we also used value .05 for rejection
of the null hypothesis.
Figure 3 shows the prevalence interval estimates (95% highest

density regions) of the selected present or probable psychopatholo-
gies. Each creative domain was also compared with the control
group of athletes. The odds ratios of psychopathology occurrence in
each group compared with the control are represented in the figures
as text labels. The asterisk notation indicates e values lower than
.05, .01, and .001, respectively. Dot indicates nonsignificant result
with e value lower than .1. Figure 3 reports that artists were more
than 2.71 times more likely to experience any mental illness over
the course of their life compared with eminent athletes, whereas

scientists had approximately the same odds as athletes. Artists were
most at risk for depression/bipolar (ORs = 6.28 and 9.19 for present
and present/probable, respectively) compared with controls. The
only disorder for which scientists were at elevated risk was the more
inclusive present/probable rate for depression/bipolar (OR = 4.53).
For all other disorders, scientists were either equally likely or less
likely to experience them compared with athlete controls. For exam-
ple, creative scientists were significantly less likely (ORs = .21 – .23)
to be afflicted with substance related and addictive disorders than
athletes.

In Figure 4 we present results broken down by subgroups. As is
evident in Figure 4, the elevated risk of mental illness in the artist
group primarily is a result of writers and visual artists. Looking at
only “present” rates of specific disorders, writers and visual artists
were 4.52 times more likely than athletes to suffer from any pres-
ent mental illness over the course of their lifetime. In fact, no other
subgroup was more or less at risk compared with athlete controls.
In addition, it was primarily the writers and visual artists who
were at increased risk of depression/bipolar (OR = 8.11), anxiety
(OR = 7.53), and suicide/attempt (OR = 15.79).

When we expand the analysis to include “probable” or “pres-
ent” rates, results were much the same except for depression and
substance abuse. Musicians, actors, writers, visual artists, poets,
mathematicians, and technologists were all more likely to be sus-
pected of having depression and/or bipolar compared with fa-
mous athletes (ORs = 3.82 – 10.86). Visual artists, writers and
poets were more likely to suffer anxiety disorder (OR = 3.58)
and be suicidal or attempt suicide (OR = 6.18) than athletes.
Moreover, physicists and chemists were less likely than athletes
to suffer substance related and addictive disorders (OR = .04).

Table 4
Sample Frequencies of Lifetime Rates of Psychopathology Across Professional Domains

Disorder
Athletes
(n = 26)

Biologists
(n = 7)

Math. and
Technol.
(n = 17)

Musicians and
Actors
(n = 50)

Physicists
and Chemists

(n = 24)

Psych. &
Social
Science
(n = 21)

Visual Artists,
Writers, and

Poets
(n = 54) p

Synesthesia 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.2) 0.0 (0.0) 0.0 (0.0) 18.5 (18.5) 1.00
Adjustment disorder 0.0 (0.0) 0.0 (0.0) 0.0 (17.6) 0.0 (0.4) 0.0 (4.2) 0.0 (0.0) 0.0 (7.4) .152
Alcoholism 7.7 (11.5) 0.0 (0.0) 11.8 (11.8) 20.0 (14) 0.0 (4.2) 9.5 (9.5) 27.8 (38.9) .002
Drug use/dependency 7.7 (7.7) 0.0 (0.0) 5.9 (11.8) 26.0 (16) 0.0 (0.0) 4.8 (4.8) 11.1 (11.1) .002
Depressive disorders 7.7 (7.7) 14.3 (28.6) 23.5 (41.2) 20.0 (19) 20.8 (41.7) 23.8 (42.9) 48.1 (64.8) ,.001
Bipolar disorder 7.7 (7.7) 0.0 (0.0) 5.9 (11.8) 0.0 (0.4) 4.2 (4.2) 0.0 (0.0) 18.5 (5.6) .770
Anxiety disorder 7.7 (11.5) 0.0 (0.0) 11.8 (17.6) 0.8 (24.0) 8.3 (20.8) 0.0 (14.3) 13 (38.9) .046*
Obsessive compulsive 0.0 (0.0) 0.0 (0.0) 0.0 (11.8) 0.2 (0.6) 0.0 (4.2) 0.0 (0.0) 0.0 (5.6) .526
Schizophrenia 7.7 (7.7) 0.0 (0.0) 5.9 (5.9) 0.0 (0.2) 0.0 (0.0) 0.0 (4.8) 18.5 (18.5) .909
Somatic disorder 7.7 (11.5) 0.0 (0.0) 0.0 (5.9) 0.0 (0.4) 0.0 (0.0) 0.0 (4.8) 18.5 (3.7) .587
Autism spectrum disorder 0.0 (0.0) 0.0 (0.0) 0.0 (5.9) 0.0 (0.0) 0.0 (12.5) 0.0 (0.0) 0.0 (1.8) .038*
Suicide/suicide attempt 0.0 (0.0) 0.0 (0.0) 0.0 (5.9) 0.0 (0.4) 4.2 (12.5) 0.0 (0.0) 7.4 (11.1) .227
Sleep disorder 7.7 (15.4) 0.0 (14.3) 0.0 (5.9) 0.4 (1.2) 0.0 (0.0) 0.0 (0.0) 3.7 (16.7) .185
Eating disorder 0.0 (0.0) 0.0 (0.0) 0.0 (5.9) 0.0 (0.6) 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) .242
Personality disorder 7.7 (7.7) 0.0 (0.0) 5.9 (11.8) 0.6 (7) 8.3 (16.7) 0.0 (4.8) 0.0 (3.7) .261
Gambling disorder 15.4 (19.2) 0.0 (0.0) 0.0 (0.0) 0.2 (0.2) 0.0 (0.0) 0.0 (0.0) 3.7 (3.7) .008*
Conduct disorder 7.7 (7.7) 0.0 (0.0) 0.0 (0.0) 0.2 (0.4) 4.2 (16.7) 0.0 (0.0) 3.7 (3.7) .106
Kleptomania 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (18.5) 1.00
Posttraumatic stress 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.2) 0.0 (0.0) 0.0 (0.0) 18.5 (3.7) .852
Paraphilia 0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 0.0 (0.2) 0.0 (0.0) 0.0 (0.0) 3.7 (3.7) .852
Any psychopathology 46.1 (57.7) 14.3 (28.6) 52.9 (70.6) 52.0 (78.0) 33.3 (62.5) 33.3 (57.1) 66.7 (87.0) .003*

Note. Percentages outside brackets involve present cases of psychopathology occurrence. Percentages inside the brackets include both present and proba-
ble cases. Column p contains Monte Carlo simulated p values testing null hypothesis that relative frequencies of present or probable psychopathology
occurrences are uniformly distributed across compared domains.
*pBonf . .05.

10 FEIST, DOSTAL, AND KWAN

There were no differences between the creative groups and ath-
letes in sleep disorders.

Discussion

The intention of the current study was to examine the more com-
plicated and moderated relationships between creativity and psycho-
pathology by updating and replicating Ludwig’s (1992) biographical
analysis of world-class creative artists and scientists. Our expectation
was that artistic creative professions in general would possess higher
levels of psychopathology than creative scientists. We also predicted
that scientists would not differ from the base rates of psychopathol-
ogy found in the U.S. population, whereas artists would.
The current study controlled for researcher bias by removing

the biographical material of its subjects’ identities. The previous
study conducted by Ludwig (1992) was executed with the
researcher knowing the identity of each subject, and may have
been biased by previous working knowledge of each professional.
Certain professions that were given new classifications as the older
categorizations, as designated in Ludwig (1992, 1995), may have
been incorrectly assigned. For example, historians and philoso-
phers were considered scientists by Ludwig. Although history and

philosophy are scholarly subjects, they are not typically considered
sciences.

The current study also sought to streamline the process of finding
relevant information in books by digitizing each biography into a
searchable digital media. This would allow for the researchers to oper-
ate at an increased pace by eliminating irrelevant text very quickly.
Transforming each book into a digital format also made it possible to
censor the names of each creator to limit any previous knowledge that
could bias the rating group.

The results of the current study generally corroborated the findings
reported in Ludwig’s and other studies, lending further support to
previously established hypotheses. Despite using an entirely new set
of subjects, not included in Ludwig’s (1992) sample, artists still pos-
sessed higher rates of psychopathological traits than scientists, ath-
letes, and the U.S. population in general. Scientists were consistently
rated lower on symptoms of psychopathology than artists, despite
equal eminence. These results held true in both inclusive and exclu-
sive requirements for classification into the mentally ill group. How-
ever, the difference between artists and athletes was not significant in
the more exclusive interpretation of the data. Rates of drug abuse and
anxiety also differed between artists, scientists, and athletes depend-
ing on whether inclusive or exclusive criterion were used to define
what constituted psychopathology. In both cases, fewer subjects

Figure 3
Prevalence of Selected Mental Disorders (Present and/or Probable) in Creative Artists and Scientists Compared
With Eminent Athletes

PSYCHOPATHOLOGY AND CREATIVITY 11

Figure 4
Prevalence of Selected Mental Disorders (Present and/or Probable) in Creative Domains and Athletes

Note. Text labels indicate odds ratios for psychopathology occurrence in creative domains compared to the athletes. The aster-
isks indicate whether the 1-a centered highest density region for posterior density of regression weight contains zero (no effect)
when a equals 5% (*), 1% (**), or 0.1% (***).

12 FEIST, DOSTAL, AND KWAN

qualified for inclusion into the mentally ill group when exclusive cri-
teria were used. However, the differences between groups grew
larger in the case of anxiety and smaller in the case of drug use, thus
moderating the results.
Artists also showed greater rates of alcoholism, drug abuse,

depression, and OCD than those found in the U.S. population. Again,
statistical significance changed for a few of psychopathologies
depending on the strictness of criterion for inclusion. Anxiety among
artists was considered lower in the stricter assessment but still gained
significance due to the high rate of anxiety reported in the U.S. popu-
lation. Rates of OCD also fell for artists and scientists under stricter
criterion and detectable differences were no longer found.
Because we had a small comparison sample of famous but not

professionally creative athletes, we could also address the question
of whether fame itself—isolated from creativity—is a contributing
factor to psychopathology. Because athletes were generally less
likely than artists and even scientists to develop psychopathology
at some point during their lives, we can tentatively conclude that
fame per SE is not the driving force behind psychopathology.
There was one exception to this, namely anxiety. Athletes exhib-
ited higher lifetime rates of anxiety disorders than the general pop-
ulation. An interesting question therefore becomes “Do higher
rates of anxiety precede or follow athletic fame?” That is, is anxi-
ety a cause or effect of athletic eminence?
Assuming the relationship between some forms of creativity

and some forms of pathology are robust and real, then the ques-
tion becomes why might these two traits covary? Recent litera-
ture from biological and evolutionary approaches have
suggested biological bases and even potentially adaptive func-
tions of the relationship between creative thought and behavior
and psychopathology. For instance, research has reported a pol-
ymorophism of a particular gene involved in psychosis that is
associated with high levels of creativity and high IQ (Kéri,
2009). The gene in question is neuregulin 1, which is a candi-
date gene for psychosis and affects neuroplasticity, glial func-
tion, and neuronal development in general. One form of the
gene, the T/T genotype, was related to both high creativity and
risk for psychosis. Based on this and other evidence, Kozbalt
and colleagues (2017) argued that one possible reason why a
maladaptive trait may still exist in humans is its shared genetic
linkage with creative behavior (cf. Akiskal & Akiskal, 2007;
Greenwood, 2020; Nettle, 2001, 2006; Power et al., 2015). Sim-
ilarly, other empirical and theoretical evidence supports the idea
that the milder levels of mental illness, for example cyclothy-
mia, confer advantages such as increase fluency of ideas that
make creative thought more likely (Carson, 2011; 2014; Green-
wood, 2020). Carson (2011, 2014), in fact, proposed that the
“shared vulnerability” traits of openness, impulsivity, schizo-
typy, cognitive disinhibition, hypomania, and cyclothymia are
the traits that connect psychopathology (risk factors) and crea-
tive (protective factors) behavior. In sum, there are various neu-
rological, evolutionary, and adaptive factors that may undergird
the associations between some forms and degrees of creativity
and some forms and degrees of psychopathology.

Caveats and Limitations

A number of confounding variables limit the results of this
study. One such limitation is sample bias. In the case of the current

study, writers and publishers may be more inclined to pursue biog-
raphies for particularly interesting people to tell more compelling
stories. Because someone with a history of psychopathology may
serve as a more desirable subject for a biography than someone
who is not, healthier professionals may have fewer books written
about them. Indeed, the study also contained a much smaller num-
ber of scientists than artists, which may be due to writers and pub-
lishers favoring more artists rather than scientists since the latter
may be perceived as less interesting or hold less recognition in the
general public.

The level of fame could not be held constant through all three
groups. Although some scientists such as Stephen Hawking and
Richard Feynman are particularly well known, not all eminent sci-
entists are easily recognizable to the public (e.g., Alfred Tarski,
Grigori Perelman, George Beadle). Most of the actors (e.g., Mar-
lon Brando, Robert DeNiro, Sophia Loren) and musicians (e.g.,
Ella Fitzgerald, Prince, Diana Ross, Johnny Cash) were well
known to the general public. Athletes, although more recognizable
than scientists, tend to dwindle in fame after retirement. Because
the careers of most athletes are particularly short, their highest
point of fame tends to come earlier in their lives rather than later.
This is incongruent with scientists as fame for their achievements
tend to come later, after their work has been recognized. Both ath-
letes and scientists may also possess lower levels of fame than per-
formance artists such as musicians and actors.

Another limitation of the current study is the gender imbalance
in the creative sample. The biographies of men in the sample out-
numbered women 164 to 35 (18% female). There are historically
fewer biographies written about women than men, especially the
sciences. Moreover, women have been less likely to reach the
highest levels of their professions, whether they have biographies
written about them or not—the famous “glass ceiling.” For exam-
ple, women have historically been seriously underrepresented in
mathematics and sciences. Only 8.8% 15.8% of tenure-track posi-
tions among top universities are held by women in math-centric
domains (Ceci & Williams, 2011), and only 20% of physics PhDs
were awarded to women as recently as 2017 (Porter & Ivie, 2019).
During the time that many eminent people in this sample were
most active professionally—the 1950s to 1980s—the percentage
of women earning PhDs in physics was between 3% to 7% (Porter
& Ivie, 2019). Yet not all professions are so imbalanced. For
example, our sample had five women of 68 (7%) in the STEM dis-
ciplines but 29 of 104 (28%) women in the arts. In acting, the cur-
rent sample had a ratio of nine of 20 (45%), and in musical
performance it was eight of 26 (31%).

This finding begs the question of why are women underrepre-
sented in certain fields more than others and in particular at the top
of their fields? There is an extensive scientific literature on this
question that goes well-beyond the scope of this article (Cheung &
Halpern, 2010). Suffice it to say that social and cultural biases
about marriage, child rearing, and performance play a very large
role in “glass ceiling” effect (Cheung & Halpern, 2010). For
example, in a nationwide study, biology, chemistry, and physics
professors were found to consider men as both more hirable and
competent (Moss-Racusin et al., 2012).

Similarly, the sample was skewed racially, with being of
White-European ancestry (83% of our sample, compared with
60% in the U.S. population). Indeed, 100% of the Science-Tech-
nology group was White. Part of this bias comes from the bias in

PSYCHOPATHOLOGY AND CREATIVITY 13

published biographies, especially during the time frame of our
study. Moreover, basing research on biographies will inherently
require older samples given the delay between creative accom-
plishments and publication of a biography. Future research will
need to continue to determine whether these trends hold with
more ethnically diverse samples.
Additionally, determining how to interpret historical and bio-

graphical texts is a challenge for psychological study (Citlak,
2016; Czechowski et al., 2016). Biographies still require interpre-
tation as historians of certain types of professions may differ from
others in what report. Some professions may encourage exagger-
ated stories, particularly of drug use, to sell their fame (Lucijani�c
et al., 2010). Musicians such as rappers and rock stars may benefit
from rumors of psychopathology as increased notoriety would
increase exposure and thus raise the likelihood of album sales. Sci-
entists do not typically benefit from fame in the same way artists
do, as they typically work to discover new knowledge rather than
sell products or develop a fan-base, thus there is less incentive to
exaggerate claims of illness or drug use.
Furthermore, we must acknowledge the fact that different disor-

ders are easier to rate from biographies than others and are more
likely to show up in biographies than others. The former consists of
more behaviorally expressed disorders such as drug or alcohol
addiction, violence, suicide, and depression, whereas the latter con-
sists of more internalized or private disorders, such as PTSD, sleep
disorders or even more moderate degrees of anxiety disorders.
We also need to make clear that our raters were not licensed

clinical psychologists but rather trained undergraduate research
assistants. To be sure, they had to go through a reliability training
process that involved learning the DSM–5 criteria for the 19 disor-
ders and they could not begin rating until they obtained the .80
interrater reliability threshold. Finally, the raters were blind to the
subject of the biography, and we had two independent raters code
each biography. Nevertheless, these are not assessments by li-
censed clinical psychologists.

Future Directions

We make little claim that this investigation settles the “debate”
over the “mad-genius.” At best, it confirms one aspect of it,
namely the higher rate of pathology and the different pathologies
in the creative arts than other creative domains. Many questions
remain. For example, as we mentioned above, the biggest question
left unresolved is the gender question. Are these patterns that we
found in a heavily male-dominated sample the same in famous
creative women? Our dataset does not allow this question to be
satisfactorily answered. Moreover, because of the restricted sam-
ple size, certain analyses were not possible among smaller groups
and specific professions. Additionally, no comparisons could be
made for fiction writers against nonfiction writers, limiting the
conclusions that could be made. Thus, more specific examinations
of individual professions can be made as the dataset grows larger.
Additional demographic variables that may influence professional
vocation and creative output will also be collected. These variables
include birth order, religious affiliation, ethnicity, and marital sta-
tus of parents. Owing to time constraints, the collection of these
data lay beyond the scope of the current study.
In conclusion, the results of this study provide support and repli-

cation for the findings of previous biographical investigations of

highly creative people. The use of digital resources allowed for the
researchers to limit bias through the use of censors to hide the
identity of each creator. The classification and grouping of each
profession were also reworked for further accuracy. As is true of
all research, however, for each question answered, others arise and
await further attention from future investigators. The topic of psy-
chopathology and world-class creative achievement is a rich and
complex topic and will provide material for researchers for years
to come.

References

Abraham, A. (2017). Neurocognitive mechanisms underlying creative
thinking: Indications from studies of mental illness. In J. C. Kaufman
(Ed.), Creativity and mental illness (pp. 79–101). Cambridge University
Press.

Acar, S., Chen, X., & Cayirdag, N. (2018). Schizophrenia and creativity: A
meta-analytic review. Schizophrenia Research, 195, 23–31. https://doi
.org/10.1016/j.schres.2017.08.036

Acar, S., & Runco, M. A. (2012). Psychoticism and creativity: A meta-ana-
lytic review. Psychology of Aesthetics, Creativity, and the Arts, 6(4),
341–350. https://doi.org/10.1037/a0027497

Acar, S., & Sen, S. (2013). A multilevel meta-analysis of the relationship
between creativity and schizotypy. Psychology of Aesthetics, Creativity,
and the Arts, 7(3), 214–228. https://doi.org/10.1037/a0031975

Akinola, M., & Mendes, W. B. (2008). The dark side of creativity: Biolog-
ical vulnerability and negative emotions lead to greater artistic creativ-
ity. Personality and Social Psychology Bulletin, 34(12), 1677–1686.
https://doi.org/10.1177/0146167208323933

Akiskal, H. S., & Akiskal, K. K. (2007). In search of Aristotle: Temperament,
human nature, melancholia, creativity and eminence. Journal of Affective
Disorders, 100(1–3), 1–6. https://doi.org/10.1016/j.jad.2007.04.013

Amabile, T. M. (1996). Creativity in context: Update to the social psychol-
ogy of creativity. Westview Press.

American Psychiatric Association. (2013). Diagnostic and statistical
manual of mental disorders (5th ed.).

Andreasen, N. C. (1987). Creativity and mental illness: Prevalence rates in
writers and their first-degree relatives. The American Journal of Psychi-
atry, 144(10), 1288–1292. https://doi.org/10.1176/ajp.144.10.1288

Andreasen, N. C. (1978). Creativity and psychiatric illness. Psychiatric
Annals, 8(3), 23–45. https://doi.org/10.3928/0048-5713-19780301-05

Andreasen, N. C., & Glick, I. D. (1988). Bipolar affective disorder and cre-
ativity: Implications and clinical management. Comprehensive Psychia-
try, 29(3), 207–217. https://doi.org/10.1016/0010-440X(88)90044-2

Baas, M., Nijstad, B. A., Boot, N. C., & De Dreu, C. K. W. (2016). Mad
genius revisited: Vulnerability to psychopathology, biobehavioral
approach-avoidance, and creativity. Psychological Bulletin, 142(6),
668–692. https://doi.org/10.1037/bul0000049

Baca-Garcia, E., Perez-Rodriguez, M. M., Keyes, K. M., Oquendo, M. A.,
Hasin, D. S., Grant, B. F., & Blanco, C. (2010). Suicidal ideation and
suicide attempts in the United States: 1991-1992 and 2001-2002. Molec-
ular Psychiatry, 15(3), 250–259. https://doi.org/10.1038/mp.2008.98

Baron-Cohen, S., Wheelwright, S., Burtenshaw, A., & Hobson, E. (2007).
Mathematical talent is linked to autism. Human Nature, 18(2), 125–131.
https://doi.org/10.1007/s12110-007-9014-0

Barron, F. (1963). Creativity and psychological health. D. Van Nostrand.
Batey, M., & Furnham, A. (2008). The relationship between measures of

creativity and schizotypy. Personality and Individual Differences, 45(8),
816–821. https://doi.org/10.1016/j.paid.2008.08.014

Beaussart, M. L., White, A. E., Pullman, A., & Kaufman, J. C. (2017).
Reviewing recent empirical findings on creativity and mental illness. In
J. C. Kaufman (Ed.), Creativity and mental illness (pp. 42–59). Cam-
bridge University Press.

14 FEIST, DOSTAL, AND KWAN

https://doi.org/10.1016/j.schres.2017.08.036

https://doi.org/10.1037/a0027497

https://doi.org/10.1037/a0031975

https://doi.org/10.1177/0146167208323933

https://doi.org/10.1016/j.jad.2007.04.013

https://doi.org/10.1176/ajp.144.10.1288

https://doi.org/10.3928/0048-5713-19780301-05

https://doi.org/10.1016/0010-440X(88)90044-2

https://doi.org/10.1037/bul0000049

https://doi.org/10.1038/mp.2008.98

https://doi.org/10.1007/s12110-007-9014-0

https://doi.org/10.1016/j.paid.2008.08.014

Becker, G. (2017). A socio-historical overview of the creativity-pathology
connection: From antiquity to contemporary times. In J. C. Kaufman (Ed.),
Creativity and mental illness (pp. 3–24). Cambridge University Press.

Billington, J., Baron-Cohen, S., & Wheelwright, S. (2007). Cognitive style
predicts entry into physical sciences and humanities: Questionnaire and
performance tests of empathy and systemizing. Learning and Individual
Differences, 17(3), 260–268. https://www.doi.org/10.1016/j.lindif.2007.02
.004

Brownstein, M., & Solyom, L. (1986). The dilemma of Howard Hughes:
Paradoxical behavior in compulsive disorders. Canadian Journal of Psy-
chiatry, 31(3), 238–240. https://doi.org/10.1177/070674378603100311

Burch, G. S. J., Pavelis, C., Hemsley, D. R., & Corr, P. J. (2006). Schizo-
typy and creativity in visual artists. British Journal of Psychology, 97(Pt
2), 177–190. https://doi.org/10.1348/000712605X60030

Carson, S. (2014). Leveraging the “mad genius” debate: Why we need a
neuroscience of creativity and psychopathology. Frontiers in Human
Neuroscience, 8, 771. https://doi.org/10.3389/fnhum.2014.00771

Carson, S. H. (2011). Creativity and psychopathology: A shared vulner-
ability model. Canadian Journal of Psychiatry, 56(3), 144–153. https://
doi.org/10.1177/070674371105600304

Carson, S. H., Peterson, J. B., & Higgins, D. M. (2003). Decreased latent
inhibition is associated with increased creative achievement in high-
functioning individuals. Journal of Personality and Social Psychology,
85(3), 499–506. https://doi.org/10.1037/0022-3514.85.3.499

Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of
women’s underrepresentation in science. Proceedings of the National
Academy of Sciences of the United States of America, 108(8),
3157–3162. https://doi.org/10.1073/pnas.1014871108

Cheung, F. M., & Halpern, D. F. (2010). Women at the top: Powerful lead-
ers define success as work þ family in a culture of gender. American
Psychologist, 65(3), 182–193. https://doi.org/10.1037/a0017309

Citlak, A. (2016). The Lvov-Warsaw School: The forgotten tradition of
historical psychology. History of Psychology, 19(2), 105–124. https://
doi.org/10.1037/hop0000029

Claridge, G., McCreery, C., Mason, O., Bentall, R., Boyle, G., Slade, P., &
Popplewell, D. (1996). The factor structure of “schizotypal’ traits: A
large replication study. British Journal of Clinical Psychology, 35(1),
103–115. https://doi.org/10.1111/j.2044-8260.1996.tb01166.x

Cox, A. J., & Leon, J. L. (1999). Negative schizotypal traits in the relation
of creativity to psychopathology. Creativity Research Journal, 12(1),
25–36. https://doi.org/10.1207/s15326934crj1201_4

Czechowski, K., Miranda, D., & Sylvestre, J. (2016). Like a rolling stone:
A mixed- methods approach to linguistic analysis of Bob Dylan’s lyrics.
Psychology of Aesthetics, Creativity, and the Arts, 10(1), 99–113.
https://doi.org/10.1037/aca0000045

Damian, R. I., & Simonton, D. K. (2015). Psychopathology, adversity, and
creativity: Diversifying experiences in the development of eminent Afri-
can Americans. Journal of Personality and Social Psychology, 108(4),
623–636. https://doi.org/10.1037/pspi0000011

Dunani, M., & Rodgers, P. (1999). Physics: Past, present, future. Physics
World, 12(12), 7–14. https://doi.org/10.1088/2058-7058/12/12/2

Eysenck, H. (1993). Creativity and personality: Suggestions for a theory.
Psychological Inquiry, 4(3), 147–178. https://doi.org/10.1207/s15327
965pli0403_1

Eysenck, H. J. (1995). Genius: The natural history of creativity (Vol. 12).
Cambridge University Press. https://doi.org/10.1017/CBO9780511752247

Feist, G. J. (1998). A meta-analysis of personality in scientific and artistic
creativity. Personality and Social Psychology Review, 2(4), 290–309.
https://doi.org/10.1207/s15327957pspr0204_5

Feist, G. J. (2012). Affective states and affective traits in creativity: Evi-
dence for non-linear relationships. In M. A. Runco (Ed.), The creativity
research handbook (Vol. 3, pp. 61–102). Hampton Press.

Feist, G. J. (2017). The creative personality: Current understandings and
debates. In J. Plucker (Ed.), Creativity and innovation: Current under-
standings and debates (pp. 181–198). Prufrock Press.

Fisher, J. E. (2015). Challenges in determining whether creativity and men-
tal illness are associated. Frontiers in Psychology, 6, 163. https://doi
.org/10.3389/fpsyg.2015.00163

Flaherty, A. W. (2005). Frontotemporal and dopaminergic control of idea
generation and creative drive. The Journal of Comparative Neurology,
493(1), 147–153. https://doi.org/10.1002/cne.20768

Focquaert, F., Steven, M. S., Wolford, G. L., Colden, A., & Gazzaniga, M.
S. (2007). Empathizing and systemizing cognitive traits in the sciences
and humanities. Personality and Individual Differences, 43(3), 619–625.

Fodor, E. M., & Laird, B. A. (2004). Therapeutic intervention, bipolar in-
clination, and literary creativity. Creativity Research Journal, 16(2–3),
149–161. https://doi.org/10.1080/10400419.2004.9651449

Forgeard, M. J. C., Mecklenburg, A. C., Lacasse, J. J., & Jayawickreme, E.
(2017). Bringing the whole universe to order: Creativity, healing, and
posttraumatic growth. In J. C. Kaufman (Ed.), Creativity and mental ill-
ness (pp. 321–342). Cambridge University Press.

Furnham, A., Batey, M., Anand, K., & Manfield, J. (2008). Personality,
hypomania, intelligence and creativity. Personality and Individual Dif-
ferences, 44(5), 1060–1069. https://doi.org/10.1016/j.paid.2007.10.035

Gardner, J. (1973). The arts and human development. Wiley.
Glazer, E. (2009). Rephrasing the madness and creativity debate: What is

the nature of the creativity construct? Personality and Individual Differ-
ences, 46(8), 755–764. https://doi.org/10.1016/j.paid.2009.01.021

Gostoli, S., Cerini, V., Piolanti, A., & Rafanelli, C. (2017). Creativity,
bipolar disorder vulnerability and psychological well-being: A prelimi-
nary study. Creativity Research Journal, 29(1), 63–70. https://doi.org/10
.1080/10400419.2017.1263511

Greenwood, T. A. (2020). Creativity and bipolar disorder: A shared genetic
vulnerability. Annual Review of Clinical Psychology, 16, 239–264.
https://doi.org/10.1146/annurev-clinpsy-050718-095449

Gwet, K. (2002). Inter-rater reliability: Dependency on trait prevalence
and marginal homogeneity. Statistical Methods for Inter-Rater Reliabil-
ity Assessment Series, 2(1), 9.

Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the
presence of high agreement. British Journal of Mathematical & Statistical
Psychology, 61(Pt 1), 29–48. https://doi.org/10.1348/000711006X126600

Hoffman, M. D., & Gelman, A. (2014). The No-U-turn sampler: Adap-
tively setting path lengths in Hamiltonian Monte Carlo. Journal of
Machine Learning Research, 15(1), 1593–1623.

Hope, A. C. A. (1968). A simplified Monte Carlo significance test proce-
dure. Journal of the Royal Statistical Society: Series B, Methodological,
30, 582–598. https://doi.org/10.1111/j.2517-6161.1968.tb00759.x

Hudson, J. I., Hiripi, E., Pope, H. G., Jr., & Kessler, R. C. (2007). The
prevalence and correlates of eating disorders in the National Comorbid-
ity Survey Replication. Biological Psychiatry, 61(3), 348–358. https://
doi.org/10.1016/j.biopsych.2006.03.040

Jamison, K. R. (1996). Touched with fire. Simon & Schuster.
Jamison, K. R., Gerner, R. H., Hammen, C., & Padesky, C. (1980). Clouds

and silver linings: Positive experiences associated with primary affective
disorders. The American Journal of Psychiatry, 137(2), 198–202.
https://doi.org/10.1176/ajp.137.2.198

Jeste, D. V., Harless, K. A., & Palmer, B. W. (2000). Chronic late-onset
schizophrenia-like psychosis that remitted: Revisiting Newton’s psycho-
sis? The American Journal of Psychiatry, 157(3), 444–449. https://doi
.org/10.1176/appi.ajp.157.3.444

Johnson, S. L., Murray, G., Fredrickson, B., Youngstrom, E. A., Hinshaw,
S., Bass, J. M., Deckersbach, T., Schooler, J., & Salloum, I. (2012). Cre-
ativity and bipolar disorder: Touched by fire or burning with questions?
Clinical Psychology Review, 32(1), 1–12. https://doi.org/10.1016/j.cpr
.2011.10.001

PSYCHOPATHOLOGY AND CREATIVITY 15

https://www.doi.org/10.1016/j.lindif.2007.02.004

https://doi.org/10.1177/070674378603100311

https://doi.org/10.1348/000712605X60030

https://doi.org/10.3389/fnhum.2014.00771

https://doi.org/10.1177/070674371105600304

https://doi.org/10.1037/0022-3514.85.3.499

https://doi.org/10.1073/pnas.1014871108

https://doi.org/10.1037/a0017309

https://doi.org/10.1037/hop0000029

https://doi.org/10.1111/j.2044-8260.1996.tb01166.x

https://doi.org/10.1207/s15326934crj1201_4

https://doi.org/10.1037/aca0000045

https://doi.org/10.1037/pspi0000011

https://doi.org/10.1088/2058-7058/12/12/2

https://doi.org/10.1207/s15327965pli0403_1

https://doi.org/10.1017/CBO9780511752247

https://doi.org/10.1207/s15327957pspr0204_5

https://doi.org/10.3389/fpsyg.2015.00163

https://doi.org/10.1002/cne.20768

https://doi.org/10.1080/10400419.2004.9651449

https://doi.org/10.1016/j.paid.2007.10.035

https://doi.org/10.1016/j.paid.2009.01.021

https://doi.org/10.1080/10400419.2017.1263511

https://doi.org/10.1146/annurev-clinpsy-050718-095449

https://doi.org/10.1348/000711006X126600

https://doi.org/10.1111/j.2517-6161.1968.tb00759.x

https://doi.org/10.1016/j.biopsych.2006.03.040

https://doi.org/10.1176/ajp.137.2.198

https://doi.org/10.1176/appi.ajp.157.3.444

https://doi.org/10.1016/j.cpr.2011.10.001

Johnson, S. L., Murray, G., Hou, S., Staudenmaier, P. J., Freeman, M. A.,
& Michalak, E. E., & the CREST.BD. (2015). Creativity is linked to
ambition across the bipolar spectrum. Journal of Affective Disorders,
178, 160–164. https://doi.org/10.1016/j.jad.2015.02.021

Johnson, S. L., Tharp, J. A., & Holmes, M. K. (2015). Understanding crea-
tivity in bipolar I disorder. Psychology of Aesthetics, Creativity, and the
Arts, 9(3), 319–327. https://doi.org/10.1037/a0038852

Kaufman, J. C. (2017). Creativity and mental illness. Cambridge Univer-
sity Press.

Kaufman, J. C., & Beghetto, R. A. (2009). Beyond big and little: The four
c model of creativity. Review of General Psychology, 13(1), 1–12.
https://doi.org/10.1037/a0013688

Kaufmann, G., & Kaufmann, A. (2017). When good is bad and bad is
good: Mood, bipolarity, and creativity. In J. C. Kaufman (Ed.), Creativ-
ity and mental illness (pp. 205–235). Cambridge University Press.

Kelter, R. (2020). fbst: The Full Bayesian Significance Test and the e-Value
(R package version 1.0). https://CRAN.R-project.org/package=fbst

Kéri, S. (2009). Genes for psychosis and creativity: A promoter polymor-
phism of the neuregulin 1 gene is related to creativity in people with high
intellectual achievement. Psychological Science, 20(9), 1070–1073.
https://doi.org/10.1111/j.1467-9280.2009.02398.x

Kessler, R. C., Angermeyer, M., Anthony, J. C., DE Graaf, R.,
Demyttenaere, K., Gasquet, I., DE Girolamo, G., Gluzman, S., Gureje,
O., Haro, J. M., Kawakami, N., Karam, A., Levinson, D., Medina Mora,
M. E., Oakley Browne, M. A., Posada-Villa, J., Stein, D. J., Adley
Tsang, C. H., Aguilar-Gaxiola, S., . . . Ustün, T. B. (2007). Lifetime
prevalence and age-of-onset distributions of mental disorders in the
world health organization’s world mental health survey initiative. World
Psychiatry, 6(3), 168–176.

Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., &
Walters, E. E. (2005). Lifetime prevalence and age-of-onset distribu-
tions of DSM–IV disorders in the National Comorbidity Survey Replica-
tion. Archives of General Psychiatry, 62(6), 593–602. https://doi.org/10
.1001/archpsyc.62.6.593

Kinney, D., & Richards, R. (2017). Creativity as “compensatory advant-
age”: bipolar and schizophrenic liability, the inverted-U hypothesis, and
practical implications. In J. C. Kaufman (Ed.), Creativity and mental ill-
ness (pp. 295–317). Cambridge University Press.

Ko, Y., & Kim, J. (2008). Scientific geniuses’ psychopathology as a mod-
erator in the relation between creative contribution types and eminence.
Creativity Research Journal, 20(3), 251–261. https://doi.org/10.1080/
10400410802278677

Kozbalt, A., Kaufman, S. B., Walder, D. J., Ospina, L. H., & Kim, J. U.
(2017). The evolutionary genetics of the creativity-psychosis connec-
tion. In J. C. Kaufman (Ed.), Creativity and mental illness (pp.
102–132). Cambridge University Press.

Kyaga, S., Lichtenstein, P., Boman, M., Hultman, C., Långström, N., &
Landén, M. (2011). Creativity and mental disorder: Family study of
300,000 people with severe mental disorder. The British Journal of Psy-
chiatry, 199(5), 373–379. https://doi.org/10.1192/bjp.bp.110.085316

LeBoutillier, N., Barry, R., & Westley, D. (2014). The role of schizotypy
in predicting performance on figural and verbal imagery-based measures
of creativity. Creativity Research Journal, 26(4), 461–467. https://doi
.org/10.1080/10400419.2014.961778

Lev-Ran, S., Imtiaz, S., Rehm, J., & Le Foll, B. (2013). Exploring the asso-
ciation between lifetime prevalence of mental illness and transition from
substance use to substance use disorders: Results from the National Epi-
demiologic Survey of Alcohol and Related Conditions (NESARC). The
American Journal on Addictions, 22(2), 93–98. https://doi.org/10.1111/j
.1521-0391.2013.00304.x

Lucijani�c, M., Breitenfeld, D., Mileti�c, J., Buljan, D., Ozimec-Vulinec, Š.,
& Akrap, A. (2010). Rock musicians’ Club 27. Alcoholism: Journal on
Alcoholism and Related Addictions, 46(2), 109–113.

Ludwig, A. (1995). The price of greatness: Resolving the creativity and
madness controversy. Guilford Press.

Ludwig, A. (1998). Method and madness in the arts and sciences. Creativity
Research Journal, 11(2), 93–101. https://doi.org/10.1207/s15326934crj1102_1

Ludwig, A. M. (1992). Creative achievement and psychopathology: Com-
parison among professions. American Journal of Psychotherapy, 46(3),
330–356. https://doi.org/10.1176/appi.psychotherapy.1992.46.3.330

Martín-Brufau, R., & Corbalán, J. (2016). Creativity and psychopathology:
Sex matters. Creativity Research Journal, 28(2), 222–228. https://doi
.org/10.1080/10400419.2016.1165531

Mason, O., Claridge, G., & Jackson, M. (1995). New scales for the assess-
ment of schizotypy. Personality and Individual Differences, 18(1),
7–13. https://doi.org/10.1016/0191-8869(94)00132-C

Merikangas, K. R., Akiskal, H. S., Angst, J., Greenberg, P. E., Hirschfeld,
R. M., Petukhova, M., & Kessler, R. C. (2007). Lifetime and 12-month
prevalence of bipolar spectrum disorder in the National Comorbidity
Survey replication. Archives of General Psychiatry, 64(5), 543–552.
https://doi.org/10.1001/archpsyc.64.5.543

Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., &
Handelsman, J. (2012). Science faculty’s subtle gender biases favor
male students. Proceedings of the National Academy of Sciences of the
United States of America, 109(41), 16474–16479. https://doi.org/10
.1073/pnas.1211286109

Motto, A. L., & Clark, J. R. (1992). The paradox of genius and madness:
Seneca and his influence. Cuadernos de filología clásica. Estudios Lat-
inos, (2), 189–200.

Nasar, S. (2011). A beautiful mind. Simon & Schuster.
Nettle, D. (2001). Strong imagination: Madness, creativity and human na-

ture. Oxford University Press.
Nettle, D. (2006). Schizotypy and mental health amongst poets, visual

artists, and mathematicians. Journal of Research in Personality, 40(6),
876–890. https://doi.org/10.1016/j.jrp.2005.09.004

Nock, M. K., & Kessler, R. C. (2006). Prevalence of and risk factors for suicide
attempts versus suicide gestures: Analysis of the National Comorbidity Sur-
vey. Journal of Abnormal Psychology, 115(3), 616–623. https://doi.org/10
.1037/0021-843X.115.3.616

Nowakowska, C., Strong, C. M., Santosa, C. M., Wang, P. W., & Ketter,
T. A. (2005). Temperamental commonalities and differences in euthy-
mic mood disorder patients, creative controls, and healthy controls.
Journal of Affective Disorders, 85(1-2), 207–215. https://doi.org/10
.1016/j.jad.2003.11.012

Paek, S. H., Abdulla, A. M., & Cramond, B. (2016). A meta-analysis of the
relationship between three common psychopathologies—ADHD, anxiety,
and depression—and indicators of little-c creativity. Gifted Child Quar-
terly, 60(2), 117–133. https://doi.org/10.1177/0016986216630600

Perälä, J., Suvisaari, J., Saarni, S. I., Kuoppasalmi, K., Isometsä, E.,
Pirkola, S., Partonen, T., Tuulio-Henriksson, A., Hintikka, J., Kieseppä,
T., Härkänen, T., Koskinen, S., & Lönnqvist, J. (2007). Lifetime preva-
lence of psychotic and bipolar I disorders in a general population. Ar-
chives of General Psychiatry, 64(1), 19–28. https://doi.org/10.1001/
archpsyc.64.1.19

Pereira, C. A. B., & Stern, J. M. (1999). Evidence and credibility: Full
Bayesian significance test for precise hypotheses. Entropy, 1(4),
99–110. https://doi.org/10.3390/e1040099

Pereira, C. A. B., & Stern, J. M. (2020). The e-value: a fully Bayesian sig-
nificance measure for precise statistical hypotheses and its research pro-
gram. São Paulo Journal of Mathematical Sciences. Advance online
publication. https://doi.org/10.1007/s40863-020-00171-7

Perry, I. H. (1947). Vincent van Gogh’s illness: — case record. Bulletin of
the History of Medicine, 21, 146–172.

Porter, A. M., & Ivie, R. (2019). Women in physics and astronomy, 2019.
https://www.aip.org/statistics/reports/women-physics-and-astronomy-2019
#:�:text=Highlights,and%2040%25%20of%20astronomy%20doctorates

16 FEIST, DOSTAL, AND KWAN

https://doi.org/10.1016/j.jad.2015.02.021

https://doi.org/10.1037/a0038852

https://doi.org/10.1037/a0013688

https://CRAN.R-project.org/package=fbst

https://doi.org/10.1111/j.1467-9280.2009.02398.x

https://doi.org/10.1001/archpsyc.62.6.593

https://doi.org/10.1080/10400410802278677

https://doi.org/10.1192/bjp.bp.110.085316

https://doi.org/10.1080/10400419.2014.961778

https://doi.org/10.1111/j.1521-0391.2013.00304.x

https://doi.org/10.1207/s15326934crj1102_1

https://doi.org/10.1176/appi.psychotherapy.1992.46.3.330

https://doi.org/10.1080/10400419.2016.1165531

https://doi.org/10.1016/0191-8869(94)00132-C

https://doi.org/10.1001/archpsyc.64.5.543

https://doi.org/10.1073/pnas.1211286109

https://doi.org/10.1016/j.jrp.2005.09.004

https://doi.org/10.1037/0021-843X.115.3.616

https://doi.org/10.1016/j.jad.2003.11.012

https://doi.org/10.1177/0016986216630600

https://doi.org/10.1001/archpsyc.64.1.19

https://doi.org/10.3390/e1040099

https://doi.org/10.1007/s40863-020-00171-7

https://www.aip.org/statistics/reports/women-physics-and-astronomy-2019#:&hx223C;:text=Highlights,and%2040%25%20of%20astronomy%20doctorates

Post, F. (1994). Creativity and psychopathology. A study of 291 world-fa-
mous men. The British Journal of Psychiatry, 165(1), 22–34. https://doi
.org/10.1192/bjp.165.1.22

Power, R. A., Steinberg, S., Bjornsdottir, G., Rietveld, C. A., Abdellaoui,
A., Nivard, M. M., Johannesson, M., Galesloot, T. E., Hottenga, J. J.,
Willemsen, G., Cesarini, D., Benjamin, D. J., Magnusson, P. K. E.,
Ullén, F., Tiemeier, H., Hofman, A., van Rooij, F. J. A., Walters, G. B.,
Sigurdsson, E., . . . Stefansson, K. (2015). Polygenic risk scores for
schizophrenia and bipolar disorder predict creativity. Nature Neuro-
science, 18(7), 953–955. https://doi.org/10.1038/nn.4040

Ramey, C. H., & Weisberg, R. W. (2004). The” poetical activity” of Emily
Dickinson: A further test of the hypothesis that affective disorders foster
creativity. Creativity Research Journal, 16(2-3), 173–185. https://doi
.org/10.1080/10400419.2004.9651451

Reiter-Palmon, R., & Schoenbeck, M. (2020). Creativity equals creativity-
–or does it? How creativity is measured influences our understanding of
creativity. In V. Dorfler & M. Stierand (Eds.), Handbook of research
methods on creativity (pp. 290–300). Edward Elgar Publishing. https://
doi.org/10.4337/9781786439659.00031

Richards, R. (1990). Everyday creativity, eminent creativity, and health:
‘Afterview’ for CRJ issues on creativity and health. Creativity Research
Journal, 3(4), 300–326. https://doi.org/10.1080/10400419009534363

Richards, R. (1994). Creativity and bipolar mood swings: Why the associa-
tion? In M. P. Shaw & M. A. Runco (Eds.), Creativity and affect (pp.
44–72). Ablex Publishing.

Richards, R., Kinney, D. K., Lunde, I., Benet, M., & Merzel, A. P. (1988).
Creativity in manic-depressives, cyclothymes, their normal relatives,
and control subjects. Journal of Abnormal Psychology, 97(3), 281–288.
https://doi.org/10.1037/0021-843X.97.3.281

Robins, L. N., Helzer, J. E., Weissman, M. M., Orvaschel, H., Gruenberg, E.,
Burke, J. D., Jr., & Regier, D. A. (1984). Lifetime prevalence of specific
psychiatric disorders in three sites. Archives of General Psychiatry, 41(10),
949–958. https://doi.org/10.1001/archpsyc.1984.01790210031005

Rothenberg, A. (1990). Creativity and madness: New findings and old ster-
eotypes. Johns Hopkins University Press.

Ruiter, M., & Johnson, S. L. (2015). Mania risk and creativity: A multi-
method study of the role of motivation. Journal of Affective Disorders,
170(1), 52–58. https://doi.org/10.1016/j.jad.2014.08.049

Runco, M., & Jaeger, G. J. (2012). The standard definition of creativity.
Creativity Research Journal, 24(1), 92–96. https://doi.org/10.1080/
10400419.2012.650092

Santosa, C. M., Strong, C. M., Nowakowska, C., Wang, P. W., Rennicke,
C. M., & Ketter, T. A. (2007). Enhanced creativity in bipolar disorder
patients: A controlled study. Journal of Affective Disorders, 100(1–3),
31–39. https://doi.org/10.1016/j.jad.2006.10.013

Sawyer, R. K. (2011). Explaining creativity: The science of human innova-
tion. Oxford University Press.

Schlesinger, J. (2009). Creative myth conceptions: A closer look at the evi-
dence for the “mad genius” hypothesis. Psychology of Aesthetics, Crea-
tivity, and the Arts, 3(2), 62–72. https://doi.org/10.1037/a0013975

Schlesinger, J. (2012). The insanity hoax: Exposing the myth of the mad
genius. Shrinktunes Media.

Schlesinger, J. (2017). Building connections on sand: The cautionary chap-
ter. In J. C. Kaufman (Ed.), Creativity and mental illness (pp. 60–75).
Cambridge University Press.

Schuldberg, D. (1990). Schizotypal and hypomanic traits, creativity, and
psychological health. Creativity Research Journal, 3(3), 218–230.
https://doi.org/10.1080/10400419009534354

Seneca. (2007). Dialogues and essays: Tranquility of mind (J. Davie,
Trans.). Oxford.

Silvia, P. J., & Kaufman, J. C. (2010). Creativity and mental illness. In
J. C. Kaufman & R. J. Sternberg (Eds.), The Cambridge handbook of
creativity (pp. 381–394). Cambridge University Press. https://doi.org/10
.1017/CBO9780511763205.024

Silvia, P. J., & Kimbrel, N. A. (2010). A dimensional analysis of creativity
and mental illness: Do anxiety and depression symptoms predict creative
cognition, creative accomplishments, and creative self-concepts? Psy-
chology of Aesthetics, Creativity, and the Arts, 4(1), 2–10. https://doi
.org/10.1037/a0016494

Simeonova, D. I., Chang, K. D., Strong, C., & Ketter, T. A. (2005). Crea-
tivity in familial bipolar disorder. Journal of Psychiatric Research,
39(6), 623–631. https://doi.org/10.1016/j.jpsychires.2005.01.005

Simonton, D. K. (2006). Creativity and madness. Talk presented to Psy-
chology Forum, The Commonwealth Club, San Francisco, CA.

Simonton, D. K. (2010). So you want to become a creative genius? You
must be crazy. In D. H. Cropley, A. J. Cropley, J. C. Kaufman, & M. A.
Runco (Eds.), The dark side of creativity (pp. 218–234). Cambridge Uni-
versity Press. https://doi.org/10.1017/CBO9780511761225.012

Simonton, D. K. (2013). Creative thought as blind variation and selective
retention: Why creativity is inversely related to sightedness. Journal of
Theoretical and Philosophical Psychology, 33(4), 253–266. https://doi
.org/10.1037/a0030705

Simonton, D. K. (2014). More method in the mad-genius controversy: A
historiometric study of 204 historic creators. Psychology of Aesthetics,
Creativity, and the Arts, 8(1), 53–61. https://doi.org/10.1037/a0035367

Simonton, D. K. (2016). Defining creativity: Don’t we also need to define
what is not creative? Journal of Creative Behavior, 52, 80–90. https://
doi.org/10.1002/jocb.137

Simonton, D. K. (2017a). Creative genius and psychopathology: Creativity
as positive and negative personality. In G. J. Feist, R. Reiter-Palmon, &
J. C. Kaufman (Eds.), The Cambridge handbook of creativity and per-
sonality research (pp. 235–250). Cambridge University Press. https://
doi.org/10.1017/9781316228036.013

Simonton, D. K. (2017b). The mad (creative) genius: What do we know af-
ter a century of historiometric research? In J. C. Kaufman (Ed.), Creativ-
ity and mental illness (pp. 25–41). Cambridge University Press.

Simonton, D. K. (2019). Creativity and psychopathology: The tenacious
mad-genius controversy updated. Current Opinion in Behavioral Scien-
ces, 27, 17–21. https://doi.org/10.1016/j.cobeha.2018.07.006

Simonton, D. K., & Song, A. V. (2009). Eminence, IQ, physical and men-
tal health, and achievement domain: Cox’s 282 Geniuses revisited. Psy-
chological Science, 20(4), 429–434. https://doi.org/10.1111/j.1467-9280
.2009.02313.x

Stan Development Team. (2016). rstanarm: Bayesian applied regression
modeling via Stan. R package version 2.13.1. http://mc-stan.org/

Stein, D. J., Phillips, K. A., Bolton, D., Fulford, K. W. M., Sadler, J. Z., &
Kendler, K. S. (2010). What is a mental/psychiatric disorder? From
DSM–IV to DSM-V. Psychological Medicine, 40(11), 1759–1765.
https://doi.org/10.1017/S0033291709992261

Sternberg, R. J. (1988). A three-facet model of creativity. In R. J. Sternberg
& R. J. Sternberg (Eds.), The nature of creativity: Contemporary psycho-
logical perspectives (pp. 125–147). Cambridge University Press.

Swain, J. E., & Swain, J. D. (2017). Non-linearity in creativity and mental
illness: The mixed blessings of chaos, catastrophe, and noise in brain
and behavior. In J. C. Kaufman (Ed.), Creativity and mental illness (pp.
133–144). Cambridge University Press.

Taylor, C. L. (2017). Creativity and mood disorder: A systematic review
and meta-analysis. Perspectives on Psychological Science, 12(6),
1040–1076. https://doi.org/10.1177/1745691617699653

Thomson, P. (2017). Trauma, attachment, and creativity. In T. Marks-
Tarlow, M. Solomon, & D. J. Siegel (Eds.), Play and creativity in psy-
chotherapy (pp. 167–190). W.W. Norton.

Thomson, N. D., Wurtzburg, S. J., & Centifanti, L. C. (2015). Empathy or
science? Empathy explains physical science enrollment for men and
women. Learning and Individual Differences, 40(2), 115–120.

Thulin, M. (2014). Decision-theoretic justifications for Bayesian hypothe-
sis testing using credible sets. Journal of Statistical Planning and Infer-
ence, 146(3), 133–138. https://doi.org/10.1016/j.jspi.2013.09.014

PSYCHOPATHOLOGY AND CREATIVITY 17

https://doi.org/10.1192/bjp.165.1.22

https://doi.org/10.1038/nn.4040

https://doi.org/10.1080/10400419.2004.9651451

https://doi.org/10.4337/9781786439659.00031

https://doi.org/10.1080/10400419009534363

https://doi.org/10.1037/0021-843X.97.3.281

https://doi.org/10.1001/archpsyc.1984.01790210031005

https://doi.org/10.1016/j.jad.2014.08.049

https://doi.org/10.1080/10400419.2012.650092

https://doi.org/10.1016/j.jad.2006.10.013

https://doi.org/10.1037/a0013975

https://doi.org/10.1080/10400419009534354

https://doi.org/10.1017/CBO9780511763205.024

https://doi.org/10.1037/a0016494

https://doi.org/10.1016/j.jpsychires.2005.01.005

https://doi.org/10.1017/CBO9780511761225.012

https://doi.org/10.1037/a0030705

https://doi.org/10.1037/a0035367

https://doi.org/10.1002/jocb.137

https://doi.org/10.1017/9781316228036.013

https://doi.org/10.1016/j.cobeha.2018.07.006

https://doi.org/10.1111/j.1467-9280.2009.02313.x

http://mc-stan.org/

https://doi.org/10.1017/S0033291709992261

https://doi.org/10.1177/1745691617699653

https://doi.org/10.1016/j.jspi.2013.09.014

Thys, E., Sabbe, B., & De Hert, M. (2014). Creativity and psychopathol-
ogy: A systematic review. Psychopathology, 47(3), 141–147. https://doi
.org/10.1159/000357822

U.S. Census Bureau. (2019). Quick facts: United States. Race and
Hispanic Origin. https://www.census.gov/quickfacts/fact/table/U.S
./PST045219

Verhaeghen, P., Joorman, J., & Khan, R. (2005). Why we sing the blues:
The relation between self-reflective rumination, mood, and creativity.
Emotion, 5(2), 226–232. https://doi.org/10.1037/1528-3542.5.2.226

Verhaeghen, P., Joormann, J., & Aikman, S. N. (2014). Creativity, mood,
and the examined life: Self-reflective rumination boosts creativity,
brooding breeds dysphoria. Psychology of Aesthetics, Creativity, and the
Arts, 8(2), 211–218. https://doi.org/10.1037/a0035594

Wei, X., Yu, J. W., Shattuck, P., & Blackorby, J. (2017). High school math and
science preparation and postsecondary STEM participation for students with
an autism spectrum disorder. Focus on Autism and Other Developmental
Disabilities, 32(2), 83–92. https://doi.org/10.1177/1088357615588489

Wills, G. I. (2003). Forty lives in the bebop business: Mental health in a
group of eminent jazz musicians. The British Journal of Psychiatry,
183(3), 255–259. https://doi.org/10.1192/bjp.183.3.255

Wongpakaran, N., Wongpakaran, T., Wedding, D., & Gwet, K. L. (2013).
A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating
inter-rater reliability coefficients: A study conducted with personality
disorder samples. BioMed Central Medical Research Methodology, 13,
61. https://doi.org/10.1186/1471-2288-13-61

Wuthrich, V., & Bates, T. C. (2001). Schizotypy and latent inhibition:
Non-linear linkage between psychometric and cognitive markers. Per-
sonality and Individual Differences, 30(5), 783–798. https://doi.org/10
.1016/S0191-8869(00)00071-4

Received December 10, 2020
Revision received July 22, 2021

Accepted August 13, 2021 n

18 FEIST, DOSTAL, AND KWAN

https://doi.org/10.1159/000357822

https://www.census.gov/quickfacts/fact/table/U.S./PST045219

https://doi.org/10.1037/1528-3542.5.2.226

https://doi.org/10.1037/a0035594

https://doi.org/10.1177/1088357615588489

https://doi.org/10.1192/bjp.183.3.255

https://doi.org/10.1186/1471-2288-13-61

https://doi.org/10.1016/S0191-8869(00)00071-4

Clinical Medicine Insights: Psychiatry
Volume 13: 1–5
© The Author(s) 2022
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/11795573211069912

Does fear mediate the neuroticism-psychopathology
link for adults living through the COVID-19 pandemic?

Sherman A. Lee1 and Mary C. Jobe2
1Christopher Newport University, Newport News, VA, USA.
2The George Washington University, Washington, DC, USA.

ABSTRACT

BACKGROUND: COVID-19 has globally increased psychological distress. Although research has shown a clear link between neuroticism and
psychopathology, pandemic fears—manifesting as fear of death and coronavirus anxiety, have not been examined as mediating factors for
explaining this connection during the pandemic.

METHODS: Therefore, to fill this void in the literature, this study examined 259 U.S. MTurk adult workers in May 2020 using an online questionnaire.
The study used the Patient Health Questionnaire, the 8-item Big Five Inventory neuroticism subscale, a single-item fear of death measure, and the
Coronavirus Anxiety Scale as well as collected demographic information to perform correlational and meditation

analyses.

RESULTS: The results showed that both coronavirus anxiety and fear of death partially mediated the relationship between neuroticism and
symptoms of depression and generalized anxiety. The results also found that those high in trait neuroticism who were fearful of death or had
coronavirus anxiety showed heightened levels of depression and general anxiety.

CONCLUSION: This study’s findings were consistent with previous research and current work on pandemic-related distress. In addition, the results
of these findings can help bring to light the connectedness of these psychopathological constructs with fears surrounding the pandemic—which
can be useful to both researchers and mental health professionals alike.

KEYWORDS: Neuroticism, COVID-19, fear of death, coronavirus, anxiety, depression

RECEIVED: February 7, 2021. ACCEPTED: December 10, 2021.

TYPE: Original Research

DECLARATION OF CONFLICTING INTERESTS: The author(s) declared no potential conflicts
of interest with respect to the research, authorship, and/or publication of this article.

FUNDING: The author(s) received no financial support for the research, authorship, and/or
publication of this article.

ETHICAL APPROVAL: All procedures performed in this study were in accordance with the ethical
standards of Christopher Newport University’s ethics and IRB approval committee. In addition, the
procedures are in accordance with the Declaration of Helsinki or ethical equivalent. In addition,
informed consent was obtained from all individual adult participants included in the study.

CORRESPONDING AUTHOR: Mary C. Jobe, The George Washington University, 2125 G Street
NW, Washington, DC, 20052, USA. Email: mary.jobe.16@cnu.edu

Introduction
As the pandemic persists, so does the worsening of people’s

mental health and well-being.1 For example, during the first

months of the pandemic, 24.4% of Americans reported clinical

levels of depression, while 29.8% reported clinical levels of

anxiety.2 One factor that has been found to be strongly associated

with depression and generalized anxiety during the COVID-19

pandemic is neuroticism.3 According to the Five-Factor model,

neuroticism describes a broad dimension of personality concerned

with tendencies to experience negative affect, and disturbed

thoughts and behaviors that accompany emotional distress.4 The

finding that neuroticism is associated with adjustment difficulties

during the COVID-19 pandemic should not be surprising given

that individuals high in this personality trait have long been

known to suffer from a wide-range of mental and physical health

conditions5 as well as psychological distress during previous

pandemics.6,7 However, what is not clear is what the psycho-

logical mechanisms are that explain why individuals high in this

trait are experiencing heightened levels of psychological distress

during this particular global health crisis.

The COVID-19 pandemic has been shrouded by fear and

anxiety with millions of people dead from this highly infectious

disease. Consequently, many people living in this pandemic fear

for their lives and the coronavirus itself, as the virus can lead to

the death and suffering of oneself and their loved ones. Ac-

cordingly, research during this pandemic has shown that both

death anxiety and coronavirus anxiety are both positively cor-

related with depression and generalized anxiety.8,9 Moreover,

past7 and current research3,10 has shown that neuroticism is

strongly tied to pandemic-related fears and psychopathology.

That said, although research has shown a clear link between

neuroticism and psychopathology (i.e., anxiety and depression),

pandemic fears—such as fear of death and coronavirus anxiety,

have not been examined as mediating factors for explaining this

connection during the pandemic. Thus, this study will aim to

address this; we anticipate that fear of death and coronavirus

anxiety will mediate the relationship between neuroticism and

psychopathology in a sample of adults during the COVID-19

pandemic. These causal implications can enlighten the rela-

tionships these constructs hold—helping researchers better

Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial
4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without

further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).

https://uk.sagepub.com/en-gb/journals-permissions

https://doi.org/10.1177/11795573211069912

https://orcid.org/0000-0001-7106-7120

mailto:mary.jobe.16@cnu.edu

https://creativecommons.org/licenses/by-nc/4.0/

understand COVID-19’s psychological impacts and possible

avenues for treatment.

Method
Participants and procedures

Data from 259 adult MTurk
11

workers in the U.S. who

completed an online survey on May 15 and 16, 2020 were used

in this IRB approved study. The sample consisted of 116

women and 143 men, with a median age of 33.00 years (ranging

from 18 to 65). Most of the participants were White (n = 165;

63.7%), had earned at least a Bachelor’s degree (n = 190; 73.4%),

had not tested positive for COVID-19 (n = 227; 87.6%), and

did not know someone with COVID-19 (n = 147; 56.8%). The

participants provided consent and received payment ($0.50) for

their involvement in this study.

Measures

Background information. Participants were asked to report their

age, gender, race, level of education, diagnosis of COVID-19,

and personal knowledge of someone with COVID-19.

Psychopathology. Symptoms of psychopathology were mea-

sured using the four-item Patient Health Questionnaire-4.12

Participants indicated how frequently they experienced symp-

toms of depression (e.g., feeling down, depressed, or hopeless; α =
.80) and generalized anxiety (e.g., feeling nervous, anxious, or on

edge; α = .78) over the last 2 weeks using a 4-point scale.
Neuroticism. The generalized tendency to experience negative

emotions was measured using the 8-item neuroticism subscale of

the Big Five Inventory.
13

Participants indicated how much they

agreed or disagreed with descriptions of neuroticism (e.g., I see

myself as someone who worries a lot) using a 4-point scale (α = .81).
Fear of Death. Fear of death was measured using a single-item

Fear of Death measure.14 Although a single-item, it has been

found to reliably assess fear of death and moderately be associated

with multi-item death anxiety scales.14 For the item, participants

indicated how much they agreed or disagreed with the statement,

“I am afraid of death” using a 4-point scale. Most of the par-

ticipants reported that they agree a little (28.2%), followed by

neither agree nor disagree (19.7%), strongly agree (18.9%), disagree

a little (17.8%), and strongly disagree (15.4%) to the item.

Coronavirus anxiety. Dysfunctional anxiety over the coro-

navirus was measured using the 5-item Coronavirus Anxiety

Scale.15 Participants indicated how frequently they experienced

physiologically based symptoms of fear and anxiety over the

coronavirus (e.g., I felt dizzy, lightheaded, or faint, when I read or

listened to news about the coronavirus) over the last 2 weeks using

a 4-point scale (α = .94).

Statistical procedures

Statistical analyses were calculated using SPSS version 26.0,

except for the mediation analyses, which were run using AMOS

version 25.0. We tested mediators one at a time to determine

independent effects16 and employed bias-corrected bootstrap

procedures using 2,000 resamples to the models.17 We chose a

bootstrap resampling method because its calculation of confi-

dence intervals is not biased by sample size, effect size, or level of

statistical significance.18

Results
Correlations

Zero-order correlations were run to examine the bivariate asso-

ciations between the measures of psychopathology, neuroticism,

and the proposed mediators of fear of death and coronavirus

anxiety (see Table 1). The results revealed that depression was

correlated with neuroticism (r = .64), fear of death (r = .40),

coronavirus anxiety (r = .66), and generalized anxiety (r = .76).

Generalized anxiety was also correlated with neuroticism (r =

.68), fear of death (r = .47), and coronavirus anxiety (r = .62).

Neuroticism was correlated with fear of death (r = .40), and

coronavirus anxiety (r = .42). Fear of death was correlated with

coronavirus anxiety (r = .33), supporting their related but

distinct expressions of pandemic fear. These intercorrelation

patterns support the inclusion of the variables in the mediation

analyses.

Mediation analyses

Four mediation analyses were conducted to examine the sep-

arate influences of proposed mediators on the association be-

tween neuroticism and psychopathology (i.e., depression and

generalized anxiety). The first model tested coronavirus anxi-

ety’s mediating influence on the relationship between neurot-

icism and depression (see Figure 1). The bootstrap results

showed that the standardized indirect (mediated) effect of

neuroticism on depression was significantly different from zero

(P = .001, 95% CI [.15, .27]). Therefore, this model demon-

strated that coronavirus anxiety partially mediated the

neuroticism-depression link (β from .64 to .44) with a stan-
dardized indirect effect of .20.

The second model tested coronavirus anxiety’s mediating

influence on the relationship between neuroticism and gener-

alized anxiety (see Figure 2). The bootstrap results showed that

the standardized indirect (mediated) effect of neuroticism on

generalized anxiety was significantly different from zero (P =

.001, 95% CI [.12, .23]). Therefore, this model demonstrated

that coronavirus anxiety partially mediated the neuroticism-

generalized anxiety link (β from .68 to .51) with a standardized
indirect effect of .17.

The third model tested death anxiety’s mediating influence

on the relationship between neuroticism and depression (see

Figure 3). The bootstrap results showed that the standardized

indirect (mediated) effect of neuroticism on depression was

significantly different from zero (P = .001, 95% CI [.03, .12]).

Therefore, this model demonstrated that fear of death partially

2 Clinical Medicine Insights: Psychiatry
n n

T
a
b
le

1
.
D
e
sc
ri
p
tiv
e
st
a
tis
tic
s
a
n
d
ze

ro
-o
rd
e
r
co

rr
e
la
tio

n
s.

M
S
D

1
2

3
4

5
6

7
8

1
0

1
D
e
p
re
ss

io
n

2
.6
1

1
.9
2

—

2
G
e
n
e
ra
liz
e
d

a
n
xi
e
ty

2
.4
7

1
.7
7

.7
6
**
*

—

3
N
e
u
ro
tic
is
m

2
3
.3
7

6
.1
5

.6
4

**
*

.6
8
**
*

—

4
F
e
a
r
o
f
d
e
a
th

3
.1
7

1
.3
5

.4
0
**
*

.4
7
**
*

.4
0
**
*
—

5
C
o
ro
n
a
vi
ru
s

a
n
xi
e
ty

.0
4

5
.9
2

.6
6
**
*

.6
2
**
*

.4
2
**
*

.3
3

**
*

—

6
A
g
e

3
5
.4
6

1
1

.3
0

�.
1
0

�.
1
2

�.
2
1
**

�.
0
9

.0
1

—

7
G
e
n
d
e
r

.5
5

.5
0

.0
3

�.
0
3

�.
2
1
**

.0
2

.1
4
*

.0
1
—

8
R
a
ce

.6
4

.4
8

.0
0

.0
5

.0
0

�.
1
2
*

�.
0
5

.1
8
**

�.
0
2

—

9
E
d
u
ca

tio
n

.7
3

.4
4

.1
4
*

.2
2
**
*

.0
4

.1
5
*

.2
8
**
*

.0
4
.1
4
*

.0
7

—
1
0

D
ia
g
n
o
si
s

.1
2

.3
3

.2
0
**

.1
9
**

.1
1

.1
0

.3
4
**
*

.0
1

.0
8

�.
0
1

.0
9

—
1
1

K
n
o
w
le
d
g
e

.4
3

.5
0
.1
1
.1
4
*
.0
4
.0
8
.2
0
**
�.
1
0
.0
7
�.
0
2
�.
0
2

.2
6
**
*

N
o
te
.
N
=
2
5
9
;
G
e
n
d
e
r
(0

=
m
a
le
;
1
=
fe
m
a
le
);
R
a
ce

(0
=
n
o
n
-W

h
ite

;
1
=
W
h
ite

);
E
d
u
ca

tio
n
(0

=
L
e
ss

th
a
n
a
B
a
ch

e
lo
r’
s
d
e
g
re
e
;
1
=
B
a
ch

e
lo
r’
s
d
e
g
re
e
a
n
d
h
ig
h
e
r)
;
D
ia
g
n
o
si
s
(0

=
n
o
t
d
ia
g
n
o
se

d
w
ith

C
O
V
ID
-1
9
;
1
=
d
ia
g
n
o
se

d
w
ith

C
O
V
ID
-1
9
);
K
n
o
w
le
d
g
e
(0

=
d
o
e
s
n
o
t
p
e
rs
o
n
a
lly

kn
o
w

m
e
o
n
e
w
ith

C
O
V
ID
-1
9
;
1
=
p
e
rs
o
n
a
lly

kn
o
w
s
so

m
e
o
n
e
w
ith

C
O
V
ID
-1
9
).

*P
< .0 5 ; tw o -t a ile d , ** P < .0 1 ; tw o -t a ile d , ** *P

< .0 0 1 ; tw o -t a ile d .

3Lee and Jobe
n n

mediated the neuroticism-depression anxiety link (β from .64
to .57) with a standardized indirect effect of .07. The last

model tested fear of death’s mediating influence on the re-

lationship between neuroticism and generalized anxiety (see

Figure 4). The bootstrap results showed that the standardized

indirect (mediated) effect of neuroticism on generalized

anxiety was significantly different from zero (P = .001, 95%

CI [.05, .15]). Therefore, this model demonstrated that fear

of death partially mediated the neuroticism-generalized

anxiety link (β from .68 to .59) with a standardized indi-
rect effect of .09.

Discussion
Overall, the COVID-19 pandemic has had an impact on mental

health. In addition, past literature has demonstrated that in

general and during pandemic times, neuroticism may play a role

in who may be more likely to experience such psychopathology

(i.e., anxiety and depression).5,19-21 Our study aimed to assess

what other mechanisms may help to explain this neuroticism-

psychopathology relationship—by exploring pandemic fears,

specifically fear of death and coronavirus anxiety. In sum, the

results demonstrated that the potential mediators explained

some of the reason as to why individuals high in trait neu-

roticism experienced elevated psychopathology symptoms

during the COVID-19 crisis. Our findings are consistent with

Nikčević and colleagues’ (2021) predicted model for COVID-19

anxiety as a mediator in the neuroticism-generalized anxiety and

neuroticism-depression relationships; however, the results of their

study did not yield significant findings between these variables.22

Moreover, our study’s findings support past literature and provide

further analysis, for using mediations to explain the fear rela-

tionships, within the pandemic context, between these commonly

associated variables: neuroticism and psychopathology. In addition,

the results demonstrate that those high in trait neuroticism, in

particular, who are fearful of death or have coronavirus anxiety may

also show these heightened levels of depression and general

anxiety. Literature has shown such psychopathology has been

increasing during COVID-1923,24 and that those high in trait

neuroticism are a vulnerable population during pandemics.7,25 This

study synthesizes these relationships, supporting Lee and Crunk

(2020)3 in showing how fears can explain these associations. After

accounting for pandemic specific fears, researchers and mental

health professionals are able to understand the bigger picture as to

why individuals high in trait neuroticism may be especially sus-

ceptible to such psychopathology, using these findings.

There are many possible approaches to treating people with

depression, generalized anxiety, and issues associated with

neuroticism during the COVID-19 pandemic. One method that

has been shown to successfully treat individuals suffering from

psychological distress and high in neuroticism, while considering

pandemic spatial distancing practices, has been the use of

Figure 4. Mediating effect of fear of death on the association between
neuroticism and generalized anxiety. Note. Two-sided bias-corrected

bootstrap procedure (95% confidence intervals; 2,000 samples). Above

values reflect

standardized regression coefficients. *** P < .001.

Figure 3. Mediating effect of fear of death on the association between
neuroticism and depression. Note. Two-sided bias-corrected bootstrap

procedure (95% confidence intervals; 2,000 samples). Above values reflect

standardized regression coefficients. *** P < .001.

Figure 2. Mediating effect of coronavirus anxiety on the association between
neuroticism and generalized anxiety. Note. Two-sided bias-corrected

bootstrap procedure (95% confidence intervals; 2,000 samples). Above

values reflect standardized regression coefficients. *** P < .001.

Figure 1. Mediating effect of coronavirus anxiety on the association between
neuroticism and depression. Note. Two-sided bias-corrected bootstrap

procedure (95% confidence intervals; 2,000 samples). Above values reflect
standardized regression coefficients. *** P < .001.

4 Clinical Medicine Insights: Psychiatry
n n

telehealth. Hedman et al (2014)26 found that using internet-

based cognitive behavior therapy, especially for individuals high

in neuroticism, has shown to be effective in and even lessened

both psychological distress and tendencies of those high in trait

neuroticism. Adopting this method of therapy to address pan-

demic fears and psychopathology for individuals high in trait

neuroticism may be both practical and beneficial, especially as the

pandemic persists and mental health issues rise.

This research has a major limitation worth noting. Specif-

ically, this study was constrained by a relatively small conve-

nience sample. Future research would benefit from a probability

sampling approach that would result in obtaining a large,

representative sample where more sophisticated mediation

analyses that examines both independent and simultaneous

effects could be applied. In addition, fear of death was only

assessed using a single item; further research could examine this

construct using longer measures. Further this research was

conducted using self-report measures, which could be subject to

possible social desirability. Notwithstanding these limitations,

our research reports important data that contribute to our

understanding of the mental health consequences of the pan-

demic. By understanding the causal implications of this study’s

findings, researchers can further explore COVID-19’s psy-

chological effects in relation to psychopathology and pandemic

fears; and mental health professionals can examine the effects of

neuroticism when adopting telehealth therapy and other ef-

fective approaches to help address these fears.(26)

Acknowledgements
The authors would like to thank those who participated.

ORCID iD
Mary C. Jobe  https://orcid.org/0000-0001-7106-7120

REFERENCES
1. Panchal N, Kamal R, Orgera K, et al. Mental health and substance abuse. Kaiser

Family Foundation. 2020. https://www.kff.org/coronavirus-covid-19/issue-brief/

the-implications-of-covid-19-for-mental-health-and-substance-use/.

2. Centers for Disease Control and Prevention. The household pulse survey. CDC.

2020. https://www.cdc.gov/nchs/covid19/pulse/mental-health.htm.

3. Lee SA, Crunk EA. Fear and psychopathology during the COVID-19 crisis:

Neuroticism, hypochondriasis, reassurance-seeking, and coronaphobia as fear fac-

tors. Omega J Death Dying 2020;1-14. doi:10.1177/0030222820949350. https://

doi.org/10.1177/0030222820949350.

4. McCrae RR, Costa PT. Validation of the five-factor model of personality across

instruments and observers. J Pers Soc Psychol. 1987;52(1):81-90. doi:10.1037/0022-

3514.52.1.81.

5. Lahey BB. Public health significance of neuroticism. Am Psychol. 2009;64(4):

241-256. doi:10.1037/a0015309.

6. Lung F-W, Lu Y-C, Chang Y-Y, Shu B-C. Mental symptoms in different health

professionals during the SARS Attack: A follow-up study. Psychiatr Q. 2009;80:

107-116.

7. Taylor S. The Psychology of Pandemics: Preparing for the Next Global Outbreak of

Infectious Disease. Newcastle upon Tyne, United Kingdom: Cambridge Scholars

Publishing; 2019.

8. Lee SA, Jobe MC, Mathis AA, Gibbons JA. Incremental validity of coronaphobia:

Coronavirus anxiety explains depression, generalized anxiety, and death anxiety. J

Anxiety Disord. 2020;74:102268. doi:10.1016/j.janxdis.2020.102268.

9. Yildirim M, Güler A. Positivity explains how COVID-19 perceived risk increases

death distress and reduces happiness. Pers Indiv Differ. 2020;168:110347. doi:10.

1016/j.paid.2020.110347.

10. Menzies RE, Menzies RG. Death anxiety in the time of COVID-19: Theoretical

explanations and clinical implications. The Cognitive Behaviour Therapist. 2020;13:

e19. doi:10.1017/S1754470X20000215.

11. Buhrmester MD, Talaifar S, Gosling SD. An evaluation of Amazon’s mechanical

turk, its rapid rise, and its effective use. Perspect Psychol Sci. 2018;13(2):149-154. doi:

10.1177/1745691617706516.

12. Kroenke K, Spitzer RL, Williams JBW, Lowe B. An ultra-brief screening scale for

anxiety and depression: The PHQ-4. Psychosomatics. 2009;50(6):613-621.

13. John OP, Srivastava S. The big five trait taxonomy: History, measurement, and

theoretical perspectives. In: Pervin LA, John OP, eds. Handbook of Personality:

Theory and Research. 2nd ed.. New York, NY: Guilford Press; 1999:102-138.

14. Abdel-Khalek AM. Single- versus multi-item scales in measuring death anxiety.

Death Stud. 1998;22(8):763-772. doi:10.1080/074811898201254.

15. Lee SA. Coronavirus anxiety scale: A brief mental health screener for COVID-19

related anxiety. Death Stud. 2020;44(7):393-401. doi:10.1080/07481187.2020.

1748481.

16. Kenny DA. Mediation; 2016. http://davidakenny.net/cm/mediate.htm.

17. Shrout PE, Bolger N. Mediation in experimental and nonexperimental studies: New

procedures and recommendations. Psychol Methods. 2002;7(4):422-445. doi:10.

1037/1082-989X.7.4.422.

18. Mallinckrodt B, Abraham WT, Wei M, Russell DW. Advances in testing the

statistical significance of mediation effects. J Counsel Psychol. 2006;53(3):372-378.

doi:10.1037/0022-0167.53.3.372.

19. Savolanien I, Oksa R, Savela N, Celuch M, Oksanen A. COVID-19 anxiety—a

longitudinal survey study of psychological and situational risks among Finnish

workers. Int J Environ Res Publ Health. 2021;18(2):794. doi:10.3390/

ijerph18020794.

20. Roelofs J, Huibers M, Peeters F, Arntz A. Effects of neuroticism on depression and

anxiety: Rumination as a possible mediator. Pers Indiv Differ. 2008;44(3):576-586.

doi:10.1016/j.paid.2007.09.019.

21. Choi EPH, Hui BPH, Wan EYF. Depression and anxiety in Hong Kong during

COVID-19. Int J Environ Res Publ Health. 2020;17(10):3740. doi:10.3390/

ijerph17103740.

22. Nikčević AV, Marino C, Kolubinski DC, Leach D, Spada MM. Modelling the

contribution of the Big Five personality traits, health anxiety, and COVID-19

psychological distress to generalised anxiety and depressive symptoms during the

COVID-19 pandemic. J Affect Disord. 2020;279(15):578-584.

23. Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence

of depression symptoms in US adults before and during the COVID-19 pandemic.

JAMA Netw Open. 2020;3(9):e2019686. doi:10.1001/jamanetworkopen.2020.

19686.

24. Salari N, Hosseinian-Far A, Jalali R, et al. Prevalence of stress, anxiety, depression

among the general population during the COVID-19 pandemic: A systematic

review and meta-analysis. Glob Health. 2020;16(57):57. doi:10.1186/s12992-020-

00589-w.

25. Khorsravi M. Neuroticism as a marker of vulnerability to COVID-19 infection.

Psychiatry Investigation. 2020;17(7):710-711. doi:10.30773/pi.2020.0199.

26. Hedman E, Andersson G, Lindefors N, et al. Personality change following internet-

based cognitive behavior therapy for severe health anxiety. PLoS One. 2014;9(12):

e113871. doi:10.1371/journal.pone.0113871.

5Lee and Jobe
n n

https://orcid.org/0000-0001-7106-7120

The Implications of COVID-19 for Mental Health and Substance Use

https://www.cdc.gov/nchs/covid19/pulse/mental-health.htm

https://doi.org/10.1177/0030222820949350

https://doi.org/10.1037/0022-3514.52.1.81

https://doi.org/10.1037/a0015309

https://doi.org/10.1016/j.janxdis.2020.102268

https://doi.org/10.1016/j.paid.2020.110347

https://doi.org/10.1017/S1754470X20000215

https://doi.org/10.1177/1745691617706516

https://doi.org/10.1080/074811898201254

https://doi.org/10.1080/07481187.2020.1748481

http://davidakenny.net/cm/mediate.htm

https://doi.org/10.1037/1082-989X.7.4.422

https://doi.org/10.1037/0022-0167.53.3.372

https://doi.org/10.3390/ijerph18020794

https://doi.org/10.1016/j.paid.2007.09.019

https://doi.org/10.3390/ijerph17103740

https://doi.org/10.1001/jamanetworkopen.2020.19686

https://doi.org/10.1186/s12992-020-00589-w

https://doi.org/10.30773/pi.2020.0199

https://doi.org/10.1371/journal.pone.0113871

Copyright of Clinical Medicine Insights: Psychiatry is the property of Sage Publications Inc.
and its content may not be copied or emailed to multiple sites or posted to a listserv without
the copyright holder’s express written permission. However, users may print, download, or
email articles for individual use.

Does fear mediate the neuroticism-psychopathology link for adults living through the COVID-19 pandemic?

Introduction
Method
Participants and procedures
Measures
Statistical procedures
Results
Correlations
Mediation analyses
Discussion
Acknowledgements
ORCID iD
References

A Dyadic Perspective on Psychopathology and
Young Adult

Physical Dating Aggression

Ann Lantagne and Wyndol Furman
Department of Psychology, University of Denver

Objective: Although psychopathology has been broadly implicated as a risk factor for dating aggression,
very little work has examined the externalizing and internalizing symptoms of both romantic partners to
more fully understand associations between psychopathology and physical dating aggression among young
adult couples. The present study examined the effects of each partner’s psychopathology on physical dating
aggression, the conjoint influence of both partners’ psychopathology, and whether the effects of psycho-
pathology on aggression depended upon the nature of the relationship. Method: Actor Partner Interdepen-
dence Models were used to examine associations between psychopathology and physical dating aggression
among 127 young adult couples (M age = 22.04 years). Actor Partner Interdependence Moderation Models
(APIMoMs) were then tested to determine whether negative relationship characteristics exacerbated the
effects of psychopathology on aggression. Results: Both males’ and females’ externalizing and internaliz-
ing symptoms were associated with dating aggression. Evidence of homophily was found, and actor partner
interactions revealed that couples in which both partners had high externalizing symptoms were at greater
risk, whereas when either partner had low symptoms, the risk was mitigated. Relationship risk factors
interacted with externalizing symptoms to predict female physical dating aggression, and with internalizing
symptoms to predict partner aggression. Conclusion: Findings lend support to the merits of using a dyadic
approach to examine individual risk factors and combinations of individual and relationship risk factors in
predicting young adult physical dating aggression. Results could potentially inform clinical work on
patterns and combinations of risk factors characteristic of high-risk young adult couples.

Keywords: dating aggression, dating violence, intimate partner violence, romantic relationships, dating

Supplemental materials: https://doi.org/10.1037/vio0000386.supp

Rates of physical dating aggression peak among young adult
couples; in fact, more than half of individuals ages 18–24 report that
they have experienced violence in a relationship (Halpern et al.,
2001; O’Leary, 1999). Psychopathology is a known risk factor for
such aggression (Devries et al., 2013). Indeed, elevated externaliz-
ing symptoms, such as conduct problems and delinquency, and
internalizing symptoms, such as depression and anxiety, are associ-
ated with dating aggression during young adulthood (Vezina &
Hebert, 2007). Existing work, however, has primarily focused on
only one individual’s psychopathology and links with aggression
(Capaldi et al., 2012). Notably, each partner’s characteristics shape
the relationship: among young adult couples, both partners’ antiso-
cial behaviors and depressive symptoms are uniquely associated
with male and female physical dating aggression (Kim & Capaldi,
2004). Young adults also tend to select partners with similar levels

of psychopathology, suggesting that certain combinations of part-
ners may be at greater risk (Kim & Capaldi, 2004). By examining
risk among couples, physical dating aggression can be predicted
above and beyond a single individual’s risk, yet limited work has
taken such a dyadic approach during young adulthood.

One of the primary purposes of the present study was to supple-
ment the limited dyadic work examining both partners’ psychopa-
thology and physical dating aggression during this time. By including
both partners’ risk factors, the present study sought to further our
understanding of whether individual risk factors for dating aggression
can be conceptualized as a dyadic process; specifically, we examined
whether partner psychopathology contributed above and beyond the
association between an individual’s psychopathology and their own
dating aggression. We also examined whether partner similarity
(homophily) occurred for psychopathology and explored the conjoint
influence of both romantic partners’ psychopathology on the risk for
aggression. Finally, the present study extended existing research by
testing a dyadic moderation model to investigate whether relationship
risk factors exacerbated associations between psychopathology and
aggression. Results have the potential to further our understanding of
psychopathology as a risk factor for physical dating aggression
among young adult couples.

A Dyadic Approach to Dating Aggression

The dynamic developmental systems perspective (DDS; Capaldi
et al., 2005) provides a dyadic conceptualization of young adult
physical dating aggression across multiple levels of risk factors.

This article was published Online First April 29, 2021.

Ann Lantagne https://orcid.org/0000-0002-9435-7080
Preparation of this manuscript was supported by Grant 050106 from the

National Institute of Mental Health (Wyndol Furman, P.I.) and Grant 049080
from the National Institute of Child Health and Human Development
(Wyndol Furman, P.I.). Appreciation is expressed to the Project Star staff
for their assistance in collecting the data, and to the Project Star participants
and their partners, friends and families.
Correspondence concerning this article should be addressed to Ann

Lantagne, Department of Psychology, University of Denver, Denver,
CO 80209, United States. Email: annlantagne@gmail.com

Psychology of Violence

© 2021 American Psychological Association 2021, Vol. 11, No. 6, 569–579
ISSN: 2152-0828 https://doi.org/10.1037/vio0000386

569

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed

b
y
th
e
A
m
er
ic
an

P
sy
ch
o
lo
g
ic
al

A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed

p
u
b
li
sh
er
s.

T
h
is
ar
ti
cl
e
is
in
te
nd
ed

so
le
ly

fo
r
th
e
p
er
so
n
al

u
se

o
f
th
e
in
di
v
id
u
al

u
se
r
an
d
is
n
o
t
to

b
e
d
is
se
m
in
at
ed

b
ro
ad
ly
.

https://doi.org/10.1037/vio0000386.supp

https://orcid.org/0000-0002-9435-7080

mailto:annlantagne@gmail.com

https://doi.org/10.1037/vio0000386

At the core level of the perspective are the individual characteristics
and behaviors each partner brings to the relationship, such as
psychopathology or personality traits. The next level incorporates
the relationship risk factors associated with aggression, including
characteristics such as conflict, jealousy, attachment styles, and
satisfaction (Capaldi et al., 2012). The DDS perspective emphasizes
that to understand the risk for dating aggression, the effect of both
partners’ risk factors on their own and their partner’s aggression
must be examined. The perspective also posits that interactions
between the individual and relationship levels can occur (Capaldi
et al., 2012; Kim & Capaldi, 2004; Whitaker et al., 2010).
One statistical approach that allows researchers to examine both

partners’ risk factors and the associations with dating aggression is
the Actor Partner Interdependence Model (APIM; Kenny, 1996). In
APIM, actor effects determine how much each individual’s psycho-
pathology influences their own physical dating aggression, and
partner effects reflect associations between an individual’s psycho-
pathology and the partner’s physical dating aggression. Actor
effects are measured while controlling for partner effects, and
vice versa. Actor partner interactions take into account the interplay
between the two romantic partners’ risk and assess whether combi-
nations of partners’ risk factors result in a greater risk (Figure 1).
Finally, additional moderators can be incorporated into APIM as
well (Garcia et al., 2015).

Psychopathology as a Risk Factor

Two domains of psychopathology associated with physical dating
aggression are externalizing and internalizing symptoms. First, a
number of externalizing symptoms, including delinquency, antiso-
cial behaviors, conduct problems, and general aggression are known
risk factors for young adult dating aggression (Andrews et al., 2000;
Ehrensaft et al., 2003). Across dyadic studies, discrepant patterns
have been found. Among adult cohabiting couples, male and female
antisocial behaviors predict female physical aggression but not male
aggression (Marshall et al., 2011). In contrast, among young adult

couples in which male partners were at high risk for delinquency,
male and female antisocial behaviors predict male physical aggres-
sion, whereas only female antisocial behaviors predict female
aggression (Kim & Capaldi, 2004). Second, internalizing symptoms
are associated with greater reactivity, irritability, withdrawal, nega-
tivity, and perceived alienation, which may underlie an increased
risk for aggression (McCabe & Gotlib, 1993). Dyadic studies
including adult couples highlight significant partner effects, such
that males’ depressive symptoms predict female aggression and
females’ depressive symptoms predict male aggression (Marshall
et al., 2011). Among young adult couples, only females’ depres-
sive symptoms predict male and female aggression concurrently
(Kim & Capaldi, 2004).

Moreover, young adults and their partners tend to have similar
levels of psychopathology, a phenomenon known as homophily
(Kim & Capaldi, 2004; Merikangas, 1982). Homophily can occur by
either assortative mating, in which individuals select partners with
similar levels of psychopathology, or socialization, in which part-
ners exert an influence on one another. Such pairing can increase the
number of couples composed of two individuals who each have a
higher risk for dating aggression. Furthermore, interactions between
the partners’ psychopathology can occur, resulting in higher risk for
aggression. Existing work among young adults has found additive
effects of each partner’s psychopathology on aggression but may
have been underpowered to find interactions between partners
(Kim & Capaldi, 2004).

To date, dyadic studies have focused on psychopathology and
physical dating aggression exclusively among adult couples that
were cohabiting or among high-risk young adult couples that had
lasting relationships (Kim & Capaldi, 2004; Marshall et al., 2011).
During young adulthood, dating aggression often culminates in
breakups, on-again-off-again dynamics, and relationship instability
(Halpern-Meekin et al., 2013; Rhoades et al., 2011). It will there-
fore be informative to supplement existing work by examining
patterns across a range of young adult relationships. Additionally,
by testing the conjoint influence of both partners’ psychopathology
in an adequately powered sample, the field will have a better
understanding of whether certain young adult couples may be at
greater risk.

Relationship Characteristics as Moderators

Although psychopathology is a relatively stable characteristic
(Ferdinand et al., 1995), aggression fluctuates within relationships.
Fifteen percent of individuals who did not initially engage in
physical dating aggression became aggressive a year and a half
later within that relationship, whereas around half of individuals
who were aggressive remained aggressive over time (O’Leary et al.,
1989). As such, psychopathology’s associations with dating aggres-
sion may be exacerbated by the presence of contextual variables
such as relationship characteristics, which vary in young adult
relationships (Ferdinand et al., 1995; Karney & Bradbury, 1995).

Notably, the Vulnerability Stress Adaptation model posits that
individuals with vulnerabilities such as psychopathology who also
experience stressful relationships are most likely to have poor
relationship outcomes (Karney & Bradbury, 1995; Marshall et al.,
2011). The intersection of multiple risk factors could cause a tipping
point in which aggression becomes more likely (Foran & O’Leary,
2008). To the best of our knowledge, only one study has examined

Figure 1
The Actor-Partner Interdependence Model (APIM)

Note. Paths labeled a indicate actor effects and paths labeled p indicate
partner effects. Paths label axp reflect actor partner interactions. Double-
headed arrows represented correlated variables. e1 and e2 represent residual
(explained) portion of perpetration score.

570 LANTAGNE AND FURMAN

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

both individual and relationship risk factors. Woodin et al. (2013)
found that lower perceived relationship bond interacted with depres-
sion to predict dating aggression among college couples; however,
existing work has not examined additional relationship risk factors,
explored the interaction with both internalizing and externalizing
symptoms, or used a dyadic approach. By examining the moderating
effect of relationship characteristics, the field can obtain a better
understanding of the roles of both individual and relationship risk
factors and identify potential targets for prevention and intervention.

The Present Study

Existing work has typically examined risk factors for physical
dating aggression by focusing on only one partner; about 5% of
studies have examined risk factors among couples (Capaldi et al.,
2012). In a prior study (Lantagne & Furman, 2020), we used a
dyadic approach to examine the relationship level risk factors from
the Dynamic Developmental Systems theory and found that rela-
tionship characteristics had main effects on dating aggression.
Specifically, our findings demonstrated that both partners’ anxious
relational styles and jealousy predicted male and female aggression;
female satisfaction and female negative interactions predicted male
and female aggression, and each individual’s avoidant style pre-
dicted their partner’s aggression.
The present study used this same dyadic sample as our prior study

to further contribute to the limited dyadic work on risk factors for
dating aggression by first examining a separate level of risk factors
from the Dynamic Developmental Systems theory—the individual
level. We used APIMs to determine associations between each
romantic partner’s externalizing and internalizing symptoms, as
well as interactions between partners’ psychopathology, in predict-
ing young adult physical dating aggression. We then applied an
innovative statistical technique, the Actor Partner Interdependence
Moderation Model (APIMoM) to determine whether interactions
can occur across levels of risk factors. Specifically, we examined the
moderating effect of relationship characteristics on the links
between psychopathology and aggression within couples.
The present study extends our prior work and other existing work

in several important directions. First, a dyadic approach furthers our
understanding of a known risk factor by determining the unique and
conjoint influence of both partners’ psychopathology on young
adult dating aggression. The present study also extends existing
work on psychopathology by examining whether homophily occurs
and whether there are interactions between both partners’ psycho-
pathology, which could place some couples at particularly high risk.
Additionally, by using a dyadic approach to examine individual risk
factors for dating aggression, we determine the utility, and therefore
the generalizability, of such an approach across a variety of risk
factors and multiple levels of risk. Finally, the present study extends
existing dyadic models by incorporating additional variables as
moderators for risk factors for dating aggression. An examination
of such moderation enables us to obtain a more nuanced conceptu-
alization of the role of psychopathology as a risk factor by helping us
to understand whether the effects of psychopathology on dating
aggression have a differing association depending upon the presence
of relationship risk factors. This could lend support to existing
theories that posit that there can be an interplay between levels of
risk factors for aggression.

Hypotheses

Hypotheses included that homophily would occur, such that
individuals would pair with partners who had similar levels of
psychopathology (Hypothesis 1). We hypothesized that higher
physical dating aggression would be associated with higher levels
of an individual’s externalizing symptoms (Hypothesis 2; “a” or
actor path in Figure 1) and their partner’s externalizing symptoms
Hypothesis 3; “p”or partner path in Figure 1). Parallel hypotheses
were posited for internalizing symptoms (Hypothesis 4 & Hypoth-
esis 5). We anticipated female and male psychopathology would
interact to predict physical dating aggression above and beyond the
main effects of each individual’s psychopathology (Hypothesis 6;
“axp” or actor partner interaction path in Figure 1). Moderation
effects were also expected, such that associations between psycho-
pathology and aggression would be strongest when partners expe-
rienced both elevated psychopathology and stressful relationship
characteristics (Hypothesis 7, high psychopathology × high nega-
tive interactions; Hypothesis 8, high psychopathology × high
jealousy; Hypothesis 9, high psychopathology × high anxious
styles; Hypothesis 10, high psychopathology × high avoidant styles;
Hypothesis 11, high psychopathology × low satisfaction, “X1M1”
and “X2M2” paths in Figure 2).

Method

Participants

The participants included in this study and our prior dyadic study
(Lantagne & Furman, 2020) were a dyadic subsample drawn from a
larger longitudinal study examining the role of adolescent and
young adult interpersonal relationships on psychosocial adjustment
(Project STAR; see Furman et al., 2009). Two hundred 10th graders
(100 females) were recruited from a Western metropolitan area in
the United States. To obtain a diverse sample, letters were sent to
families across 36 zip codes of ethnically diverse neighborhoods and
brochures were distributed to students enrolled in three high schools
Interested families were contacted and compensated $25 to hear a
description of the project, with the goal of selecting a quota sample
with a distribution of racial and ethnic groups that approximated that
of the U.S. and had equal rates of males and females. As many
families were contacted who did not have a 10th grader, an
ascertainment rate could not be determined. 85.5% of the families
that heard a description of the project expressed interest and
participated in the Wave 1 assessment. In Wave 1, scores for the
overall sample were compared to comparable national norms of
representative samples for eleven measures of adjustment. Our
sample was more likely to have tried marijuana (54% vs. 40%);
otherwise, sample scores did not differ from national scores on the
other 11 measures, including frequency of marijuana usage (see
Furman et al., 2009).

Procedure and Selection of Dyadic Sample

The dyadic sample from Lantagne & Furman (2020) was used in
the present study. Participants were eligible to invite their romantic
partners to participate in our study if the romantic relationship was
currently three months or longer. 75.7% of the eligible partners
participated (N = 293 of 387). Independent samples t tests were
used to assess for differences between the participants whose partner

PSYCHOPATHOLOGY AND AGGRESSION 571

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

participated and those whose partner did not. Those whose partner
participated reported more committed relationships, t(387) = 4.37,
p = .01 and more dating aggression perpetration and victimization,
t(387) = 2.14, p = .01 & t(387) = 2.90, p = .01, respectively. There
were no differences in internalizing and externalizing symptoms.
The dyadic sample was drawn from the first wave during young

adulthood (Waves 5–8) in which the participant had a romantic
partner complete questionnaires. Data were reorganized into scores
for males and females and only heterosexual dating couples were
included to allow APIM analyses of distinguishable dyads. Ten
LGBQ couples were excluded; additionally, seven married couples
were excluded. The resulting sample consisted of 127 couples
(participants’ M age = 22.04 yrs., range = 19.54—26.87 yrs;
average relationship length = 17.77 months, SD = 16.42); 31.5%
of the dyads were cohabiting/engaged. 74% were White, non-
Hispanic, 7.7% African American, 1.85% Asian American, 12.1%
Hispanic,0.7% Native American, and 3.65% biracial. The local
Institutional Review Board approved the study. U.S. Department
of Health and Human Services Certificates of Confidentiality pro-
tected the confidentiality of the data.

Measure

Psychopathology

Each partner completed 35 externalizing and 39 internalizing
items from the Adult Self-Report (Achenbach, 1997); the external-
izing scale included items about aggressive behavior, rule-breaking,
and intrusiveness, whereas the internalizing scale included items
about anxiety/depression, withdrawal, and somatic complaints.

Participants rated how descriptive each item was on a 3-point scale
ranging from 0 = not true of me to 2 = very true or often true of me.
Higher scores indicated greater symptomatology. The ASR has
acceptable internal reliability and construct validity (mean Cron-
bach’s α= .82 for externalizing & α = .88 for internalizing).

Relationship Characteristics

The following relationship characteristics were included in the
present study as moderators. These characteristics were selected
from the prior dyadic study by Lantagne & Furman (2020) because
they were the theoretically driven measures in our data set that
allowed us to examine the constructs of interest and to permit
comparisons across the two studies.

Negative Interactions. Each partner completed the Network
of Relationships Inventory: Behavioral Systems Version (NRI;
Furman & Buhrmester, 2009). Six items assessed negative inter-
actions in the romantic relationship, including conflict, criticism,
and antagonism (e.g., “How much do you and this person get on
each other’s nerves?”). Ratings were made on a 5-point Likert scale
ranging from 1 = Strongly Disagree to 5 = Strongly Agree. Items
were averaged to create a total score; higher scores represented
greater negative interactions; M Cronbach’s α = .91. Validational
evidence is presented in Furman & Buhrmester (2009).

Relationship Satisfaction. A version of Norton’s (1983) Qual-
ity of Marriage Index was adapted to assess young adult relationship
satisfaction. A sample item was “My romantic partner and I have a
good relationship.” The measure consists of five 7-point Likert items
(1 = Strongly Disagree/Not at all true to 7 = Strongly Agree/Very
true), and one 10-point Likert item that assessed overall satisfaction

Figure 2
Actor Partner Interdependence Moderation Model (APIMoMs) for Interaction Between Externalizing Symptoms and Relationship
Satisfaction on Dating Aggression

Male Physical
Dating Aggression (Y1)

Female Physical
Dating Aggression (Y2)

Male Externalizing
Symptoms (X1)

Female Externalizing
Symptoms (X2)

Male Relationship
Satisfaction (M1)

Female Relationship
Satisfaction (M2)

Female Externalizing x
Female Satisfaction (X2M2)

Male Externalizing x
Male Satisfaction (X1M1)

Note. Double-headed arrows represent correlations; e1 and e2 represent residual portion of dating aggression.

572 LANTAGNE AND FURMAN

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

with relationship (1 = Extremely Unhappy, 10 = Extremely
Happy). Scores were transformed to a 5-point scale so all items
had the same range and averaged to derive a total score, with higher
scores indicating greater satisfaction; M Cronbach’s α = .95. The
measure has acceptable internal reliability and construct validity
(Baxter & Bullis, 1986).
Jealousy. Pfeiffer and Wong’s (1989) Multidimensional Jeal-

ousy Scale assessed emotional, cognitive, and behavioral jealousy.
Each partner used a 5-point Likert scale (1 = Never to 5 = All the
time) to rate 24 items; a sample item was “I suspect my romantic
partner may be attracted to someone else”; M Cronbach’s α = .89.
The scale has good convergent and discriminant validity (Pfeiffer &
Wong, 1989).
Relational Styles. The Behavioral Systems Questionnaire as-

sessed romantic relational styles (BSQ; Furman & Wehner, 1999).
Each partner used a 5-point Likert scale to rate agreement with 27
statements ranging from 1 = Strongly Disagree to 5 = Strongly
Agree. Two scores were calculated based on previous factor analy-
ses (see Jones & Furman, 2011): (a) an avoidant score, consisting of
the average of 9 dismissing items (e.g., I do not ask my romantic
partner to comfort me) and 9 secure items (reverse scored) and (b)
an anxious score, consisting of the average of 9 preoccupied
items (e.g., I am afraid that my romantic partner thinks I am
too dependent). The scales have acceptable internal reliability
(M Cronbach’s α = .89 for avoidant & .84 for anxious styles)
and validity (Furman et al., 2002).

Physical Dating Aggression

Participants reported on their own and their partner’s use of
physical aggression using Simon and Furman’s (2010) adaptation
of Kurdeck’s Conflict Resolution Style Inventory (Kurdeck, 1994).
Four items assessed physical aggression (Forcefully pushing or
shoving, Slapping or hitting, Throwing items that could hurt, and
Kicking, biting or hair pulling). The items were similar in content to
items on the Conflict Tactics Scale (Straus et al., 1996), but used the
CRSI’s 7-point format (ranging from 1=never; 7= always) to assess
how often each partner engaged in such aggression in arguments
during the past year (M α = .82 for perpetration & .91 for victimi-
zation). Higher scores indicated greater frequency of dating aggres-
sion. Both partners’ reports were used to yield male physical dating
aggression (male self-report & partner report of females, r = .56,
p = .001) and female aggression (female self-report & partner
report of males, r = .35, p = .001). This measure has demonstrated
construct validity over several studies (Collibee et al., 2018;
Collibee & Furman, 2016; Novak & Furman, 2016; Simon &
Furman, 2010) and finds similar rates of dating aggression as other
measures (e.g., 46% of adolescents experienced victimization and
52% perpetration; see Novak & Furman, 2016).

Data Analysis Plan

All variables were examined to ensure acceptable skew and
kurtosis (Behrens, 1997), and outliers were Winsorized. Hypotheses
were tested using MPlus 8.0 (Muthén & Muthén, 2012). Per
guidelines around standardizing dyadic data in SEM (Kenny
et al., 2006), predictor and outcome variables were standardized
by grand mean centering across the entire sample and dividing by
the standard deviation across entire sample. APIMoMs were

conducted to determine whether the effects of an individual’s
psychopathology on their own and their partner’s physical dating
aggression is stronger depending on the nature of the relationship.

Power was assessed using APIM Power R (Ackerman & Kenny,
2016). Given medium-size effects (β = .30) and an alpha of .05, the
power for internalizing and externalizing with 127 dyads is .95 and
.96. Power exceeded .80 for β’s of .25 or greater. APIM Power R
does not assess interaction effects, but Kenny & Cook (1999) noted
that the sample size requirements for multiple regression analyses
apply to APIMs estimated via SEM. Thus, we examined the power
of interaction effects using Cohen’s (1988) calculations for multiple
regression. The estimates of power for the interactions were virtually
identical to those for the main effects.

Results

Descriptive statistics and bivariate correlations are presented in
Table 1. 24.4% of couples endorsed male physical perpetration and
34.6% endorsed female perpetration. Rates of female and male
physical dating aggression were highly correlated, r = .75,
p < .001. Regarding clinical cutoffs, 11.8% of our sample were in the clinical range for internalizing symptoms and 9.3% were in the clinical range for externalizing symptoms. Males reported higher levels of externalizing symptoms than females, t(123) = 1.99, p = .04; females reported higher levels of internalizing symptoms than males, t(123) = −3.05, p = .003. Evidence of homophily was found (Hypothesis 1): male and female externalizing symptoms were correlated (r = .28, p < .01) and male and female internaliz- ing symptoms were also correlated (r = .23, p = .02).

Psychopathology

Overall, higher psychopathology was associated with higher
physical dating aggression. Female externalizing symptoms had
an actor effect on female aggression, and male externalizing symp-
toms had an actor effect on male aggression (Hypothesis 2; β = .25,
p < .01 & β = .33, p = .001, respectively). Male externalizing symptoms had a partner effect on female aggression as well (Hypothesis 3; β = .40, p < .001). Additionally, male internalizing symptoms had an actor effect on male aggression, and female internalizing symptoms had an actor effect on female aggression (Hypothesis 4; β = .28, p = .001 & β = .21, p = .03, respectively). Both male and female internalizing symptoms had partner effects (Hypothesis 5; β = .39, p < .001 & β = .16, p = .05, respectively).

Regarding actor partner interactions, the product terms of male by
female externalizing or internalizing symptoms were then exam-
ined. Significant interactions were interpreted using Preacher et al.’s
(2006) computational tools. The estimated effect of one individual’s
psychopathology on their dating aggression was plotted at three
levels of their partner’s psychopathology: low levels (one standard
deviation below the mean), average levels (at the mean), and high
levels (one standard deviation above the mean). Consistent with our
hypothesis (Hypothesis 6), male and female externalizing symptoms
interacted to predict both male (β = .14, p = .04) and female
aggression (β = .25, p < .001; Figure 3). For couples in which females had average or high externalizing symptoms, male aggres- sion increased as male externalizing symptoms increased, t(121) = 2.94, p < .01 for average female externalizing; t(121) = 3.81, p < .001 for high). Similarly, for couples in which males had average

PSYCHOPATHOLOGY AND AGGRESSION 573

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

or high externalizing symptoms, female aggression increased as female
externalizing symptoms increased, t(121) = 3.21, p = .002 for aver-
age male externalizing; t(121) = 4.46, p < .001 for high male exter- nalizing). Contrary to hypotheses, there were no interactions between male and female internalizing symptoms.

Relationship Characteristics as Moderators

Relationship characteristics were then examined as moderators of
associations between psychopathology and dating aggression.

Product interaction terms were created as follows: (1) female
externalizing (or internalizing) symptoms by female relationship
characteristic and (2) male externalizing (or internalizing) symp-
toms by male relationship characteristic (Figure 2). All product
interaction terms were entered after main effects to avoid the
limitations of interpreting conditional main effects (Cohen et al.,
2003; Little, 2013). Main effects of relationship characteristics were
reported in Lantagne and Furman (2020) and thus not presented
here. Due to the number of interaction effects found, only significant
simple slopes are reported in the text; however, nonsignificant
simple slopes are presented as supplemental material.

Female Externalizing Symptoms

Female satisfaction interacted with female externalizing symp-
toms to predict female aggression (see Figure 2, β = −.18, p = .02;
Hypothesis 11), and female jealousy interacted with female exter-
nalizing symptoms to predict female aggression (β = .22, p = .01;
Hypothesis 8). As female externalizing symptoms increased, female
aggression increased for females with low satisfaction, t(115) =
2.76, p = .01. As female externalizing symptoms increased, female
aggression increased for females with high jealousy, t(115) =
2.69, p = .01.

Male Externalizing Symptoms

Male satisfaction interacted with male externalizing symptoms to
predict female aggression (see Figure 2, β = −.22, p = .002;
Hypothesis 11), and male jealousy interacted with male externaliz-
ing symptoms to predict female aggression (β = .18, p = .003;
Hypothesis 8). Male anxious styles and negative interactions inter-
acted with male externalizing symptoms to predict female and
male aggression (β = .16, p = .03 for male anxious styles to
male aggression; β = .26, p = .001 for male anxious styles to
female aggression; β = .17, p = .007 for male negative interactions

Table 1
Means and Standard Deviations of Psychopathology, and Bivariate Correlations of Psychopathology
With Relationship Characteristics and Dating Aggression

Variables 1 2 3 4

Male internalizing symptoms (1) —
Female internalizing symptoms (2) .23* —
Male externalizing symptoms (3) .63** .24** —
Female externalizing symptoms (4) .25** .72** .28** —
Male negative interactions (5) .29** .25** .37** .40**
Female negative interactions (6) .25** .51** .31** .58**
Male satisfaction (7) −.33** −.30** −.27** −.35**
Female satisfaction (8) −.29** −.38** −.21* −.40**
Male jealousy (9) .37** .11 .31** .21*
Female jealousy (10) .18* .44** .26** .49**
Male anxious styles (11) .46** .23* .50** .32**
Female anxious styles (12) .26** .52** .23* .41**
Male avoidant styles (13) .23** .18* .28** .14
Female avoidant styles (14) .11 .15 .14 .15
Male dating aggression (15) .34** .25** .33** .22*
Female dating aggression (16) .38** .27** .46** .37**
Mean (SD in parentheses) 0.28 (0.24) 0.38 (0.26) 0.33 (0.22) 0.28 (0.22)

Note. The correlations were conducted on the measures prior to standardizing them for the APIM analyses. Please
see Lantagne and Furman (2020) for correlations among relationship characteristics and dating aggression.
† p < .10. * p < .05. ** p < .01. *** p < .001.

Figure 3
Actor by Partner Interactions Between Male and Female External-
izing Symptoms on Male Physical Dating Aggression

574 LANTAGNE AND FURMAN

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

https://doi.org/10.1037/vio0000386.supp

to male aggression; β = .25, p < .001 for male negative inter- actions to female aggression; Hypothesis 9 & Hypothesis 7, respectively). First, as male externalizing symptoms increased, female aggres-

sion increased for males with low or average satisfaction (low
t(115) = 5.25, p < .0001 & average t(115) = 4.24, p < .0001). Next, as male externalizing symptoms increased, female aggression increased for males with high or average levels of jealousy (high t(115) = 5.28, p = .0001 & average t(115) = 4.14, p < .0001). For anxious styles, as male externalizing symptoms increased,

female aggression increased for males with high or average anxious
styles, t(115) = 4.21, p = .0001 for high & t(115) = 2.58, p = .01
for average. Similar patterns were found for predicting male aggres-
sion: as male externalizing symptoms increased, male aggression
increased for males with high anxious styles, t(115) = 2.63,
p = .01. Finally, as males’ externalizing symptoms increased,
female aggression also increased for males with high or average
negative interactions, t(115) = 4.95, p <.001 for high & t(115) = 3.30, p < .001 for average. Regarding male physical dating aggres- sion, as males’ externalizing symptoms increased, male aggression increased for males with high negative interactions, t(115) = 2.92, p = .004.

Female Internalizing Symptoms

Female satisfaction and avoidant relational styles interacted with
female internalizing symptoms to predict male dating aggression
(β = −0.17, p = .008 for satisfaction & β = .23, p = .004 for
avoidant styles; Hypothesis 11 & Hypothesis 10 respectively).
As female internalizing symptoms increased, male aggression
increased for females with low satisfaction, t(115) = 2.47,
p = .02. Next, as female internalizing symptoms increased, male
aggression increased for females with high or average avoidant
styles, t(115) = 4.20, p < .001 for high & t(115) = 2.33, p =.02 for average.

Male Internalizing Symptoms

Male satisfaction and anxious relational styles interacted with
male internalizing symptoms to predict female aggression
(β = −.30, p = .001 for satisfaction & β = .20, p = .01 for anxious
relational styles; Hypothesis 11 & Hypothesis 9 respectively).
Male negative interactions interacted with male internalizing symp-
toms to predict female and male aggression (β = .30, p < .001 for female aggression & β = .16, p = .01 for male aggression; Hypothesis 7). First, as female internalizing symptoms increased, female

aggression increased for males with low or average satisfaction,
t(115) = 3.86, p = .001 for low & t(115) = 2.25, p = .03 for
average. Next, as male internalizing symptoms increased, female
aggression increased for males with high anxious styles,
t(115) = 2.34, p = .02. Finally, as male internalizing symptoms
increased, female aggression increased for males with high or
average negative interactions, t(115) = 4.71, p = .001 for high &
t(115) = 2.29, p < .02 for average. Similarly, as male internalizing symptoms increased, male aggression increased for males with high reports of negative interactions, t(115) = 3.12, p = .002.

Discussion

The present study contributes to the limited work examining
associations between psychopathology and dating aggression in
young adult couples. Each partner’s psychopathology was associ-
ated with their own and their partner’s aggression. Males and
females externalizing symptoms interacted to predict both partners’
aggression, placing certain couples at greater risk. Additionally,
relationship characteristics interacted with externalizing symptoms
to predict female aggression, and with internalizing symptoms to
predict partner aggression. Present findings add merit to conceptu-
alizing the risk for young adult dating aggression as dyadic,
depending on both partners’ psychopathology and the nature of
the relationship.

Main Effects of Psychopathology and Actor by Partner
Interactions

One goal of the present study was to extend the limited dyadic
work examining young adult psychopathology and physical dating
aggression. Present findings are largely consistent with existing
work on externalizing symptoms (Kim & Capaldi, 2004) and with
our second and third hypotheses: males’ externalizing symptoms
predicted male aggression and both partners’ externalizing symp-
toms predicted female aggression. Whereas existing work has
shown that only females’ depressive symptoms predict male aggres-
sion (Kim & Capaldi, 2004), present findings were also consistent
with our fourth and fifth hypotheses and indicated that both partners’
internalizing symptoms are associated with male and female aggres-
sion. Prior research has focused on relationships of high-risk young
adult males; the present study may have found different patterns due
to greater variability in psychopathology within our community
sample. Despite differences, overall findings demonstrate that both
partners’ psychopathology are risk factors for male and female
physical dating aggression.

Regarding gender, past work has also found that male internaliz-
ing symptoms are less influential in predicting dating aggression
(Kim & Capaldi, 2004). Notably, the present study demonstrated
that both male and female internalizing symptoms predict each
partner’s aggression. In fact, both partners’ externalizing symptoms
also uniquely predicted male and female aggression, highlighting
the importance of a dyadic approach. The present study may have
uncovered effects for both males’ and females’ psychopathology
because we examined each individual’s risk factor on aggression
while controlling for the partner’s effect.

Moreover, consistent with our sixth hypothesis regarding actor
partner interactions, rates of aggression were highest when homo-
phily occurred and both partners had high levels of externalizing
symptoms. Hostile and aggressive patterns of interacting may be
more prevalent and prolonged for these couples, as both partners
could have lower impulse control and emotion regulation abilities
(Capaldi & Kim, 2007; Keenan-Miller et al., 2007). In contrast,
when one partner had low levels of externalizing symptoms, the
other partner’s level of symptoms was not predictive of aggression.
For such couples, the effects of the partner’s externalizing symp-
toms on aggression were mitigated. Findings are consistent with
existing work, which has found that if an individual pair with a
normative partner, adaptive functioning improves, whereas if an

PSYCHOPATHOLOGY AND AGGRESSION 575

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

individual couple with a deviant partner, maladaptive functioning
continues (Pickles & Rutter, 1991; Quinton et al., 1993).

Interactions Between Psychopathology and Relationship
Characteristics

The present study then examined relationship characteristics as
moderators. Male satisfaction and negative interactions interacted
with externalizing symptoms to predict male aggression. Female
satisfaction and jealousy interacted with externalizing symptoms to
predict female aggression. The risk for aggression is high for those
with externalizing symptoms; when combined with low satisfaction,
high conflict, or high jealousy, it may reach a tipping point where the
impulse exceeds inhibition (Reyes et al., 2015). Additionally, con-
sistent with our hypotheses on the moderating role of relationship
characteristics, male negative interactions, jealousy, anxious styles,
and satisfaction interacted with externalizing symptoms to predict
female aggression (Hypothesis 7, Hypothesis 8, Hypothesis 9, &
Hypothesis 1, respectively). As discussed in our prior dyadic study
of this sample (Lantagne & Furman, 2020), such characteristics may
be indices of relationship insecurity. Relationships in which males
experience both higher insecurity and externalizing behaviors may
cultivate a particularly taxing interpersonal context. Such relation-
ship dynamics may also challenge the use of effective communica-
tion strategies within the dyad (Capaldi et al., 2005). Females have
reported that one of the most pervasive explanations for their own
aggression is to show anger, which may be salient in such relation-
ships (O’Keefe, 1997).
Findings for internalizing symptoms were consistent with exist-

ing literature (Longmore et al., 2014), and showed that internalizing
symptoms predicted partner aggression, an association that was
most pronounced in the presence of negative relationship character-
istics. Female avoidant styles and low satisfaction interacted with
internalizing symptoms to predict male aggression (Hypothesis 10 &
Hypothesis 11, respectively). Male negative interactions, jealousy,
anxious styles, and low levels of satisfaction interacted with
internalizing symptoms to predict female aggression (Hypothesis 7,
Hypothesis 8, Hypothesis 9, & Hypothesis 11, respectively). Inter-
nalizing symptoms may impede an individual’s sense of self-
efficacy and self-worth, increasing the odds of entering or remaining
in an unhealthy relationship (Cleveland et al., 2003; Vezina &
Hebert, 2007). Studies suggest that individuals who feel depressed
often stay in poor relationships to avoid losing an interpersonal
connection (Vicary et al., 1995). Additionally, internalizing symp-
toms can impair interpersonal competence (Longmore et al., 2014;
Stroud et al., 2008). Indeed, individuals with high internalizing
symptoms often demonstrate poor problem-solving in romantic
relationships (Vujeva & Furman, 2011). Individuals with poor
communication skills or problem-solving abilities, who are also
in a romantic relationship with negative characteristics may also
experience greater escalation in conflict, resulting in dating aggres-
sion (Riggs & O’Leary, 1989). Globally, for couples in which one
partner has high internalizing symptoms, romantic interactions are
rated by objective outsiders as displaying greater hostility, irritabil-
ity, negative affect, and negative communication (McCabe &
Gotlib, 1993). Thus, the impact of psychopathology on relationship
interactions, when combined with the presence of a negative
relationship may result in greater risk.

Across internalizing and externalizing symptoms, seven signifi-
cant interactions between relationship characteristics and psycho-
pathology were found for males and four for females. Disinhibitory
psychopathology has been found to be a unique risk factor for males
(Ehrensaft et al., 2003), whereas other factors were uniquely pre-
dictive for females. It may be that for males, the combination of
psychopathology and relationship characteristics culminates in dis-
inhibition, whereas other processes are at play for females. Notably,
across genders, the interplay between characteristics and psychopa-
thology suggests that not all individuals who experience psychopa-
thology are involved in aggressive relationships. Rather, consistent
with a theory of multifinality (Cicchetti & Rogosch, 1996), complex
combinations of male and female psychopathology and relationship
characteristics culminate in both partners’ aggression.

In sum, the interactions between individual and relationship risk
factors highlight the interplay across multiple levels of the dynamic
developmental systems theory (Capaldi et al., 2012). In a previous
study of this same sample (Lantagne & Furman, 2020), we identified
a number of relationship characteristics that were dyadic risk factors;
here we found internalizing and externalizing symptoms were
additional dyadic risk factors. However, the risk for dating aggres-
sion is not simply a linear or additive risk: relationship character-
istics appear to work synergistically with psychopathology (Moffitt
et al., 2001) such that individuals who have high levels of psycho-
pathology and who are in stressful relationships are at greater risk for
aggression. Present findings are consistent with theories on multiple
risk factors, which posit that the presence of any single risk factor for
dating aggression can be exacerbated by the presence of additional
risk factors (Kim & Capaldi, 2004). Finally, our findings also add
merit to conceptualizing risk for dating aggression as dynamic
(Collibee & Furman, 2016). The majority of existing studies
have examined the risk for dating aggression as an invariant and
static factor rather than as a risk that changes over time and across
partners. Notably, the degree of current psychopathology and
relationship characteristics can vary between and within young
adult relationships, underscoring the dynamic nature of risk.

Limitations

The present study had several limitations. First, it was cross-
sectional; longitudinal research examining dyadic models of dating
aggression over time is needed to test the temporal order of the
associations between psychopathology and aggression. Externaliz-
ing and internalizing symptoms also tend to co-occur, and indivi-
duals who experience both often have the poorest overall adjustment
(Capaldi & Stoolmiller, 1999). While participant and partner reports
of dating aggression were included, we only incorporated self-
reports of predictors. Additionally, our measure of dating aggression
assesses a range of conflict tactics but has been used less often in
studies of dating aggression. It will be important to replicate the
actor and partner effects with other measures such as the Conflict
Tactics Scale 2 (Straus et al., 1996).

We conducted a number of statistical analyses and thus, it is likely
that some Type I errors may have occurred. Statisticians have
persuasively argued that corrections for Type I error do not solve
the problem and, in fact, present other problems (Garamszegi, 2006;
Nakagawa, 2004; Rothman, 1990; Saville, 1990). Thomas (1998)
argues that simply describing which tests of significance have been
performed and why is the best way to manage multiple comparisons.

576 LANTAGNE AND FURMAN

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

In that respect, 24 of our 52 tests of actor, partner, and interaction
effects were significant.
Finally, because the present study drew from a community

sample, there is greater generalizability of our findings than
much existing work. However, our sample consisted of early adult
couples, and it will be important to replicate findings across
adolescence and adulthood to determine developmental differences
in associations. Additionally, our study exclusively included het-
erosexual couples. Dating aggression is also prevalent among
LGBQ couples and it will be imperative to extend dyadic ap-
proaches to LGBQ couples.

Research Implications

Findings from the present study lend additional support to the
utility of using a dyadic approach to more fully understand the
complex relationship among risk factors and dating aggression in
young adult couples. Not only were actor and partner effects found,
but actor partner interactions were found. The effects of high levels
of externalizing symptoms were mitigated when the partner had low
levels. Results also demonstrated that the combination of psycho-
pathology and poor relationship characteristics substantially
increased the likelihood of dating aggression. Our findings under-
score that dating aggression should be conceptualized as a multi-
determined behavior (Capaldi et al., 2012; Foran & O’Leary, 2008);
future studies should expand analyses to include an array of pre-
dictors for both male and female dating aggression, as well as the
interplay among variables, to further our understanding of this
complex phenomenon.

Clinical Implications

Present findings imply that there are several critical points of
intervention. One point would be to focus on decreasing psychopa-
thology; studies indicate that individuals with lower levels of
depression are less likely to be victimized (Halpern et al., 2009).
Alternatively, as different patterns of relationship characteristics
moderated internalizing and externalizing symptoms, relationship
prevention programs that target multiple risk factors may be most
effective (Longmore et al., 2014). Finally, findings could shift the
focus of existing prevention programs from individuals to young
couples (Capaldi & Kim, 2007). Programs may be most effective if
targeting both partners’ externalizing symptoms, as present findings
indicate risk for dating aggression is higher for couples in which
both partners have elevated symptoms.

Conclusion

Taken together, the current study makes several notable con-
tributions to the field on young adult physical dating aggression.
Findings highlight that both partners’ psychopathology is risk
factor. The interplay between partners’ externalizing symptoms
underscores that when couples consist of two individuals with
high externalizing symptoms, the risk for dating aggression in-
creases. Results also demonstrate that psychopathology does not
inevitably lead to dating aggression; rather, the co-occurrence of
individual and relationship characteristics shapes risk. In sum, by
considering combinations of risk factors among couples, researchers
may be better able to predict who is at greatest risk for physical

dating aggression, with which partners, and in which relationships
(Collibee & Furman, 2016; Reese-Weber & Johnson, 2013).

References

Achenbach, T. M. (1997). Manual for the Young Adult Self-Report & Young
Adult Behavior Checklist. University of Vermont Department of
Psychiatry.

Ackerman, R. A., & Kenny, D. A. (2016, December). APIMPower: An
interactive tool for Actor-Partner Interdependence Model power analysis
[Computer software]. https://robert-a-ackerman.shinyapps.io/apimpower/

Andrews, J. A., Foster, S. L., Capaldi, D., & Hops, H. (2000). Adolescent
and family predictors of physical aggression, communication, and satis-
faction in young adult couples: A prospective analysis. Journal of
Consulting and Clinical Psychology, 68(2), 195–208. https://doi.org/10
.1037/0022-006X.68.2.195

Baxter, L. A., & Bullis, C. (1986). Turning points in developing romantic
relationships. Human Communication Research, 12(4), 469–493. https://
doi.org/10.1111/j.1468-2958.1986.tb00088.x

Behrens, J. T. (1997). Principles and procedures of exploratory data analysis.
Psychological Methods, 2(2), 131–160. https://doi.org/10.1037/1082-
989X.2.2.131

Capaldi, D. M., & Kim, H. K. (2007). Typological approaches to violence in
couples: A critique and alternative conceptual approach. Clinical Psychol-
ogy Review, 27(3), 253–265. https://doi.org/10.1016/j.cpr.2006.09.001

Capaldi, D. M., Knoble, N. B., Shortt, J. W., & Kim, H. K. (2012). A
systematic review of risk factors for intimate partner violence. Partner
Abuse, 3(2), 231–280. https://doi.org/10.1891/1946-6560.3.2.231

Capaldi, D. M., Shortt, J. W., & Kim, H. K. (2005). A lifespan developmental
systems perspective on aggression toward a partner. In W. M. Pinsof &
J. L. Lebow (Eds.), Family psychology: Art of the science (pp. 141–167).
Oxford University Press.

Capaldi, D. M., & Stoolmiller, M. (1999). Co-occurrence of conduct
problems and depressive symptoms in early adolescent boys: III. Predic-
tion to young-adult adjustment. Development and Psychopathology,
11(1), 59–84. https://doi.org/10.1017/S0954579499001959

Cicchetti, D., & Rogosch, F. A. (1996). Equifinality and multifinality in
developmental psychopathology. Development and Psychopathology,
8(4), 596–600. https://doi.org/10.1017/S0954579400007318

Cleveland, H. H., Herrera, V. M., & Stuewig, J. (2003). Abusive males and
abused females in adolescent relationships: Risk factor similarity and
dissimilarity and the role of relationship seriousness. Journal of Family
Violence, 18, 325–339. https://doi.org/10.1023/A:1026297515314

Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(2nd ed.). Lawrence Erlbaum.

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple
regression/correlation analysis for the behavioral sciences (3rd ed.).
Lawrence Erlbaum.

Collibee, C., & Furman, W. (2016). Chronic and acute relational risk factors
for dating aggression in adolescence and young adulthood. Journal of
Youth and Adolescence, 45, 763–776. https://doi.org/10.1007/s10964-
016-0427-0

Collibee, C., Furman, W., & Shoop, J. (2018). Risky interactions: Relational
and developmental moderators of substance use and dating aggression.
Journal of Youth and Adolescence, 48, 102–113. https://doi.org/10.1007/
s10964-018-0950-2

Devries, K. M., Mak, J. Y., Bacchus, L. J., Child, J. C., Falder, G., Petzold,
M., Astbury, J., & Watts, C. (2013). Intimate partner violence and incident
depressive symptoms and suicide attempts: A systematic review longitu-
dinal studies. PLoS Medicine, 10(5), Article e1001439. https://doi.org/10
.1371/journal.pmed.1001439

Ehrensaft, M. K., Cohen, P., Brown, J., Smailes, E., Chen, H., & Johnson,
J. G. (2003). Intergenerational transmission of partner violence: A 20 year

PSYCHOPATHOLOGY AND AGGRESSION 577

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

https://robert-a-ackerman.shinyapps.io/apimpower/

https://doi.org/10.1037/0022-006X.68.2.195

https://doi.org/10.1111/j.1468-2958.1986.tb00088.x

https://doi.org/10.1037/1082-989X.2.2.131

https://doi.org/10.1016/j.cpr.2006.09.001

https://doi.org/10.1891/1946-6560.3.2.231

https://doi.org/10.1017/S0954579499001959

https://doi.org/10.1017/S0954579400007318

https://doi.org/10.1023/A:1026297515314

https://doi.org/10.1007/s10964-016-0427-0

https://doi.org/10.1007/s10964-018-0950-2

https://doi.org/10.1371/journal.pmed.1001439

prospective study. Journal of Consulting and Clinical Psychology, 71(4),
741–753. https://doi.org/10.1037/0022-006X.71.4.741

Ferdinand, R. F., Verhulst, F. C., & Wiznitzer, M. (1995). Continuity and
change of self-reported problem behaviors from adolescence into young
adulthood. Journal of the American Academy of Child & Adolescent
Psychiatry, 34(5), 680–690. https://doi.org/10.1097/00004583-199505000-
00020

Foran, H. M., & O’Leary, K. D. (2008). Alcohol and intimate partner
violence: A meta-analytic review. Clinical Psychology Review, 28(7),
1222–1234. https://doi.org/10.1016/j.cpr.2008.05.001

Furman, W., & Buhrmester, D. (2009). Methods and measures: The network
of relationships inventory: Behavioral systems version. International
Journal of Behavioral Development, 33(5), 470–478. https://doi.org/10
.1177/0165025409342634

Furman, W., Low, S., & Ho, M. (2009). Romantic experience and psycho-
social adjustment in middle adolescence. Journal of Clinical Child and
Adolescent Psychology, 38(1), 75–90. https://doi.org/10.1080/
15374410802575347

Furman, W., Simon, V. A., Shaffer, L., & Bouchey, H. A. (2002). Adoles-
cents’ working models and styles for relationships with parents, friends,
and romantic partners. Child Development, 73(1), 241–255. https://
doi.org/10.1111/1467-8624.00403

Furman, W., & Wehner, E. A. (1999). The behavioral systems questionnaire-
revised [Unpublished measure]. University of Denver.

Garamszegi, L. Z. (2006). Comparing effect sizes across variables: Gener-
alization without the need for Bonferroni correction. Behavioral Ecology,
17(4), 682–687. https://doi.org/10.1093/beheco/ark005

Garcia, R. L., Kenny, D. A., & Ledermann, T. (2015). Moderation in the
actor-partner interdependence model. Personal Relationships, 22(1), 8–
29. https://doi.org/10.1111/pere.12060

Halpern, C. T., Oslak, S. G., Young, M. L., Martin, S. L., & Kupper, L. L.
(2001). Partner violence among adolescents in opposite-sex romantic
relationships: Findings from the National Longitudinal Study of Adoles-
cent Health. American Journal of Public Health, 91(10), 1679–1685.
https://doi.org/10.2105/AJPH.91.10.1679

Halpern, C. T., Spriggs, A. L., Martin, S. L., & Kupper, L. L. (2009). Patterns
of intimate partner violence victimization from adolescence to young
adulthood in a nationally representative sample. Journal of Adolescent
Health, 45(5), 508–516. https://doi.org/10.1016/j.jadohealth.2009.03.011

Halpern-Meekin, S., Manning, W. D., Giordano, P. C., & Longmore, M. A.
(2013). Relationship churning, physical violence, and verbal abuse in
young adult relationships. Journal of Marriage and Family, 75(1), 2–12.
https://doi.org/10.1111/j.1741-3737.2012.01029.x

Jones, M. C., & Furman, W. (2011). Representations of romantic relation-
ships, romantic experience and sexual behavior in adolescence. Personal
Relationships, 18(1), 144–164. https://doi.org/10.1111/j.1475-6811.2010
.01291.x

Karney, B. R., & Bradbury, T. N. (1995). The longitudinal course of marital
quality and stability: A review of theory, methods, and research. Psycho-
logical Bulletin, 118(1), 3–34. https://doi.org/10.1037/0033-2909.118.1.3

Keenan-Miller, D., Hammen, C., & Brennan, P. (2007). Adolescent psycho-
social risk factors for severe intimate partner violence in young adulthood.
Journal of Consulting and Clinical Psychology, 75(3), 456–463. https://
doi.org/10.1037/0022-006X.75.3.456

Kenny, D. A. (1996). Models of non-interdependence in dyadic research.
Journal of Social and Personal Relationships, 13(2), 279–294. https://
doi.org/10.1177/0265407596132007

Kenny, D. A., & Cook, W. (1999). Partner effects in relationship research:
Conceptual issues, analytic difficulties, and illustrations. Personal
Relationships, 6(4), 433–448. https://doi.org/10.1111/j.1475-6811.1999
.tb00202.x

Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis.
Guilford Press.

Kim, H. K., & Capaldi, D. M. (2004). The association of antisocial behavior
and depressive symptoms between partners and risk for aggression in
romantic relationships. Journal of Family Psychology, 18(1), 82–96.
https://doi.org/10.1037/0893-3200.18.1.82

Kurdeck, L. A. (1994). Conflict resolution styles in gay, lesbian, heterosex-
ual nonparent, and heterosexual parent couples. Journal of Marriage and
the Family, 56(3), 705–722. https://doi.org/10.2307/352880

Lantagne, A., & Furman, W. (2020). More than the sum of two partners: A
dyadic perspective on young adult physical dating aggression. Psychology
of Violence, 10(4), 379–389. https://doi.org/10.1037/vio0000267

Little, T. D. (2013). Longitudinal structural equation modeling. Guilford
Press.

Longmore, M. A., Manning, W. D., Giordano, P. C., & Copp, J. E. (2014).
Intimate partner victimization, poor relationship quality, and depressive
symptoms during young adulthood. Social Science Research, 48, 77–89.
https://doi.org/10.1016/j.ssresearch.2014.05.006

Marshall, A. D., Jones, D. E., & Feinburg, M. (2011). Enduring vulner-
abilities, relationship attributions, and couple conflict: An integrative
model of the occurrence and frequency of intimate partner violence.
Journal of Family Psychology, 25(5), 709–718. https://doi.org/10.1037/
a0025279

McCabe, S. B., & Gotlib, I. H. (1993). Interactions of couples with and
without a depressed spouse: Self-reported and observations of problem-
solving situations. Journal of Social and Personal Relationships, 10(4),
589–599. https://doi.org/10.1177/0265407593104007

Merikangas, K. R. (1982). Assortative mating for psychiatric disorders and
psychological traits. Archives of General Psychiatry, 39(10), 1173–1180.
https://doi.org/10.1001/archpsyc.1982.04290100043007

Moffitt, T. E., Robins, R. W., & Caspi, A. (2001). A couples analysis of
partner abuse with implications for abuse-prevention policy. Criminology &
Public Policy, 1(1), 5–36. https://doi.org/10.1111/j.1745-9133.2001
.tb00075.x

Muthén, L. K., & Muthén, B. O. (2012). Mplus.
Nakagawa, S. (2004). A farewell to Bonferroni: The problems of low
statistical power and publication bias. Behavioral Ecology, 15(6),
1044–1045. https://doi.org/10.1093/beheco/arh107

Norton, R. (1983). Measuring marital quality: A critical look at the depen-
dent variable. Journal of Marriage and the Family, 45(1), 141–151.
https://doi.org/10.2307/351302

Novak, J., & Furman, W. (2016). Partner violence during adolescence and
young adulthood: Individual and relationship level risk factors. Journal of
Youth and Adolescence, 45, 1849–1861. https://doi.org/10.1007/s10964-
016-0484-4

O’Keefe, M. (1997). Predictors of dating violence among high school
students. Journal of Interpersonal Violence, 12(4), 546–568. https://
doi.org/10.1177/088626097012004005

O’Leary, K. D. (1999). Developmental and affective issues in assessing and
treating partner aggression. Clinical Psychology: Science and Practice,
6(4), 400–414. https://doi.org/10.1093/clipsy.6.4.400

O’Leary, K. D., Barling, J., Arias, I., Rosenbaum, A., Malone, J., & Tyree, A.
(1989). Prevalence and stability of physical aggression between spouses:
A longitudinal analysis. Journal of Consulting and Clinical Psychology,
57(2), 263–268. https://doi.org/10.1037/0022-006X.57.2.263

Pfeiffer, S. M., & Wong, P. T. P. (1989). Multidimensional jealousy. Journal
of Social and Personal Relationships, 6(2), 181–196. https://doi.org/10
.1177/026540758900600203

Pickles, A., & Rutter, M. (1991). Statistical and conceptual models
of “turning points” in developmental processes. In D. Magnusson, L.
Bergman, G. Rudinger, & B. Torestad (Eds.), Problems and methods in
longitudinal research: Stability and change (pp. 131–165). Cambridge
University Press.

Preacher, K. J., Curran, P. J., & Bauer, D. J. (2006). Computational tools for
probing interactions in multiple linear regression, multilevel modeling,

578 LANTAGNE AND FURMAN

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

https://doi.org/10.1037/0022-006X.71.4.741

https://doi.org/10.1097/00004583-199505000-00020

https://doi.org/10.1016/j.cpr.2008.05.001

https://doi.org/10.1177/0165025409342634

https://doi.org/10.1080/15374410802575347

https://doi.org/10.1111/1467-8624.00403

https://doi.org/10.1093/beheco/ark005

https://doi.org/10.1111/pere.12060

https://doi.org/10.2105/AJPH.91.10.1679

https://doi.org/10.1016/j.jadohealth.2009.03.011

https://doi.org/10.1111/j.1741-3737.2012.01029.x

https://doi.org/10.1111/j.1475-6811.2010.01291.x

https://doi.org/10.1037/0033-2909.118.1.3

https://doi.org/10.1037/0022-006X.75.3.456

https://doi.org/10.1177/0265407596132007

https://doi.org/10.1111/j.1475-6811.1999.tb00202.x

https://doi.org/10.1037/0893-3200.18.1.82

https://doi.org/10.2307/352880

https://doi.org/10.1037/vio0000267

https://doi.org/10.1016/j.ssresearch.2014.05.006

https://doi.org/10.1037/a0025279

https://doi.org/10.1177/0265407593104007

https://doi.org/10.1001/archpsyc.1982.04290100043007

https://doi.org/10.1111/j.1745-9133.2001.tb00075.x

https://doi.org/10.1093/beheco/arh107

https://doi.org/10.2307/351302

https://doi.org/10.1007/s10964-016-0484-4

https://doi.org/10.1177/088626097012004005

https://doi.org/10.1093/clipsy.6.4.400

https://doi.org/10.1037/0022-006X.57.2.263

https://doi.org/10.1177/026540758900600203

and latent curve analysis. Journal of Educational and Behavioral Statis-
tics, 31(4), 437–448. https://doi.org/10.3102/10769986031004437

Quinton, D., Pickles, A., Maughan, B., & Rutter, M. (1993). Partners, peers,
and pathways: Assortative pairing and continuities in conduct disorder.
Development and Psychopathology, 5(4), 763–783. https://doi.org/10
.1017/S0954579400006271

Reese-Weber, M., & Johnson, A. I. (2013). Examining dating violence from
the relational context. In H. R. Cunningham & W. F. Berry (Eds.),
Handbook on the psychology of violence (pp. 53–76). Nova Publishing.

Reyes, H. L., Foshee, V. A., Tharp, A. T., Ennett, S. T., & Bauer, D. J.
(2015). Substance use and physical dating violence: The role of contextual
moderators. American Journal of Preventive Medicine, 49(3), 467–475.
https://doi.org/10.1016/j.amepre.2015.05.018

Rhoades, G. K., Kamp-Dush, C. M., Atkins, D. C., Stanley, S. M., &
Markman, H. J. (2011). Breaking up is hard to do: The impact of
unmarried relationship dissolution on mental health and life satisfaction.
Journal of Family Psychology, 25(3), 366–374. https://doi.org/10.1037/
a0023627

Riggs, D. S., & O’Leary, K. D. (1989). The development of a model of
courtship aggression. In M. A. Pirog-Good & J. E. Stets (Eds.), Violence in
dating relationships: Emerging social issues (pp. 53–71). Praeger.

Rothman, K. (1990). No adjustments are needed for multiple comparisons.
Epidemiology (Cambridge, Mass.), 1(1), 43–46.

Saville, R. (1990). Multiple comparison procedures: The practical solution.
The American Statistician, 40(2), 174–180.

Simon, V. A., & Furman, W. (2010). Interparental conflict and adolescents’
romantic relationships. Journal of Research on Adolescence, 20(1), 188–
209. https://doi.org/10.1111/j.1532-7795.2009.00635.x

Straus, M. A., Hamby, S. L., Boney-McCoy, S., & Sugarman, D. B. (1996).
The revised conflict tactics scale (CTS2): Development and preliminary
psychometric data. Journal of Family Issues, 17(3), 283–316. https://
doi.org/10.1177/019251396017003001

Stroud, C. B., Davila, J., & Moyer, A. (2008). The relationship between
stress and depression in first onsets versus recurrences: A meta analytic
review. Journal of Abnormal Psychology, 117(1), 206–213. https://
doi.org/10.1037/0021-843X.117.1.206

Thomas, V. P. (1998). What’s wrong with Bonferroni adjustments. British
Medical Journal, 316(1236), 1236–1238. https://doi.org/10.1136/bmj.316
.7139.1236

Vezina, J., & Hebert, M. (2007). Risk factors for victimization in romantic
relationships of young women: A review of empirical studies and im-
plications for prevention. Trauma, Violence & Abuse, 8(1), 33–66. https://
doi.org/10.1177/1524838006297029

Vicary, J. R., Klingaman, L. R., & Harkness, W. L. (1995). Risk
factors associated with date rape and sexual assault of adolescent girls.
Journal of Adolescence, 18(3), 289–306. https://doi.org/10.1006/jado
.1995.1020

Vujeva, H. M., & Furman, W. (2011). Depressive symptoms and romantic
relationship qualities from adolescence through emerging adulthood: A
longitudinal examination of influences. Journal of Clinical Child and
Adolescent Psychology, 40(1), 123–135.

Whitaker, D. J., Le, B., & Niolon, P. H. (2010). Persistence and desistance of
the perpetration of physical aggression across relationships: Findings from
a national study of adolescents. Journal of Interpersonal Violence, 25(4),
591–609. https://doi.org/10.1177/0886260509334402

Woodin, E. M., Caldeira, V., & O’Leary, K. D. (2013). Dating aggression in
emerging adulthood: Interactions between relationship processes and
individual vulnerabilities. Journal of Social and Clinical Psychology,
32(6), 619–650. https://doi.org/10.1521/jscp.2013.32.6.619

Received October 27, 2019
Revision received February 20, 2021

Accepted March 8, 2021 ▪

PSYCHOPATHOLOGY AND AGGRESSION 579

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

https://doi.org/10.3102/10769986031004437

https://doi.org/10.1017/S0954579400006271

https://doi.org/10.1016/j.amepre.2015.05.018

https://doi.org/10.1037/a0023627

https://doi.org/10.1111/j.1532-7795.2009.00635.x

https://doi.org/10.1177/019251396017003001

https://doi.org/10.1037/0021-843X.117.1.206

https://doi.org/10.1136/bmj.316.7139.1236

https://doi.org/10.1177/1524838006297029

https://doi.org/10.1006/jado.1995.1020

https://doi.org/10.1177/0886260509334402

https://doi.org/10.1521/jscp.2013.32.6.619

A Dyadic Perspective on Psychopathology and Young Adult Physical Dating Aggression

Outline placeholder
A Dyadic Approach to Dating Aggression
Psychopathology as a Risk Factor
Relationship Characteristics as Moderators
The Present Study
Hypotheses
Method
Participants
Procedure and Selection of Dyadic Sample
Measure
Psychopathology
Relationship Characteristics
Negative Interactions
Relationship Satisfaction
Jealousy
Relational Styles

Physical Dating Aggression
Data Analysis Plan
Results
Psychopathology
Relationship Characteristics as Moderators
Female Externalizing Symptoms
Male Externalizing Symptoms
Female Internalizing Symptoms
Male Internalizing Symptoms

Discussion
Main Effects of Psychopathology and Actor by Partner Interactions
Interactions Between Psychopathology and Relationship Characteristics
Limitations
Research Implications
Clinical Implications
Conclusion
References

Vol.:(0123456789)

Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-

www.nature.com/scientificreports

On the transience or stability
of subthreshold psychopathology
Marieke J. Schreuder*, Johanna T. W. Wigman, Robin N. Groen, Marieke Wichers &
Catharina A. Hartman

Symptoms of psychopathology lie on a continuum ranging from mental health to psychiatric
disorders. Although much research has focused on progression along this continuum, for most
individuals, subthreshold symptoms do not escalate into full-blown disorders. This study investigated
how the stability of psychopathological symptoms (attractor strength) varies across severity levels
(homebase). Data were retrieved from the TRAILS TRANS-ID study, where 122 at-risk young adults
(mean age 23.6 years old, 57% males) monitored their mental states daily for a period of six months
(± 183 observations per participant). We estimated each individual’s homebase and attractor strength
using generalized additive mixed models. Regression analyses showed no association between
homebases and attractor strengths (linear model: B = 0.02, p = 0.47, R2 < 0.01; polynomial model: B < 0.01, p = 0.61, R2 < 0.01). Sensitivity analyses where we (1) weighed estimates according to their uncertainty and (2) removed individuals with a DSM-5 diagnosis from the analyses did not change this finding. This suggests that stability is similar across severity levels, implying that subthreshold psychopathology may resemble a stable state rather than a transient intermediate between mental health and psychiatric disorder. Our study thus provides additional support for a dimensional view on psychopathology, which implies that symptoms differ in degree rather than kind.

Psychopathology is increasingly recognized as a dimensional phenomenon1–5. From such a dimensional perspec-
tive, psychiatric disorders reflect the extreme end of a severity continuum ranging from the absence of symptoms
to the presence of severe symptoms. Along this continuum lie subthreshold symptoms, which fall short of the
diagnostic criteria for a clinical disorder but may still cause burden and functional impairments3,6,7.

A dimensional view on psychopathology implies that the differences between subthreshold symptoms and
their full-threshold counterparts are quantitative rather than qualitative. This is supported by studies showing
that subthreshold symptoms and full-blown psychiatric disorders have a similar etiology, structure (based on
symptom interrelations8), and treatment response (i.e., phenomenological continuity9). For instance, mild psy-
chiatric traits and disorders share similar genetic risk factors, illustrated by the finding that 80% of the covariance
between subthreshold symptoms and psychiatric disorders is attributable to genetic overlap10. Similarly, the brain
regions associated with subthreshold and clinical manifestations of psychopathology are largely overlapping11.
Environmental risk factors, such as childhood abuse and stressful life events, have also been related to both
sub- and full-threshold expressions of psychopathology6,12 Finally, like psychiatric disorders, subthreshold symp-
toms are associated with distress and declined functioning6,7,12,13, which can improve following psychological
treatment14. In sum, there is substantive evidence that the distinction between subthreshold symptoms and
psychiatric disorders seems to be a matter of degree—e.g., reflected in the number of symptoms and affected
individuals—rather than kind15,16.

Subthreshold symptoms are commonly considered clinically relevant not only because of the above-men-
tioned similarities to psychiatric disorders, but also because of their prognostic significance1. That is, individuals
with subthreshold symptoms are two to five times more likely to develop a psychiatric disorder compared to
individuals without such symptoms1,17,18. This implies that, for some individuals, subthreshold symptoms reflect a
temporary phase between having no symptoms and having a psychiatric disorder. Yet, longitudinal cohort studies
have shown that for the majority of individuals, subthreshold symptoms do not escalate into full-blown disorders.
Specifically, the proportion of individuals with subthreshold symptoms that meet the criteria for a psychiatric
disorder when assessed several years later ranges between 14 and 35% (depression1,2,19), 14–15% (anxiety1,20),
32% (bipolar disorder21), 25% (psychosis22), and 36–38% (substance abuse1). For other individuals, subthresh-
old symptoms may either remit or persist. Such persistence contradicts the common notion that subthreshold
symptoms are transient. Instead, subthreshold symptoms could—at least for some individuals—reflect stable

OPEN

Interdisciplinary Center for Psychopathology and Emotion regulation, Department of Psychiatry, University
Medical Center Groningen, University of Groningen, Hanzeplein, 19713 GZ Groningen, The Netherlands. *email:
m.j.schreuder@umcg.nl

http://crossmark.crossref.org/dialog/?doi=10.1038/s41598-021-02711-3&domain=pdf

Vol:.(1234567890)

Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3

www.nature.com/scientificreports/

states rather than transient transitionary phenomena. This introduces the possibility of yet another qualitative
similarity between subthreshold symptoms and psychiatric disorders: both might be stable phenotypes.

So far, the stability of psychopathological symptoms has mostly been investigated across very short timescales
(e.g., hour-to-hour) and relatively long timescales (e.g., year-to-year). The present study aims to investigate the
day-to-day stability of psychopathological symptoms across six months using a complex systems perspective23–26.
According to this perspective, symptoms might manifest as stable states, for instance labelled as mental health,
subthreshold psychopathology, or psychiatric disorder27–29. These stable states—commonly referred to as attrac-
tors—can be thought of as the set point to which systems tend to return again and again upon perturbations (i.e.,
stressful or pleasant events). In healthy individuals, for instance, events may lead to temporary dips or uplifts in
mood, but eventually, a state of mental health (i.e., their attractor) is restored. Attractors result from regulatory
processes, reflected in interactions between elements of the system (e.g., feedback loops between mental states24).
In the presence of strong regulatory processes, systems are resistant to change. This translates to a strong tendency
to remain in an attractor (e.g., one with low symptom severity). As regulatory processes weaken, transitions from
one attractor to another become more likely. Hence, the stability of an attractor can be inferred from regulatory
processes, known as attractor strength30. Strong attractors (or, attractors with high attractor strength) can be
considered stable and persistent. Weaker attractors, in contrast, are less stable and may therefore quickly disap-
pear. It follows that strong attractors without symptoms of psychopathology can be considered favorable, as they
reflect stable mental health. Strong attractors featured by severe symptoms of psychopathology, in contrast, may
be unfavorable, as they reflect persistent mental ill-health. Finally, weak attractors can be considered transient
conditions that easily disappear.

If subthreshold symptoms indeed reflect a stable attractor that behaves similar to the attractors with low
and high symptom severity, the strengths of these attractors should be similar. This would mean that there is no
clear association between the symptom severity of attractors (referred to as homebases) and attractor strengths
(Fig. 1a). If, on the other hand, subthreshold symptoms reflect more transient phenomena (i.e., temporary states
between low and severe symptoms), there should be a quadratic relation between homebases and attractor
strengths (Fig. 1b). We investigated this hypothesis in an intensive longitudinal study where 122 at-risk young
adults monitored transdiagnostic (subthreshold) symptoms daily for a period of six months. Since subthreshold
symptoms are considered diffuse, representing a mix of symptoms from different psychopathological domains,
we focused on attractors of overall symptom severity, rather than attractors of specific symptom domains31.

Materials and methods
Participants. Participants were recruited from the clinical cohort of an ongoing study, named TRacking
Adolescents’ Individual Lives Survey (TRAILS32). At the time of inclusion in the clinical cohort of TRAILS
(TRAILS-CC), participants were between 10 and 12 years old and had been referred to mental health care
services. Because of this history, they were considered at increased risk to develop mental health problems.
Since their inclusion, participants completed bi- or tri-annual follow-up assessments. When TRAILS-CC par-
ticipants were approximately 23.6 years old (range 21–24), they were invited to take part in an add-on diary

Figure 1. Illustration of the association between homebases and attractor strengths under two different
scenarios. The homebase corresponds to the severity of symptoms that characterize an attractor. (A) If the
subthreshold attractor is comparable to the healthy and disordered attractors in strength, there is no clear
association between homebases and attractor strengths. (B) If the subthreshold attractor is transient, there is a
quadratic relation between homebases and attractor strengths.

3
Vol.:(0123456789)
Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3
www.nature.com/scientificreports/

study (TRAILS TRANS-ID). Of the 443 eligible participants, 134 (30.2%) were included in TRAILS TRANS-ID.
The present study included the 122 individuals who completed the diary period. A more elaborate description
of these participants, as well as other methodological details, has been published elsewhere33. All participants
provided written informed consent. This study was approved by the medical ethics committee of the University
Medical Center Groningen (reference no. 2017/203). All procedures contributing to this work comply with the
ethical standards of the relevant national and institutional committees on human experimentation and with the
Helsinki Declaration of 1975, as revised in 2008.

Procedure. Participants completed daily questionnaires every evening for a period of six consecutive
months. Each questionnaire consisted of 58 items pertaining to the past day (e.g., ‘Today, I felt tired’) that were
rated on a visual analogue scale (VAS) ranging from 0 to 100. These questionnaires, or diaries, were sent via a
text messages to participants’ mobile phones. Prior to and immediately after the diary period (i.e., at baseline and
post), a semi-structured diagnostic interview was orally administered (mini-SCAN). This interview was used to
assess whether individuals met the diagnostic criteria for a DSM-5 disorder (for details on the procedure, please
see33). The post assessment covered the entire diary period, and therefore, this assessment was used for sensitiv-
ity analyses (see “Data analysis” and Supplement).

Data analysis. The diary procedure yielded a maximum of 183 measurements of 58 mental states per indi-
vidual, for 122 individuals (i.e., > 1.2 million observations in total). The data pertaining to the 35 negative mental
states assessed in our study—listed in the Supplement—were selected for analyses. Together, these mental states
were considered reflective of individuals’ overall symptom severity. We estimated overall symptom severity (sx)
for individual i at time t by computing the mean rating across the individual’s negative mental states (ms) at time
t, so that sxi,t = Σmsi,t/35. Subsequently, a generalized additive mixed model (GAMM) was fitted34,35. Specifically,
symptom severity of individual i at time t was predicted by this individual’s (1) intercept, (2) autoregressive
parameter, and (3) non-linear trend in symptom severity over time (for details, see supplementSupplement).
This model yielded an estimated homebase and attractor strength for each individual separately, while taking
into account each individual’s change in symptoms over time. The homebase is given by the person-specific
intercept (which is conceptually similar, but not equal to, the person’s mean), and reflects the symptom sever-
ity that characterizes an individual’s attractor36. As such, relatively low homebases can be considered adaptive,
while higher homebases may be maladaptive. The attractor strength reflects the regulatory forces that maintain
the attractor, and is given by person-specific estimates of the inversed autoregressive parameter (i.e., the effect of
symptom severity at t-1 on symptom severity at t)36. This operationalization of homebases and attractor strengths
has also been adopted in earlier studies36,37, and can be considered a discrete-time translation of the parameters
described in the DynAffect model30 and the PersDyn model38, which are formalized in continuous time.

The relation between homebases and attractor strengths was tested with regression analyses. Specifically, we
compared models where attractor strength was predicted by homebase vs. squared homebase (i.e., polynomial
regression). This allowed for differentiating between the scenarios displayed in Fig. 1. We repeated these regres-
sion analyses in two sensitivity analyses. First, we fitted weighted regressions to account for the uncertainty in
the estimates and attractor strengths. The weights in these models were proportional to the sum of the range of
the confidence intervals around the homebases and attractor strengths. Second, we checked the effect of (co-
morbid) full-blown disorders by omitting individuals who met the criteria for at least one DSM-5 diagnosis
from the analyses. This was done to allow for the possibility that mental states, and the stability thereof, might
have a different meaning for individuals with versus without psychiatric disorders39. By re-running analyses in
individuals without disorders, we could verify whether findings followed from between-individual differences in
e.g., the “threshold” for reporting a certain mental state. Individuals with a DSM-5 diagnosis were selected based
on the mini-SCAN assessed at post, which covered the presence of psychiatric disorders during the entire diary
period. All analyses were performed in R (version 4.0.2) using the package mgcv (version 1.8.33).

Results
Participants (N = 122, 56.6% male) were on average 23.64 years old (SD = 0.67, range = 22.26–24.81) and had on
average completed 163.39 diary assessments (88.6%, SD = 17.12, range = 116–190). At baseline, 37 individuals
(30.33%) met the criteria for at least one DSM-5 diagnosis. After the diary period, 34 individuals (27.87%) had
a DSM-5 disorder, of whom 23 (67.65%) were also diagnosed at baseline. Most prevalent were mood disorders
(n = 24 and 23 at baseline and post, respectively), followed by anxiety disorders (n = 6 and 12) and ADHD (n = 6
and 8).

The fitted values and the distribution of residuals indicated that assumptions of the GAMM were not violated
(see Supplement for details). The GAMM had an adjusted R2 of 77% and yielded homebases that varied between
2.85 and 46.51, with a mean of 17.81 (SD = 9.80). Attractor strengths varied between 1.52 and 28.83 (mean = 4.16,
SD = 3.35). Neither homebases nor attractor attractor strengths were related to the within-person variability in
observations (Supplement 2, GAMM details). Individuals who met criteria for a DSM-5 diagnosis at post had a
higher homebase (mean = 21.57) compared to non-diagnosed individuals (mean = 16.36, t(120) = 2.70, P < 0.01, Cohen’s d = 0.55), but did not differ in terms of attractor strength (mean = 4.26 vs. 4.12, respectively; t(120) = 0.21, P = 0.84, Cohen’s d = 0.04).

Regression analyses indicated that there was no clear association between homebase and attractor strength
(linear model: B = 0.02, P = 0.47, R2 < 0.01; polynomial model: B < 0.01, P = 0.61, R2 < 0.01; Fig. 2). This finding did not change after taking into account the uncertainty in the estimates nor after removing individuals with a DSM-5 diagnosis from the analyses (see Supplement).

Vol:.(1234567890)
Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3
www.nature.com/scientificreports/

Discussion
Symptoms of psychopathology have been proposed to lie on a severity continuum, where the absence of symp-
toms and psychiatric disorders mark the extreme ends. This has been supported by the notion that subthreshold
and full-threshold psychopathological symptoms show a comparable etiology6,9–12 and treatment outcome14. This
study investigated the stability of psychopathological symptoms, i.e., their attraction strength, along the severity
continuum. We found that the stability of symptoms assessed daily over a period of six months is independent of
the severity of symptoms. This provides additional support for a dimensional view on psychopathology, which
implies that subthreshold and full-threshold psychopathological symptoms differ in degree (i.e., severity) rather
than in kind (e.g., stability). In conclusion, just like some individuals may experience constant mental health
or psychopathology, others may get stuck in subthreshold psychopathology. Subthreshold symptoms may thus
resemble stable states, rather than transient conditions that mark the progression from relatively healthy towards
disordered states (or vice versa).

A dimensional view on psychopathology does not necessarily preclude the existence of discrete, stable states
along the severity continuum15,40. Present findings show that such states not only lie on the extreme ends of
the continuum—reflecting mental health and mental disorder—but may just as well occupy the regions in
between these extremes—reflecting subthreshold psychopathology. It follows that the clinical relevance of sub-
threshold symptoms does not just lie in their associated burden3,6,7 and their tendency to precede full-threshold
symptoms1,17,18 (their prognostic significance), but also in their stability. Stability here refers to a property of
an attractor in a complex dynamic system, namely attractor strength. In the context of psychopathology, an
attractor can be considered a set of mental states to which a system tends to return upon perturbations (e.g.,
pleasant/stressful events23,27,29). An attractor has a certain homebase, which may describe mild vs. more severe
psychopathological symptoms, and strength, which reflects the regulatory processes that maintain the attractor.
Relatively strong (stable) attractors with low homebase can be considered adaptive, as they illustrate a healthy
system that is resilient to external perturbations. In contrast, attractors with higher symptom severity may be
maladaptive, illustrated by the current finding that individuals with a DSM-5 disorder had higher homebases
than those without a disorder.

We have shown that maladaptive attractors do not differ from more adaptive attractors in terms of strength.
Yet, previous work has reported that individuals with a psychiatric disorder may have weaker attractors compared
to healthy controls, implying a negative association between attractor strengths and homebases36,37. Similarly,
studies that used alternative measures of stability (i.e., adjusted square of successive differences41–43 and prob-
ability of acute change42,43) found higher instability in patients compared to controls. However, this difference was
likely driven by the standard deviation, meaning that patients and controls may differ primarily in the dispersion
of mental states as opposed to the stability of mental states41,44. An explanation for the discrepancy between earlier

Figure 2. Association between the homebase and attractor strength of symptoms of psychopathology.
Homebases and attractor strengths were estimated from a generalized additive mixed model using six months
of daily diary data from 122 young adults. Individuals who received any DSM-5 diagnosis after the diary period
are printed in blue. The black line shows the association between homebases and attractor strengths based
on a linear model; the grey line shows the fit of a polynomial model. Neither model indicated an association
between homebases and attractor strengths. For illustrative purposes, four outliers (individuals with an attractor
strength of > 10) were omitted from this figure. Including these individuals did not change the results (see the
Supplemented Figure).

Vol.:(0123456789)
Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3
www.nature.com/scientificreports/

and present findings could be that the at-risk youth in the present study are more impaired than the healthy
controls and/or less impaired than the patient samples in former studies36,37, which in turn may have restricted
the ranges in attractor strengths and homebases in the current study. However, the large variability in symptom
severity in the present sample suggests that a restricted range of psychopathological symptom severity is unlikely
to underlie current findings. Instead, the differences across studies concerning attractor strengths potentially
follow from considerable differences in sampling frequency (i.e., assessments with a 1-day interval vs. 15-min/1-
h interval) and duration (i.e., 6 months vs. one to two days): while individuals with psychiatric disorders may
have a lower hour-to-hour stability of emotions compared to non-affected individuals36,37, their day-to-day
stability of mental states may be, as indicated by the current findings, similar. Hence, while stability on a short
timescale could be adaptive—for instance because it signals adequate emotional responsivity to environmental
cues45—the meaning of stability on a longer timescale depends on the homebase that is maintained. Specifically,
stability on a longer timescale can be either adaptive (when it maintains mental health) or maladaptive (when
it maintains sub- or full-threshold psychopathological symptoms). In conclusion, the meaning of psychological
dynamics—such as the stability of mental states—crucially depends on the timescale under consideration. An
important goal for further research is therefore to investigate the timescale at which dynamics such as attractor
strengths are informative of psychopathology.

Besides the timescale of assessments, the dynamics of psychopathological symptoms might be affected by the
type of symptoms under consideration. It could for instance be hypothesized that certain symptom domains (e.g.,
anxiety) are more stable than others (e.g., psychosis, mania46). At present, little is known about such between-
domain differences: it has been reported that panic disorder and major depression show higher homebases (but
similar attractor strengths) compared to borderline, post-traumatic stress and eating disorders37,47, while nega-
tive psychotic symptoms may have a stronger attraction (but similar homebase) compared to positive symptoms
of psychosis48. However, small sample sizes and methodological heterogeneity preclude firm conclusions. To
investigate dissociations between homebases and attractor strengths across clinical stages and psychopathological
domains, future studies should aim to include individuals with a wide range of symptoms of varying severity. The
current study did so by including youth who experienced a widely varying degree of (mental health) problems
and a wide variety of mental states.

It should be noted that although we collected intensive longitudinal data—which allows for addressing within-
individual processes, including changes in homebases or attractor strengths over time within individuals—we
investigated differences in homebases and attractor strengths between individuals. Our approach fits the notion
that the boundaries between mental health, subthreshold psychopathology, and full-threshold psychopathology
are based on differences between rather than within individuals. This can be illustrated as follows: if an individual
consistently experiences more mental health problems than others (i.e., between-person difference), without
ever deviating from their own homebase (i.e., without within-person differences), they can still meet the criteria
for a mental disorder. Conversely, another individual who substantially differs from their own homebase (i.e.,
within-person difference), but not from mentally healthy individuals (i.e., without between-person differences),
will not qualify for a mental disorder. Hence, it makes sense to study subthreshold psychopathology at a between-
individual level, while adjusting for within-person fluctuations in symptoms over time. Nevertheless, it would
be interesting to extend the current work by investigating the within-person association between the severity
and stability of psychopathological symptoms. A second consideration is that, unlike the majority of earlier
studies on subthreshold symptoms, the present study considered attractors on a continuum of symptom sever-
ity, and did not classify individuals into subgroups based on pre-set cut-offs. This is particularly advantageous
given the considerable heterogeneity in definitions of “subthreshold” psychopathology that plagues research
on this topic7,13,49. Arguably, our decision to not categorize came at the cost of an unclear clinical significance
of the homebase estimates, which were based on daily ratings of negative mental states. However, the fact that
individuals with a DSM-5 diagnosis had significantly higher homebases than those without a diagnosis sup-
ports our inferences. Another potential limitation of the current study is that the aggregation of symptoms into
global psychopathology might have obscured domain-specific associations between the homebase and strength
of attractors. However, our operationalization was in line with the notion that subthreshold psychopathology
may not be domain-specific31, and therefore, fitted with our aim to study the dynamics of symptoms of varying
severity. Finally, our estimates of attractor strengths require that the timescale of assessments (daily) matches the
timescale of the process of interest (i.e., strength of attraction, or the speed with which a homebase is restored).
This issue is not specific to the current study, but rather applicable to all intensive longitudinal studies: within-
person dynamics (including homebases and attractor strengths) can only be estimated with sufficient sampling
frequency50. Although the present timescale (daily) is in line with our interest in long-term stability of symp-
toms—as opposed to momentary fluctuations in emotions41,44—further work on the role of timescales in studies
on symptom dynamics is hopefully awaited (for a recent example, see Sperry and Kwapil42).

The lack of an association between homebases and attractor strengths found in the present study implies
that individuals can get stuck anywhere on the severity continuum. Attractors do not, however, eternally persist:
they may change over time, and such changes may involve a shift from subthreshold to full-threshold psycho-
pathological symptoms or vice versa. Future research is needed to establish what triggers such shifts. After a
shift towards a maladaptive attractor (one with a high homebase) has occurred, it is imperative to understand
what maintains the attractor. A complex systems perspective on psychopathology implies that attractors emerge
from interactions between mental states—meaning that individuals with stronger attractors would be expected
to show greater connectivity between mental states24. An alternative avenue for further research concerns the
comparison of attractors of different domains of psychopathology, which could expose how specific domains
progress and persist, and may inform treatment.

Vol:.(1234567890)
Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3
www.nature.com/scientificreports/

Received: 14 July 2021; Accepted: 18 November 2021

References
1. Shankman, S. A. et al. Subthreshold conditions as precursors for full syndrome disorders: A 15-year longitudinal study of multiple

diagnostic classes. J. Child Psychol. Psychiatry 50, 1485–1494 (2009).
2. Fergusson, D. M., Horwood, L. J., Ridder, E. M. & Beautrais, A. L. Subthreshold depression in adolescence and mental health

outcomes in adulthood. Arch. Gen. Psychiatry 62, 66–72 (2005).
3. Keenan, K. et al. Subthreshold symptoms of depression in preadolescent girls are stable and predictive of depressive disorders. J.

Am. Acad. Child Adolesc. Psychiatry 47, 1433–1442 (2008).
4. McGorry, P. D., Hartmann, J. A., Spooner, R. & Nelson, B. Beyond the “at risk mental state” concept: Transitioning to transdiag-

nostic psychiatry. World Psychiatry 17, 133–142 (2018).
5. McGorry, P. D., Hickie, I. B., Yung, A. R., Pantelis, C. & Jackson, H. J. Clinical staging of psychiatric disorders: A heuristic frame-

work for choosing earlier, safer and more effective interventions. Aust. N. Z. J. Psychiatry 40, 616–622 (2006).
6. Wesselhoeft, R., Sørensen, M. J., Heiervang, E. R. & Bilenberg, N. Subthreshold depression in children and adolescents: A systematic

review. J. Affect. Disord. 151, 7–22 (2013).
7. Haller, H., Cramer, H., Lauche, R., Gass, F. & Dobos, G. J. The prevalence and burden of subthreshold generalized anxiety disorder:

A systematic review. BMC Psychiatry 14, 128 (2014).
8. Groen, R. N., Wichers, M., Wigman, J. T. W. & Hartman, C. A. Specificity of psychopathology across levels of severity: A transdi-

agnostic network analysis. Sci. Rep. 9, 1–10 (2019).
9. Markon, K. E., Chmielewski, M. & Miller, C. J. The reliability and validity of discrete and continuous measures of psychopathology:

A quantitative review. Psychol. Bull. 137, 856–879 (2011).
10. Taylor, M. J. et al. Association of genetic risk factors for psychiatric disorders and traits of these disorders in a Swedish population

twin sample. JAMA Psychiat. 76, 280–289 (2019).
11. Besteher, B., Gaser, C. & Nenadić, I. Brain structure and subclinical symptoms: A dimensional perspective of psychopathology in

the depression and anxiety spectrum. Neuropsychobiology 79, 270–283 (2020).
12. Meeks, T. W., Vahia, I. V., Lavretsky, H., Kulkarni, G. & Jeste, D. V. A tune in “a minor” can “b major”: A review of epidemiology,

illness course, and public health implications of subthreshold depression in older adults. J. Affect. Disord. 129, 126–142 (2011).
13. Balázs, J. & Keresztény, Á. Subthreshold attention deficit hyperactivity in children and adolescents: A systematic review. Eur. Child

Adolesc. Psychiatry 23, 393–408 (2014).
14. Cuijpers, P., Smit, F. & van Straten, A. Psychological treatments of subthreshold depression: A meta-analytic review. Acta Psychiatr.

Scand. 115, 434–441 (2007).
15. Meehl, P. E. Factors and taxa, traits and types, differences of degree and differences in kind. J. Pers. 60, 117–174 (1992).
16. Flett, G. L., Vredenburg, K. & Krames, L. The continuity of depression in clinical and nonclinical samples. Psychol. Bull. 121,

395–416 (1997).
17. Lee, Y. Y. et al. The risk of developing major depression among individuals with subthreshold depression: A systematic review and

meta-analysis of longitudinal cohort studies. Psychol. Med. 49, 92–102 (2018).
18. Bertha, E. A. & Balazs, J. Subthreshold depression in adolescence: a systematic review. Eur. J. Child Adolesc. Psychiatry 22, 589–603

(2013).
19. Tuithof, M. et al. Course of subthreshold depression into a depressive disorder and its risk factors. J. Affect. Disord. 241, 206–215

(2018).
20. Bosman, R. C. et al. Prevalence and course of subthreshold anxiety disorder in the general population: A three-year follow-up

study. J. Affect. Disord. 247, 105–113 (2019).
21. Papachristou, E. et al. The predictive value of childhood subthreshold manic symptoms for adolescent and adult psychiatric out-

comes. J. Affect. Disord. 212, 86–92 (2017).
22. Kaymaz, N. et al. Do subthreshold psychotic experiences predict clinical outcomes in unselected non-help-seeking population-

based samples? A systematic review and meta-analysis, enriched with new results. Psychol. Med. 42, 2239–2253 (2012).
23. Wichers, M., Schreuder, M. J., Goekoop, R. & Groen, R. N. Can we predict the direction of sudden shifts in symptoms? Transdi-

agnostic implications from a complex systems perspective on psychopathology. Psychol. Med. 49, 380–387 (2019).
24. Cramer, A. O. J. et al. Major depression as a complex dynamic system. PLoS ONE 11, 1–20 (2016).
25. Olthof, M. et al. Critical fluctuations as an early-warning signal for sudden gains and losses in patients receiving psychotherapy

for mood disorders. Clin. Psychol. Sci. https:// doi. org/ 10. 1177/ 21677 02619 865969 (2019).
26. Schiepek, G. Complexity and nonlinear dynamics in psychotherapy. Eur. Rev. 17, 331–356 (2009).
27. Jeronimus, B. F. Dynamic system perspectives on anxiety and depression. In Psychosocial Development in Adolescence: Insights

from the Dynamic Systems Approach (eds Kunnen, E. S. et al.) (Routledge, 2019). https:// doi. org/ 10. 1128/ MCB. 01405- 08.
28. Shapiro, Y. & Scott, J. R. Dynamical systems therapy (DST): Complex adaptive systems in psychiatry and psychotherapy. In Hand-

book of Research Methods in Complexity Science (eds Mitleton-Kelly, E. et al.) (Edward Elgar Publishing Limited, 2018).
29. Hayes, A. M. & Andrews, L. A. A complex systems approach to the study of change in psychotherapy. BMC Med. 18, 1–13 (2020).
30. Kuppens, P., Oravecz, Z. & Tuerlinckx, F. Feelings change: Accounting for individual differences in the temporal dynamics of affect.

J. Pers. Soc. Psychol. 99, 1042–1060 (2010).
31. van Os, J. The dynamics of subthreshold psychopathology: Implications for diagnosis and treatment. Am. J. Psychiatry 170, 695–698

(2013).
32. Huisman, M. et al. Cohort profile: The dutch “tracking adolescents” individual lives’ survey’; TRAILS. Int. J. Epidemiol. 37, 1227–

1235 (2008).
33. Schreuder, M. J., Groen, R. N., Wigman, J. T. W., Hartman, C. A. & Wichers, M. Measuring psychopathology as it unfolds in daily

life: Addressing key assumptions of intensive longitudinal methods in the TRAILS TRANS-ID study. BMC Psychiatry 20, 1–14
(2020).

34. Wood, S. N. Generalized Additive Models: An Introduction with R (Springer, 2006).
35. Hastie, T. & Tibshirani, R. Generalized additive models. Stat. Sci. 1, 297–318 (1986).
36. Ebner-Priemer, U. W. et al. Unraveling affective dysregulation in borderline personality disorder: A theoretical model and empirical

evidence. J. Abnorm. Psychol. 124, 186–198 (2015).
37. Santangelo, P. S. et al. Analyzing subcomponents of affective dysregulation in borderline personality disorder in comparison to

other clinical groups using multiple e-diary datasets. Borderline Pers. Disord. Emotion Dysregul. 3, 1–13 (2016).
38. Sosnowska, J., Kuppens, P., de Fruyt, F. & Hofmans, J. A dynamic systems approach to personality: The personality dynamics

(PersDyn) model. Pers. Individ. Differ. 144, 11–18 (2019).
39. Helmchen, H. & Linden, M. Subthreshold disorders in psychiatry: Clinical reality, methodological artifact, and the double-

threshold problem. Compr. Psychiatry 41, 1–7 (2000).
40. Linscott, R. J. & van Os, J. Systematic reviews of categorical versus continuum models in psychosis: Evidence for discontinuous

subpopulations underlying a psychometric continuum. Implications for DSM-V, DSM-VI, and DSM-VII. Annu. Rev. Clin. Psychol.
6, 391–419 (2010).

https://doi.org/10.1177/2167702619865969

https://doi.org/10.1128/MCB.01405-08

Vol.:(0123456789)
Scientific Reports | (2021) 11:23306 | https://doi.org/10.1038/s41598-021-02711-3
www.nature.com/scientificreports/

41. Koval, P., Pe, M. L., Meers, K. & Kuppens, P. Affect dynamics in relation to depressive symptoms: variable, unstable or inert?.
Emotion 13, 1132–1142 (2013).

42. Sperry, S. H. & Kwapil, T. R. Bipolar spectrum psychopathology is associated with altered emotion dynamics across multiple
timescales. Emotion https:// doi. org/ 10. 1037/ emo00 00759 (2020).

43. Trull, T. J., Lane, S. P., Koval, P. & Ebner-Priemer, U. W. Affective dynamics in psychopathology. Emot. Rev. 7, 355–361 (2015).
44. Bos, E. H., de Jonge, P. & Cox, R. F. A. Affective variability in depression: Revisiting the inertia–instability paradox. Br. J. Psychol.

https:// doi. org/ 10. 1111/ bjop. 12372 (2018).
45. Kashdan, T. B. & Rottenberg, J. Psychological flexibility as a fundamental aspect of health. Clin. Psychol. Rev. 30, 865–878 (2010).
46. Sperry, S. H., Walsh, M. A. & Kwapil, T. R. Emotion dynamics concurrently and prospectively predict mood psychopathology. J.

Affect. Disord. 261, 67–75 (2020).
47. Heller, A. S., Davidson, R. J. & Fox, A. S. Parsing affective dynamics to identify risk for mood and anxiety disorders. Emotion 19,

283–292 (2019).
48. Westermann, S. et al. Untangling the complex relationships between symptoms of schizophrenia and emotion dynamics in daily

life: Findings from an experience sampling pilot study. Psychiatry Res. 257, 514–518 (2017).
49. Cuijpers, P. & Smit, F. Subthreshold depression as a risk indicator for major depressive disorder: A systematic review of prospective

studies. Acta Psychiatr. Scand. 109, 325–331 (2004).
50. Haslbeck, J. M. B. & Ryan, O. Recovering within-person dynamics from psychological time series. Multivariate Behav. Res. 56,

1–32 (2021).

Acknowledgements
We thank everyone who contributed to this study.

Author contributions
M.J.S. and R.N.G. collected data. M.J.S. performed analyses and interpreted results together with J.T.W.W. and
C.A.H. M.J.S. drafted the manuscript. M.W. collected funding. All authors revised the manuscript and approved
of the final version.

Funding
The infrastructure for the TRacking Adolescents’ Individual Lives Survey (TRAILS) is funded by the Nether-
lands Organization for Scientific Research (NWO), ZonMW, GB-MaGW, the Dutch Ministry of Justice, the
European Science Foundation, the European Research Council, BBMRI-NL, and the participating universities.
Additionally, this research was supported by the Netherlands Organization for Scientifc Research (NWO) (R.N.
Groen, research talent Grant Number 406.16.507 and J.T.W. Wigman Veni Grant Number 016.156.019), and the
European Research Council (ERC) under the European Union’s Horizon 2020 research and innovative program
(M. Wichers, Grant Number 681466).

Competing interests
The authors declare no competing interests.

Additional information
Supplementary Information The online version contains supplementary material available at https:// doi. org/
10. 1038/ s41598- 021- 02711-3.

Correspondence and requests for materials should be addressed to M.J.S.

Reprints and permissions information is available at www.nature.com/reprints.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International
License, which permits use, sharing, adaptation, distribution and reproduction in any medium or

format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons licence, and indicate if changes were made. The images or other third party material in this
article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the
material. If material is not included in the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.

https://doi.org/10.1037/emo0000759

https://doi.org/10.1111/bjop.12372

https://doi.org/10.1038/s41598-021-02711-3

www.nature.com/reprints

http://creativecommons.org/licenses/by/4.0/

On the transience or stability of subthreshold psychopathology

Materials and methods
Participants.
Procedure.
Data analysis.
Results
Discussion
References
Acknowledgements

https://doi.org/10.1177/09593543211021157

Theory & Psychology
2021, Vol. 31(6) 842 –866

Article

reuse guidelines:

sagepub.com/journals-permissions
DOI: 10.1177/09593543211021157

journals.sagepub.com/home/tap

Describing disorder: The
importance and advancement
of compositional explanations
in psychopathology

Hannah Hawkins-Elder and Tony Ward
Victoria University of Wellington

Abstract
Understanding the makeup of mental disorders has great value for both research and practice
in psychopathology. The richer and more detailed our compositional explanations of mental
disorder—that is, comprehensive accounts of client signs and symptoms—the more information
we have to inform etiological explanations, classification schemes, clinical assessment, and
treatment. However, at present, no explicit compositional explanations of psychopathology
have been developed and the existing descriptive accounts that could conceivably fill this role—
DSM/ICD syndromes, transdiagnostic and dimensional approaches, symptom network models,
historical accounts, case narratives, and the Research Domain Criteria (RDoC)—fall short in
critical ways. In this article, we discuss what compositional explanations are, their role in scientific
inquiry, and their importance for psychopathology research and practice. We then explain why
current descriptive accounts of mental disorder fall short of providing such an explanation and
demonstrate how effective compositional explanations could be constructed.

Keywords
composition, description, explanation, symptoms, theory

Mental disorder represents a serious and expanding global health problem; demonstrat-
ing high prevalence internationally (Steel et al., 2014) and accounting for a considerable
proportion of global disease burden (Whiteford et al., 2015). Being able to assess and
treat these problems effectively is therefore critically important. Researchers and clini-
cians also have an ethical obligation to seek accurate understandings of mental disorders,
so as not to provide mistaken accounts of people’s genuine concerns. Both goals rely

Corresponding author:
Hannah Hawkins-Elder, Victoria University of Wellington, Kelburn Parade, Wellington, 6140, New Zealand.
Email: hannah.hawkinselder@vuw.ac.nz

1021157TAP0010.1177/09593543211021157Theory & PsychologyHawkins-Elder and Ward
research-article2021

Article

https://uk.sagepub.com/en-gb/journals-permissions

https://journals.sagepub.com/home/tap

mailto:hannah.hawkinselder@vuw.ac.nz

http://crossmark.crossref.org/dialog/?doi=10.1177%2F09593543211021157&domain=pdf&date_stamp=2021-06-09

Hawkins-Elder and Ward 843

heavily on the explanations we have for mental disorders: our theoretical accounts of
how they originated and why they persist.

Explanations are invaluable: they tell us what to look for in assessment, what to target
in treatment, and how to do so successfully. At present, however, psychopathological
explanations are often unsound: constructed in idiosyncratic ways and containing numer-
ous conceptual flaws (see Hawkins-Elder & Ward, 2020a, 2020b,). The comprehensive
description of explanatory targets—known as compositional explanation—is signifi-
cantly underemphasized, and often neglected, in psychopathology (Gillett, in press).
Describing the nature of psychopathological symptoms and signs holds significant value
for explanation, classification, research, and practice (Wilshire et al., in press). Despite
this, no explicit compositional explanations of psychopathological phenomena have
been created and existing descriptive accounts present relatively poor alternatives; lack-
ing sufficient depth, theoretical structure, or evidence base.

Current diagnostic syndromes provide only a short list of vaguely defined “core”
features (e.g., DSM-5, American Psychiatric Association, 2013; ICD-11, World Health
Organization, 2019), thereby limiting the richness of information theorists possess
about the disorder. Alternative classificatory perspectives, such as transdiagnostic,
dimensional, and symptom network models, although in some cases providing greater
taxonomic validity, still fail to richly describe psychopathological problems at all the
relevant levels. In contrast, more “clinical” descriptions, such as historical accounts
(i.e., the first identification or early scientific conceptualizations of a disorder) and
clinical case narratives (e.g., case studies) tend to describe disorders in more depth, but
lack detail and theoretical organization and are often empirically outdated. The
Research Domain Criteria (RDoC; Cuthbert & Insel, 2013) is perhaps the current
option that most closely approximates a compositional explanation. However, criti-
cally, this framework is not geared towards the theoretical conceptualization of psy-
chopathological phenomena and therefore, although excellent at guiding empirical
investigation into psychopathology, is unable to provide coherent compositional expla-
nations—at least, not on its own.

Our intentions with this article are therefore threefold. First, to highlight the impor-
tance of compositional explanations of psychopathology by outlining their role in scien-
tific inquiry and clinical practice. Second, to demonstrate that none of our current
descriptive accounts (i.e., DSM/ICD syndromes, transdiagnostic and dimensional classi-
fications, symptom network models, historical accounts, case narratives, and the RDoC)
are capable of successfully acting as compositional explanations. Finally, based on this
discussion, to suggest how we could construct effective compositional explanations of
psychopathology.

Compositional explanations: What they are and what they do

In this section, we discuss the nature of compositional explanations and their role in
scientific inquiry and clinical practice. However, we first must clarify what we mean by
“explanation.”

844 Theory & Psychology 31(6)

What is an “explanation”?

There is some disagreement in science over what exactly constitutes an explanation, as
well as how exactly explanations relate to models and theories. Trout (2016) provides a
simple and helpful definition of explanation: “an explanation is the description of under-
lying causal factors that bring about an effect” (p. 18). More specifically, an explanation
is an account that provides an understanding of a phenomenon’s causes, composition,
context, or consequences (Faye, 2014; Ruphy, 2016). Models and theories, in contrast,
are conceptual representations of phenomena in the world: tools that can be put to the
task of explanation (Mantzavinos, 2016; Savulescu et al., 2020).

For our purposes, a theory is an integrated system of concepts and ideas that can be
used to explain why some phenomena occur and persist. Theories are more general than
models, usually detailing more abstract phenomena (e.g., “human behaviour” in general
vs. specific types or instances) and able to explain different subsets of phenomena
(Bailer-Jones, 2003). Models, in contrast, represent more concrete empirical phenomena
(i.e., identified factors, systems, or processes such as “clinical depression” or “binge eat-
ing”); typically, in idealized and simplified ways (Bailer-Jones, 2003; Haig, 2014).
Theories may inform the development of models, and models may represent localized
applications of theories (Bailer-Jones, 2003). For example, the coercion cycle model
represents the localized application of operant conditioning theory to the phenomenon of
child conduct problems (Dishion & Patterson, 2015).

We define an explanation herein as the entire bank of explanatory knowledge about a
phenomenon—that is, the most complete explanatory account (Craver & Kaplan, 2020).
A model or theory may serve as an explanation if it represents the entire bank of knowl-
edge about a phenomenon: when it adequately represents all aspects of the phenomenon
without being overly complicated or sacrificing critical detail. However, this is rarely the
case. More often, models and theories represent partial explanations (Bailer-Jones,
2003). When the phenomenon of interest is more complex, multiple theories and models
will typically be needed to fully explain it (Kendler et al., 2020).

What are compositional explanations?

Philosophers of science make a distinction between causal and compositional explana-
tions (e.g., Craver, 2007; Kaiser & Krickel, 2016). A causal explanation depicts the fac-
tors that result in a subsequent effect—for example, heating water (a cause) until it boils
(an effect). A psychopathological example is the proposition that traumatic experience
can cause individuals to experience intrusive memories, elevated arousal, and avoidance
behaviour (i.e., a “posttraumatic stress” syndrome). Causal explanations in psychopa-
thology may also include accounts of relations that maintain the disorder, such as mutual
reinforcement between symptoms (e.g., insomnia and low mood; Konjarski et al., 2018)
or behavioural reinforcement “cycles” (e.g., the coercion cycle; Dishion & Patterson,
2015).

In contrast, a compositional explanation describes underlying structures and interac-
tions that make up a phenomenon; viewed as part of it rather than “causing” it (Craver,
2007; Gillett, in press; Kendler et al., 2020). For example, the symptom “low mood” is

Hawkins-Elder and Ward 845

likely to be composed of processes at the phenomenological, subpersonal, neurobiologi-
cal, and physiological levels (Ward & Clack, 2019). It is important to note that composi-
tional explanations may, in some cases, contain causal relations, depending on the
phenomenon being explained. For example, a compositional explanation of a syndrome,
like clinical depression, would necessarily include description of causal relations between
symptoms (e.g., low mood and insomnia), as these relationships are part of the constitu-
tion of that syndrome (although arguably do not cause it). However, these same relations
could form part of an etiological explanation depending on the focus of inquiry. For
instance, if, instead of describing the composition of depression, we were trying to
explain the development and persistence of low mood in depressed individuals, we might
ascribe causal or etiological significance to insomnia somewhere within that explana-
tion, but we would not say that insomnia in any way constituted low mood. Hence, the
role of factors and processes within an explanation varies depending on the question
being asked.

What is the role of compositional explanation in theory, research, and
practice?

Compositional explanations play a critical role in all aspects of clinical inquiry. First,
they hold significant value for etiological explanation. The more detailed our composi-
tional understanding of a phenomenon, the more information we have to provide clues
about its etiology (Hawkins-Elder & Ward, 2020b). For example, if we were trying to
explain the existence of a cake—knowing only that it was a cake—we could reason that
its etiology probably involved components and processes common to most cakes (e.g.,
flour, sugar, being baked). However, with further detail about how it is composed—for
example, chocolate sponge, cream-filled—we have additional clues to help refine our
etiological reasoning; strengthening ideas about the involvement of some factors and
processes (e.g., flour, being baked) and suggesting new ones (e.g., whipping cream,
cocoa). When our compositional understandings are “thin” (less detailed) it can promote
errors in causal reasoning: relevant causal factors and processes may be neglected or
deliberately omitted, and flawed or irrelevant ones may be included. For example, we
could develop multiple etiological theories about our cake, hypothesizing various baking
processes, when all the while we were dealing with an ice cream cake—a type of cake
indeed, but one involving none of our postulated causal processes.

Compositional explanations also hold value for research and practice. Their primary
value for research is via classification. Although compositional explanations are attempts
to describe disorder phenomena, they do not claim to know the best method of classify-
ing the psychopathological phenomena with which they are concerned. However,
because of their informational value, compositional explanations are highly useful for
those aiming to develop taxonomies of mental disorder (Wilshire et al., in press): their
rich descriptions may help to signal connections between syndromes or symptoms, and
thereby suggest novel and improved ways of organizing them. Compositional explana-
tions lay out the psychopathological landscape in comprehensive detail, allowing it to be
thoroughly surveyed by those who wish to classify it.

846 Theory & Psychology 31(6)

In clinical practice, compositional explanations hold value for both assessment and
treatment. For one, their value for etiological explanations and classification systems has
flow-on effects for assessment and treatment. Improvement of classification systems will
likely be beneficial for diagnosis and the prescription of appropriate clinical interven-
tions. Likewise, better etiological explanations will likely improve clinical formulations
and intervention strategies based on them. Compositional explanations also hold inde-
pendent value for clinical practice. Possessing more information about a psychopatho-
logical problem provides us with more features to look for in assessment. It may also
help identify potential therapeutic issues. For example, knowing that a particular disor-
der often involves cognitive inflexibility or poor attentional control might contraindicate
interventions requiring high levels of cognitive effort. Likewise, knowing that a particu-
lar symptom involves difficulty sensing physical sensations may influence how an inter-
vention is administered (e.g., devoting extra time during a mindfulness intervention to
helping the client identify physical sensations).

Current “approximate” compositional explanations

There are several types of account within the psychopathology space that, due to their
descriptive nature, could potentially serve as compositional explanations: namely, (a)
DSM-5/ICD-11, (b) transdiagnostic and dimensional approaches, (c) symptom network
models, (d) historical accounts, (e) clinical case narratives, and (f) the Research Domain
Criteria (RDoC) framework. We will now address each in turn and explain why, on our
view, none effectively serve as compositional explanations.

Diagnostic syndromes: DSM-5 and ICD-11. The DSM-5 (American Psychiatric Associa-
tion, 2013) and ICD-11 (World Health Organization, 2019) are perhaps the most promi-
nent attempts to conceptualize and describe mental disorders. Both group disorders into
discrete syndromes (collections of symptoms and signs) comprising a set of diagnostic
criteria (e.g., borderline personality disorder, anorexia nervosa). They are frequently
used as compositional explanations in research and theory: empirical inquiry is often
oriented around DSM-5 categories and etiological models commonly use them as the
foundation for explanation. However, these syndromes fall short of providing a compo-
sitional explanation in two important ways.

First, DSM/ICD syndromes lack explanatory scope. Each is characterized by a rela-
tively small number of descriptively “thin” criteria spanning but a few levels of analysis
(e.g., behavioural, cognitive, emotional). For example, the criteria for anorexia nervosa
(listed in Table 1) describe only a few features, despite research identifying many others
common to these individuals, such as alexithymia (Nowakowski et al., 2013; Westwood
et al., 2017), interoceptive deficits (Stinson, 2019), cognitive deficits (Hedges et al.,
2019), and autistic traits (Westwood et al., 2016). Furthermore, both the DSM and ICD
outline only those features that are most clinically salient—that is, those most readily
observable in practice or perceived as “most central” to the disorder’s pathology.
Although appropriate and often useful in practice, this omits other relevant features that
may be harder to identify (e.g., emotional comprehension, executive functioning,

Hawkins-Elder and Ward 847

interoceptive ability) or have less apparent relevance (e.g., attentional bias, central coher-
ence), but are nonetheless characteristic of the disorder.

Second, DSM/ICD syndromes lack explanatory depth, as the features/symptoms
listed by them are typically thinly described. For instance, “disturbance in the way in
which one’s body weight or shape is experienced” (American Psychiatric Association,
2013, p. 339) is a necessary criterion for anorexia nervosa (see Table 1), however there
is no detail about the exact nature of this “disturbance.” For example, is it a distortion in
sensory perception or cognitive evaluation (Mölbert et al., 2017)? Does it encompass the
body in general or does it tend to be focused on specific areas (Cash & Deagle, 1997)?
Body image is recognized to be a “multi-faceted construct consisting of a variety of
measured dimensions” (Thompson, 2004, p. 8), including perceptual, conceptual, and

Table 1. DSM-5 diagnostic criteria for anorexia nervosa and bulimia nervosa.

DSM criteria: Anorexia Nervosa

A. Restriction of energy intake relative to requirements, leading to a significantly low
body weight in the context of age, sex, developmental trajectory, and physical health.
Significantly low weight is defined as a weight that is less than minimally normal or, for
children and adolescents, less than that minimally expected.

B. Intense fear of gaining weight or of becoming fat, or persistent behaviour that interferes
with weight gain, even though at a significantly low weight.

C. Disturbance in the way in which one’s body weight or shape is experienced, undue
influence of body weight or shape on self-evaluation, or persistent lack of recognition of
the seriousness of the current low body weight.

Specifiers:
Restricting type: during the last 3 months, the individual has not engaged in recurrent episodes
of binge eating or purging behaviour (i.e., self-induced vomiting or the misuse of laxatives,
diuretics, or enemas). This subtype describes presentations in which weight loss is accomplished
primarily through dieting, fasting, and/or excessive exercise.
Binge-eating/purging type: during the last 3 months, the individual has engaged in recurrent
episodes of binge eating or purging behaviour (i.e., self-induced vomiting or the misuse of
laxatives, diuretics, or enemas).
DSM criteria: Bulimia Nervosa
A. Recurrent episodes of binge eating. An episode of binge eating is characterized by both of

the following:
1. Eating, in a discrete period of time (e.g., within any 2-hour period), an amount of food that

is definitely larger than what most individuals would eat in a similar period of time under
similar circumstances.

2. A sense of lack of control over eating during the episode (e.g., a feeling that one cannot
stop eating or control what or how much one is eating).

B. Recurrent inappropriate compensatory behaviours in order to prevent weight gain, such
as self-induced vomiting; misuse of laxatives, diuretics, or other medications; fasting; or
excessive exercise.

C. The binge eating and inappropriate compensatory behaviours both occur, on average, at
least once a week for 3 months.

D. Self-evaluation is unduly influenced by body shape and weight.
E. The disturbance does not occur exclusively during episodes of anorexia nervosa.

848 Theory & Psychology 31(6)

emotional (Stinson, 2019). Therefore, the precise nature of any proposed “disturbance”
would need to be more specifically detailed.

Transdiagnostic approaches and dimensional approaches. Transdiagnostic approaches
advocate dispensing with existing diagnostic syndromes in favour of broader classifica-
tions based on shared characteristics. In some cases, this involves collapsing said syn-
dromes into a broader disorder category (e.g., anxiety disorders), in others basing
classification on some common factor (e.g., the internalizing/externalizing model; Krue-
ger & Eaton, 2015). Dimensional approaches are based around spectra or “scales” rather
than discrete categories, such as the Five Factor Model of personality disorders (Widiger
& Costa, 2013). Approaches may be both dimensional and transdiagnostic, such as the
Hierarchical Taxonomy of Psychopathology (HiTOP) model: a hierarchical organization
of mental disorder, consisting of transdiagnostic spectra at the top (e.g., general psycho-
pathology, internalizing/externalizing) and syndromal subfactors (e.g., eating problems),
shared symptoms/signs, and traits at progressively lower levels (see Kotov et al., 2017).

Although transdiagnostic and dimensional approaches may provide useful alternative
means for classifying psychological problems, they do not necessarily describe psycho-
pathological phenomena any more fully than diagnostic syndromes. In some cases, they
even provide weaker descriptions. For example, the internalizing/externalizing model,
although highlighting links between diagnostic categories and thus traversing arbitrary
diagnostic boundaries, provides even less information about mental disorders. Describing
a problem as an “internalizing disorder,” although useful for some purposes, gives very
little information about its precise nature (e.g., whether it involves anxiety, mood, eating,
etc.), or the minutiae of its presentation (i.e., the factors and mechanisms that comprise
the problem). The HiTOP model provides somewhat more information than diagnostic
syndromes thanks to its hierarchical structure, which conceptualizes psychopathological
problems at both more general levels (e.g., spectra levels) and more specific levels (e.g.,
symptoms, signs, and traits). However, this model still lacks the richness of information
necessary for a compositional explanation: symptoms, signs, and traits are not broken
down into lower level factors or mechanisms, nor are any relevant relationships between
them modelled. Furthermore, although disorders are conceptualized at broader, transdi-
agnostic scales, they are not described contextually at higher levels of analysis (e.g.,
sociocultural, interpersonal, political); layers of meaning necessary to fully comprehend
any psychopathological problem.

Symptom network models. The network theory of mental disorder proposes that psycho-
pathological symptoms should be conceptualized as causing each other (e.g., persecu-
tory delusions resulting in paranoia, subsequently leading to social withdrawal) rather
than caused by an underlying “disease” process (e.g., delusions, paranoia, and social
withdrawal as arising from a common cause, such as a neurobiological dysfunction or
genetic mutation; Borsboom, 2017). Symptom network models (SNMs) apply this the-
ory to specific syndrome clusters—often, but not always, DSM/ICD syndromes. A net-
work structure is generated by depicting the causal links between symptoms of that
condition, including their strength and direction (Borsboom, 2017). SNMs can also
model the relationships between symptoms across disorders (e.g., eating disorders and

Hawkins-Elder and Ward 849

depression/anxiety; Smith et al., 2018), which makes them particularly useful in account-
ing for comorbidity (Fried et al., 2017). However, although SNMs provide a useful and
interesting description of psychopathological symptom relationships, they still fail to
provide effective compositional explanations.

SNMs, like DSM/ICD syndromes, lack explanatory scope and depth. Although the
relationships between symptoms are elaborated within these models, the nature of the
symptoms themselves is not fully explained: each is represented largely at the phenom-
enological level, rather than at each level of analysis (e.g., molecular, neural, physiologi-
cal, cognitive/psychological, interpersonal, sociocultural). For example, anhedonia, a
key symptom of depression, can be represented at the phenomenological level as involv-
ing both decreased “liking” and decreased “wanting,” at the cognitive level as a reduced
hedonic capacity, reduced reward motivation, and errors in reward learning, at the neural
level as dysfunction in the “hedonic network” and mesolimbic pathways, and at the
molecular level as reductions in opioid and dopaminergic activity (see Clack & Ward,
2020). Compared to a full analysis such as this, the descriptions of symptoms given in
SNMs are significantly underpowered. They may act as partial compositional explana-
tions, certainly—as models depicting the relationships between psychopathological
symptoms—but lack the depth of detail required to fully explain the constitution of the
disorders with which they are concerned.

Historical accounts. We refer here to descriptions of disorder states that accompanied the
first identification of a psychiatric syndrome or were developed around the time of the
DSM-III (published in 1980), which represented a paradigm shift towards our current
conceptualization of mental disorders (Mayes & Horwitz, 2005). Examples include Rus-
sell’s (1979) initial characterization of bulimia nervosa and Bruch’s (1973, 1978/1982)
early descriptive accounts of anorexia nervosa,1 considered the first “modern descrip-
tion” of the disorder (Marks, 2019). These sorts of accounts typically consist of a set of
clinical case studies from which the author draws broader conclusions. For example,
Russell’s (1979) initial characterization of bulimia nervosa involved 30 patients, three of
whom were presented as illustrative case studies, from which he drew conclusions about
the disorder’s typical features, such as demographics, symptomology, medical complica-
tions, and psychopathological correlates. These accounts are often more descriptively
comprehensive than the classificatory approaches above. However, they are nevertheless
unsuitable to serve as compositional explanations.

Most problematic is that their explanatory scope extends beyond composition.
Although they do describe the presentation of a disorder—as a compositional explana-
tion should—they often branch into hypothesizing its etiology as well. For example, as
well as describing the disorder’s presentation, Bruch’s (1978/1982) account of anorexia
nervosa makes numerous etiological claims—for example, “the child’s inability for con-
structive self-assertion and the associated deficits in personality development are the
outcome of interactional patterns that began early in life” (p. 37)—including several
chapters highlighting precipitating factors and speculating on the causal role of family
dynamics (e.g., chapters “The Perfect Childhood” and “How It Starts”). Although this
information may hold relevance in a clinical context, theoretically it conflates the theo-
retical tasks of compositional and etiological explanation. Although these tasks are

850 Theory & Psychology 31(6)

related—each informing the other—they are conceptually distinct, requiring different
modes of theoretical reasoning: causal versus compositional (see above). Attempting to
achieve both within a single account is therefore likely to create convolution and pro-
mote logical errors, thereby impairing the integrity of each task.

Historical accounts also tend to lack empirical foundation: being either the first or one
of the earliest descriptions of a disorder, there was typically little empirical research to
inform their construction. They are therefore most often based on a small number of case
studies which, although potentially the best option available at the time, falls short of
modern scientific standards. For example, there is typically extensive sampling bias:
samples are generally comprised solely of the author’s existing patients, and therefore
(due to reduced access to psychiatric treatment at the time; Mechanic, 2007) likely to be
skewed towards those of higher socioeconomic status and European descent. Furthermore,
cases are often aggregated in pseudoscientific or anecdotal ways to illustrate the author’s
points, rather than analysed in a valid statistical manner.

There has been little structured effort to update or expand such accounts in line with
contemporary research, despite many still being used to inform it. Although some aspects
of historical accounts can now be empirically verified, there are still many claims that
current research fails to support or actively refutes. For example, Bruch’s (1978/1982)
account of anorexia nervosa describes the disorder as affecting “the daughters of well-to-
do, educated, and successful families, not only in the United States but in many other
affluent countries” (p. vii), implying that anorexia nervosa exclusively affects WEIRD
populations (i.e., Western Educated Industrialized Rich Democratic; Henrich et al.,
2010). However, research now indicates anorexia nervosa is not a culture-bound syn-
drome (Keel & Klump, 2003; Pike et al., 2014), and shows no reliable association with
ethnicity or socioeconomic status (Hadassah Cheng et al., 2018; Schaumberg et al.,
2017).

Finally, as with DSM/ICD syndromes, historical accounts tend to refer to only the
most salient features of a disorder (i.e., the phenomenological level) and therefore neglect
those that are more deep-seated and less easily observed (e.g., neural network dysfunc-
tion, alterations in hormone or neurotransmitter systems), though nonetheless relevant.

Case narratives. Case narratives are often provided to students and clinicians in text-
books or treatment manuals to demonstrate how a disorder typically presents. Their
descriptions usually include information about the characteristic symptoms and signs,
demographics (e.g., age, gender), relevant history (e.g., familial, medical, psychiatric),
and triggers for that disorder. For example:

Anna, a 15-year old girl of European descent, presented with extreme weight loss and low
appetite. Her BMI had fallen from 19 (healthy for her age group) to 16 within the last six
months, such that she was substantially underweight. A recent check-up revealed no underlying
medical explanations for her weight-loss. Anna’s mother reported she had been refusing to join
family meals, confining herself often to her bedroom, and eating a drastically reduced diet. She
had also stopped spending as much time with friends and increased her exercise regime
significantly—running for 1–2 hours every day, in addition to competitive swimming training.
When her parents expressed their concerns to Anna, she tended to either burst into tears or

Hawkins-Elder and Ward 851

shout at them. Anna was unconcerned about her weight-loss and denied that her eating or
exercise behaviour was a problem. She expressed significant body dissatisfaction and drive for
thinness, complaining she was “too fat” and wishing she were thinner. According to her mother,
Anna had had many challenges throughout her development and had previously seen a child
psychologist for anxiety.

As with both diagnostic syndromes and historical accounts, case narratives tend to
refer only to features of the disorder that are most clinically salient. For example, the
exemplar above refers largely to the phenomenological experience of the client, Anna,
and neglects to include information about any physiological, neural, or molecular pro-
cesses. This makes sense given that such accounts are intended as prototypical examples
of clinical presentation, and in practice one would not routinely engage in the methods of
investigation required to identify more deep-seated structural phenomena (e.g., fMRI,
CSF sampling). However, as previously discussed, it significantly limits their explana-
tory value.

Furthermore, although case narratives refer to significantly more features of the dis-
order than most classificatory approaches, they still lack depth in their descriptions. They
fail to go into any features in detail—relying on brief, superficial descriptions despite the
fact that these constructs are often multidimensional (e.g., body image; see above)—and
continue to refer to the thinly defined constructs entrenched within psychopathological
research (e.g., “drive for thinness,” “body image dissatisfaction”). Case examples are
also just that: examples. Each presents a specific instance of a disorder. Hence, although
many features may be represented, it is unlikely that all features relevant to the disorder
will be included, as real-life cases seldom (if ever) present with every feature associated
with the condition.

The Research Domain Criteria (RDoC). The RDoC is a clinically independent research
framework intended to guide empirical investigation into psychological mechanisms
(Cuthbert & Insel, 2013). It was developed as a reaction to the publication of the DSM-5,
which many perceived as being a conservative development on the previous edition
(DSM-IV) that retained many of the problems originally identified (e.g., reification;2
Whooley, 2014). The RDoC presents an “alternative nosological framework” (Whooley,
2014, p. 100) that seeks to advance psychopathology research—specifically, neurobio-
logical investigation (Cuthbert & Insel, 2013; Whooley, 2014). The RDoC assumes that
mental disorders are “brain disorders” born out of dysfunctions in neural circuitry, and
therefore aims to build a nosology of mental disorder from the “bottom-up” using current
neuroscience research (Cuthbert & Insel, 2013; Whooley, 2014). The hope is that by
doing so we will develop more valid diagnostic categories, anchored in neurobiology
(Lilienfeld, 2014; Whooley, 2014).

The RDoC framework provides a two-dimensional “matrix” to guide psychopathol-
ogy research, consisting of six psychological “domains” of investigation—(a) negative
valance systems (e.g., threat, loss), (b) positive valence systems (e.g., approach motiva-
tion, reward learning), (c) cognitive systems (e.g., attention, working memory), (d) sys-
tems for social processes (e.g., attachment, social communication), (e) sensorimotor
systems (e.g., action selection, initiation, execution, habit development), and (f) arousal/

852 Theory & Psychology 31(6)

modulatory systems (e.g., sleep-wake, arousal)—as well as seven “units of analysis”—
(a) genes, (b) molecules, (c) cells, (d) neural circuits, (e) physiology, (f) behaviour, and
(g) self-report (Cuthbert & Insel, 2013; Lilienfeld, 2014). The RDoC assumes that men-
tal disorders result from disruptions in the normal-range functioning of these processes
and thus applies basic understandings of psychology and neuroscience to psychopatho-
logical problems (Lilienfeld, 2014).

It is important that the role of the RDoC within the scientific inquiry process be accu-
rately understood. The RDoC is a research framework intended to scaffold investigation
into psychological processes—both their function and dysfunction—in order to obtain
insight into psychopathology and thereby “inform future classification schemes [empha-
sis added]” (Insel et al., 2010, p. 748). Hence, although able to generate substantial data
about psychological and psychopathological processes, the RDoC does not work to con-
ceptualize these theoretically—that is, to create a coherent compositional account that
links these findings together in relevant and meaningful ways. It therefore does not
directly produce compositional explanations.

Even if it did include such a synthesis, the RDoC matrix is not directly geared towards
studying psychopathological processes. Although intended to provide insight into mental
disorder, one of the core philosophies of the RDoC is that investigation should be directed
towards broader psychological processes (e.g., positive valence systems, cognitive sys-
tems)—how they both function and malfunction, and thereby may contribute to the
development and maintenance of mental disorder—rather than specific psychopatho-
logical problems (Cuthbert & Insel, 2013). Hence, the picture developed by the RDoC
framework is more likely to be a comprehensive understanding of these systems—
including their role in psychopathology—rather than synthesized descriptions of particu-
lar mental disorders. This is not to say the RDoC cannot contribute valuably to the
development of compositional explanations—the wealth of compositional data the
framework has the power to generate would have great value for their construction.
However, it is critical to note that the RDoC also encompasses etiological investigation
(e.g., genetic research), and does not clearly distinguish between these two processes. As
previously discussed, this conflates two distinct theoretical tasks (i.e., causal vs. compo-
sitional explanation) and may lead to problems farther along in the explanatory
process.

The RDoC is also significantly neurocentric; asserting that mental disorders be con-
sidered “brain disorders” born out of dysfunctions in neurocircuitry (Cuthbert & Insel,
2013; Lilienfeld, 2014; Whooley, 2014). Although this approach has some benefits—for
example, highlighting the role of neurobiological processes in mental disorder (at times
discounted or neglected) and providing a platform for investigating neurobiological
aspects of psychopathology—it largely sidelines other levels of explanation (e.g., phe-
nomenological, sociocultural) despite their equal relevance to mental disorder. Of the
seven “units of analysis” prescribed by the RDoC, five are biologically based—genes,
molecules, cells, neurocircuitry, physiology—implying that neurobiological factors hold
far greater explanatory weight (Lilienfeld, 2014).

Furthermore, even though one of the investigative domains concerns social/interper-
sonal phenomena (i.e., systems for social processes), broader sociocultural structures and
influences are not addressed within the framework. Cultural factors are well-evidenced

Hawkins-Elder and Ward 853

as playing a significant role in multiple aspects of psychopathology (e.g., etiology, main-
tenance, symptom expression) and we would argue that mental disorders cannot be
understood independently of their social and cultural context. For example, some symp-
toms associated with posttraumatic stress disorder can be viewed as adaptive when
viewed within specific contexts, including the precipitating trauma event (e.g., hyper-
vigilance, physical hyperarousal, and emotional detachment may be useful in combat
situations). Hence, even though the RDoC might lead to richer neurobiological under-
standings of psychopathology, it also risks decontextualizing mental distress such that
lower levels of analysis (e.g., cellular, neural) are not considered within the broader
context of the problem and higher levels (e.g., psychological, phenomenological, socio-
cultural, etc.) end up significantly underspecified despite their explanatory relevance
(Whooley, 2014).

Building better descriptions

As we have seen, current options for compositional explanation are insufficient; failing
to demonstrate the necessary empirical adequacy, depth, and scope. In this section there-
fore, we demonstrate how we believe effective compositional explanations could be con-
structed. To begin, we first outline the theoretical framework used to guide these ideas:
the Phenomena Detection Method (PDM; Ward & Clack, 2019).

Guiding framework: The phenomena detection method

The PDM (Ward & Clack, 2019) is a metatheoretical framework for the detection and
modelling of “clinical phenomena” (e.g., symptoms) which is not dependent on existing
classification systems such as the DSM-5 or ICD (discussed above). Critically, it empha-
sizes the importance of developing compositional explanations of symptoms, making it
highly relevant to the current problem. It has four phases: (1) formulating client com-
plaints and/or accompanying signs, (2) discerning and analysing patterns in data related
to these symptoms (i.e., detecting clinical phenomena), (3) constructing multiple models
of the phenomenon using different levels or units of analysis, and (4) linking in etiologi-
cal factors to develop causal explanations. Phases 1–3 are relevant to the construction of
compositional explanations and thus inform our reasoning in this section.

Another important aspect of the PDM is its promotion of, and adherence to, a plural-
istic account of scientific explanation, which states that scientific explanations should
involve a collection of theoretical models that represent the constitution or causes of a
symptom at and across different spatial and temporal scales, instead of trying to repre-
sent everything using a single model. This approach is known as model pluralism and has
been widely recognized as a promising way forward in both the biological and social
sciences, in which the phenomena of interest are of a high level of complexity (Hochstein,
2016; Mitchell & Dietrich, 2006; Potochnik, 2010; Ruphy, 2016). The PDM endorses
this by prescribing the construction of multiple compositional or causal models of the
explanatory target at a range of spatial scales and levels of abstraction.

854 Theory & Psychology 31(6)

From syndromes to symptoms

A key impediment to current descriptive accounts acting as compositional explanations
is the fact that most are built around DSM/ICD syndromes, which are widely acknowl-
edged to possess numerous conceptual flaws, such as symptomatic heterogeneity and
rampant comorbidity (e.g., Nielsen & Ward, 2020; Whooley, 2014). Building composi-
tional explanations using these constructs is therefore problematic: any resultant descrip-
tion, no matter how rich or detailed, will lack a certain amount of validity as the very
construct being described is conceptually flawed. Continuing to use these constructs
theoretically also further entrenches them in research and practice, thereby impeding the
development of better classificatory approaches. Hence, on our view, the first step
towards building better compositional explanations is to transition away from these syn-
dromes as the foci of explanation.

Several theorists argue that, to move forward, psychological explanation should, at
least for now, focus on symptoms rather than syndromes (Berrios, 2013; Borsboom,
2017; Ward & Clack, 2019; Wilshire et al., in press). For instance, instead of trying to
describe and explain the syndrome bulimia nervosa, which comprises a cluster of diverse
symptoms, one would instead focus on a single symptom of that pathology, such as binge
eating. This approach makes sense conceptually, as symptoms and signs have greater
validity than DSM/ICD syndromes; arguably representing genuine phenomena as
opposed to artificial categories. Compared to these syndromes, symptoms have more
defined boundaries, less heterogeneity, and greater stability. For example, the symptom
binge eating is more obviously distinct from other symptoms (e.g., self-starvation, purg-
ing) than the syndrome bulimia nervosa is from other eating disorder diagnoses (e.g.,
anorexia nervosa). A client shifting from this symptom presentation to an alternative
presentation (e.g., self-starvation) or to a state of recovery is also likely to be much more
psychologically meaningful than a transition from one eating disorder diagnosis to
another, which can currently be accomplished by changes in arbitrary factors like BMI.
Practically, it is also useful to reduce the scope of our explanatory focus: it is much easier
to detail the composition of a single symptom than a large and diverse collection of them
(i.e., a syndrome).

Symptoms and signs also make for more appropriate foci at an ethical level, as they
represent the actual concerns of clients: each is a valid and important aspect of the cli-
ent’s difficulties that we should aim to understand. At the coarser grain size of syn-
dromes, although we are getting a concise and practical account, we may neglect the
description and explanation of some symptoms in favour of providing a brief and
uncomplicated overall account. At finer grain sizes, such as the neurobiological (e.g.,
the RDoC), although generating useful and detailed information that can be used to
inform theoretical conceptualization of clinical phenomena, we are no longer centring
our accounts on client problems—which arguably should be our paramount concern as
clinicians—and risk decontextualizing their distress (Whooley, 2014). Although to
fully describe symptoms we no doubt need to investigate and describe phenomena at
smaller scales (e.g., neurobiological, molecular) and consider their relationship to
other symptoms (e.g., syndromes, symptom networks), we argue that the appropriate

Hawkins-Elder and Ward 855

starting point for compositional explanations should, at least for now, be psychopatho-
logical symptoms.

Starting with data

Compositional explanations should be constructed using empirical evidence: reasoning
abductively from data to identify constructs and processes relevant to the phenomenon in
question (Haig, 2014). This begins with an unbiased gathering of relevant data—for
example, cross-sectional research from a variety of disciplines involving those present-
ing with that symptom/sign—that is of high methodological quality—for example,
RCTs, meta-analyses, systematic reviews, methodologically rigorous single studies—
which is then mined for patterns that might represent compositional constructs or pro-
cesses (Hawkins-Elder & Ward, 2020b).

For instance, within the symptom binge eating, we may theorize the existence of the
phenomenon impaired inhibitory control based on meta-analyses and systematic reviews
showing that individuals who exhibit binge eating demonstrate poorer performance on
planning (e.g., Farstad et al., 2016), decision-making (e.g., Guillaume et al., 2015; Wu
et al., 2016), and set-shifting tasks (e.g., Wu et al., 2016), higher self-reported impulsiv-
ity (e.g., Farstad et al., 2016; Steward et al., 2017), and frequent engagement in other
impulsive or reckless behaviours (e.g., self-harm, substance abuse; Peebles et al., 2011).
Having a range of different, and methodologically robust, data all pointing to the exist-
ence of impaired inhibitory control means we can be more confident that this phenome-
non is genuinely present, rather than the false product of biased reasoning or
methodological error. A full model constructed in this manner will therefore be a more
accurate representation of the phenomenon of interest.

Multilevel explanation

Compositional explanations should describe their phenomenon at all relevant levels of
analysis—for example, molecular, neurological, cognitive, phenomenological, interper-
sonal, contextual/sociocultural, and so forth. Indeed, Zachar (2008) describes psycho-
pathological phenomena as structures with “many overlapping levels” and argues that
“having alternative models better reflects the domain of psychiatric disorders” (pp. 339–
340). Compositional explanations should therefore be similarly multilevel in order to
adequately reflect this.

To accomplish this, we recommend building models in a “stacked” format, beginning
with the phenomenological level (at which the symptom/sign is reported/observed) and
moving outwards, considering each level of analysis in turn, to identify factors and pro-
cesses that might be constitutionally relevant. This is useful because factors and pro-
cesses at one level may partially constitute or be otherwise related to those at another and
building outwards in this manner may help the researcher to make these connections.

Consider the symptom binge eating: at the phenomenological level, we can recount
how this symptom is experienced by the client based on self-report data from empirical
research: privacy is important (secretive eating; Lydecker & Grilo, 2019), emotion is
often involved (emotional eating; e.g., Leehr et al., 2015; Ricca et al., 2012), individuals

856 Theory & Psychology 31(6)

typically perceive a lack of control over their eating (e.g., Colles et al., 2008), and may
experience strong physical hunger and hedonic craving for food (e.g., Ng & Davis, 2013;
Witt & Lowe, 2014; see Figure 1). From this level, we can then consider factors and
processes suggested by empirical research at lower levels that may comprise this symp-
tom. For example, at a cognitive level, the reported hunger and craving could be repre-
sented as a heightening of appetite3 (see Figure 1). This may be partially constituted at
the physiological level by an impaired appetite feedback system (e.g., imbalances in
hunger and satiety hormones, Culbert et al., 2016; altered vagal nerve transmission,
Peschel et al., 2016) and at the neurological level by interoceptive network deficits (e.g.,
insular dysfunction, Gasquoine, 2014; Klabunde et al., 2017) and alterations in reward
pathways (e.g., Avena & Bocarsly, 2012; Frank, 2013; Wierenga et al., 2014). Further
down at the molecular level, serotonin dysregulation (e.g., Compan et al., 2012) may be
partially responsible for the experience of hunger surrounding a binge (due to its role in
appetite regulation; e.g., Lam et al., 2010), and the experience of intense craving may be
influenced by dysregulation in opioid and dopaminergic systems implicated in reward
and addiction (e.g., Berridge, 2009; Majuri et al., 2017). At a higher level, it is also worth
considering how sociocultural factors may influence how the symptom is experienced
(e.g., enabling or inhibiting certain behaviours, altering symptom content). For instance,
overeating, for metabolic or hedonic reasons, is somewhat dependent on socioeconomic
food security (e.g., Anderson-Fye, 2018), as food must be available in reasonable abun-
dance for it to be overconsumed. Similarly, the content of cravings is likely to be influ-
enced by the individual’s cultural environment (e.g., Osman & Sobal, 2006).

We can conduct the same process for each aspect of a symptom to build a multilevel
explanation of its constitution (see Figure 1). Such an explanation provides a rich descrip-
tion of the symptom, as it details the relevant factors and processes at all levels and
considers how these may comprise or influence each other.

Detailing domains

An adequate compositional explanation should describe all aspects of the phenomenon
in a high level of detail. One way of doing this is to build smaller compositional models
of constructs “nested” within the larger account. This helps to avoid overcomplicating
the broader model with specifics, allowing it to present a streamlined overview, but
ensures that all constructs are sufficiently outlined and the overall explanation is descrip-
tively rich.

Consider the multilevel model of binge eating previously sketched out: although this
model provides a good overview of constitutional factors and processes at each level, the
constructs referred to within each domain are still in need of further definition. By con-
structing nested “submodels” of these phenomena, we can more clearly define the factors
and processes invoked and thereby enrich the overall account. For example, consider
heightened appetite (identified at the cognitive level). Based on the literature, we can
construct a more detailed compositional submodel of this construct (see Figure 2).

Individuals who binge eat are more sensitive to the effects of reward (e.g., Harrison
et al., 2010; Wierenga et al., 2014), including reward from food (e.g., Schag et al., 2013).
They are therefore likely to have greater hedonic hunger (e.g., Witt & Lowe, 2014) than

Hawkins-Elder and Ward 857

nonbinge-eating individuals. Binge eaters also demonstrate impaired satiety (e.g., Sysko
et al., 2007). Their metabolic hunger is therefore also likely greater, as they have less
indication when they are full. Finally, binge eaters reliably demonstrate poorer interocep-
tion—ability to sense and interpret internal sensations—than nonbinge eaters (e.g.,
Jenkinson et al., 2018; Klabunde et al., 2017). They may therefore struggle to detect
physical cues, including appetitive signals, and accurately interpret them, sometimes
misattributing physical sensations as hunger or satiety. This may make it harder for them
to appropriately modulate eating behaviour according to their body’s metabolic needs
(e.g., Herbert & Pollatos, 2018).

Submodels at one level can also be linked to submodels at other levels to further
enrich the explanation. For example, we might construct a physiological submodel of the
impaired appetite feedback system (see Figure 3), involving alterations in the baseline
levels and responses of appetitive hormones (e.g., Culbert et al., 2016; Prince et al.,
2009), increased gastric capacity and delayed gastric emptying (e.g., Klein & Walsh,

Figure 1. Illustration of a multilevel approach to compositional explanation, using the
symptom binge eating.

858 Theory & Psychology 31(6)

2004), and decreased ascending vagal nerve transmission (e.g., Simmons & DeVille,
2017). This can then be linked to the cognitive model by relating the lower level physi-
ological processes to those at the higher level. For example, the impaired satiety described
at the cognitive level is likely partially constituted by these hormonal and gastric differ-
ences (see Figure 3; e.g., Berthoud, 2008; Zanchi et al., 2017). Aberrant hormonal func-
tioning could also partially comprise the increased sensitivity to food-related reward due
to the influence of some hormones on dopaminergic networks (e.g., leptin; Cassioli et al.,
2020). Vagal nerve dysregulation may likewise play a part in impaired interoception, as
it plays a key role in transmitting sensory information from the body to the brain (Craig,
2002). This kind of intermodel linking further enriches the overall compositional picture,
contributing to a more in-depth account of the symptom’s composition.

Conclusion

In this article, we suggested how better compositional explanations could be constructed
by focusing on symptoms rather than syndromes, using empirical research, and creating
detail-rich models spanning all levels of analysis. At present, we lack theoretically ori-
ented and descriptively rich accounts of how psychopathological problems are consti-
tuted. The absence of these compositional explanations is of significant concern, as they
hold genuine value for both research and practice. The kind of nested modelling outlined
above is a good example of how originally conceptually thin phenomena can be elabo-
rated into rich, multilayered compositional accounts. Developing a network of composi-
tional models at different levels of analysis may yield insight into the structures and
processes constituting disorders that could, ultimately, result in stronger etiological
explanations, more accurate taxonomies, and more precisely targeted treatment. In our
view, greater attention to the compositional explanation of psychopathological symp-
toms is a crucial step towards ameliorating the social costs and personal suffering of
mental illness.

Figure 2. Example of a compositional submodel within the symptom binge eating, detailing the
nested cognitive phenomenon heightened appetite.

Hawkins-Elder and Ward 859

Acknowledgements

The authors would like to thank the EPC Lab at Victoria, as well as Alexander Moses for his assis-
tance in designing the figures for the paper.

Declaration of Conflicting Interests

The authors declare that there is no conflict of interest.

Funding

The authors received no financial support for the research, authorship, and/or publication of this
article.

ORCID iDs

Hannah Hawkins-Elder https://orcid.org/0000-0002-3511-3908
Tony Ward https://orcid.org/0000-0002-6292-2364

Figure 3. Example model linking cognitive (heightened appetite) and physiological (impaired
appetite feedback system) submodels within the symptom binge eating.

https://orcid.org/0000-0002-3511-3908

https://orcid.org/0000-0002-6292-2364

860 Theory & Psychology 31(6)

Notes

1. The most well-known of these being her book The Golden Cage: The Enigma of Anorexia
Nervosa (Bruch, 1978/1982), first published in 1978.

2. Reification refers to the process of considering or representing an abstract concept or idea as
a material or concrete entity (Hyman, 2010).

3. “Appetite” refers to an individual’s drive to eat, for either hedonic or metabolic purposes
(Booth, 2003).

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders
(5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Anderson-Fye, E. P. (2018). Cultural influences on body image and eating disorders. In W. S.
Agras & A. Robinson (Eds.), The Oxford handbook of eating disorders (2nd ed.). Oxford
University Press.

Avena, N. M., & Bocarsly, M. E. (2012). Dysregulation of brain reward systems in eating dis-
orders: Neurochemical information from animal models of binge eating, bulimia nervosa,
and anorexia nervosa. Neuropharmacology, 63(1), 87–96. https://doi.org/10.1016/j.neurop-
harm.2011.11.010

Bailer-Jones, D. M. (2003). When scientific models represent. International Studies in the
Philosophy of Science, 17(1), 59–74. https://doi.org/10.1080/02698590305238

Berridge, K. C. (2009). “Liking” and “wanting” food rewards: Brain substrates and roles in
eating disorders. Physiology & Behavior, 97(5), 537–550. https://doi.org/10.1016/j.phys-
beh.2009.02.044

Berrios, G. E. (2013). Formation and meaning of mental symptoms: History and epistemology.
Dialogues in Philosophy, Mental and Neuro Sciences, 6(2), 39–48. https://philarchive.org/
rec/BERFAM-2

Berthoud, H. (2008). Vagal and hormonal gut–brain communication: From satiation to satis-
faction. Neurogastroenterology & Motility, 20(s1), 64–72. https://doi.org/10.1111/j.1365-
2982.2008.01104.x

Booth, D. A. (2003). Food, nutrition, and appetite. In B. Caballero (Ed.), Encyclopedia of food
sciences and nutrition (2nd ed.; pp. 5098–5102). Academic Press. https://www.sciencedirect.
com/referencework/9780122270550/encyclopedia-of-food-sciences-and-nutrition

Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16(1), 5–13.
https://doi.org/10.1002/wps.20375

Bruch, H. (1973). Eating disorders: Obesity, anorexia nervosa and the person within. Basic Books.
Bruch, H. (1982). The golden cage: The enigma of anorexia nervosa. Harvard University Press.

(Original work published 1978)
Cash, T. F., & Deagle, E. A. (1997). The nature and extent of body-image disturbances in anorexia

nervosa and bulimia nervosa: A meta-analysis. International Journal of Eating Disorders,
22(2), 107–125. https://doi.org/10.1002/(SICI)1098-108X(199709)22:2<107::AID- EAT1>3.0.CO;2-J

Cassioli, E., Rossi, E., Squecco, R., Baccari, M. C., Maggi, M., Vignozzi, L., Comeglio, P., Gironi,
V., Lelli, L., Rotella, F., Monteleone, A. M., Ricca, V., & Castellini, G. (2020). Reward and
psychopathological correlates of eating disorders: The explanatory role of leptin. Psychiatry
Research, 290, Article 113071. https://doi.org/10.1016/j.psychres.2020.113071

Clack, S., & Ward, T. (2020). Modeling the symptoms of psychopathology: A pluralistic
approach. New Ideas in Psychology, 59, Article 100799. https://doi.org/10.1016/j.newidea-
psych.2020.100799

https://doi.org/10.1176/appi.books.9780890425596

https://doi.org/10.1016/j.neuropharm.2011.11.010

https://doi.org/10.1080/02698590305238

https://doi.org/10.1016/j.physbeh.2009.02.044

https://philarchive.org/rec/BERFAM-2

https://doi.org/10.1111/j.1365-2982.2008.01104.x

https://www.sciencedirect.com/referencework/9780122270550/encyclopedia-of-food-sciences-and-nutrition

https://doi.org/10.1002/wps.20375

https://doi.org/10.1002/(SICI)1098-108X(199709)22:2

https://doi.org/10.1016/j.psychres.2020.113071

https://doi.org/10.1016/j.newideapsych.2020.100799

Hawkins-Elder and Ward 861

Colles, S. L., Dixon, J. B., & O’Brien, P. E. (2008). Loss of control is central to psychologi-
cal disturbance associated with binge eating disorder. Obesity, 16(3), 608–614. https://doi.
org/10.1038/oby.2007.99

Compan, V., Laurent, L., Jean, A., Macary, C., Bockaert, J., & Dumuis, A. (2012). Serotonin
signalling in eating disorders. Wiley Interdisciplinary Reviews: Membrane Transport and
Signaling, 1(6), 715–729. https://doi.org/10.1002/wmts.45

Craig, A. D. (2002). How do you feel? Interoception: The sense of the physiological condition of
the body. Nature Reviews Neuroscience, 3(8), 655–666. https://doi.org/10.1038/nrn894

Craver, C. F. (2007). Explaining the brain: Mechanisms and the mosaic unity of neuroscience.
Oxford University Press.

Craver, C., & Kaplan, D. M. (2020). Are more details better? On the norms of completeness for
mechanistic explanations. The British Journal for the Philosophy of Science, 71(1), 287–319.
https://doi.org/10.1093/bjps/axy015

Culbert, K. M., Racine, S. E., & Klump, K. L. (2016). Hormonal factors and disturbances in
eating disorders. Current Psychiatry Reports, 18(7), 65–81. https://doi.org/10.1007/s11920-
016-0701-6

Cuthbert, B. N., & Insel, T. R. (2013). Toward the future of psychiatric diagnosis: The seven pil-
lars of RDoC. BMC Medicine, 11(1), Article 126. https://doi.org/10.1186/1741-7015-11-126

Dishion, T. J., & Patterson, G. R. (2015). The development and ecology of antisocial behavior in
children and adolescents. In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathol-
ogy: Vol. 3. Risk, disorder, and adaptation (2nd ed., pp. 503–541). John Wiley & Sons.

Farstad, S. M., McGeown, L. M., & von Ranson, K. M. (2016). Eating disorders and person-
ality 2004–2016: A systematic review and meta-analysis. Clinical Psychology Review, 46,
91–105. https://doi.org/10.1016/j.cpr.2016.04.005

Faye, J. (2014). The nature of scientific thinking: On interpretation, explanation, and understand-
ing. Palgrave Macmillan.

Frank, G. K. W. (2013). Altered brain reward circuits in eating disorders: Chicken or egg? Current
Psychiatry Reports, 15(10), Article 396. https://doi.org/10.1007/s11920-013-0396-x

Fried, E. I., van Borkulo, C. D., Cramer, A. O. J., Boschloo, L., Schoevers, R. A., & Borsboom,
D. (2017). Mental disorders as networks of problems: A review of recent insights. Social
Psychiatry and Psychiatric Epidemiology, 52(1), 1–10. https://doi.org/10.1007/s00127-016-
1319-z

Gasquoine, P. G. (2014). Contributions of the insula to cognition and emotion. Neuropsychology
Review, 24(2), 77–87. https://doi.org/10.1007/s11065-014-9246-9

Gillett, C. (in press). Using compositional explanations to understand compositional levels: An
integrative account. In D. S. Brooks, J. DiFrisco, & W. C. Wimsatt (Eds.), Levels of organiza-
tion in the biological sciences. MIT Press.

Guillaume, S., Gorwood, P., Jollant, F., Van den Eynde, F., Courtet, P., & Richard-Devantoy,
S. (2015). Impaired decision-making in symptomatic anorexia nervosa and bulimia ner-
vosa patients: A meta-analysis. Psychological Medicine, 45(16), 3377–3391. https://doi.
org/10.1017/S003329171500152X

Hadassah Cheng, Z., Perko, V. L., Fuller-Marashi, L., Gau, J. M., & Stice, E. (2018). Ethnic differ-
ences in eating disorder prevalence, risk factors, and predictive effects of risk factors among
young women. Eating Behaviors, 32, 23–30. https://doi.org/10.1016/j.eatbeh.2018.11.004

Haig, B. D. (2014). Investigating the psychological world: Scientific method in the behavioural
sciences. The MIT Press.

Harrison, A., O’Brien, N., Lopez, C., & Treasure, J. (2010). Sensitivity to reward and punish-
ment in eating disorders. Psychiatry Research, 177(1–2), 1–11. https://doi.org/10.1016/j.psy-
chres.2009.06.010

https://doi.org/10.1038/oby.2007.99

https://doi.org/10.1002/wmts.45

https://doi.org/10.1038/nrn894

https://doi.org/10.1093/bjps/axy015

https://doi.org/10.1007/s11920-016-0701-6

https://doi.org/10.1186/1741-7015-11-126

https://doi.org/10.1016/j.cpr.2016.04.005

https://doi.org/10.1007/s11920-013-0396-x

https://doi.org/10.1007/s00127-016-1319-z

https://doi.org/10.1007/s11065-014-9246-9

https://doi.org/10.1017/S003329171500152X

https://doi.org/10.1016/j.eatbeh.2018.11.004

https://doi.org/10.1016/j.psychres.2009.06.010

862 Theory & Psychology 31(6)

Hawkins-Elder, H., & Ward, T. (2020a). Explanations for eating disorders: A critical analysis.
Behaviour Change, 37(2), 93–110. https://doi.org/10.1017/bec.2020.6

Hawkins-Elder, H., & Ward, T. (2020b). Theory construction in the psychopathology
domain: A multiphase approach. Theory & Psychology, 30(1), 77–98. https://doi.org/
10.1177/0959354319893026

Hedges, D., Farrer, T. J., Bigler, E. D., & Hopkins, R. O. (2019). Cognition in anorexia nervosa and
bulimia nervosa. In D. Hedges, T. J. Farrer, E. D. Bigler, & R. O. Hopkins (Eds.), The brain at
risk: Associations between disease and cognition (pp. 67–83). Springer Nature Switzerland.

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Beyond WEIRD: Towards a broad-based behav-
ioral science. Behavioral and Brain Sciences, 33(2–3), 111–135. https://doi.org/10.1017/
S0140525X10000725

Herbert, B. M., & Pollatos, O. (2018). The relevance of interoception for eating behavior and eat-
ing disorders. In M. Tsakiris & H. De Preester (Eds.), The interoceptive mind: From homeo-
stasis to awareness (pp. 165–186). Oxford University Press.

Hochstein, E. (2016). One mechanism, many models: A distributed theory of mechanistic explana-
tion. Synthese, 193(5), 1387–1407. https://doi.org/10.1007/s11229-015-0844-8

Hyman, S. E. (2010). The diagnosis of mental disorders: The problem of reification. Annual Review of
Clinical Psychology, 6, 155–179. https://doi.org/10.1146/annurev.clinpsy.3.022806.091532

Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., Sanislow, C., & Wang,
P. (2010). Research domain criteria (RDoC): Toward a new classification framework for
research on mental disorders. American Journal of Psychiatry, 167(7), 748–751. https://doi.
org/10.1176/appi.ajp.2010.09091379

Jenkinson, P. M., Taylor, L., & Laws, K. R. (2018). Self-reported interoceptive deficits in eat-
ing disorders: A meta-analysis of studies using the eating disorder inventory. Journal of
Psychosomatic Research, 110, 38–45. https://doi.org/10.1016/j.jpsychores.2018.04.005

Kaiser, M. I., & Krickel, B. (2016). The metaphysics of constitutive mechanistic phenomena. The
British Journal for the Philosophy of Science, 68(3), 745–779. https://doi.org/10.1093/bjps/
axv058

Keel, P. K., & Klump, K. L. (2003). Are eating disorders culture-bound syndromes? Implications
for conceptualizing their etiology. Psychological Bulletin, 129(5), 747–769. https://doi.
org/10.1037/0033-2909.129.5.747

Kendler, K. S., Parnas, J., & Zachar, P. (2020). (Eds.). Levels of analysis in psychopathology:
Cross-disciplinary perspectives. Cambridge University Press.

Klabunde, M., Collado, D., & Bohon, C. (2017). An interoceptive model of bulimia nervosa: A
neurobiological systematic review. Journal of Psychiatric Research, 94, 36–46. https://doi.
org/10.1016/j.jpsychires.2017.06.009

Klein, D. A., & Walsh, B. T. (2004). Eating disorders: Clinical features and pathophysiology.
Physiology & Behavior, 81(2), 359–374. https://doi.org/10.1016/j.physbeh.2004.02.009

Konjarski, M., Murray, G., Lee, V. V., & Jackson, M. L. (2018). Reciprocal relationships between
daily sleep and mood: A systematic review of naturalistic prospective studies. Sleep Medicine
Reviews, 42, 47–58. https://doi.org/10.1016/j.smrv.2018.05.005

Kotov, R., Krueger, R. F., Watson, D., Achenbach, T. M., Althoff, R. R., Bagby, R. M., Brown,
T. A., Carpenter, W. T., Caspi, A., Clark, L. A., Eaton, N. R., Forbes, M. K., Forbush, K. T.,
Goldberg, D., Hasin, D., Hyman, S. E., Ivanova, M. Y., Lynam, D. R., Markon, K., Miller, J.
D., . . . Zimmerman, M. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP):
A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4),
454–477. https://doi.org/10.1037/abn0000258

Krueger, R. F., & Eaton, N. R. (2015). Transdiagnostic factors of mental disorders. World
Psychiatry, 14(1), 27–29. https://doi.org/10.1002/wps.20175

https://doi.org/10.1017/bec.2020.6

https://doi.org/10.1177/0959354319893026

https://doi.org/10.1017/S0140525X10000725

https://doi.org/10.1007/s11229-015-0844-8

https://doi.org/10.1146/annurev.clinpsy.3.022806.091532

https://doi.org/10.1176/appi.ajp.2010.09091379

https://doi.org/10.1016/j.jpsychores.2018.04.005

https://doi.org/10.1093/bjps/axv058

https://doi.org/10.1037/0033-2909.129.5.747

https://doi.org/10.1016/j.jpsychires.2017.06.009

https://doi.org/10.1016/j.physbeh.2004.02.009

https://doi.org/10.1016/j.smrv.2018.05.005

https://doi.org/10.1037/abn0000258

https://doi.org/10.1002/wps.20175

Hawkins-Elder and Ward 863

Lam, D. D., Garfield, A. S., Marston, O. J., Shaw, J., & Heisler, L. K. (2010). Brain serotonin
system in the coordination of food intake and body weight. Pharmacology Biochemistry and
Behavior, 97(1), 84–91. https://doi.org/10.1016/j.pbb.2010.09.003

Leehr, E. J., Krohmer, K., Schag, K., Dresler, T., Zipfel, S., & Giel, K. E. (2015). Emotion regu-
lation model in binge eating disorder and obesity—A systematic review. Neuroscience &
Biobehavioral Reviews, 49, 125–134. https://doi.org/10.1016/j.neubiorev.2014.12.008

Lilienfeld, S. O. (2014). The Research Domain Criteria (RDoC): An analysis of methodologi-
cal and conceptual challenges. Behaviour Research and Therapy, 62, 129–139. https://doi.
org/10.1016/j.brat.2014.07.019

Lydecker, J. A., & Grilo, C. M. (2019). I didn’t want them to see: Secretive eating among adults
with binge-eating disorder. International Journal of Eating Disorders, 52(2), 153–158.
https://doi.org/10.1002/eat.23002

Majuri, J., Joutsa, J., Johansson, J., Voon, V., Alakurtti, K., Parkkola, R., Lahti, T., Alho, H.,
Hirvonen, J., Arponen, E., Forsback, S., & Kaasinen, V. (2017). Dopamine and opioid neu-
rotransmission in behavioral addictions: A comparative PET study in pathological gambling
and binge eating. Neuropsychopharmacology, 42(5), 1169–1177. https://doi.org/10.1038/
npp.2016.265

Mantzavinos, C. (2016). Explanatory pluralism. Cambridge University Press.
Marks, A. (2019). The evolution of our understanding and treatment of eating disorders over the

past 50 years. Journal of Clinical Psychology, 75(8), 1380–1391. https://doi.org/10.1002/
jclp.22782

Mayes, R., & Horwitz, A. V. (2005). DSM-III and the revolution in the classification of men-
tal illness. Journal of the History of the Behavioral Sciences, 41(3), 249–267. https://doi.
org/10.1002/jhbs.20103

Mechanic, D. (2007). Mental health services then and now. Health Affairs, 26(6), 1548–1550.
https://doi.org/10.1377/hlthaff.26.6.1548

Mitchell, S. D., & Dietrich, M. R. (2006). Integration without unification: An argument for plural-
ism in the biological sciences. The American Naturalist, 168(Suppl. 6), S73–S79. https://doi.
org/10.1086/509050

Mölbert, S. C., Klein, L., Thaler, A., Mohler, B. J., Brozzo, C., Martus, P., Karnath, H., Zipfel,
S., & Giel, K. E. (2017). Depictive and metric body size estimation in anorexia nervosa and
bulimia nervosa: A systematic review and meta-analysis. Clinical Psychology Review, 57,
21–31. https://doi.org/10.1016/j.cpr.2017.08.005

Ng, L., & Davis, C. (2013). Cravings and food consumption in binge eating disorder. Eating
Behaviors, 14(4), 472–475. https://doi.org/10.1016/j.eatbeh.2013.08.011

Nielsen, K., & Ward, T. (2020). Phenomena complexes as targets of explanation in psychopathol-
ogy: The relational analysis of phenomena approach. Theory & Psychology, 30(2), 164–185.
https://doi.org/10.1177/0959354320906462

Nowakowski, M. E., McFarlane, T., & Cassin, S. E. (2013). Alexithymia and eating disorders:
A critical review of the literature. Journal of Eating Disorders, 1(1), Article 21. https://doi.
org/10.1186/2050-2974-1-21

Osman, J. L., & Sobal, J. (2006). Chocolate cravings in American and Spanish individuals:
Biological and cultural influences. Appetite, 47(3), 290–301. https://doi.org/10.1016/j.
appet.2006.04.008

Peebles, R., Wilson, J. L., & Lock, J. D. (2011). Self-injury in adolescents with eating disor-
ders: Correlates and provider bias. Journal of Adolescent Health, 48(3), 310–313. https://doi.
org/10.1016/j.jadohealth.2010.06.017

https://doi.org/10.1016/j.pbb.2010.09.003

https://doi.org/10.1016/j.neubiorev.2014.12.008

https://doi.org/10.1016/j.brat.2014.07.019

https://doi.org/10.1002/eat.23002

https://doi.org/10.1038/npp.2016.265

https://doi.org/10.1002/jclp.22782

https://doi.org/10.1002/jhbs.20103

https://doi.org/10.1377/hlthaff.26.6.1548

https://doi.org/10.1086/509050

https://doi.org/10.1016/j.cpr.2017.08.005

https://doi.org/10.1016/j.eatbeh.2013.08.011

https://doi.org/10.1177/0959354320906462

https://doi.org/10.1186/2050-2974-1-21

https://doi.org/10.1016/j.appet.2006.04.008

https://doi.org/10.1016/j.jadohealth.2010.06.017

864 Theory & Psychology 31(6)

Peschel, S. K. V., Feeling, N. R., Vögele, C., Kaess, M., Thayer, J. F., & Koenig, J. (2016).
A meta-analysis on resting state high-frequency heart rate variability in bulimia nervosa.
European Eating Disorders Review, 24(5), 355–365. https://doi.org/10.1002/erv.2454

Pike, K. M., Hoek, H. W., & Dunne, P. E. (2014). Cultural trends and eating disorders. Current
Opinion in Psychiatry, 27(6), 436–442. https://doi.org/10.1097/yco.0000000000000100

Potochnik, A. (2010). Levels of explanation reconceived. Philosophy of Science, 77(1), 59–72.
https://doi.org/10.1086/650208

Prince, A. C., Brooks, S. J., Stahl, D., & Treasure, J. (2009). Systematic review and meta-analysis
of the baseline concentrations and physiologic responses of gut hormones to food in eating
disorders. American Journal of Clinical Nutrition, 89(3), 755–765. https://doi.org/10.3945/
ajcn.2008.27056

Ricca, V., Castellini, G., Fioravanti, G., Lo Sauro, C., Rotella, F., Ravaldi, C., Lazzeretti, L., &
Faravelli, C. (2012). Emotional eating in anorexia nervosa and bulimia nervosa. Comprehensive
Psychiatry, 53(3), 245–251. https://doi.org/10.1016/j.comppsych.2011.04.062

Ruphy, S. (2016). Scientific pluralism reconsidered. University of Pittsburgh Press.
Russell, G. (1979). Bulimia nervosa: An ominous variant of anorexia nervosa. Psychological

Medicine, 9(3), 429–448. https://doi.org/10.1017/s0033291700031974
Savulescu, J., Roache, R., & Davies, W. (Eds.). (2020). Psychiatry reborn: Biopsychosocial psy-

chiatry in modern medicine. Oxford University Press.
Schag, K., Teufel, M., Junne, F., Preissl, H., Hautzinger, M., Zipfel, S., & Giel, K. E. (2013).

Impulsivity in binge eating disorder: Food cues elicit increased reward responses and dis-
inhibition. PLOS ONE, 8(10), Article e76542. https://doi.org/10.1371/journal.pone.0076542

Schaumberg, K., Welch, E., Breithaupt, L., Hübel, C., Baker, J. H., Munn-Chernoff, M. A.,
Yilmaz, Z., Ehrlich, S., Mustelin, L., Ghaderi, A., Hardaway, A. J., Bulik-Sullivan, E. C.,
Hedman, A. M., Jangmo, A., Nilsson, I. A. K., Wiklund, C., Yao, S., Seidel, M., & Bulik, C.
M. (2017). The science behind the academy for eating disorders’ nine truths about eating dis-
orders. European Eating Disorders Review, 25(6), 432–450. https://doi.org/10.1002/erv.2553

Simmons, W. K., & DeVille, D. C. (2017). Interoceptive contributions to healthy eating and obesity.
Current Opinion in Psychology, 17, 106–112. https://doi.org/10.1016/j.copsyc.2017.07.001

Smith, K. E., Crosby, R. D., Wonderlich, S. A., Forbush, K. T., Mason, T. B., & Moessner,
M. (2018). Network analysis: An innovative framework for understanding eating disorder
psychopathology. International Journal of Eating Disorders, 51(3), 214–222. https://doi.
org/10.1002/eat.22836

Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014).
The global prevalence of common mental disorders: A systematic review and meta-analysis
1980–2013. International Journal of Epidemiology, 43(2), 476–493. https://doi.org/10.1093/
ije/dyu038

Steward, T., Mestre-Bach, G., Vintro-Alcaraz, C., Aguera, Z., Jimenez-Murcia, S., Granero, R.,
& Fernandez-Aranda, F. (2017). Delay discounting of reward and impulsivity in eating dis-
orders: From anorexia nervosa to binge eating disorder. European Eating Disorders Review,
25(6), 601–606. https://doi.org/10.1002/erv.2543

Stinson, C. (2019). The absent body in psychiatric diagnosis, treatment, and research. Synthese,
196, 2153–2176. https://doi.org/10.1007/s11229-017-1507-8

Sysko, R., Devlin, M. J., Walsh, B. T., Zimmerli, E., & Kissileff, H. R. (2007). Satiety and test meal
intake among women with binge eating disorder. International Journal of Eating Disorders,
40(6), 554–561. https://doi.org/10.1002/eat.20384

Thompson, J. K. (2004). The (mis)measurement of body image: Ten strategies to improve assess-
ment for applied and research purposes. Body Image, 1(1), 7–14. https://doi.org/10.1016/
S1740-1445(03)00004-4

https://doi.org/10.1002/erv.2454

https://doi.org/10.1097/yco.0000000000000100

https://doi.org/10.1086/650208

https://doi.org/10.3945/ajcn.2008.27056

https://doi.org/10.1016/j.comppsych.2011.04.062

https://doi.org/10.1017/s0033291700031974

https://doi.org/10.1371/journal.pone.0076542

https://doi.org/10.1002/erv.2553

https://doi.org/10.1016/j.copsyc.2017.07.001

https://doi.org/10.1002/eat.22836

https://doi.org/10.1093/ije/dyu038

https://doi.org/10.1002/erv.2543

https://doi.org/10.1007/s11229-017-1507-8

https://doi.org/10.1002/eat.20384

https://doi.org/10.1016/S1740-1445(03)00004-4

Hawkins-Elder and Ward 865

Trout, J. D. (2016). Wondrous truths: The improbable triumph of science. Oxford University Press.
Ward, T., & Clack, S. (2019). From symptoms of psychopathology to the explanation of clini-

cal phenomena. New Ideas in Psychology, 54, 40–49. https://doi.org/10.1016/j.newidea-
psych.2019.01.004

Westwood, H., Eisler, I., Mandy, W., Leppanen, J., Treasure, J., & Tchanturia, K. (2016). Using
the autism-spectrum quotient to measure autistic traits in anorexia nervosa: A systematic
review and meta-analysis. Journal of Autism and Developmental Disorders, 46(3), 964–977.
https://doi.org/10.1007/s10803-015-2641-0

Westwood, H., Kerr-Gaffney, J., Stahl, D., & Tchanturia, K. (2017). Alexithymia in eating dis-
orders: Systematic review and meta-analyses of studies using the Toronto Alexithymia
Scale. Journal of Psychosomatic Research, 99, 66–81. https://doi.org/10.1016/j.jpsy-
chores.2017.06.007

Whiteford, H. A., Ferrari, A. J., Degenhardt, L., Feigin, V., & Vos, T. (2015). The global burden
of mental, neurological and substance use disorders: An analysis from the global burden of
disease study 2010. PLOS ONE, 10(2), Article e0116820. https://doi.org/10.1371/journal.
pone.0116820

Whooley, O. (2014). Nosological reflections: The failure of DSM-5, the emergence of RDoC, and
the decontextualization of mental distress. Society and Mental Health, 4(2), 92–110. https://
doi.org/10.1177/2156869313519114

Widiger, T. A., & Costa, P. T., Jr. (Eds.). (2013). Personality disorders and the five-fac-
tor model of personality (3rd ed.). American Psychological Association. https://doi.
org/10.1037/13939-000

Wierenga, C. E., Ely, A., Bischoff-Grethe, A., Bailer, U. F., Simmons, A. N., & Kaye, W. H.
(2014). Are extremes of consumption in eating disorders related to an altered balance
between reward and inhibition? Frontiers in Behavioral Neuroscience, 8, Article 410. https://
doi.org/10.3389/fnbeh.2014.00410

Wilshire, C. E., Ward, T., & Clack, S. (2021). Symptom descriptions in psychopathology: How
well are they working for us? Clinical Psychological Science. Advance online publication.
https://doi.org/10.1177/2167702620969215

Witt, A. A., & Lowe, M. R. (2014). Hedonic hunger and binge eating among women with eating
disorders. International Journal of Eating Disorders, 47(3), 273–280. https://doi.org/10.1002/
eat.22171

World Health Organization. (2019). International statistical classification of diseases and related
health problems (11th ed.). https://icd.who.int/

Wu, M., Brockmeyer, T., Hartmann, M., Skunde, M., Herzog, W., & Friederich, H. (2016). Reward-
related decision making in eating and weight disorders: A systematic review and meta-
analysis of the evidence from neuropsychological studies. Neuroscience and Biobehavioral
Reviews, 61, 177–196. https://doi.org/10.1016/j.neubiorev.2015.11.017

Zachar, P. (2008). Real kinds but no true taxonomy: An essay in psychiatric systematics. In K. S.
Kendler & J. Parnas (Eds.), Philosophical issues in psychiatry: Explanation, phenomenology,
and nosology (pp. 327–367). Johns Hopkins University Press.

Zanchi, D., Depoorter, A., Egloff, L., Haller, S., Mählmann, L., Lang, U. E., Drewe, J., Beglinger,
C., Schmidt, A., & Borgwardt, S. (2017). The impact of gut hormones on the neural circuitry
of appetite and satiety: Systematic review. Neuroscience and Biobehavioral Reviews, 80,
457–475. https://doi.org/10.1016/j.neubiorev.2017.06.013

https://doi.org/10.1016/j.newideapsych.2019.01.004

https://doi.org/10.1007/s10803-015-2641-0

https://doi.org/10.1016/j.jpsychores.2017.06.007

https://doi.org/10.1371/journal.pone.0116820

https://doi.org/10.1177/2156869313519114

https://doi.org/10.1037/13939-000

https://doi.org/10.3389/fnbeh.2014.00410

https://doi.org/10.1177/2167702620969215

https://doi.org/10.1002/eat.22171

https://icd.who.int/

https://doi.org/10.1016/j.neubiorev.2015.11.017

https://doi.org/10.1016/j.neubiorev.2017.06.013

866 Theory & Psychology 31(6)

Author biographies

Hannah Hawkins-Elder, BSc (Hons), is a PhD and clinical psychology student in the school of
psychology at Victoria University of Wellington. Her thesis focuses on the explanation of disor-
dered eating; specifically, taking a metatheoretical approach to how explanations of eating disor-
der phenomena are constructed and utilized in practice. Her recent publications include “The
Explanation of Eating Disorders: A Critical Analysis” in Behaviour Change (2020) and (with T.
Ward) “From Competition to Co-Operation: Shifting the ‘One Best Model’ Perspective” in Theory
& Psychology (2021).

Tony Ward, DipClinPsyc, PhD, is a professor of clinical psychology in the school of psychology
at Victoria University of Wellington, New Zealand. He has over 430 research publications and his
current research focuses on the development of explanatory models in psychopathology and foren-
sic psychology. His recent publications include “Why Theoretical Literacy is Essential for Forensic
Research and Practice” in Criminal Behaviour and Mental Health (2020), “The Classification of
Crime: Towards Pluralism” in Aggression and Violent Behavior (2020), and “Modeling the
Symptoms of Psychopathology: A Pluralistic Approach” in New Ideas in Psychology (2020).

Professional Psychology: Research and Practice

Examining Cognitive Performance and Psychopathology in Individuals
Undergoing Parental Competency Evaluations
Christian Terry and Len Lecci
Online First Publication, December 23, 202

. http://dx.doi.org/10.1037/pro0000436

CITATION
Terry, C., & Lecci, L. (2021, December 23). Examining Cognitive Performance and Psychopathology in Individuals
Undergoing Parental Competency Evaluations. Professional Psychology: Research and Practice. Advance online
publication. http://dx.doi.org/10.1037/pro0000436

Examining Cognitive Performance and Psychopathology in
Individuals Undergoing Parental Competency Evaluations

Christian Terry and Len Lecci
Department of Psychology, University of North Carolina Wilmington

In the determination of parental fitness, or competency of an individual to care for a child, psychological
assessments are often utilized. Moreover, research suggests that parental competency examinees are distinct
from child custody examinees with respect to psychopathology and should be studied as a separate group.
To that end, the present study examined the cognitive functioning of 136 parental competency examinees
who were undergoing court-ordered evaluations, as well as examined the relationship between cognitive
functioning (as assessed by the Wechsler Adult Intelligence Scale-IV [WAIS-IV]) and psychopathology (as
assessed by the Minnesota Multiphasic Personality Inventory-2 [MMPI-2]). Overall, the parental compe-
tency sample had lower education and lower cognitive functioning (particularly Full Scale Intelligence
Quotient [FSIQ] and Working Memory Index [WMI]) than the normative sample. MMPI-2 scores
paralleled those of previous findings for parental competency examinees, and MMPI-2 Scales 8, 0, and
7 were significantly related to WAIS-IV performance, with lower cognitive scores associated with greater
psychopathology. Implications include recognition of the role that cognitive functioning may play in parents
being referred for parental competency evaluations, the interaction of comorbid psychopathology and lower
cognitive functioning, as well as informing treatment recommendations for individuals with co-occurring
psychopathology and cognitive deficits.

Public Significance Statement
Parental competency examinees appear to have significantly lower scores in overall intellectual
functioning and working memory on the Wechsler Adult Intelligence Scale-IV (WAIS-IV) relative
to normative values, and lower cognitive functioning was found to be associated with higher
psychopathology on Minnesota Multiphasic Personality Inventory-2 (MMPI-2) scales related to unusual
thoughts/attitudes, social isolation, and anxiety. These findings indicate co-occurring psychopathology
and cognitive deficits in those referred for parental competency evaluations and this can inform
treatment recommendations.

Keywords: parental competency evaluation, psychopathology, cognition, MMPI-2, WAIS-IV

In evaluating parental fitness, or competency of an individual to
care for a child, psychological assessments are often utilized to aid in
this determination (Budd, 2001; Conley, 2004). Two formal assess-
ments that may be included as part of a parental competency test
battery are the

Minnesota Multiphasic Personality Inventory-2

(MMPI-2; Butcher, 2010) and the Wechsler Adult Intelligence
Scale-IV (WAIS-IV; Wechsler, 2008). Despite the MMPI-2 and
WAIS-IV being among the most commonly administered psycho-
logical assessment instruments in the United States (Ball et al.,
1994), there has been limited research on their relationship with one

another. Indeed, although it has been shown that psychopathology
predicts executive functioning (Snyder et al., 2015), it is unclear
whether MMPI-2 scores meaningfully relate to WAIS-IV perfor-
mance and how this may manifest specifically for parental compe-
tency examinees. Moreover, little is known regarding cognition in
individuals referred for parental competency evaluations. The pres-
ent study seeks to examine the following three issues in a parental
competency sample: (a) levels of cognitive functioning as assessed
by the WAIS-IV, (b) psychopathology as assessed by the MMPI-2,
and (c) the potential overlap among these constructs. Thus, the

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed

b
y
th
e
A
m
er
ic
an

P
sy
ch
o
lo
g
ic
al

A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed

p
u
b
li
sh
er
s.

T
h
is
ar
ti
cl
e
is
in
te
nd
ed

so
le
ly

fo
r
th
e
p
er
so
n
al

u
se

o
f
th
e
in
di
v
id
u
al

u
se
r
an
d
is
n
o
t
to

b
e
d
is
se
m
in
at
ed

b
ro
ad
ly
.

Christian Terry https://orcid.org/0000-0002-3904-0371
CHRISTIAN TERRY received his MA in psychology from the University of

North Carolina Wilmington (UNCW). He is presently completing his
internship at Larned State Hospital in Larned, Kansas to attain his PhD in
clinical psychology from UNCW. His areas of professional interest include
clinical neuropsychological assessment, psychological assessment, and
mindfulness-based interventions.
LEN LECCI received his PhD in clinical psychology from Arizona State

University. He is presently a professor of psychology at the University of

North Carolina Wilmington and director of clinical services at MARS
Memory-Health Network. His research and clinical work focuses on assess-
ment, memory disorders, concussion, health anxiety, and bias.
The authors have no known conflicts of interest to disclose.
Anonymous data used in this study are available at the following link:

https://osf.io/7j6rx/?view_only=baae7d98065a4c47b996c14d34b1b5b1.
CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Len

Lecci, Department of Psychology, University of North Carolina Wilmington,
P.O. Box 5612, 601 S College Road, Wilmington, NC 28403, United States.
Email: leccil@uncw.edu

Professional Psychology: Research and Practice

https://orcid.org/0000-0002-3904-0371

https://osf.io/7j6rx/?view_only=baae7d98065a4c47b996c14d34b1b5b1

mailto:leccil@uncw.edu

https://doi.org/10.1037/pro0000436

purpose of this investigation is to highlight the value of a broader
psychological assessment that includes cognitive testing for parental
competency evaluations, as both cognitive difficulties and their
comorbidity with psychopathology can otherwise be overlooked
(Meyer, 2002).

Parental Competency Evaluations and Intellectual
Functioning

The process of evaluating competency of an individual to parent
is complex. As McGaw et al. (2010) outline, it is typically the case
that parents undergo a competency evaluation secondary to con-
cerns related to neglect rather than intentional abuse. The neglect
often co-occurs with and/or stems from substance use or other
mental health disorders (Resendes & Lecci, 2012), and the latter
can include intellectual limitations.
Although ranges for Intellectual Quotients (IQ) were previously

used to categorize individuals into mild, moderate, and severe
levels of intellectual disabilities using the fourth edition of the
Diagnostic and Statistical Manual of Mental Disorders (DSM-IV;
American Psychiatric Association, 1994), the most recent edition of
the DSM removes IQ ranges as a criterion and instead emphasizes
the consideration of other aspects of client functioning in relation to
their objective cognitive performance to aid in diagnoses (American
Psychiatric Association, 2013). Using the new criteria, the severity
of an individual’s intellectual disability is determined based on
their cognitive performance (Criterion A), evidence that these
cognitive deficits are impairing the individuals’ ability to function
(Criterion B), and onset of these deficits within the developmental
period (Criterion C). Thus, while IQ testing can aid in documenting
impaired cognitive performance (Criterion A), identifying that an
individual’s intellectual functioning is such that they may be unable
to care for a child may also contribute to meeting Criterion B.
Of course, this interpretation is made only when comparing multi-
ple sources of evidence, as IQ testing alone is insufficient to
determine high-risk versus low-risk parents (McGaw et al.,
2010). Moreover, given the potential comorbidity of cognitive
limitations and psychopathology, it is imperative to elucidate the
relation between psychopathology and cognition in parental com-
petency examinees. Unpacking this relationship may aid any
psychological assessor in considering the intersection of cognition
and psychopathology in their patients and making appropriate
recommendations for needed interventions and prognosis for
change.
Originally, it was argued that parental competency and custody

examinees are functionally equivalent (Stredny et al., 2006). How-
ever, more recent research has determined that there are fundamental
differences at least in regard to psychopathology as measured by the
MMPI-2 (Resendes & Lecci, 2012). Specifically, competency ex-
aminees relative to child custody examinees tend to exhibit a
different pattern of defensiveness (elevated L, but lower K),
more inconsistent responding (elevated variable response inconsis-
tency [VRIN]; true response inconsistency [TRIN]), more unusual
responses (elevated F and Fb), and psychopathology related to
social introversion, depressive symptoms, difficulty incorporating
and accepting societal standards, and unusual thoughts or attitudes
(i.e., elevated MMPI-2 Scales 0, 2, 4, and 8, respectively, with
differences reflecting large effect sizes ranging from Cohen’s
d values of 1.01–1.21).

According to Resendes and Lecci (2012),

parental competency evaluations typically involve a legal intervention
by a government agency in order to protect the child (e.g., allegations of
abuse, neglect, etc.), and criminal charges may co-occur. In contrast,
child custody evaluations are civil cases that largely involve parental
disagreement about legal and/or physical custody, without necessarily
involving problems with the basic parenting abilities of either par-
ent. (p. 1055)

In the former, the utilization of both cognitive and psychopatho-
logical assessment is necessary to answer referral questions, such as
those related to neglect that may occur secondary to cognitive
limitations (e.g., WAIS-IV) and psychological/substance use dis-
orders (e.g., MMPI-2).

Cognitive Functioning and Psychopathology

With respect to the overlap between psychopathology and cog-
nitive abilities, poorer cognitive functioning is often a specific
symptom of a psychological disorder (e.g., diminished concentra-
tion in major depressive disorder [MDD], disorganized speech and
behavior in schizophrenia). Further, the literature consistently
shows that greater psychopathology is associated with lower exec-
utive functioning. For instance, Stordal et al. (2005) determined that
those experiencing more depressive or schizophrenia-related symp-
toms as measured by the Brief Psychiatric Rating Scale-Expanded
(Overall & Gorham, 1962) and the General Psychopathology
Subscale of the Positive and Negative Syndrome Scale (Kay et al.,
1987) exhibited poorer executive function skills as measured by the
Wisconsin Card Sorting Test (Heaton et al., 1991), Stroop Color
Word Test (Stroop, 1992), Paced Auditory Serial Addition Test
(Gronwall, 1977), Digits Backwards subscale of the WAIS-IV
(Wechsler, 2008), and Controlled Oral Word Association Test
(Benton et al., 1994). Gass (1991) found that anxiety, as measured
by MMPI Scale 7, significantly predicted poorer performance on the
Speech Perception Test of the Halstead–Reitan (Broshek & Barth,
2000) above and beyond age and education. In all though, other
MMPI clinical scales were generally not associated with cognitive
performance in their sample of veterans.

Snyder et al. (2015) performed a comprehensive literature review
of studies assessing the relationship between various psychopathol-
ogies and executive functioning, including inhibition, attentional
shifting, updating, and working memory. They identified that the
diagnosis of schizophrenia was the strongest predictor of diminished
performance for each of these constructs of executive functioning,
and that bipolar and MDD inversely predicted executive function,
albeit to a lesser degree. Findings were mixed on the relationship
between anxiety and executive functioning (Snyder et al., 2015),
and for MDD, the findings corroborate research explicitly focusing
on the WAIS-IV (Wechsler, 2008). Finally, the initial validation
of the WAIS-IV included investigating the cognitive abilities of
“special groups” (Wechsler, 2008), one of which was individuals
diagnosed with MDD. Results indicated that those with MDD
scored lower than matched controls on the Processing Speed Index
(PSI; Cohen’s d = .26), and these findings align with prior research
(Gorlyn et al., 2006).

Only one known study has directly assessed WAIS profiles of
parental competency examinees (McCartan & Gudjonsson, 2016).
Researchers analyzed 144 individuals who were evaluated for

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

2 TERRY AND LECCI

parental competency in the U.K. (i.e., Child Care Proceedings) and
who were administered the WAIS-III. Results indicated that their
parental competency sample had Full Scale Intelligence Quotient
(FSIQ) scores just over 1 SD (M = 83.64) below the normative
sample (M = 100, SD = 15). The study also assessed gender
differences in personality and psychopathology as measured by
the Millon Clinical Multiaxial Inventory, 3rd Edition (MCMI-III)
but did not assess the association between cognition and psychopa-
thology. This review turns now to the only known studies to directly
evaluate the relationship between psychopathology assessed with
the MMPI and cognitive performance as measured by the WAIS.
Gass and Gutierrez (2017) sought to compare the MMPI-2 with

the WAIS-IV in male veterans using the content scales of the
MMPI-2. Inclusion criteria for the MMPI-2 were that they com-
pleted at least 557 of the 567 items, did not respond randomly or
display a response bias (i.e., VRIN and TRIN < 80 T), did not exaggerate symptoms (i.e., F < 90 T, Fb < 100 T), and did not respond defensively (i.e., K < 60 T). Because the researchers performed correlation analyses between the WAIS-IV indices and MMPI-2 content scales, they condensed the 15 MMPI-2 content scales into three factors of Internalized Emotional Dysfunction (IED; e.g., low self-esteem, low energy, depression), Externalized Emotional Dysfunction (EED; e.g., antisocial attitudes, aggres- siveness, poor anger control), and Fear (i.e., physical health worries, paranoia, and intense fearfulness). None of the WAIS-IV indices correlated with IED and EED, however the Fear factor was signifi- cantly inversely correlated with Perceptual Reasoning Index (PRI), Verbal Comprehension Index (VCI), and FSIQ. The authors also found that those with T-scores greater than 64 on the MMPI-2 Fear factor scales performed more poorly on the VCI and FSIQ indices relative to those with T-scores below 56. It was noted that the Fear factor correlated most strongly with clinical Scales 8, 7, and 3 (Gass & Gutierrez, 2017). Contrary to Gass and Gutierrez’s (2017) findings, Morasco et al.

(2006) found no significant relationships among WAIS-III indices
and MMPI-2 clinical scales in a sample of young adults receiving
psychoeducational evaluations. Thus, it appears that there is limited
and inconsistent evidence of any association between these two
frequently employed measures, and the findings may be specific to
the population under investigation.

The Present Study

Although research has established connections between specific
disorders and executive function abilities, limited research has
assessed whether degree of psychopathology (as opposed to the
dichotomous presence/absence of a disorder) may influence cogni-
tion more broadly. Further, explorations of the relationship between
psychopathology and cognition have been very limited in parental
competency samples, and past research examining the association
between WAIS and MMPI scores (e.g., Gass & Gutierrez, 2017)
may have been limited by employing stringent validity cutoff scores.
The latter issue may be especially relevant for parental competency
evaluations, as some degree of defensiveness has been documented
in such samples (e.g., Resendes & Lecci, 2012). Moreover, average
L scale scores for child custody examinees have also been shown to
be as high as 60 (Bagby et al., 1999, as cited in Graham, 1990).
Given the high comorbidity of psychopathology in those with lower
cognitive functioning (Peña-Salazar et al., 2018), it is critical to

examine the interaction of comorbities in psychological examinees,
especially given that psychopthology and cognitive difficulties can
both individually and in combination undermine functional abilities,
including with respect to parenting. Further, the effect of these
comorbidites on treatment outcomes should be considered when
providing treatment recommendations. Thus, the present study
seeks to document the cognitive (WAIS-IV) scores of parental
competency examinees, their level of psychopathology as assessed
by the MMPI-2, and overlap thereof by making the following
predictions:

1. Given that cognitive limitations are sometimes noted
among the referral questions from Child Protective Services,
average WAIS-IV performance for the competency sample
is predicted to be below that of the normative sample across
all WAIS-IV indices and subscales. Lower WAIS scores
(by approximately 1 SD) would replicate the findings from a
similar U.K. sample (McCartan & Gudjonsson, 2016).

2. To evaluate representativeness with respect to psychopa-
thology, the present competency sample will be compared
to scores obtained in previous parental competency samples
(Resendes & Lecci, 2012; Stredny et al., 2006). It is predicted
that (a) the present sample will generally align with prior
samples and (b) that the present competency sample will
be significantly different from Bathurst et al.’s (1997)
custody sample across all MMPI-2 validity and clinical
scales, reinforcing the argument that competency and cus-
tody examinees require separate interpretive considerations
with respect to psychopathology.

3. We will explore correlations among WAIS-IV indices and
MMPI-2 clinical scale scores. Based on prior findings
regarding MDD and diminished PSI performance, it is
predicted that greater scores on MMPI-2 clinical Scale 2
will predict poorer performance on the WAIS-IV PSI
beyond age and education (Gorlyn et al., 2006). Further,
the relationship between MMPI-2 Scale 8 and WAIS-IV
performance will be explored, given prior findings indicating
a negative relationship between thought disorders (e.g.,
schizophrenia) and cognitive functioning (Snyder et al.,
2015; Stordal et al., 2005).

Method

Participants and Procedure

Participants were 136 individuals aged 19–67 (Mage = 31.93,
SD = 8.93) whom the court system required to complete a psycho-
logical evaluation to aid in the determination of parental compe-
tency. The majority already had their child(ren) removed from the
home due to concerns from Child Protective Services that one or
more problematic circumstances may be present which limit the
individual’s ability to parent. These circumstances included sus-
pected domestic violence (41.9%), substance use (56.6%), child
neglect (37.5%), physical abuse (14%), sexual abuse (6.6%), psy-
chiatric instability (44.9%), and cognitive incompetence (14.7%).
(Note: These reflect nonmutually exclusive concerns, as 73% of
individuals had more than one listed problem). Participants were
predominantly female (70.6%) and Caucasian (53.7%), followed by

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

PARENTAL COMPETENCY WAIS-IV AND MMPI-2 SCORES 3

African American (37.5%) and Native American (1.5%). Average
education was 11.65 years (SD = 1.89) and ranged from 6 to 16 years.
The psychological evaluations were completed by a licensed clinical
psychologist over the course of approximately 5 hr, reflecting
consecutive evaluations occurring from 2012 to 2019. Data were
archivally extracted and analyzed using SPSS Statistics software, and
the study was approved by the University of North Carolina Wil-
mington’s Institutional Review Board (IRB; #20-0110). Anonymous
data used in this study are available at the following link: https://osf
.io/7j6rx/?view_only=baae7d98065a4c47b996c14d34b1b5b1

Measures

Wechsler Adult Intelligence Scale-IV

The fourth edition of the WAIS-IV (Wechsler, 2008) is one of the
most widely used measures of cognitive performance/intelligence
and includes the administration of 10 standard scales which take
approximately 1.5–2 hr to complete. These 10 scales load onto four
broader indices: VCI, PRI, Working Memory Index (WMI), and
PSI. These four indices load onto a primary factor called the FSIQ
(Wechsler, 2008).

Minnesota Multiphasic Personality Inventory-2

The second edition of the MMPI (i.e., MMPI-2) is a 567-item
measure of personality and psychopathology which takes approxi-
mately 1–2 hr to complete (Graham, 1990). It contains embedded
validity measures to defensive responding (e.g., K, S, and L scales),
possible exaggeration (e.g., F, Fp, FBS scales), or indiscriminate
responding (e.g., VRIN, TRIN scales). Additionally, 10 clinical scales
are related to a range of psychological tendencies and possible
psychopathology (e.g., hypochondriasis, depression, somatization in
response to stress, antisocial traits, traditional masculine/feminine
roles, paranoia, psychological turmoil, thought disturbances, hypoma-
nia, and social introversion). The present study focuses on the validity
and primary clinical scales. Typically, a T-score of 65 (1.5 SDs >
average) is the clinical cutoff for clinical interpretation (Graham,
1990), but scores below 65 can indicate meaningful variability.
Although not presented here, participants also completed semi-

structured clinical and parenting interviews and a mental status
exam, and records were available from other providers.

Results

Because the present research focuses on the cognitive functioning
of a parental competency sample, and because cognitive functioning
is related to educational attainment (Kaufman et al., 2009), it is
important to examine the educational background of the present
sample relative to the normative sample for the WAIS-IV to rule this
out as a confounding variable. To determine whether our sample
matched the normative sample in terms of average education, we
first aggregated the percentages of those within each age and
education group reported in the WAIS-IV technical manual. We
then compared these percentages for our sample.
Level of education in the present competency sample is much

lower than in the WAIS-IV normative sample. Specifically, 18.5%
of the competency sample had less than 9 years of education,
compared to only 4.6% of the normative sample. About 14.9%
of the competency sample had 9–11 years of education, compared to

8.7% of the normative sample. While most of the competency
(65.9%) and normative (60.1%) samples had between 12 and
15 years of education, less than 1% of the competency sample
had greater than 15 years of education, compared to 26.6% of the
normative sample. A chi-square analysis was conducted among the
percentage of individuals in the normative versus competency
sample across each educational category and this was significant,
χ2 (3, N = 136) = 56.03, p < .001. These differences suggest that the present competency sample has a significantly greater number of individuals with less than 12 years of education than the normative WAIS-IV sample.

Also, 12.5% of the competency sample scored in the extremely
low range of cognitive functioning (i.e., FSIQ < 70), whereas approximately 2% of the normative sample had FSIQ scores in this range. The 12.5% extremely low FSIQ percentage aligns with the fact that cognitive concerns were noted in the Child Protective Services referrals for 14.7% of the sample. Given that cognitive concerns are a common issue in competency evaluations (identified in 12.5% of the referred cases), this subset of individuals is a contributing factor to the overall competency sample’s WAIS-IV index scores being lower than the normative sample.

Because educational attainment has consistently been shown to
positively predict cognition, we first examined this relationship in
our competency sample. A bivariate correlation was performed
between total years of education and WAIS-IV indices and revealed
significant positive correlations between education and VCI:
r(134) = .23, p = .008; PRI: r(134) = .20, p = .018; WMI: r(134) =
.32, p < .001; PSI: r(134) = .24, p = .005; and FSIQ: r(134) = .29, p < .001. Considering the significant correlation between education and all WAIS-IV indices, and that the education of the competency sample is markedly lower than that of the normative sample, we controlled for education in later analyses that included WAIS-IV scores as a variable.

Hypothesis 1

To compare the WAIS-IV scores of the competency sample with
normative values, single-sample t-tests were conducted. All
WAIS-IV index scores and FSIQ were significantly lower in the
competency sample compared to the normative sample (i.e., stan-
dard score of 100; see Table 1). The lowest index scores were the
WMI, M = 88.30, t(135) = −9.30, p < .001, Cohen’s d = −0.80, and FSIQ, M = 87.79, t(135) = −9.65, p < .001, Cohen’s d = −0.83. Across WAIS-IV indices, our competency sample performs between the 21st and 28th percentile on average. Single-sample t-tests were also conducted between subscale scores and normative (i.e., 10) scaled scores. Again, all subscale scores were significantly lower than the normative scores, with the lowest being Arithmetic and Coding (see Table 1). This indicates that the parental compe- tency sample shows consistently lower cognitive functioning rela- tive to normative standards. These findings persisted after controlling for education.1

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

1 Single-sample t-tests performed in Hypothesis 1 were also conducted
using education-adjusted means obtained from an analysis of covariance
(ANCOVA). However, education-adjusted means were minimally different
than nonadjusted means (i.e., changes were less than one standard point), and
results were minimally different. Thus, these findings are not explicated here.

4 TERRY AND LECCI

https://osf.io/7j6rx/?view_only=baae7d98065a4c47b996c14d34b1b5b1

Hypothesis 2

MMPI-2 validity and clinical scale scores for the present sample
were compared with those from two prior parental competency
samples (Resendes & Lecci, 2012; Stredny et al., 2006). The
hypothesis that scores would be the same between these samples
was largely supported (see Table 2), except that when comparing the
present sample to Stredny et al.’s (2006) competency sample, Scale
8 is significantly higher and Scale 7 is marginally higher for the
present sample. Thus, while slight elevations are seen on two
MMPI-2 scaled scores related to anxiety and unusual thoughts/
attitudes, scores for the present sample are largely the same as those
from two previous parental competency samples, suggesting that
this is a representative sample with respect to MMPI-2-assessed
psychopathology.
The second portion of this hypothesis, that the present compe-

tency sample will be significantly different from Bathurst et al.’s
(1997) custody sample across all MMPI-2 validity and clinical scale
scores, was also supported. Moreover, the observed differences
resulted in at least medium and in most cases large effect sizes. This
finding reinforces the argument that competency and custody ex-
aminees are markedly different (i.e., 12 of 14 comparisons resulted
in statistically significant and substantial differences) and thus likely
require separate interpretive considerations.

Hypothesis 3

As Gass and Gutierrez (2017) have emphasized, obtaining an
accurate analysis of the relationship between psychopathology and
cognitive performance may only be done when examinees are as
effortful as possible when taking the test. In the case of parental
competency evaluations, it is quite typical for examinees to exhibit

full effort on the WAIS-IV while also exhibiting a degree of
defensiveness evidenced by elevations on MMPI-2 validity scales
L and to a lesser degree K. Therefore, inclusion criteria for Hypoth-
esis 3 based on the MMPI-2 are that participants completed at least
557 of the 567 items, did not respond randomly or display a response
bias (i.e., VRIN and TRIN < 80), did not exaggerate symptoms (i.e., F < 90, Fb < 100; which would be extremely rare for such court- ordered parental competency evaluations), and did not respond in a highly defensive manner (i.e., L and K < 80). Three participants (2.9%2) were excluded due to VRIN T-scores greater than or equal to 80, 11 (10.7%) for TRIN T-scores greater than or equal to 80, 12 (8.8%) for F T-scores greater than or equal to 90, eight (6.5%) for Fb T-scores greater than or equal to 100, 24 (17.6%) for L T-scores greater than or equal to 80, and one (0.7%) for a K T-score greater than or equal to 80. These exclusions were not mutually exclusive and after applying the above criteria, 92 of the 136 participants were left for the exploratory correlations.

A partial correlation, controlling for education, was conducted
(see Table 3). The most prominent finding was that Scale 8 (unusual
thoughts/attitudes) was significantly negatively correlated with all
WAIS-IV indices, with the strongest correlation occurring with
WMI, r(90) = −.40, p < .01. This correlation was explored further via hierarchical regression, entering education into Block 1 and Scale 8 into Block 2 to predict WMI. Assumptions for performing the analysis were met. Scatterplots indicated linearity, no skew or kurtosis was detected on p-p and q-q plots, data were homoscedastic, and no violations of independence were noted (e.g., Durbin– Watson = 1.66). Moreover, multicollinearity was not detected. The regression indicated that both education (7.1%, β = 2.53, p < .01, power = .99) and MMPI-2 Scale 8 (13.6%, β = −.48, p < .001, power = .99) significantly explained 20.7% of the variance in WMI performance, R2 = .207, F(2, 89) = 12.85, p < .001. This indicates that after controlling for premorbid cognitive achievement (education), at least one form of MMPI-2-assessed psychopathology predicts poorer cognitive efficiency (working memory) as assessed by the WAIS-IV.

The second most noteworthy finding was the consistently nega-
tive correlations between Scale 0 (social introversion) and all
WAIS-IV indices, that is, r(90) = −.30 to −.33, p < .01, except for PSI, r(90) = −.18, p > .05. The final MMPI-2 scale to
significantly correlate with a majority of WAIS-IV indices was
Scale 7 (psychasthenia), which negatively correlated with PRI,
r(90) = −.27, p < .05; WMI, r(90) = −.28, p < .01; and FSIQ, r(90) = −.24, p < .05. The latter two findings indicate that social introversion and anxiety are both associated with decreased cogni- tive performance. Four other modest correlations emerged between MMPI-2 scales and WAIS-IV indices. Specifically, Scale 9 (hypo- manic activation) was negatively correlated with VCI and FSIQ, Scale 1 (hypochondriasis) was negatively correlated with PSI, and Scale 3 (hysteria) was positively correlated with VCI (this was the only significant positive correlation to emerge).

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

Table 1
Independent Samples t-Tests Between WAIS-IV Normative and
Competency Sample Means

WAIS-IV df
Competency sample

M (SD)
Independent samples t-test

(Cohen’s d)

VCI 135 90.63 (14.91) −7.33* (−0.63)
PRI 135 90.22 (14.95) −7.63* (−0.65)
WMI 135 88.30 (14.68) −9.30* (−0.80)
PSI 135 90.10 (14.25) −8.10* (−0.69)
FSIQ 135 87.79 (14.75) −9.65* (−0.83)
BD 135 7.99 (2.96) −7.92* (−0.68)
SI 135 8.17 (2.86) −7.48* (−0.64)
DS 135 8.18 (2.85) −7.45* (−0.64)
MR 135 9.00 (3.24) −3.60* (−0.31)
VC 135 8.07 (2.83) −7.97* (−0.68)
AR 106 7.10 (2.42) −12.37* (−1.20)
SS 131 8.45 (3.23) −5.53* (−0.48)
CD 133 7.81 (2.66) −9.52* (−0.82)
CO 131 8.83 (3.22) −4.17* (−0.36)

Note. Independent samples t-tests were conducted between normative
scores and competency samples’ scores. WAIS-IV = Wechsler Adult
Intelligence Scale-IV; df = degrees of freedom; VCI = Verbal
Comprehension Index; PRI = Perceptual Reasoning Index; WMI =
Working Memory Index; PSI = Processing Speed Index; FSIQ = Full
Scale Intelligence Quotient; BD = Block Design; SI = Similarities; DS =
Digit Span; MR = Matrix Reasoning; VC = Vocabulary; AR = Arithmetic;
SS = Symbol Search; CD = Coding; CO = Comprehension.
* p < .01.

2 Percentages of excluded cases were based on the total (i.e., n = 136)
sample for F, L, and K but were based on a smaller subset of individuals for
VRIN (n = 103), TRIN (n = 103), and Fb (n = 124) due to unavailability of
T-scores for the remaining participants as they only completed the first 370
items of the MMPI-2. For these latter three scales, those with missing data
were still included in the exploratory correlations as long as F, L, and K were
valid.

PARENTAL COMPETENCY WAIS-IV AND MMPI-2 SCORES 5

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

T
ab

le
2

C
o
m
p
a
ri
n
g
th
e

M
M
P
I-
2

S
ca
le

s
fo
r
C
u
st
o
d
y
a
n
d

C
o
m
p
et
en
cy

S
a
m
p
le
s

A
B

C
D

A
v
er
su
s
B

A
v
er
su
s
C

v
er
su
s
D

M
M
P
I-
2
C
o
m
p
et
en
cy

sa
m
p
le

M
(S
D
);

(N
=

1
36
)

C
h
il
d
cu
st
od
y

sa
m
p
le
M
(S
D
);

B
at
h
u
rs
t
et

al
.,
1
9
9
7

(N
=

5
08
)

C
o
m
p
et
en
cy
sa
m
p
le

M
(S
D
);
R
es
en
d
es

&
L
ec
ci
,
2
0
1
2

(N
=

1
3
6
)

C
o
m
p
et
en
cy
sa
m
p
le
M
(S
D
);

S
tr
ed
n
y
et

al
.,
2
00
6

(N
=

1
2
7
)

In
d
ep
en
d
en
t

sa
m
p
le
s

t-
te
st

(C
oh
en
’s

d
)

In
d
ep
en
d
en
t

sa
m
pl
es

t-
te
st

(C
o
h
en
’s

d
)
In
d
ep
en
d
en
t

sa
m
p
le
s
t-
te
st

(C
oh
en
’s
d
)

L
6
5.
6
5
(1
3
.2
7
)

5
6
.0
1
(1
0
.5
4
)

6
2
.6

(1
4
.1
)

6
4
.3
7
(1
2
.6
4)

8
.9
4*

(.
8
6
)

1
.8
4
(.
2
2
)

.8
1
(.
1
0
)

F
6
0.
6
8
(1
7
.8
2
)

4
4
.6
7
(6
.8
2
)

5
8
.9

(1
6
.0
1
)

5
8
.5
9
(1
9
.1
2)

1
6.
3
0
*
(1
.5
7
)

.8
7
(.
1
1
)

.9
3
(.
1
1
)

K
5
2.
9
7
(1
2
.8
0
)

5
8
.6
8
(8
.6
1
)

5
2
.6

(1
1
.0
1
)

5
1
.5
0
(1
2
.2
7)

−
6
.1
3*

(−
.5
9
)

.2
6
(.
0
3
)

.9
7
(.
1
2
)

F
b

5
7.
3
8
(1
8
.3
0
)

4
4
.1
2
(4
.1
7
)

5
6
.1

(1
5
.0
7
)

5
9
.1
7
(1
7
.9
3)

1
4.
8
6
*
(1
.4
9
)

.6
2
(.
0
8
)

−
.8
0
(−
.1
0
)

1
(H

s)
5
7.
3
4
(1
2
.0
1
)

4
8
.3
9
(7
.1
0
)

5
6
.6

(1
1
.3
)

5
4
.2
5
(1
1
.6
1)

1
1.
0
7
*
(1
.0
7
)

.5
2
(.
0
6
)

2
.1
6
(.
2
6
)

2
(D

)
5
7.
7
9
(1
1
.1
0
)

4
6
.6
2
(7
.1
1
)

5
7
.2

(1
1
.5
)

5
5
.8
1
(1
1
.0
8)

1
4.
2
6
*
(1
.3
8
)

.4
3
(.
0
5
)

1
.4
7
(.
1
8
)

3
(H

y
)

5
3.
7
9
(1
2
.4
2
)

5
2
.3
1
(7
.8
9
)

5
4
.6

(1
1
.5
)

5
2
.7
4
(1
2
.3
2)

1
.7
0
(.
1
6
)

−
.5
6
(−
.0
7)

.7
0
(.
0
9
)

4
(P
d)

6
2.
0
4
(1
1
.5
2
)

5
0
.8
7
(7
.3
5
)

6
3
.3

(1
2
.4
)

6
0
.2
6
(1
2
.8
9)

1
3.
7
7
*
(1
.3
3
)

−
.8
7
(−
.1
1)

1
.2
0
(.
1
5
)

5
(M

f)
5
2.
7
7
(1
0
.3
5
)

5
0
.5
6
(8
.8
3
)

5
2
.8

(1
0
.5
)

5
8
.4
6
(1
6
.0
9)

2
.5
0
(.
2
4
)

−
.0
2
(−
.0
03
)

−
3
.4
7*

(−
.4
2
)

6
(P
a)

6
0.
3
0
(1
5
.7
4
)

5
2
.4
4
(8
.9
6
)

5
9
.4

(1
3
.7
)

5
8
.4
6
(1
3
.7
2)

7
.5
8*

(.
7
3
)

.5
0
(.
0
6
)

1
.0
3
(.
1
2
)

7
(P
t)

5
5.
3
6
(1
1
.8
5
)

4
7
.1
8
(6
.7
7
)

5
4
.3

(1
1
.2
)

5
0
.9
1
(1
4
.5
7)

1
0.
4
5
*
(1
.0
1
)

.7
6
(.
0
9
)

2
.7
6*
*
(.
3
4
)

8
(S
c)

5
9.
4
0
(1
3
.7
9
)

4
6
.8
7
(6
.6
2
)

5
6
.9

(1
1
.9
)

5
4
.6
2
(1
0
.6
6)

1
5.
0
3
*
(1
.4
5
)

1
.6
0
(.
1
9
)

3
.2
0*

(.
3
9
)

9
(M

a)
5
5.
5
6
(9
.5
3
)

4
8
.3
5
(7
.5
7
)

5
4
.3

(1
0
.3
)

5
4
.8
7
(1
1
.6
7)

9
.3
1*

(.
9
0
)

1
.0
5
(.
1
3
)

.5
3
(.
0
7
)

0
(S
i)

5
1.
1
6
(1
0
.0
3
)

4
2
.6
9
(7
.1
)

5
1
.7

(1
0
.4
)

5
1
.6
8
(9
.4
5
)

1
1.
2
4
*
(1
.0
8
)

−
.4
4
(−
.0
5)

−
.4
4
(−
.0
5
)

N
o
te
.

A
B
o
n
fe
rr
o
n
ia
lp
h
a
co
rr
ec
ti
o
n

w
as

ap
p
li
ed

to
th
e
v
al
ue

o
f
.0
5
fo
r
1
4
o
n
e-
ta
il
ed

t-
te
st
s
ac
ro
ss
ea
ch

o
f
th
e
th
re
e
co
m
p
ar
is
o
n
s,
y
ie
ld
in
g
a
v
al
ue

o
f
.0
0
3
5
7.
C
o
lu
m
n
A
v
al
u
es

ar
e
fr
o
m
th
e
p
re
se
n
ts
tu
d
y
(F
b
M

an
d
S
D
is
ca
lc
u
la
te
d
fr
o
m
1
2
4
p
ar
ti
ci
p
an
ts
fo
r
co
lu
m
n
A
,a
n
d
al
l
o
th
er
sc
al
es

ar
e
fr
o
m
N
=
1
36
).
C
o
lu
m
n
B
v
al
ue
s
ar
e
fr
o
m

B
at
h
u
rs
t
et
al
.,
1
9
9
7
,T

ab
le
1
.C

o
lu
m
n
C
v
al
u
es

ar
e
fr
o
m
R
es
en
d
es

&
L
ec
ci
,2
0
1
2
,

T
ab
le
1
.C

o
lu
m
n
D
v
al
ue
s
ar
e
fr
o
m
S
tr
ed
n
y
et
al
.,
2
00
6
,T

ab
le
1
.M

M
P
I-
2
=
M
in
n
es
o
ta
M
ul
ti
p
h
as
ic
P
er
so
n
al
it
y
In
v
en
to
ry
-2
.W

e
co
n
ta
ct
ed

S
tr
ed
n
y
et
al
.,
2
00
6
re
g
ar
d
in
g
th
e
A
v
er
su
s
D
S
ca
le
5
d
is
cr
ep
an
cy

to
v
er
if
y
th
at

S
ca
le

5
w
as

5
8
.4
6
in

th
ei
r
sa
m
p
le

(a
s
S
ca
le

6
al
so

w
as

5
8
.4
6
),
an
d
th
ey

w
er
e
u
na
b
le

to
v
er
if
y.

T
h
u
s,
th
is
d
is
cr
ep
an
cy

is
n
ot

fu
rt
h
er

ex
p
lo
re
d
.

*
p
<

.0
0
3
.

*
*
T
h
e
p
v
al
u
e
ac
hi
ev
ed

fo
r
th
e
A

v
er
su
s
D

an
al
y
si
s
o
f
S
ca
le

7
w
as

p
=

.0
0
6
.

6 TERRY AND LECCI

Contrary to the previous literature and our prediction, no corre-
lation was found between Scale 2 (depressive symptoms) and PSI.
This effect also failed to emerge when dichotomizing MMPI-2
Scale 2 scores into high and low, to be more commensurate with the
previous literature (Gass & Gutierrez, 2017). Thus, depressive
symptoms and processing speed were unrelated.

Discussion

The rationale for examining the cognitive abilities of those
undergoing parental competency evaluations is that limited cogni-
tive functioning has been raised as a concern by Child Protective
Services. Moreover, when evaluating parental competency more
broadly, cognitive functioning is often at the forefront. For example,
court cases pertaining to termination of parental rights cite low IQ on
behalf of the parent as a barrier to parenting in the majority of cases
(Callow et al., 2017). Despite this reliance on intellectual abilities,
cognition alone is a poor predictor of parental competency (McGaw
et al., 2010). Instead, a combination of low intellectual functioning
with co-occurring risk factors (e.g., increased psychopathology) and
lack of protective factors (e.g., low social support) is more predictive
of child outcome and, hence, parental competency (Feldman et al.,
2012). Thus, the present study sought to examine (a) levels of
cognitive functioning as assessed by the WAIS-IV, (b) psychopa-
thology as assessed by the MMPI-2, and (c) the potential overlap
among these constructs, as it presents in parental competency
evaluations.
As expected, the parental competency sample had lower educa-

tion than the normative WAIS-IV sample and had a greater inci-
dence of individuals with extremely low cognitive functioning (i.e.,
12.5%) compared to the normative sample (i.e., 2%). Moreover, our
sample’s WAIS-IV performance was over half a standard deviation
below that of the normative sample. This finding indicates that
parental competency examinees score on average in the 21st to 28th
percentile of cognition. The fact that our sample’s cognitive perfor-
mance aligns with another (i.e., McCartan & Gudjonsson, 2016)
parental competency sample’s performance, and that a notable (i.e.,
12.5%) portion of this sample exhibited extremely low cognitive

function, suggests that similar rates may exist in other parental
competency samples elsewhere. Though prior findings suggest
that parental competency assessors are including standardized cog-
nitive measures in their test batteries (Conley, 2004), assessment
of cognitive functioning is not an explicit recommendation
found in parental competency assessment guidelines (American
Psychological Association [APA], 2013; Steinhauer, 1983) and
therefore these areas of potentially problematic functioning may
be overlooked. Such an oversight may lead to attributing problem-
atic parenting behaviors (e.g., neglect) to more nefarious explana-
tions (i.e., intentional neglect), when in fact the provision of
parenting classes and/or supportive services could remedy such
behaviors. Notably, when examining our sample, 35% of those with
FSIQ scores less than 70 were referred due to concerns of neglect.

Aside from the FSIQ, the WMI was normatively the lowest index
and was 0.80 SDs below average. Given that the WMI was strongly
associated with MMPI-2 Scales 7, 8, and 0, it is possible that the
WMI difficulties tend to manifest in the context of mood and thought
disturbances (Snyder et al., 2015; Stordal et al., 2005). Moreover, it
is also reasonable to assume that the more problems that a parent has
(i.e., difficulties affecting multiple domains, such as cognition and
psychopathology) the more likely they will experience dysfunction
in general and come to the attention of Child Protective Services
(e.g., recall that 14.7% of examinees in our sample were referred
for cognitive concerns, and 12.5% scored in the extremely low
range of cognitive functioning). Thus, although diminished cogni-
tion and psychopathology are each factors that may prompt or
contribute to Child Protective Services intervention, they likely
have a compounding effect on one another. This effect highlights
the importance of considering the interaction of cognition with
psychopathology.

The underperformance on WAIS-IV subscales, particularly
Arithmetic and Vocabulary, aligns with prior findings indicating
that lower education corresponds to underperformance on these
WAIS subscales (Shuttleworth-Edwards et al., 2004). In our sample,
34.6% of participants performed more than 1 SD below the norma-
tive average on Vocabulary, and 45.8% for Arithmetic. This under-
performance suggests that both reading literacy and numerical
literacy issues may also be common in parental competency cases.
Thus, careful consideration should be made regarding a client’s
educational attainment and literacy and the effect that it may have in
suppressing cognitive performance, especially in evaluations as
consequential as those pertaining to parental competency. Adapta-
tions may be necessary when illiteracy is of concern (e.g., reading
aloud questionnaires to the examinee instead of having them read for
themselves when dealing with surveys, etc.). Moreover, limitations
associated with literacy may be more amenable to interventions as
compared to cognitive limitations that may be associated with
congenital or developmental disorders.

The present sample’s scores on the MMPI-2 were largely similar
to those of two prior parental competency samples (Resendes &
Lecci, 2012; Stredny et al., 2006) with the exception that, when
compared to Stredny et al.’s (2006) competency sample, Scale 8 is
significantly higher in the present sample. This difference is likely
attributable to the present sample having more individuals with
disturbances in thinking than in Stredny et al.’s (2006) sample,
particularly given the lack of difference on Scale 8 between the
present sample and Resendes and Lecci’s (2012) sample. Significant
elevations on Scale 8 (i.e., T > 75) raises the possibility of a

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

Table 3
Partial Correlation Between WAIS-IV Indices and MMPI-2 Clinical
Scale Scores, Controlling for Education

MMPI-2 VCI PRI WMI PSI FSIQ

1 (Hs) −.03 −.06 −.15 −.25* −.12
2 (D) 0.0 −.11 −.06 −.10 −.08
3 (Hy) .25* .16 .08 .09 .18
4 (Pd) .17 .12 .04 .14 .15
5 (Mf) −.05 −.16 −.02 −.01 −.08
6 (Pa) −.07 −.11 −.18 −.16 −.13
7 (Pt) −.15 −.27* −.28** −.11 −.24*
8 (Sc) −.23* −.28** −.40** −.29** −.33**
9 (Ma) −.25* −.12 −.21 −.19 −.21*
0 (Si) −.31** −.31** −.30** −.18 −.33**

Note. WAIS-IV =Wechsler Adult Intelligence Scale-IV;MMPI-2 = Minnesota
Multiphasic Personality Inventory-2; VCI = Verbal Comprehension Index; PRI =
Perceptual Reasoning Index; WMI = Working Memory Index; PSI = Processing
Speed Index; FSIQ = Full Scale Intelligence Quotient. Scales 1–0 correspond to
MMPI-2 clinical scale T-scores.
* p < .05. ** p < .01.

PARENTAL COMPETENCY WAIS-IV AND MMPI-2 SCORES 7

psychotic disorder (Graham, 1990) or at least disturbances in
thinking about oneself or others. Schizophrenia and other psychotic
disorders are usually characterized as maintaining a chronic course,
high comorbidity with substance use and other psychiatric disorders,
and frequently necessitate formal and/or informal assistance with
daily living activities (American Psychiatric Association, 2013).
The impact of these disorders on daily functioning likely increases
the chance of a parent with such a disorder, especially when
undiagnosed and/or untreated, to come to the attention of Child
Protective Services and thereby may account for the greater inci-
dence of Scale 8 elevations in parental competency samples. The
replication of Resendes and Lecci’s (2012) findings regarding the
substantial difference between parental competency and custody
samples further highlights that parental competency examiners
should not use parental custody data as a means of comparison,
and vice versa.
Following the precautions recommended by Gass and Gutierrez

(2017), we excluded participants who exhibited invalid response
sets on the MMPI-2 prior to conducting the exploratory correlations
in Hypothesis 3. It is noteworthy that upwards of 17% of respon-
dents had invalid protocols, though this aligns with a previous
competency sample (Resendes & Lecci, 2012). As a follow-up
exploratory analysis, we sought to examine whether intellectual
functioning (i.e., WAIS-IV FSIQ) is related to MMPI-2 invalidity.
Independent samples t-tests were conducted separately between
those who were and were not excluded due to L, TRIN, and F
invalidity. FSIQ scores were significantly lower for those excluded
based on elevated TRIN and F T-scores, with the differences
reflecting large effect sizes (Cohen’s d values of 1.3 and 1.1,
respectfully). Though nonsignificant, those who were excluded
due to elevated L T-scores had lower FSIQ scores (M = 83.67)
than those who were not excluded (M = 88.68). Despite causality
being unclear, these findings implicate low intellectual functioning
as a potential source of MMPI-2 invalidity regarding TRIN, atypical
responding (F), and possibly defensiveness (L). Clinicians should
therefore be mindful to not assume that an invalid MMPI-2 profile
indicates a blatant attempt by the examinee to subvert the
assessment.
The relationship between Scale 8, 0, and 7 and WAIS-IV indices

has important implications for assessing parental competency ex-
aminees. For instance, Scale 8 is consistently negatively correlated
with WAIS-IV indices suggesting that diminished cognition may be
an accompanying factor for individuals with a greater propensity to
experience unusual thinking/attitudes. Importantly, only six parti-
cipants scored at a level on Scale 8 which would more strongly
suggest the presence of a thought disorder (T-score > 75) and a
relatively wide range of T-scores (32–93) was obtained. Thus, the
relationship between Scale 8 and WAIS-IV indices was likely not
solely (or at all) attributable to individuals with thought disorders.
Although we considered cognition to be the dependent variable in

our analyses (e.g., Scale 8 predicted WMI), the directionality of the
relationship between psychopathology and cognition is unclear. For
example, working memory deficits are a common feature of schizo-
phrenia (Eryilmaz et al., 2016), yet whether these deficits are a
consequence of or comorbid with schizophrenia is not established.
Despite nebulous causality, our findings support previous literature
indicating that impaired working memory on the WAIS-IV is seen in
individuals with disturbances of thinking, as measured by MMPI-2
Scale 8 (Snyder et al., 2015; Stordal et al., 2005). Our findings even

partially support those of Gass and Gutierrez (2017) regarding the
impact of fear-related MMPI-2 content scales (which includes
Bizarre Mentation, a scale strongly correlated with clinical
Scale 8) on the WAIS-IV PRI, VCI, and FSIQ. However, the
fact that their Fear factor did not correlate with WMI is less
consistent with present findings.

The broad negative correlation between social introversion (Scale
0) and all WAIS-IV indices except for PSI is consistent with the fact
that diminished social engagement has been found to occur in
individuals with significant psychopathology, such as those with
schizophrenia spectrum disorders (Green et al., 2018). Further, it
may also be the case that those with lower cognitive functioning
have lower social support and, thus, are more socially isolated
(Graham, 1990). Regardless of causality, the co-occurrence of low
cognition and social introversion implicates the need to provide
supportive services (e.g., support groups and other tangible inter-
personal help) to individuals with lower cognition and elevated
psychopathology.

A significant negative relationship also emerged between
MMPI-2 Scale 7 and perceptual reasoning, working memory,
and FSIQ. As a measure of psychological turmoil and anxiety, it
is reasonable that individuals elevated on this scale would exhibit
impaired cognitive performance; particularly given consistent find-
ings that anxiety can impact performance on tasks of working
memory and attention (Dorenkamp & Vik, 2018). Though some
additional, small correlations emerged, these findings must be
interpreted with caution given the exploratory nature of the analysis,
and future studies can better determine whether these relationships
are robust.

Finally, the prediction that MMPI-2 Scale 2 would negatively
correlate with PSI was not supported despite depressive symptoms
commonly predicting low processing speed in the literature (Gorlyn
et al., 2006; Snyder et al., 2015; Wechsler, 2008). One explanation
that may account for this is in the measurement of depression. While
prior studies utilized the diagnosis of MDD as a predictor for PSI,
we used a continuous measure of depressive symptoms. It may be
the case that no relationship emerged because MMPI-2 Scale 2
linearly measures depressive symptoms, rather than as a clinically
significant diagnosis (e.g., does not include a measure of significant
impairment in daily functioning). Other factors related to undergo-
ing parental competency examinations may have also contributed to
the lack of a relationship between Scale 2 and PSI, such as greater
effort typically seen in parental competency examinees, whereas
amotivation is a common symptom of depression and may manifest
more readily in other assessment contexts. The relationship between
MMPI-2 Scale 2 and PSI performance should be assessed in more
diverse samples to account for sampling bias.

We argue that the above findings underscore the need to consider
comorbidities and the interactions thereof that may be present in
psychological assessments of parental competency examinees. The
present study specifically highlights the cross section between
cognition and psychopathology, but there are countless other
comorbid factors that examiners must consider, including most
notably substance use disorders.

Limitations

Several limitations should be noted. First, our sample was a
parental competency sample with a majority of females, and thus our

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

8 TERRY AND LECCI

findings regarding cognition and psychopathology have limited
generalizability beyond this population. However, the fact that
some relationships align with findings in other populations, such
as the relationship between Scale 8 and WMI, indicates this may be a
robust finding. It should also be noted that the majority of the sample
being female is likely due to the greater tendency of females to take a
primary role in caring for the child, rather than females being more
likely to have compromised parenting. Further, our sample was
obtained archivally and, thus, we were unable to collect other
information which may have informed these analyses, such as
stand-alone validity tests or psychiatric diagnoses. Comparing
cognitive performance with both continuous measures of psycho-
pathology (e.g., MMPI-2) in addition to discrete diagnoses would
provide important insight into the relationship between psychopa-
thology and cognition. That is, identifying thresholds at which
psychopathology impacts cognition may inform decisions for
when and to what extent to provide interventions. Finally, the
newest (third) edition of the MMPI has recently been released,
and development of the fifth edition of the WAIS is underway. We
utilized the MMPI-2 instead of the MMPI-2-Restructured Form or
MMPI-3 because the data collection began in 2012, which predated
the newer versions. However, the findings of the present study
should be replicated with the MMPI-3 and WAIS-V, when avail-
able, to determine whether these relationships persist despite revi-
sions to the instruments.

Implications and Applications

Evaluations and research of parental competency should be done
with adherence to the Ethics Code (American Psychological
Association, 2017) and to guidelines laid out for conducting psy-
chological evaluations related to child protection (APA, 2013).
These guidelines include, but are not limited to, maintaining impar-
tiality, practicing with competence, using multiple data-gathering
approaches, and basing conclusions on actuarial data. The present
study provides additional actuarial data specific to parental compe-
tency evaluations when the WAIS-IV and MMPI-2 are used. These
data may inform such assessments and facilitate adherence to ethical
guidelines.
It should also be noted that, although there is a clear rationale for

the use of tests like the WAIS-IV in parental competency evalua-
tions, as cognitive limitations can contribute to less than optimal
functioning and coping, there are also clear limitations in using such
tests, especially for child custody evaluations (e.g., Brodzinsky,
1993). Measures such as the WAIS-IV and MMPI-2 are by them-
selves insufficient to inform whether an individual should be
considered to have low parental competence (McGaw et al., 2010).
Rather, assessors must integrate multiple sources of data (e.g., social
support, occupation) and consider the interaction of the examinee’s
risk and protective factors when highlighting possible areas of
strengths and weaknesses for parental competency examinees
(Callow et al., 2017; Feldman et al., 2012).
This study represents the first attempt to identify relationships

between psychopathology as measured by the MMPI-2 and cogni-
tion as measured by the WAIS-IV for parental competency exam-
inees. Additionally, we have made WAIS-IV index and subscale
data of parental competency examinees available for the first time,
which may aid evaluators in future parental competency assess-
ments as a point of comparison. The data may also be used to

determine a client’s intellectual abilities relative to the average
parental competency examinee and whether this may predict paren-
tal fitness, in conjunction with MMPI-2 performance.

Of clinical relevance is the co-occurrence of diminished cognition
in those with psychopathology, which suggests that the effective-
ness of interventions for individuals with disturbances in thinking
and/or mood is inversely related to cognitive scores. The presence of
borderline intellectual functioning in those who may evidence
symptoms of a thought disorder does not bar an individual from
being able to effectively participate in treatment (Pitschel-Walz
et al., 2009). Rather, the provision of interventions to those with
greater psychopathology and/or lower intellectual abilities may be
most effective when it is conveyed in a concrete and easily under-
stood manner, focuses on skills-based techniques (e.g., dialectical
behavior therapy), and minimizes interventions requiring high
cognitive demand (e.g., cognitive or insight-oriented therapies).
Unsurprisingly, decreased functioning has been observed in
higher-severity psychotic disorders with comorbid intellectual dys-
function compared to those with similar psychotic disorders without
intellectual impairment (Bouras et al., 2004). Thus, a good under-
standing of cognitive functioning is critical to parental competency
assessments to inform prognosis and treatment recommendations.

Importantly, noncompliance or poor treatment outcome may be
perceived as a volitional behavior on the part of the patient to not
engage in treatment. However, our findings suggest that comorbid
conditions may contribute to this noncompliance. For example, a
common treatment recommendation for parental competency ex-
aminees is to participate in parenting classes. However, if borderline
or impaired intellectual functioning is present (and especially if it
goes undetected), then the ability of that individual to understand
and engage in the course material may be jeopardized. In the very
least, lower cognitive functioning can undermine the informational
benefit that can be derived from such classes. As such, a recom-
mendation for one-on-one parent training, as opposed to a group
format, may be more appropriate. Such an understanding may
additionally foster a more compassionate approach to treatment
on behalf of the individual administering the intervention when they
encounter treatment barriers with the patient. In all, clinicians should
consider all risk (e.g., cognitive impairment, psychopathology, lack
of employment/housing) and protective (e.g., social support, dis-
ability/other assistive services) factors in evaluations of parental
competency, and treatment recommendations should likewise be
guided by such factors.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual
of mental disorders (DSM-IV).

American Psychiatric Association. (2013). Diagnostic and statistical manual
of mental disorders (DSM-5®). American Psychiatric Pub.

American Psychological Association [APA]. (2013). Guidelines for psycho-
logical evaluations in child protection matters. American Psychologist,
68(1), 20–31. https://doi.org/10.1037/a0029891

American Psychological Association. (2017). Ethical principles of psychol-
ogists and code of conduct (2002, amended effective June 1, 2010, and
January 1, 2017). https://www.apa.org/ethics/code/

Bagby, R. M., Nicholson, R. A., Buis, T., Radovanovic, H., & Fidler, B. J.
(1999). Defensive responding on the MMPI-2 in family custody and
access evaluations. Psychological Assessment, 11(1), 24–28. https://
doi.org/10.1037/1040-3590.11.1.24

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

PARENTAL COMPETENCY WAIS-IV AND MMPI-2 SCORES 9

https://doi.org/10.1037/a0029891

https://www.apa.org/ethics/code/

https://doi.org/10.1037/1040-3590.11.1.24

Ball, J. D., Archer, R. P., & Imhof, E. A. (1994). Time requirements of
psychological testing: A survey of practitioners. Journal of Personality
Assessment, 63(2), 239–249. https://doi.org/10.1207/s15327752jpa
6302_4

Bathurst, K., Gottfried, A. W., & Gottfried, A. E. (1997). Normative data for
the MMPI-2 in child custody litigation. Psychological Assessment, 9(3),
205–211. https://doi.org/10.1037/1040-3590.9.3.205

Benton, A. L., deS, K., & Sivan, A. (1994). Multilingual aphasia examina-
tion. AJA Associates.

Bouras, N., Martin, G., Leese, M., Vanstraelen, M., Holt, G., Thomas, C.,
Hindler, C., & Boardman, J. (2004). Schizophrenia-spectrum psychoses in
people with and without intellectual disability. Journal of Intellectual
Disability Research, 48(6), 548–555. https://doi.org/10.1111/j.1365-2788
.2004.00623.x

Brodzinsky, D. M. (1993). On the use and misuse of psychological testing in
child custody evaluations. Professional Psychology, Research and Prac-
tice, 24(2), 213–219. https://doi.org/10.1037/0735-7028.24.2.213

Broshek, D. K., & Barth, J. T. (2000). The halstead-reitan neuropsychologi-
cal test battery. In G. Groth-Marnat (Ed.), Neuropsychological assessment
in clinical practice: A guide to test interpretation and integration (pp.
223–262). Wiley.

Budd, K. S. (2001). Assessing parenting competence in child protection
cases: A clinical practice model. Clinical Child and Family Psychology
Review, 4(1), 1–18. https://doi.org/10.1023/A:1009548509598

Butcher, J. N. (2010). Minnesota multiphasic personality inventory. The
Corsini Encyclopedia of Psychology, 1–3. https://doi.org/10.1002/
9780470479216.corpsy0573

Callow, E., Tahir, M., & Feldman, M. (2017). Judicial reliance on parental IQ
in appellate-level child welfare cases involving parents with intellectual
and developmental disabilities. Journal of Applied Research in Intellectual
Disabilities, 30(3), 553–562. https://doi.org/10.1111/jar.12296

Conley, C. (2004). A review of parenting capacity assessment reports.
Journal of the Ontario Association of Children’s Aid Societies, 47(3),
16–23. https://www.oacas.org/pubs/oacas/journal/2003_2004winter/
2003_2004winter

Dorenkamp, M. A., & Vik, P. (2018). Neuropsychological assessment
anxiety: A systematic review. Practice Innovations, 3(3), 192–211.
https://doi.org/10.1037/pri0000073

Eryilmaz, H., Tanner, A. S., Ho, N. F., Nitenson, A. Z., Silverstein, N. J.,
Petruzzi, L. J., Goff, D. C., Manoach, D. S., & Roffman, J. L. (2016).
Disrupted working memory circuitry in schizophrenia: Disentangling
fMRI markers of core pathology vs other aspects of impaired performance.
Neuropsychopharmacology, 41(9), 2411–2420. https://doi.org/10.1038/
npp.2016.55

Feldman, M., McConnell, D., & Aunos, M. (2012). Parental cognitive
impairment, mental health, and child outcomes in a child protection
population. Journal of Mental Health Research in Intellectual Disabilities,
5(1), 66–90. https://doi.org/10.1080/19315864.2011.587632

Gass, C. S. (1991). Emotional variables and neuropsychological test
performance. Journal of Clinical Psychology, 47(1), 100–104. https://
doi.org/10.1002/1097-4679(199101)47:1<100::AID-JCLP2270470116>
3.0.CO;2-H

Gass, C. S., & Gutierrez, L. (2017). Psychological variables and Wechs-
ler adult intelligence scale-IV performance. Applied Neuropsychology.
Adult, 24(4), 357–363. https://doi.org/10.1080/23279095.2016
.1185427

Gorlyn, M., Keilp, J. G., Oquendo, M. A., Burke, A. K., Sackeim, H. A., &
John Mann, J. (2006). The WAIS-III and major depression: Absence of
VIQ/PIQ differences. Journal of Clinical and Experimental Neuropsy-
chology, 28(7), 1145–1157. https://doi.org/10.1080/13803390500246944

Graham, J. R. (1990). MMPI-2: Assessing personality and psychopathology.
Oxford University Press.

Green, M. F., Horan, W. P., Lee, J., McCleery, A., Reddy, L. F., & Wynn,
J. K. (2018). Social disconnection in schizophrenia and the general

community. Schizophrenia Bulletin, 44(2), 242–249. https://doi.org/10
.1093/schbul/sbx082

Gronwall, D. M. (1977). Paced auditory serial-addition task: A measure of
recovery from concussion. Perceptual and Motor Skills, 44(2), 367–373.
https://doi.org/10.2466/pms.1977.44.2.367

Heaton, R. K., Grant, I., & Matthews, C. G. (1991). Comprehensive norms
for an expanded Halstead-Reitan Battery: Demographic corrections,
research findings, and clinical applications. Psychological Assessment
Resources.

Kaufman, A. S., Kaufman, J. C., Liu, X., & Johnson, C. K. (2009). How do
educational attainment and gender relate to fluid intelligence, crystallized
intelligence, and academic skills at ages 22–90 years?. Archives of
Clinical Neuropsychology, 24(2), 153–163. https://doi.org/10.1093/
arclin/acp015

Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative
syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin,
13(2), 261–276. https://doi.org/10.1093/schbul/13.2.261

McCartan, D., & Gudjonsson, G. (2016). Gender differences in MCMI-III
and WAIS-III scores in parental competency examinees. Personality and
Individual Differences, 102, 36–40. https://doi.org/10.1016/j.paid.2016
.06.035

McGaw, S., Scully, T., & Pritchard, C. (2010). Predicting the unpredictable?
Identifying high-risk versus low-risk parents with intellectual disabilities.
Child Abuse & Neglect: The International Journal, 34(9), 699–710.
https://doi.org/10.1016/j.chiabu.2010.02.006

Meyer, G. J. (2002). Implications of information-gathering methods for a
refined taxonomy of psychopathology. In L. E. Beutler & M. L. Malik
(Eds.), Rethinking the DSM: A psychological perspective (pp. 69–105).
American Psychological Association; https://doi.org/10.1037/10456-003

Morasco, B. J., Gfeller, J. D., & Chibnall, J. T. (2006). The relationship
between measures of psychopathology, intelligence, and memory among
adults seen for psychoeducational assessment. Archives of Clinical
Neuropsychology, 21(4), 297–301. https://doi.org/10.1016/j.acn.2006
.03.005

Overall, J. E., & Gorham, D. R. (1962). The brief psychiatric rating scale.
Psychological Reports, 10(3), 799–812. https://doi.org/10.2466/pr0.1962
.10.3.799

Peña-Salazar, C., Arrufat, F., Santos, J. M., Novell, R., & Valdés-Stauber, J..
(2018). Psychopathology in borderline intellectual functioning: A narra-
tive review. Advances in Mental Health and Intellectual Disabilities,
12(1), 22–33. https://doi.org/10.1108/AMHID-07-2017-0031

Pitschel-Walz, G., Bäuml, J., Froböse, T., Gsottschneider, A., & Jahn, T.
(2009). Do individuals with schizophrenia and a borderline intellectual
disability benefit from psychoeducational groups?. Journal of Intellectual
Disabilities, 13(4), 305–320. https://doi.org/10.1177/1744629509353237

Resendes, J., & Lecci, L. (2012). Comparing the MMPI-2 scale scores of
parents involved in parental competency and child custody assessments.
Psychological Assessment, 24(4), 1054–1059. https://doi.org/10.1037/
a0028585

Shuttleworth-Edwards, A. B., Kemp, R. D., Rust, A. L., Muirhead, J. G.,
Hartman, N. P., & Radloff, S. E. (2004). Cross-cultural effects on IQ test
performance: A review and preliminary normative indications on WAIS-III
test performance. Journal of Clinical and Experimental Neuropsychology,
26(7), 903–920. https://doi.org/10.1080/13803390490510824

Snyder, H. R., Miyake, A., & Hankin, B. L. (2015). Advancing understand-
ing of executive function impairments and psychopathology: Bridging the
gap between clinical and cognitive approaches. Frontiers in Psychology,
6, Article 328. https://doi.org/10.3389/fpsyg.2015.00328

Steinhauer, P. D. (1983). Assessing for parenting capacity. American
Journal of Orthopsychiatry, 53(3), 468–481. https://doi.org/10.1111/j
.1939-0025.1983.tb03391.x

Stordal, K. I., Mykletun, A., Asbjørnsen, A., Egeland, J., Landrø, N. I.,
Roness, A., Rund, B. R., Sundet, K. S., Lundervold, A. J., & Lund, A.
(2005). General psychopathology is more important for executive

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

10 TERRY AND LECCI

https://doi.org/10.1207/s15327752jpa6302_4

https://doi.org/10.1037/1040-3590.9.3.205

https://doi.org/10.1111/j.1365-2788.2004.00623.x

https://doi.org/10.1037/0735-7028.24.2.213

https://doi.org/10.1023/A:1009548509598

https://doi.org/10.1002/9780470479216.corpsy0573

https://doi.org/10.1111/jar.12296

https://www.oacas.org/pubs/oacas/journal/2003_2004winter/2003_2004winter

https://doi.org/10.1037/pri0000073

https://doi.org/10.1038/npp.2016.55

https://doi.org/10.1080/19315864.2011.587632

https://doi.org/10.1002/1097-4679(199101)47:1%3C100::AID-JCLP2270470116%3E3.0.CO;2-H

https://doi.org/10.1080/23279095.2016.1185427

https://doi.org/10.1080/13803390500246944

https://doi.org/10.1093/schbul/sbx082

https://doi.org/10.2466/pms.1977.44.2.367

https://doi.org/10.1093/arclin/acp015

https://doi.org/10.1093/schbul/13.2.261

https://doi.org/10.1016/j.paid.2016.06.035

https://doi.org/10.1016/j.chiabu.2010.02.006

https://doi.org/10.1037/10456-003

https://doi.org/10.1016/j.acn.2006.03.005

https://doi.org/10.2466/pr0.1962.10.3.799

https://doi.org/10.1108/AMHID-07-2017-0031

https://doi.org/10.1177/1744629509353237

https://doi.org/10.1037/a0028585

https://doi.org/10.1080/13803390490510824

https://doi.org/10.3389/fpsyg.2015.00328

https://doi.org/10.1111/j.1939-0025.1983.tb03391.x

functioning than diagnosis. Acta Psychiatrica Scandinavica, 111(1), 22–
28. https://doi.org/10.1111/j.1600-0447.2004.00389.x

Stredny, R. V., Archer, R. P., & Mason, J. A. (2006). MMPI-2 and MCMI-III
characteristics of parental competency examinees. Journal of Personality
Assessment, 87(1), 113–115. https://doi.org/10.1207/s15327752jpa8701_10

Stroop, J. R. (1992). Studies of interference in serial verbal reactions. Journal
of Experimental Psychology: General, 121(1), 15–23. https://doi.org/10
.1037/0096-3445.121.1.15

Wechsler, D. (2008). Wechsler adult intelligence scale-Fourth Edition
(WAIS-IV) [Database record]. APA PsycTests. https://doi.org/10.1037/
t15169-000

Received April 16, 2021
Revision received September 20, 2021

Accepted September 28, 2021 ▪

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
T
h
is
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
p
er
so
n
al
u
se
o
f
th
e
in
di
v
id
u
al
u
se
r
an
d
is
n
o
t
to
b
e
d
is
se
m
in
at
ed
b
ro
ad
ly
.

PARENTAL COMPETENCY WAIS-IV AND MMPI-2 SCORES 11

https://doi.org/10.1111/j.1600-0447.2004.00389.x

https://doi.org/10.1207/s15327752jpa8701_10

https://doi.org/10.1037/0096-3445.121.1.15

https://doi.org/10.1037/t15169-000

Perspectives

Structural developmental psychology and health

promotion in the third age

Lars Bauger1,* and Rob Bongaardt2

1Department of Nursing and Health Sciences and 2Department of Health, Social and Welfare Studies,

Faculty of Health and Social Sciences, University College of Southeast Norway, Porsgrunn, Norway

*Corresponding author. E-mail: lars.bauger@hit.no

Summary

In response to the ever-increasing longevity in Western societies, old age has been divided into two

different periods, labelled the third and fourth age. Where the third age, with its onset at retirement,

mostly involves positive aspects of growing old, the fourth age involves functional decline and in-

creased morbidity. This article focuses on the entry to the third age and its potential for health promo-

tion initiatives. Well-being is an important factor to emphasize in such health promotion, and this arti-

cle views the lifestyle of third agers as essential for their well-being. The structural developmental

theory of Robert Kegan delineates how a person’s way of knowing develops throughout the life

course. This theory is an untapped and salient perspective for health promotion initiatives in the third

age. This article outlines Kegan’s approach as a tool for developing psychologically spacious health

promotion, and suggests future directions for research on the topic.

Key words: health promotion programs, quality of life, qualitative methods, older people

INTRODUCTION

Retiring from work is a major transition in life and in

many countries. It is the social marker of entering into

old age (Kloep and Hendry, 2006). The conception of

old age altered dramatically during the late 20th century

as people lived increasingly longer. As one consequence

of this, researchers now distinguish between the ‘third

age’ and ‘fourth age’ (Baltes, 1997; Baltes and Smith,

2003; Laslett, 1996). In gerontology the last stage of a

person’s life is often called the fourth age (Koss and

Ekerdt, 2016), which is a period characterized by func-

tional decline and an increased dependency. The third

age, with its onset in retirement, is seen as a period of

relatively good health with the potential of active social

engagement forming a solid base for healthy ageing

(Robinson, 2013). Even though the third age has a posi-

tive ring to it, it may come with some challenges that are

specific for this period of life. Retirement itself, whether

it comes voluntarily or, as may happen, involuntarily,

may be experienced as troubling (Daatland and Solem,

1995) and can have a negative effect on the well-being

of the retiree (Wang, 2007). Studies of retirement effects

on the person’s well-being have demonstrated that be-

tween 9-25% experience negative effects to their well-

being after retirement (Wang, 2007; Pinquart and

Schindler, 2007). In their recent review, Wang and Shi

(2014) highlighted different factors pre, during and post

VC The Author 2017. Published by Oxford University

Press.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/),

which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Health Promotion International, 2018;33:686–694

doi: 10.1093/heapro/daw104

Advance Access Publication Date: 11 January 2017

Perspectives

D
ow

nloaded from
https://academ

ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A

dam
E

llsw
orth, A

dam
E

llsw
orth on 21 S

eptem
ber 2018

Deleted Text: i

http://www.oxfordjournals.org/

retirement that affected the well-being of the retiree.

The negative factors were ill health, involuntary retire-

ment, a concern with the maintenance of social status

and contacts, and strongly identifying with one’s work

role (Wang and Shi, 2014). Health promotion may help

retirees to find a suitable place in society and improve

well-being in spite of these negative factors. In addition,

health promotion will prolong this third age period and

as a consequence likely compress morbidity during the

fourth age period (Whitehead, 2011). Whitehead (2011)

also suggests that during the fourth age, persons may

draw on existential forces to overcome adversity, forces

that are built up during earlier stages of life, including

the third age. Health promotion is apt to endorse such

existential forces. In other words, health promotion dur-

ing the third age may postpone the onset of the fourth

age, make it shorter and more endurable once the person

faces adversity.

Health promotion typically targets large populations

and may be unable to address individual differences.

The third age population, however, is characterized by

an immense heterogeneity (George, 2011; Wang, 2007),

and many third agers have acquired a unique profes-

sional competence, a specific way of living, and a net-

work that intertwines two or three generations of family

and friends. Ideally, health promotion should be individ-

ually tailored to the needs of each third ager. However,

the group’s heterogeneity renders that unfeasible. In this

article we outline a feasible approach to shaping health

promotion, directed at the intermediate range between a

large population and the unique individual. We do so by

introducing the structural developmental theory to the

healthy ageing discourse and linking this to the policy

making and practice of healthy lifestyle promotion. The

specific purpose of this article, then, is to outline a struc-

tural developmental approach to the field of health pro-

motion that targets the well-being of third agers. Before

presenting the structure of the rest of this article, we will

first delineate some central concepts.

The structural developmental theory focuses on con-

secutive stages of mental structures in a person’s life.

Such a theory is perhaps best introduced by contrasting

it to phase theories of life course development that

emphasize normative phases of life, such as birth, child-

hood, education, young adulthood, marriage, parent-

hood, working life and retirement (e.g. Erikson, 1980).

Whereas the phase developmental theory focuses on the

content of age-dependent periods of life, structural de-

velopmental stage theory underscores how this content

is put into perspective by the person – i.e. the extent to

which one takes responsibility for the unfolding of

events, and, ultimately, how the story of one’s life is told

at any particular moment in time. The development of

these perspectives is referred to as the development or

growth of complexity of mind

(Kegan, 1994).

The field of health promotion often refers to the life

style concept. However, definitions of the lifestyle con-

cept abound. We assume that lifestyle is made of the fab-

ric of a person’s attitudes, manners, behaviours and

practices, which are all woven into a Gestalt

(Cockerham, 2005; Elstad, 2000). In our view, a per-

son’s complexity of mind underpins his or her lifestyle.

We thereby emphasize coherence in what are often pre-

sented as separate lifestyle ‘factors’, such as smoking,

diet, exercise, etc. (cf. Veal, 1993). Furthermore, lifestyle

and well-being can be seen as reciprocally related – well-

being is embedded in lifestyle and takes shape through

it. Well-being is a heavily debated topic within health

psychology and we are not advocating for any of its

schools of thought. In this article, we take a broad per-

spective and focus on the subjective experience of the

phenomenon. Nevertheless, our use of well-being is in

line with how Huppert (2009 p.137) defines psychologi-

cal well-being, i.e. ‘the combination of feeling good and

functioning effectively’. Feeling good, then, is not just

concerned with happiness and contentment but addi-

tional emotions such as ‘interest, engagement, confi-

dence and affection’ (2009, p. 138), whereas functioning

effectively captures ‘the development of one’s potential,

having some control over one’s life, having a sense of

purpose (e.g. working towards valued goals), and

experiencing positive relationships’ (2009, p. 138).

The structure of this article is as follows. We first re-

view and present the key concepts of our article; the

third age, health promotion and lifestyle. Then we sum-

marize Kegan’s theory of structural development of the

mind. After that, we present the design of a study that

addresses the experience of well-being premised on com-

plexity of mind, and, finally, discuss the logical implica-

tions of a psychological developmental approach to

tailoring health promotion for third agers.

HEALTH PROMOTION AT THE ONSET OF
THE THIRD AGE

A positive perspective on the third age is well captured

by the gerontology term ‘successful ageing’. The term

gained popularity during the last decades of the 20th

century (Baltes and Smith, 2003). It was introduced by

Rowe and Kahn (1987) who reacted to the tendency in

gerontology to distinguish only between older people

with disease or disability and those without such condi-

tions. They introduced successful ageing as a positive

concept in order to address high cognitive and physical

Developmental psychology in health promotion for retirees 687

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: S

Deleted Text: (2009, p. 137)

Deleted Text: Health promotion at the onset of the third age

functioning and an active engagement with life, in addi-

tion to a low probability of disease and disability. In the

newfound optimism in the field of gerontology, the per-

ception of ageing changed from a passive experience to a

process of active engagement and participation (Baltes

and Baltes, 1990).

This more optimistic perspective on ageing has influ-

enced political discourse (Villar, 2012), as witnessed by

the introduction of the term ‘active ageing’ by the World

Health Organization (2002). The WHO defines active

ageing as ‘the process of optimizing opportunities for

health, participation and security in order to enhance

quality of life as people age’ (2002, p. 6). The WHO pol-

icy is to promote active ageing as a way to address socie-

tal and economic challenges stemming from an ageing

population as well as individual challenges associated

with getting older (World Health Organization, 2002).

Here the focus is on adding ‘more life to years, not just

years to life’ (Vaillant, 2004, p. 561), which is a hallmark

of health promotion in the third age. Wilson and Palha

(2007) argue that health promotion during this transi-

tional period will not only assist in maintaining existing

health but could also improve health and well-being sim-

ply because this is a period when one has more time to at-

tend to health-related needs than when one was working.

The third age is a period where one is left more to one’s

own devices with few established social structures and so-

cially defined roles (Freund et al., 2009). People are often

more free to do what they want, but those who do not

know or have not planned for what to do with this new

freedom could easily become ‘passive and couch ridden’

(Solem, 2012, p. 88; our translation).

It is evident that retirement is seen as an important

period for health promotion efforts. However,

retirement-specific research on health promotion is still

in its early stages. Reviewing the research, Wilson and

Palha (2007) identified 20 studies on the topic. Their

content analysis of these studies revealed four major

themes in the research on health promotion at the onset

of the third age, i.e. retirement: (1) the considerable ef-

fect of retirement and the need to support positive retire-

ment, (2) the identification and overcoming of barriers

to health promotion at retirement, (3) the best methods

to pro

mote and sustain healthy lifestyle changes among

retirees and (4) the short and long-term benefits of

health promotion at retirement (Wilson and Palha,

2007). Given the aim of the present article, we will elab-

orate on theme (3), which links successful ageing to the

promotion of healthy lifestyles.

We emphasized above that the Gestalt of a person’s

attitudes, manners, behaviours and practices can be seen

as his or her lifestyle. A lifestyle approach to health

promotion builds on the assumption that the individual

can amend this lifestyle (Elstad, 2000; Nutbeam, 1998).

Although studies show that adopting a healthy lifestyle

may be beneficial for healthy ageing, the literature re-

ports some difficulty in promoting a healthy lifestyle

through interventions (Zhang et al., 2013). The main fo-

cus has been restricted to financial planning (Osborne,

2011), whereas psychological or social changes that

might occur after retirement have received hardly any

attention (Kloep and Hendry, 2006). Health promotion

initiatives usually communicate messages about healthy

lifestyles to a large target population through health ed-

ucation booklets or pamphlets. Kreuter et al. (1999)

have criticized this way of promoting health for its ‘one-

size-fits-all’ approach, with little consideration of indi-

vidual needs and personal relevance. In response to this

criticism, there has been a growing interest in tailoring

interventions to different individual users and user

groups (Davis, 2008; Orji and Mandryk, 2014). We

share this interest and wish to contribute. Our contribu-

tion to the development of tailor-made methods to pro-

mote and sustain healthy lifestyle changes among

retirees is based on structural developmental theory,

which we describe in the following section.

STRUCTURAL DEVELOPMENTAL THEORY

Neo-Piagetian psychologist Robert Kegan developed a

structural developmental theory (1982, 1994) which

proposes that individuals interpret and make meaning of

their world in qualitatively different ways. These ways

of meaning-making develop throughout the life course

along an invariant path whereby more complex ways of

meaning-making build upon and transform earlier ways

of meaning-making. The ways of meaning-making are

termed structures or orders of mind. Kegan (1982) has

described three orders of mind that capture most of the

adult population. He refers to these orders as the social-

ized, the self-authoring, and the self-transforming mind

(Kegan, 1994). Each order captures what an individual

can take as an object – can see ‘in front of’ him or her –

and what an individual is subject to – is part of and

thereby lacks a perspective on.

Individuals who have developed a socialized order of

mind can think in abstract terms and have the capacity

to internalize the meaning systems of others, such as

family values, social values, professional culture, etc.

They have the ability to subordinate their own desires

and be guided by the norms and standards in the ideolo-

gies, institutions or people that are most important to

them (Fitzgerald and Berger, 2002). At this order of

mind, one easily sees beyond one’s own needs and can

688 L. Bauger and R. Bongaardt

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: in

Deleted Text: Structural developmental theory

adopt a larger picture, in which one is part of a socially

defined reality. Even though one has the capacity to in-

ternalize others’ points of view, one is embedded in these

points of view and is essentially dependent on them.

That is to say, the individual’s experience of being a per-

son or ‘self’ is entangled with ‘the quality of . . . internal

experiences of others’ experiences of them’ (Lewis,

2011, location 692). This means that at this order of

mind one does not ‘have the capacity to stand apart

from the values, beliefs, expectations, or definitions of

one’s tribe, community, or culture and make indepen-

dent judgments about them’ (Kegan, 1998, p. 201).

Individuals who make meaning with a self-authoring

mind have distanced themselves from the sense of being

entangled in others’ feelings and ideas about themselves.

They now have the capacity to be in charge of their own

feelings and generate an internal personal meaning sys-

tem, theory or ideology. Thus, one is able to take as an

object the values, beliefs and expectations of others

(one’s ‘tribe’, local community, or culture) that one was

subject to earlier. Individuals making meaning with this

order of mind perceive others as independent entities,

with their own integrity, distinct from themselves.

Unlike individuals at the socialized order of mind who

may struggle heavily with conflicting internalized views,

the self-authoring mind tolerates such conflicts or re-

solves these by invoking a system of self-authored values

and knowledge. This system has typically developed

over a period of years, gradually integrating the experi-

ences and reflections of personal encounters with a wide

variety of other knowledge and value systems (Kegan,

1994). This system of ‘self’ requires strong boundaries,

which may prevent the person from recognizing the con-

structed nature of the system itself. When meeting this

construction of self, others may experience it as a some-

what distant way of being, an obstacle to gaining direct

contact. However, ‘[t]his greater psychological indepen-

dence does not mean that [the person is] any less com-

mitted to you and to . . . other close relationships’

(Lewis, 2011, location 1111).

Those individuals who make meaning according to

the self-transforming mind have gained a perspective of

their own identity construction, and are no longer

‘blind’ to their self-authored identity. At this order, the

construction of identity is object to them. This implies

that they are now hesitant to see personhood as coincid-

ing with ‘a single system or form’ (Kegan, 1994, p. 313),

but rather see their system of self as incomplete and in

continuous development. At this order, individuals view

the ‘other as part of oneself’ (Souvaine et al., 1990, p.

253) and they are characterized by their embeddedness

in a multisystem perspective (Rosen, 1991). These

individuals are less likely to view the world in dichoto-

mies, and ‘suspicious of their own tendency to feel

wholly identified with one side of any opposite and to

identify the other with the other side of that opposite’

(Kegan, 1994, pp. 311-312). Meaning-making with this

order of mind concerns the reflections on the process of

making meaning itself more than the outcomes of this

process. The individual reflects on his or her own need

for meaning while acknowledging that knowledge is al-

ways partial, and he or she thrives on ‘rending every

new veil that comes into awareness, because . . . closure

and fixed boundaries [are] restrictive’ (Cook-Greuter,

1999, p. 107).

In his book In over our heads: The mental demands

of modern life, Kegan (1994) asks whether people make

meaning in accordance with society’s demands. In other

words, he asks what order of mind is required to suc-

cessfully parent, partner, work, learn, heal, and collabo-

rate as modern society frames these life tasks. He shows

that society implicitly demands a self-authoring mind

for all these tasks. In a composite study sample of adults

(Kegan, 1994, p. 195), about half of the persons did not

construct their experiences as complexly as the self-

authoring

mind.

What are the mental demands on ageing in our mod-

ern Western society? Does the ageing population meet

these demands? Currently, hardly any empirical research

exists that answers these questions. Newhouse (as refer-

enced in Kegan, 1998) suggests a number of tasks and

expectations typical of the third age: giving up a central

identity formed around work and a career, changing

from a highly structured to a less structured everyday

life, needing to create new friendships after the loss of a

ready-made social network, and remaining relatively in-

dependent of the care-taking resources of family or soci-

ety. Kegan infers from Newhouse’s list that it is ‘the self-

authoring mind that constitutes the implicit mental

threshold for successfully handling this curriculum, a

threshold many adults will not yet have reached in old

age, and not having done so, will be ‘at risk’ for poorer

outcomes thereby’ (1998, p. 209; italics in original).

Therefore, he argues that it may be ‘an absolutely crucial

educational or mental health goal serving as a protective

factor against decline and depression in old age’ (Kegan,

1998, p. 212) to develop a self-authoring mind since it is

with this order of mind that one can meet the demands

of ageing. Moreover, if it is true that more people

make meaning with a self-authoring mind, then the so-

cial institutions relevant to the third age are challenged

to provide the space for the personal paths and demands

that are so typical for individuals with this order of

mind.

Developmental psychology in health promotion for retirees 689

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: “

Deleted Text: ”

It is against the backdrop of Kegan’s theory and its

possible implications for the third age that we now turn

to outlining the research we envision. In the following

section, we juxtapose the promotion of healthy lifestyles

during the third age with Kegan’s psychological develop-

ment theory.

DEVELOPING HEALTH PROMOTION FOR
THIRD AGERS

Structural developmental theory has informed classroom

practice in educational psychology, where developmen-

tally conscious teachers are teaching in ways that en-

courage students to make meaning in an increasingly

complex way, while also meeting students at their stage

of development (Helsing et al., 2004). In the context of

business coaching and counselling, Berger (2012) refers

to this practice as keeping conversations ‘psychologi-

cally spacious’. Inspired by such thinking, we envision

health promotion initiatives to be psychologically spa-

cious and tailored to a person’s order of mind. Neither

our aim nor our interest is in highlighting or facilitating

the development towards one specific order of mind

(e.g. self-authoring). Our contribution is rather to raise

awareness of the qualitatively different ways of making

meaning in the world, and, where possible, outline how

health promotion can be formulated in developmentally

spacious ways, to enable more people to be reached and

feel included.

In order to do so, we require a knowledge base that

links a person’s lifestyle to his or her stage of structural

development. Our research will hopefully help to estab-

lish this knowledge base. The rationale for our research

is that much information can be gained from the experi-

ences of individuals who report that they have recently

transitioned successfully into the third age. In other

words, our preferred starting point is narratives concern-

ing a successful lifestyle during retirement, i.e. one that

leads to an experience of well-being. True to this

experience-oriented bottom-up approach, we employ no

specific definition of well-being. The next logical step in

our rationale is to relate these situation-specific experi-

ences to a person’s order of mind. Kegan’s measure of

order of mind indicates in general terms how a person

structures his or her life in terms of responsibility alloca-

tion and perspective taking, that is, how a person under-

stands him- or herself to play a role in his or her own

life. The assumption is that persons with different orders

of mind structure retirement-specific experiences in dif-

ferent ways, because lifestyle and the ensuing experience

of well-being are dependent upon order of mind.

More concretely, our research will unfold as follows.

We will recruit participants recently retired from work-

ing life and reporting having done so satisfactorily ac-

cording to their own expectations and standards. To

assess the participants’ orders of mind, we will conduct

subject-object interviews (SOI) (Lahey et al., 1988/

2011) with all our participants. During the SOI, ten

emotionally laden probes (e.g. ‘Can you tell me of a re-

cent experience of being quite angry about something?’)

are presented to a participant, and he or she is asked to

write down recent experiences brought to mind by the

probes. The participant then selects some of the experi-

ences to elaborate on. During the interview, the inter-

viewer listens sympathetically and confirms the content

of the participant’s experience, while also probing for

the structuring of the experience. The combination of

the emotionally laden probes and the why-questions in-

vites the participants to describe their experiences at the

borderline between what is and is not explicitly reflected

upon. An analysis of transcripts from the interview al-

lows the researcher to score where participants are on

their developmental journey according to Kegan’s devel-

opmental theory (1982, 1994). This score indicates

whether the participants are currently at one order of

mind or in transition between two orders of mind,

where four sub-stages can be distinguished. The inter-

rater reliability for the SOI ranges between 0.82 to 1.00

for agreement within one discrimination unit (Kegan,

1994; Lahey et al., 1988/2011). We have completed

training in subject-object interviewing, are experienced

and reliable scorers, and we will establish and report on

our inter-rater reliability within this study. If a partici-

pant scores at a transitional order of mind, we will allo-

cate him or her according to the dominant order. We are

interested to include all adult orders of mind in this

study, preferably three participants within each order.

However, we are aware of the difficulty of recruiting

persons who make meaning at the self-transforming

mind as they are few and far between (Kegan, 1994).

Knowing this, and given the resources necessary to con-

duct and analyse such SOIs, it is unlikely that we will be

able to recruit enough participants at the self-

transforming mind. It is likely that we can include at

least three persons at the socialized mind and three at

the self-authoring mind, as these are the two orders

where most of the adult population makes meaning

(Kegan, 1994).

We will conduct an in-depth phenomenological inter-

view with each of the participants. This form of the

open qualitative interview will allow us to reveal the

phenomenon of well-being as it emerges in the partici-

pants’ descriptions of their experiences of the

690 L. Bauger and R. Bongaardt

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: Developing health promotion for third agers

Deleted Text: “

Deleted Text: ?”)

phenomenon (Giorgi, 2009). We have found that three

such interviews suffice to make valid inferences about

the participants’ experiences with the phenomenon un-

der investigation. That is mainly because a descriptive

phenomenological analysis makes use of all data mate-

rial and is not guided by themes that are established be-

forehand. We will analyse the descriptions separately for

each of the orders of mind, resulting in so-called general

meaning structures. Such a general meaning structure re-

veals the shared meaning across many variations of how

participants experience the phenomenon in their daily

life (Giorgi, 2009). In a final analysis, we will compare

and discuss differences and similarities in the general

meaning structure of the phenomenon between the or-

ders of mind. The results of this will feed into the next

stage of the project.

SHAPING STRUCTURAL
DEVELOPMENTAL HEALTH PROMOTION

We referred earlier to a quote that a hallmark of health

promotion is the aim to bring ‘more life to years, not

just years to life’ (Vaillant, 2004, p. 561). One way to

bring more life to years is to facilitate experiences of

well-being through the promotion of a lifestyle pervaded

by such experiences. We will endeavour to make our re-

search results accessible to retirees as well as to the

policy-makers and welfare and health promotion profes-

sionals who are engaged in their well-being. What do we

expect to be able to tell them? What does our research

underscore or explicate? In the following, we present a

preliminary sketch along three lines of the contribution

value of the rationale presented above.

First, both forms of interview will most likely pro-

vide information about the shift from working life to re-

tirement. The phenomenological interview aims to

capture the general meaning structure of well-being dur-

ing early retirement. The SOI explores how the individ-

ual structures some of his or her recent experiences with

change, success, feeling torn, etc. A change of lifestyle

that comes with a major shift (such as retiring) appears

in the light of a structural developmental approach as ei-

ther solving a technical problem or overcoming an adap-

tive challenge (Heifetz and Linsky, 2002). The latter

implies a change in order of mind, whereas the former

means that the person maintains the same order of mind

while incorporating new activities in his or her daily life.

For instance, the third age could be lived so that time is

increasingly spent on previously well-established activi-

ties, or it could incorporate new activities that facilitate

or emerge with the structural development of mind. An

awareness of the differences between these changes

assists the retiree, welfare professional and policy-maker

alike in choosing or recommending one activity in fa-

vour of another.

Second, both types of interview will provide informa-

tion about how well-being takes shape in different orders

of mind. Following Labouvie-Vief et al. (1989), Noam,

Young, and Jilnina (2006) have argued that people at var-

ious levels of mental complexity may experience and un-

derstand their well-being in qualitatively different ways.

Bauer (2011) researched the content of the growth stories

told by persons with late stages of mental growth (with

what he refers to as ‘postconventional selves’). He found

that, on average, later stages of development do not nec-

essarily make a person more happy as measured by estab-

lished quantitative measures of well-being (Diener et al.,

1985), which is consistent with Kegan’s theoretical as-

sumptions. One finding, however, stands out, namely

that the individuals with the highest score of mental com-

plexity had indeed higher levels of well-being on average

when compared to the other stages (Bauer et al., 2011).

However, Bauer et al. (2011) findings are preliminary,

given the relatively small number of participants who

scored in the highest stage. Mental complexity, Bauer and

colleagues confirm, taps into different aspects of well-

being, but their research is inconclusive as to how the

first-person experience of well-being relates to mental

growth, especially concerning individuals who have not

reached the very late stages of development, i.e. the ma-

jority of the population.

Kegan (1982, pp. 267-268) has looked into what can

be called psychological ‘ill-being’ and its relation to

mental complexity. He analysed patient journals at a

psychiatric hospital and inferred three different kinds of

depression, characterized by three types of loss, respec-

tively: a loss of one’s own needs or the increasing costs

of trying to satisfy these needs, a loss of an interpersonal

relationship leading to loneliness or even a loss of parts

of oneself, and loss of control over meeting one’s own

standards. Upon first measuring mental complexity and

then relating it to these three types of depression, a

strong association between type of depression and men-

tal complexity was observed.

We aim to follow up on the interest of Noam et al.

(2006) and Bauer et al. in the link between mental com-

plexity and well-being, and use a research design in-

spired by Kegan’s study of depression. Here we will first

divide our participants up into groups according to their

SOI score, and then interview them to discover how they

experience well-being.

Third, the combination of both interviews will pro-

vide essential information to suggest new opportunities

for tailoring interventions to the intermediate range

Developmental psychology in health promotion for retirees 691

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: Shaping structural developmental health promotion

between the unique individual and larger cohorts of the

population. Tailored interventions have been defined as

follows: ‘Any combination of information or change

strategies intended to reach one specific person, based

on characteristics that are unique to that person, related

to the outcome of interest, and have been derived from

an individual assessment’ (Kreuter and Skinner, 2000,

p. 1; italics in original). For our purposes, this may be an

unattainable ideal considering the amount of resources

required. At the other end of the continuum, health pro-

motion that is specific to cohorts, though economically

more manageable, may risk not reaching all the mem-

bers of the targeted population. Consequently, we prefer

an intermediate range at which to target the population

of retirees. In other words, understanding how individ-

uals with different orders of mind experience well-being

differently allows programme developers to tailor psy-

chologically spacious programmes while avoiding indi-

vidual time-consuming assessments. Moreover, health

care and welfare professionals will benefit from an

awareness of structural development, lest they under- or

overshoot their communication with the target popula-

tion concerning health promotion activities. Therefore,

our research may also help to provide these profes-

sionals with knowledge of lifelong development and

learning as well as active ageing.

CONCLUSION

In this article we have outlined perspectives which have

as yet not been combined. We have emphasized the no-

tion of adding more life to years as well as the potential

for structural developmental thinking in health promo-

tion initiatives. This is an area largely untouched in the

health promotion literature, and we see its inclusion as a

contribution to extending the positive period of the third

age while also aiding the compression of the fourth age.

We have underscored the reciprocity of well-being

and lifestyle and have argued that the experience of

well-being may have quite different manifestations for

different persons when seen through the lenses of a

structural development approach. We have sketched a

feasible mid-range approach to tailoring health promo-

tion initiatives. This approach attends to the orders of

the mind within the target group and has the potential

to overcome the practical difficulties of developing

unique individual health promotion initiatives.

We have presented one structural developmental the-

ory within the neo-Piagetian tradition as a contrasting

view to the current phase theories employed in ageing

research, but there are many others which we have not

discussed. Notable examples of others in this tradition

are Kohlberg (1969), Fowler (1981), Commons et al.

(1998), Gilligan (1982), Basseches and Mascolo (2009),

Cook-Greuter (1999) and Loevinger and Blasi (1976).

Kegan’s theory of adult development in health promo-

tion serves our purpose well, which is why we have not

focused on other potentially appropriate theories of

adult development or mental growth. We conclude that

a sensitivity towards the complexity of mind with re-

spect to the experience of well-being will provide health-

care professionals and policy-makers with a powerful

tool in their health promotion toolbox.

ACKNOWLEDGEMENTS

We wish to thank the two reviewers for their insightful com-

ments and specific points of improvements for the article.

REFERENCES

Baltes, P. B. (1997) On the incomplete architecture of human on-

togeny: Selection, optimization, and compensation as foun-

dation of developmental theory. American Psychologist 52,

366–380.

Baltes, P. B. and Baltes, M. M. (1990). Successful ageing: A psy-

chological model. In Baltes P. B. & Baltes M. M. (Eds.),

Successful aging: Perspectives from the Behavioral Sciences

(pp. 1–34). Cambridge: Cambridge University Press.

Baltes, P. B. and Smith, J. (2003) New frontiers in the future of

aging: From successful aging of the young old to the di-

lemmas of the fourth age. Gerontology 49, 123–135.

Basseches, M. and Mascolo, M. F. (2009). Psychotherapy as a

Developmental Process. New York, NY: Routledge

Bauer, J. J. (2011). The postconventional self: Ego maturity,

growth stories. . . and happiness? In Pfaffenberger A. H.,

Marko P. W., & Combs A. (Eds.), The Postconventional

Personality: Assessing, Researching, and Theorizing Higher

Development (pp. 101–117). Albany, NY: SUNY Press.

Bauer, J. J., Schwab, J. R. and McAdams, D. P. (2011) Self-actu-

alizing: Where ego development finally feels good?. The

Humanistic Psychologist 39, 121–136.

Berger, J. G. (2012). Changing on the job: Developing leaders

for a complex world. Stanford, CA: Stanford Business

Books.

Cockerham, W. (2005) Health lifestyle theory and the conver-

gence of agency and structure. Journal of Health and Social

Behavior 46, 51–67.

Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A.

and Krause, S. R. (1998) Hierarchical complexity of tasks

shows the existence of developmental stages.

Developmental Review 18, 237–278.

Cook-Greuter, S. R. (1999). Postautonomous Ego Development:

A Study of Its Nature and Measurement. (Doctoral disserta-

tion), Harvard Graduate School of Education.

Daatland, S. O. and Solem, P. E. (1995). Velferdsgevinst eller

velferdstap ved å bli pensjonist? In Kjønstad A., Hatland

692 L. Bauger and R. Bongaardt

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

Deleted Text: Conclusion

A., & Halvorsen B. (Eds.), Det norske trygdesystemet:

Fortid, nåtid og framtid (pp. 85–112). Oslo: Ad Notam

Gyldendal.

Davis S. (2008) The influence of collectivistic and individualistic

value orientations on the acceptance of individually-tailored

Internet communications. Interface: The Journal of

Education, Community and Values 8, 17–32. http://com

mons.pacificu.edu/inter08/ (last accessed 28 December

2016).

Diener, E., Emmons, R. A., Larsen, R. J. and Griffin, S. (1985)

The satisfaction with life scale. Journal of Personality

Assessment 49, 71–75.

Elstad, J. I. (2000). Social inequalities in health and their expla-

nations (NOVA Report 9/00). www.hioa.no/content/down

load/45507/674888/file/2746_1 (last accessed 28

December 2016).

Erikson, E. H. (1980). Identity and the Life Cycle. New York,

NY: Norton.

Fitzgerald, C. and Berger, J. G. (2002). Leadership and complex-

ity of mind: The role of executive coaching. In Fitzgerald C.

& Berger J. G. (Eds.), Executive Coaching: Practices &

Perspectives (pp. 27–57). Palo Alto, CA: Davies-Black Pub.

Fowler, J. W. (1981). Stages of faith: The psychology of human

development and the quest for meaning. San Francisco, CA:

Harper & Row.

Freund A. M., Nikitin J & Ritter J. O. (2009) Psychological

Consequences of Longevity. Human Development 52, 1–37.

George, L. K. (2011). The third age: Fact or fiction – And does it

matter. In Carr, C. D. & Komp K. (Eds.), Gerontology in

the era of the third age: implications and next steps. New

York, NY: Springer.

Gilligan, C. (1982). In a different voice. Cambridge, MA:

Harvard University Press.

Giorgi, A. (2009). The descriptive phenomenological method in

psychology: A modified Husserlian approach. Pittsburgh,

PA: Duquesne University Press.

Heifetz, R. A. and Linsky, M. (2002). Leadership on the line

staying alive through the dangers of leading. Boston, MA:

Harvard Business Shool Press.

Helsing, D., Drago-Severson, E. and Kegan, R. (2004). Applying

constructive-developmental theories of adult development

to ABE and ESOL practices. In Comings J., Garner B., &

Smith C. (Eds.), Connecting research, policy, and practice

(pp. 157–197). Mahwah, NJ: Lawrence Erlbaum.

Huppert, F. A. (2009) Psychological well-being: Evidence re-

garding its causes and consequences. Applied Psychology:

Health and Well-Being 1, 137–164.

Kegan, R. (1982). The evolving self: Problem and process in hu-

man development. Cambridge, MA: Harvard University

Press.

Kegan, R. (1994). In over our heads: The mental demands of

modern life. Cambridge, MA: Harvard University Press.

Kegan, R. (1998). Epistemology, expectation, and aging: A develop-

mental analysis of the gerontological curriculum. In Lomranz J.

(Ed.), Handbook of aging and mental health: An integrative ap-

proach (pp. 197–216). New York, NY: Plenum Press.

Kloep, M. and Hendry, L. B. (2006) Pathways into retirement:

Entry or exit?. Journal of Occupational and Organizational

Psychology 79, 569–593.

Kohlberg, L. (1969). Stage and sequence: The cognitive-

developmental approach to socialization. In Goslin D. A.

(Ed.), Handbook of socialization theory and research.

Chicago, IL: Rand McNally.

Koss, C. and Ekerdt, D. J. (2016) Residential reasoning and the

tug of the fourth age. The Gerontologist. doi:10.1093/ger-

ont/gnw010.

Kreuter, M. W. and Skinner, C. S. (2000) Tailoring: what’s in a

name?. Health Education Research 15, 1–4.

Kreuter, M. W., Strecher, V. J. and Glassman B. (1999) One size

does not fit all: The case for tailoring print materials. Annals

of Behavioral Medicine 21, 276–283.

Labouvie-Vief, G., Hakim-Larson, J., DeVoe, M. and

Schoeberlein, S. (1989) Emotions and Self-Regulation: A

Life Span View. Human Development 32, 279–299.

Lahey, L., Souvaine, E., Kegan, R., Goodman, R. and Felix, S.

(1988/2011). A guide to the subject-object interview: Its admin-

istration and interpretation. Cambridge, MA: Minds at Work.

Laslett, P. (1996). A fresh map of life: the emergence of the

Third Age. Basingstoke, UK:

Macmillan.

Lewis, P. M. (2011). The discerning heart: The developmental

psychology of Robert Kegan [Kindle version]. http://www.am

azon.com/The-Discerning-Heart-Developmental-Psychology-

ebook/dp/B006F631FY (last

accessed 28 December 2016).

Loevinger, J. and Blasi, A. (1976). Ego development. San

Francisco, CA: Jossey-Bass.

Noam, G. G., Young, C. H. and Jilnina, J. (2006). Social cogni-

tion, psychological symptoms, and mental health: The

model, evidence, and contribution of ego development. In

Cicchetti D. and Cohen D. J. (Eds.), Developmental psycho-

pathology, Vol 1: Theory and method (2nd ed.) (pp.

750–794). Hoboken, NJ, US: John Wiley & Sons Inc.

Nutbeam, D. (1998) Health promotion glossary. Health

Promotion International 13, 349–364.

Orji, R. and Mandryk, R. L. (2014) Developing culturally relevant de-

sign guidelines for encouraging healthy eating behavior.

International Journal of Human-Computer Studies 72, 207–223.

Osborne, J. W. (2011) Psychological effects of the transition to

retirement. Canadian Journal of Counselling and

Psychotherapy 46.

Pinquart, M. and Schindler, I. (2007) Changes of life satisfaction

in the transition to retirement: A latent-class approach.

Psychology and Aging 22, 442–455.

Robinson O. (2013). Development Through Adulthood: An

Integrative Sourcebook. Basingstoke, UK: Palgrave

Macmillan.

Rosen H. (1991). Constructivism: Personality, psychopathol-

ogy, and psychotherapy. In Keating D. P. and Rosen H.

(Eds.), Constructivist perspectives on developmental psy-

chopathology and atypical development (pp. 149–171).

Hillsdale, NJ: Erlbaum.

Rowe, J. W. and Kahn, R. L. (1987) Human aging: Usual and

successful. Science 237, 143.

Developmental psychology in health promotion for retirees 693

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

http://commons.pacificu.edu/inter08/

http://www.hioa.no/content/download/45507/674888/file/2746_1

Solem, P. E. (2012). Ny kunnskap om aldring og arbeid (NOVA

Rapport 6/12). www.nova.no/asset/5577/1/5577_1 (last

accessed 28 December 2016).

Souvaine, E., Lahey, L. L. and Kegan, R. (1990). Life after for-

mal operations: Implications for a psychology of the self. In

Alexander C. N. & Langer E. J. (Eds.), Higher stages of hu-

man development: Perspectives on adult growth (pp.

229–257). New York, NY: Oxford university press.

Vaillant, G. E. (2004). Positive aging. In Linley P. A. & Joseph S.

(Eds.), Positive psychology in practice (pp. 561–578).

Hoboken, N.J.: Wiley.

Veal, A. J. (1993) The concept of lifestyle: A review. Leisure

Studies 12, 233–252.

Villar, F. (2012) Successful ageing and development: the contri-

bution of generativity in older age. Ageing and Society 32,

1087–1105.

Wang, M. (2007) Profiling retirees in the retirement transition

and adjustment process: Examining the longitudinal change

patterns of retirees’ psychological well-being. Journal of

Applied Psychology 92, 455–474.

Wang, M. and Shi, J. (2014) Psychological research on retire-

ment. Annual Review of Psychology 65, 209–233.

Whitehead, D. (2011) Before the cradle and beyond the grave: A

lifespan/settings-based framework for health promotion.

Journal of Clinical Nursing 20, 2183–2194.

Wilson, D. M. and Palha, P. (2007) A systematic review of pub-

lished research articles on health promotion at retirement.

Journal of Nursing Scholarship 39, 330–337.

World Health Organization. (2002). Active ageing: A policy

framework. whqlibdoc.who.int/hq/2002/WHO_NMH_NPH_

02.8 ?ua¼1 (last accessed 28 December 2016).
Zhang, S., Tao, F., Ueda, A., Wei, C. and Fang, J. (2013) The in-

fluence of health-promoting lifestyles on the quality of life of

retired workers in a medium-sized city of Northeastern

China. Environmental Health and Preventive Medicine 18,

458–465.

694 L. Bauger and R. Bongaardt

D
ow
nloaded from
https://academ
ic.oup.com
/heapro/article-abstract/33/4/686/2897769 by A
dam
E
llsw
orth, A
dam
E
llsw
orth on 21 S
eptem
ber 2018

http://www.nova.no/asset/5577/1/5577_1

Copyright of Health Promotion International is the property of Oxford University Press / USA
and its content may not be copied or emailed to multiple sites or posted to a listserv without
the copyright holder’s express written permission. However, users may print, download, or
email articles for individual use.

Linked Articles:
Bjorklund; doi: 10.1111/cdev.13019
Bjorklund; doi: 10.1111/cdev.13020
Frankenhuis and Tiokhin; doi: 10.1111/cdev.13021

Evolutionary Developmental Psychology: 2017 Redux

Cristine H. Legare
The University of Texas at Austin

Jennifer M. Clegg
Texas State University

Nicole J. Wen
The University of Texas at Austin

Bjorklund is a pioneer in bringing evolutionary
theory to developmental psychology. In doing so,
he has made major contributions to the field,
including publishing a widely adopted and influ-
ential textbook (Bjorklund & Causey, 2017). We
commend him for his groundbreaking research
and strongly agree that it is “undeniable that evolu-
tionary thinking has seeped into the minds of many
cognitive developmental psychologists” (Bjorklund,
2018, p. 14).

We suggest that evolutionary theory has
impacted developmental psychology even more
strongly than Bjorklund suggests. Many of the most
influential recent programs of research in the field
of developmental psychology, cognitive and other-
wise, take an evolutionary approach to understand-
ing the ontogeny of cognition and behavior (as just
a few recent examples: Barrett et al., 2013; Blake
et al., 2015; Broesch, Rochat, Olah, Broesch, & Hen-
rich, 2016; Clay & Tennie, 2017; Gopnik et al., 2017;
Hamlin, 2014; Henrich, 2015a; Heyes, in press;
House et al., 2013; Nielsen & Haun, 2016; Rosati &
Warneken, 2016; Santos & Rosati, 2015; van Leeu-
wen, Call, & Haun, 2014; Warneken & Tomasello,
2017; Wertz & Wynn, 2014). Additional evidence of
impact can be found in recent programs at major
conferences in the field. The Society for Research in
Child Development, the Cognitive Development
Society, and the International Congress for Infant

Studies have all featured evolutionary research in
preconferences and invited addresses in recent
years. In fact, evolution has been so successful as a
metatheory within developmental psychology,
many doing research within this tradition do not
use this label to identify their area of expertise or
theoretical approach.

Perhaps as a result of developmental psycholo-
gists conducting research that is increasingly guided
by evolutionary theory, but not explicitly labeled as
such, there is a large body of recent literature not
reviewed by Bjorklund. To give one example, the
best research on cognitive obstacles to understand-
ing evolution and recommendations for how to
teach it comes from research programs in cognitive
and developmental psychology that draw on evolu-
tionary theory. This research demonstrates that intu-
itive cognitive bias such as essentialism and
teleological reasoning impede understanding of evo-
lutionary concepts (e.g., Emmons, Smith, & Kele-
men, 2016; Evans, in press; Heddy & Sinatra, 2013;
Legare, Lane, & Evans, 2013; Lombrozo, 2013; Short
& Hawley, 2015; Shtulman, Neal, & Lindquist, 2016).

Another increasingly influential trend within
developmental, cognitive, and comparative research
guided by evolutionary theory, not covered in
Bjorklund’s review, is to examine the origins of
complexity and variability in human culture. Tack-
ling interdisciplinary questions of this kind requires
understanding the differences between human and

Commentary on Bjorklund (2017). Child Development.
Correspondence concerning this article should be addressed to

Cristine H. Legare, The University of Texas at Austin, Depart-
ment of Psychology, 1 University Station #A8000, Austin, TX,
78712. Electronic mail may be sent to legare@austin.utexas.edu.

© 2018 The Authors
Child Development © 2018 Society for Research in Child Development, Inc.
All rights reserved. 0009-3920/2018/8906-0026
DOI: 10.1111/cdev.13018

Child Development, November/December 2018, Volume 89, Number 6, Pages 2282–2287

info:doi/10.1111/cdev.13019

info:doi/10.1111/cdev.13020

info:doi/10.1111/cdev.13021

http://orcid.org/0000-0001-5655-9899

http://orcid.org/0000-0001-8471-4876

nonhuman social learning capacities, the ontogeny
of those capacities, and their expression across
diverse human populations. We propose that only
the combination of these perspectives will enable us
to fully understand the roots of human culture. We
and others advocate for a triadic approach to
understanding the evolution and ontogeny of cul-
tural learning by integrating comparative, cross-
cultural, and developmental psychological research,
with all of these lines drawing heavily on evolu-
tionary theory (Legare, 2017; Nielsen & Haun,
2016).

Our research differs from Bjorklund’s not because
we disagree about the importance and impact of evo-
lutionary theory within developmental psychology,
but instead is based on the relative emphasis we
place on the scientific importance of understanding
cultural transmission and variation. Examining cul-
tural variation would enrich Bjorklund’s discussion
of developmental cognitive neuroscience and cogni-
tive development. Claims about the universality and
ontogeny of cognitive mechanisms without data on
global diversity are unwarranted (Nielsen, Haun,
Kaertner, & Legare, 2017). Prioritizing studying cul-
tural diversity in programs of research would also
encourage researchers to utilize cutting-edge and
state-of-the-art methodologies and tools, elevating
evolutionary developmental science programs.
Below we describe an example of a comparative,
cross-cultural, and developmental program of
research on cultural learning and social group cogni-
tion and behavior, all based on evolutionary theory.

Comparative Perspectives on Cultural Learning

Cultural variation in humans is unique among ani-
mals and differs dramatically even from our closest
primate relatives (Boyette & Hewlett, 2017; Henrich,
2015b; Lew-Levy, Reckin, Lavi, Crist�obal-Azkarate,
& Ellis-Davies, 2017; Mesoudi, Chang, Murray, &
Lu, 2015; Terashima & Hewlett, 2016). Here culture
is defined as “group-typical behaviors shared by
members of a community that rely on socially
learned and transmitted information” (Laland &
Hoppitt, 2003). Although nonhuman animals may
have the ability to learn social information (Aplin,
2015; Leadbeater, 2015; Perry et al., 2003; Plotnik,
Lair, Suphachoksahakun, & de Waal, 2011; White-
head & Rendell, 2015) and to transmit group-speci-
fic behavior (Cantor et al., 2015; Garland et al.,
2013; Laland & Galef, 2009), humans display a
much wider repertoire of socially acquired and
transmitted behaviors that vary more across groups

than nonhuman animals (Dean, Vale, Laland,
Flynn, & Kendal, 2014; Johnson-Pynn, Fragaszy, &
Cummins-Sebree, 2003).

How does human cognition differ from non-
human primate cognition? One potential candidate is
cross-species variation in social cognition (van Schaik
& Burkart, 2011). Our prolonged early development
also sets humans aside from other primates. As
Bjorklund and others suggest, natural selection
favored an extended childhood to allow for increased
flexibility in cognitive development (Bjorklund, 2018;
Bjorklund & Ellis, 2014). During this extended juve-
nile period, our offspring are dependent on adults
for survival, and in turn, this dependency increases
opportunities for interaction with caregivers and
enables social learning (Hublin, 2005).

The technological and social complexity of
human populations is due to our ability for cumu-
lative cultural transmission, a process by which the
discoveries and inventions of others are built upon
to create increasingly complex reserves of socially
heritable knowledge (Henrich, 2015b). Human psy-
chological flexibility allows us to build upon estab-
lished behaviors by relinquishing old solutions and
flexibly switching to more productive or efficient
ones (Davis, Vale, Schapiro, Lambeth, & Whiten,
2016). Evidence for culture in nonhuman species
continues to grow, but there is little evidence for
the accumulation of cultural innovation in nonhu-
man animals. Recent comparative research has
examined the development of social learning and
imitative flexibility across hominin evolutionary his-
tory (Whiten, 2017). Comparative research on this
topic will increase our knowledge of how cognitive
capacity may constrain young children’s and chim-
panzees’ learning potential and technological skill,
as well as elucidating the diverse learning heuristics
that children and chimpanzees employ. Although
largely absent from Bjorklund’s commentary, re-
search contrasting children’s and other primates’
social cognition adds to our understanding of the
origins of cumulative culture in humans and evolu-
tionary theory more broadly.

Development and Diversity of Cultural Learning

Young children are adept at acquiring the beliefs
and practices of whatever group they are born into,
a cognitive capacity that requires substantial flexi-
bility. We agree with Bjorklund that the sociocogni-
tive mechanisms that children display are not the
“derivatives of ‘hard’ cognition” but a set of critical
psychological adaptations in their own right

Evolutionary Developmental Psychology 2283

(Bjorklund, 2018, p. 15). For example, children have
a number of cognitive biases that aid in the acquisi-
tion of their specific cultural practices. These biases
include preferences for learning from those who are
from similar social groups (Kinzler, Dupoux, &
Spelke, 2007), those who conform (Haun & Over,
2014) and display behavioral or cognitive consensus
with others (Claidi�ere & Whiten, 2012; Corriveau,
Fusaro, & Harris, 2009; Herrmann, Legare, Harris,
& Whitehouse, 2013), and those who display
prestige (Chudek, Heller, Birch, & Henrich, 2012;
Henrich, 2009).

Missing from Bjorklund’s commentary on the
development of the sociocognitive brain and the
social brain hypothesis (Bjorklund, 2018, p. 15) is a
discussion of the flexibility and diversity of chil-
dren’s social learning. We argue that studying the
flexibility and diversity of children’s sociocognitive
development provides insight into the evolution
and ontogeny of human culture. This same flexibil-
ity and diversity provides an interesting evolution-
ary problem—if children’s capacity for social
learning explains cultural transmission, the psycho-
logical mechanisms should be universal, but these
psychological mechanisms must also be respon-
sive to diverse ontogenetic contexts and cultural
ecologies (Apicella & Barrett, 2016; Hrdy, 2009;
Legare & Harris, 2016; Nielsen et al., 2017). To
address this problem, we must first ask: H is cul-
ture acquired?

Children possess cognitive and communication
systems that evolved to acquire the complicated
technical and social skills characteristic of human
cultures. They are attentive to social input and learn
important skills and information through observa-
tion. Another way that children acquire cultural
knowledge and practices is through imitation. As
Bjorklund mentions, we know that children are also
precocious imitators and “overimitation” may be an
adaptive learning strategy to promote the high-fide-
lity acquisition and transmission of behavior. Is
high fidelity copying an adaptation that provides
the psychological foundation of human cultural
transmission? What is the function of imitation? We
have developed an integrated cognitive psychologi-
cal and ontogenetic account of how imitation and
innovation work in tandem to drive cultural learn-
ing and facilitate our capacity for cumulative cul-
ture. We propose that the unique demands of
acquiring instrumental skills (based on physical
causation) and rituals (based on social convention)
provide insight into when children imitate, when
they innovate, and to what degree. For instrumental
learning, with an increase in experience, high-fidelity

imitation decreases and innovation increases. In con-
trast, for conventional learning, imitative fidelity
stays high, regardless of experience, and innovation
stays low (Legare & Nielsen, 2015).

What distinguishes instrumental from ritual prac-
tices is a matter of interpretation based on contextual
information and social cues. We have used both
quantitative and qualitative methodologies to exam-
ine the kind of information children use to determine
when an event provides an opportunity for learning
instrumental skills versus learning cultural conven-
tions (Clegg & Legare, 2016b; Legare, Wen, Her-
rmann, & Whitehouse, 2015), the implications of
learning instrumental skills versus learning cultural
conventions for social group behavior (Watson-Jones
& Legare, 2016; Watson-Jones, Legare, Whitehouse,
& Clegg, 2014; Wen, Herrmann, & Legare, 2016), and
socialization of instrumental skills versus cultural
conventions in early childrearing environments
(Clegg & Legare, 2017; Clegg, Wen, & Legare, 2017).
Data from cross-cultural research have demonstrated
that children use imitation flexibly to acquire the
specific practices, beliefs, and values of their groups
(Clegg & Legare, 2016a).

To understand the ontogeny of human social
learning, we must examine how it changes over the
life span and how it varies in a strategically selected
set of cultural contexts that differ along theoretically
relevant variables. How do caregiver socialization
practices and the development of social learning
capacities enable and structure cumulative cultural
transmission? We are addressing the question by
studying the impact of diverse childrearing environ-
ments, practices, and social dynamics on the devel-
opment of cultural learning. For example, we
conduct research in educational settings and home
environments in both the United States (Austin,
Texas) and Vanuatu (Tanna; Clegg & Legare,
2016a). Vanuatu, a Melanesian island nation in the
South Pacific, is one of the most remote, culturally
and linguistically diverse, and understudied coun-
tries in the world. Vanuatu provides a unique
opportunity to explore the development of cultural
learning in populations that vary in extent of Wes-
tern influence. Conducting this research cross-cultu-
rally in Vanuatu and the United States allows us to
examine the imitative foundations of cultural learn-
ing in contexts that represent key aspects of the
diversity of human childrearing practices.

Humans are uniquely able to accumulate and
build upon the cultural innovations of previous
generations (Kurzban & Barrett, 2012; Pagel, 2012;
Pradhan, Tennie, & van Schaik, 2012; Whiten &
Erdal, 2012). Teaching and imitation work in tandem

2284 Legare, Clegg, and Wen

to conserve and transmit group-specific cultural
knowledge, increasing the likelihood for modifica-
tions and innovations, thus enhancing cultural com-
plexity (Enquist, Strimling, Eriksson, Laland, &
Sjostrand, 2010). Developing a comprehensive
understanding of teaching and imitation requires the
systematic study of cultural variation in childrearing
practices (Nielsen et al., 2017). We can enrich our
understanding of the developmental origins of
cumulative cultural transmission by conducting
cross-cultural research on cognitive and social devel-
opment (Legare & Harris, 2016).

In sum, comparative, developmental, and cross-
cultural research guided by evolutionary theory
provides insight into the evolution and ontogeny of
human cognition and behavior. Evolutionary theory
has made a profound and permanent impact on the
field of developmental psychology, shaping our
own research programs, as well as those of many
others. Bjorklund deserves substantial credit for this
striking scientific success story and should be feel-
ing very well indeed.

References

Apicella, C. L., & Barrett, H. C. (2016). Cross-cultural
evolutionary psychology. Current Opinion in Psychol-
ogy, 7, 92–97. https://doi.org/10.1016/j.copsyc.2015.08.
015

Aplin, L. M. (2015). Experimentally induced innovations
lead to persistent culture via conformity in wild birds.
Nature, 518, 538–541. https://doi.org/10.1038/nature
13998

Barrett, H. C., Broesch, T., Scott, R. M., He, Z., Baillargeon,
R., Wu, D., . . . Laurence, S. (2013). Early false-belief
understanding in traditional non-Western societies. Pro-
ceedings of the Royal Society B: Biological Sciences, 280,
20122654. https://doi.org/10.1098/rspb.2012.2654

Bjorklund, D. F. (2018). A metatheory for cognitive devel-
opment (or “Piaget is dead” revisited). Child Develop-
ment, 89, 2288–2302. https://doi.org/10.1111/13019

Bjorklund, D. F., & Causey, K. (2017). Children0s thinking:
Cognitive development and individual differences (6th ed.).
Thousand Oaks, CA: Sage.

Bjorklund, D. F., & Ellis, B. J. (2014). Children, childhood,
and development in evolutionary perspective. Develop-
mental Review, 34, 225–264. https://doi.org/10.1016/j.d
r.2014.05.005

Blake, P. R., McAuliffe, K., Corbit, J., Callaghan, T. C.,
Barry, O., Bowie, A., . . . Warneken, F. (2015). The
ontogeny of fairness in seven societies. Nature, 528,
258–261. https://doi.org/10.1038/nature15703

Boyette, A. H., & Hewlett, B. S. (2017). Teaching in hun-
ter-gatherers. Review of Philosophy and Psychology.
https://doi.org/10.1007/s13164-017-0347-2

Broesch, T., Rochat, P., Olah, K., Broesch, J., & Henrich, J.
(2016). Similarities and differences in maternal respon-
siveness in three societies: Evidence from Fiji, Kenya,
and the United States. Child Development. https://doi.
org/10.1111/cdev.12501

Cantor, M., Shoemaker, L. G., Cabral, R. B., Flores, C. O.,
Varga, M., & Whitehead, H. (2015). Multilevel animal
societies can emerge from cultural transmission. Nature
Communications, 6, 8091. https://doi.org/10.1038/nc
omms9091

Chudek, M., Heller, S., Birch, S., & Henrich, J. (2012).
Prestige-biased cultural learning: Bystander’s differen-
tial attention to potential models influences children’s
learning. Evolution and Human Behavior, 33(1), 46–56.
https://doi.org/10.1016/j.evolhumbehav.2011.05.005

Claidi�ere, N., & Whiten, A. (2012). Integrating the study
of conformity and culture in humans and nonhuman
animals. Psychological Bulletin, 138, 126–145. https://
doi.org/10.1037/a0025868

Clay, Z., & Tennie, C. (2017). Is overimitation a uniquely
human phenomenon? Insights from human children as
compared to bonobos. Child Development. https://doi.
org/10.1111/cdev.12857

Clegg, J. M., & Legare, C. H. (2016a). A cross-cultural
comparison of children’s imitative flexibility. Develop-
mental Psychology, 52, 1435–1444. https://doi.org/10.
1037/dev0000131

Clegg, J. M., & Legare, C. H. (2016b). Instrumental and con-
ventional interpretations of behavior are associated with
distinct outcomes in early childhood. Child Development,
87, 527–542. https://doi.org/10.1111/cdev.12472

Clegg, J. M., & Legare, C. H. (2017). Parents scaffold flexi-
ble imitation during early childhood. Journal of Experi-
mental Child Psychology, 153, 1–14. https://doi.org/10.
1016/j.jecp.2016.08.004

Clegg, J. M., Wen, N. J., & Legare, C. H. (2017). Is non-
conformity WEIRD? Cultural variation in adults’ beliefs
about children’s competency and conformity. Journal of
Experimental Psychology: General, 146, 428–441.
https://doi.org/10.1037/xge0000275

Corriveau, K. H., Fusaro, M., & Harris, P. L. (2009).
Going with the flow: Preschoolers prefer nondissenters
as informants. Psychological Science, 20, 372–377.
https://doi.org/10.1111/j.1467-9280.2009.02291.x

Davis, S. J., Vale, G. L., Schapiro, S. J., Lambeth, S. P., &
Whiten, A. (2016). Foundations of cumulative culture in
apes: Improved foraging efficiency through relinquish-
ing and combining witnessed behaviours in chim-
panzees (Pan troglodytes). Scientific Reports, 6, 35953.
https://doi.org/10.1038/srep35953

Dean, L. G., Vale, G. L., Laland, K. N., Flynn, E. G., &
Kendal, R. L. (2014). Human cumulative culture: A
comparative perspective. Biological Review Cambridge
Philosophical Society, 89, 284–301. https://doi.org/10.
1111/brv.12053

Emmons, N. A., Smith, H., & Kelemen, D. (2016). Chang-
ing minds with the story of adaptation: Strategies for
teaching young children about natural selection. Early

Evolutionary Developmental Psychology 2285

https://doi.org/10.1016/j.copsyc.2015.08.015

https://doi.org/10.1038/nature13998

https://doi.org/10.1098/rspb.2012.2654

https://doi.org/10.1111/13019

https://doi.org/10.1016/j.dr.2014.05.005

https://doi.org/10.1038/nature15703

https://doi.org/10.1007/s13164-017-0347-2

https://doi.org/10.1111/cdev.12501

https://doi.org/10.1038/ncomms9091

https://doi.org/10.1016/j.evolhumbehav.2011.05.005

https://doi.org/10.1037/a0025868

https://doi.org/10.1111/cdev.12857

https://doi.org/10.1037/dev0000131

https://doi.org/10.1111/cdev.12472

https://doi.org/10.1016/j.jecp.2016.08.004

https://doi.org/10.1037/xge0000275

https://doi.org/10.1111/j.1467-9280.2009.02291.x

https://doi.org/10.1038/srep35953

https://doi.org/10.1111/brv.12053

Education and Development, 27, 1205–1221. https://doi.
org/10.1080/10409289.2016.1169823

Enquist, M., Strimling, P., Eriksson, K., Laland, K., &
Sjostrand, J. (2010). One cultural parent makes no cul-
ture. Animal Behaviour, 79, 1353–1362. https://doi.org/
10.1016/j.anbehav.2010.03.009

Evans, E. M. (2016). Bridging the gap: From intuitive to
scientific reasoning-the case of evolution. In K. Rutten,
S. Blancke, & R. Soetaert (Eds.), Perspectives on science
and culture West Lafayette, IN: Purdue University
Press.

Garland, E. C., Gedamke, J., Rekdahl, M. L., Noad, M. J.,
Garrigue, C., & Gales, N. (2013). Humpback whale
song on the Southern Ocean feeding grounds: Implica-
tions for cultural transmission. PLoS ONE, 8. https://d
oi.org/10.1371/journal.pone.0079422

Gopnik, A., O’Grady, S., Lucas, C. G., Griffiths, T. L.,
Wente, A., Bridgers, S., . . . Dahl, R. E. (2017). Changes
in cognitive flexibility and hypothesis search across
human life history from childhood to adolescence to
adulthood. Proceedings of the National Academy of
Sciences of the United States of America, 114, 7892–7899.
https://doi.org/10.1073/pnas.1700811114

Hamlin, J. K. (2014). Context-dependent social evaluation
in 4.5-month-old human infants: The role of domain-
general versus domain-specific processes in the devel-
opment of social evaluation. Frontiers in Psychology, 5,
1–10. https://doi.org/10.3389/fpsyg.2014.00614

Haun, D. B. M., & Over, H. (2014). Like me: A homo-
phily-based account of human culture. In T. Breyer
(Ed.), Epistemological dimensions of evolutionary psychol-
ogy (pp. 117–130). New York, NY: Springer.

Heddy, B. C., & Sinatra, G. M. (2013). Transforming mis-
conceptions: Using transformative experience to pro-
mote positive affect and conceptual change in students
learning about biological evolution. Science Education,
97, 723–744. https://doi.org/10.1002/sce.21072

Henrich, J. (2009). The evolution of costly displays, coop-
eration and religion: Credibility enhancing displays and
their implications for cultural evolution. Evolution and
Human Behavior, 30, 244–260. https://doi.org/10.1016/
j.evolhumbehav.2009.03.005

Henrich, J. (2015a). Culture and social behavior. Current
Opinion in Behavioral Sciences, 3, 84–89. https://doi.org/
10.1016/j.cobeha.2015.02.001

Henrich, J. (2015b). The secret of our success: How culture is
driving human evolution, domesticating our species, and
making us smarter. Princeton, NJ: Princeton University
Press.

Herrmann, P. A., Legare, C. H., Harris, P. L., & White-
house, H. (2013). Stick to the script: The effect of witness-
ing multiple actors on children’s imitation. Cognition,
129, 536–543. https://doi.org/10.1016/j.cognition.2013.
08.010

Heyes, C. M. (in press). Enquire within: Cultural evolu-
tion and cognitive science. Philosophical Transactions of
the Royal Society: B.

House, B. R., Silk, J. B., Henrich, J., Barrett, H. C., Scelza,
B. A., Boyette, A. H., . . . Laurence, S. (2013). Ontogeny
of prosocial behavior across diverse societies. Proceed-
ings of the National Academy of Sciences of the United
States of America, 110, 14586–14591. https://doi.org/10.
1073/pnas.1221217110

Hrdy, S. B. (2009). Mothers and others: The evolutionary ori-
gins of mutual understanding. Cambridge, MA: Harvard
University Press.

Hublin, J. J. (2005). Evolution of the human brain and
comparative paleoanthropology. In S. Dehaene, J.-R.
Duhamel, M. D. Hauser, & G. Rizzolatti (Eds.), In from
monkey brain to human brain: A Fyssen Foundation sympo-
sium (pp. 57–71). Cambridge, MA: MIT Press.

Johnson-Pynn, J., Fragaszy, D., & Cummins-Sebree, S.
(2003). Common territories in comparative and devel-
opmental psychology: Quest for shared means and
meaning in behavioral investigations. International Jour-
nal of Comparative Psychology, 16, 1–27.

Kinzler, K. D., Dupoux, E., & Spelke, E. S. (2007). The native
language of social cognition. Proceedings of the National
Academy of Sciences of the United States of America, 104,
12577–12580. https://doi.org/10.1073/pnas.0705345104

Kurzban, R., & Barrett, H. C. (2012). Origins of cumula-
tive culture. Science, 335, 1056. https://doi.org/10.
1126/science.1219232

Laland, K. N., & Galef, B. (2009). The question of animal
culture. Cambridge, MA: Harvard University Press.

Laland, K. N., & Hoppitt, W. (2003). Do animals have
culture? Evolutionary Anthropology, 12, 150–159.
https://doi.org/10.1002/evan.10111

Leadbeater, E. (2015). What evolves in the evolution of
social learning? Journal of Zoology, 295(1), 4–11.
https://doi.org/10.1111/jzo.12197

Legare, C. H. (2017). Cumulative cultural learning:
Development and diversity. Proceedings of the National
Academy of Sciences of the United States of America, 114,
7877–7883. https://doi.org/10.1073/pnas.1620743114

Legare, C. H., & Harris, P. L. (2016). The ontogeny of cul-
tural learning. Child Development, 87, 633–642. https://
doi.org/10.1111/cdev.12542

Legare, C. H., Lane, J. D., & Evans, E. M. (2013). Anthro-
pomorphizing science: How does it affect the develop-
ment of evolutionary concepts? Merrill-Palmer Quarterly,
59, 168–197. https://doi.org/10.1353/mpq.2013.0009

Legare, C. H., & Nielsen, M. (2015). Imitation and innova-
tion: The dual engines of cultural learning. Trends in
Cognitive Sciences, 19, 688–699. https://doi.org/10.
1016/j.tics.2015.08.005

Legare, C. H., Wen, N. J., Herrmann, P. A., & White-
house, H. (2015). Imitative flexibility and the develop-
ment of cultural learning. Cognition, 142, 351–361.
https://doi.org/10.1016/j.cognition.2015.05.020

Lew-Levy, S., Reckin, R., Lavi, N., Crist�obal-Azkarate, J.,
& Ellis-Davies, K. (2017). How do hunter-gatherer chil-
dren learn subsistence skills? Human Nature. https://
doi.org/10.1007/s12110-017-9302-2

2286 Legare, Clegg, and Wen

https://doi.org/10.1080/10409289.2016.1169823

https://doi.org/10.1016/j.anbehav.2010.03.009

https://doi.org/10.1371/journal.pone.0079422

https://doi.org/10.1073/pnas.1700811114

https://doi.org/10.3389/fpsyg.2014.00614

https://doi.org/10.1002/sce.21072

https://doi.org/10.1016/j.evolhumbehav.2009.03.005

https://doi.org/10.1016/j.cobeha.2015.02.001

https://doi.org/10.1016/j.cognition.2013.08.010

https://doi.org/10.1073/pnas.1221217110

https://doi.org/10.1073/pnas.0705345104

https://doi.org/10.1126/science.1219232

https://doi.org/10.1002/evan.10111

https://doi.org/10.1111/jzo.12197

https://doi.org/10.1073/pnas.1620743114

https://doi.org/10.1111/cdev.12542

https://doi.org/10.1353/mpq.2013.0009

https://doi.org/10.1016/j.tics.2015.08.005

https://doi.org/10.1016/j.cognition.2015.05.020

https://doi.org/10.1007/s12110-017-9302-2

Lombrozo, T. (2013). Evolution challenges: Integrating
research and practice in teaching and learning about
evolution. Reports of the National Center for Science Edu-
cation, 33, 5.

Mesoudi, A., Chang, L., Murray, K., & Lu, H. J. (2015).
Higher frequency of social learning in China than in
the West shows cultural variation in the dynamics of
cultural evolution. Proceedings of Biological Sciences, 282,
20152209.

Nielsen, M., & Haun, D. (2016). Why developmental psy-
chology is incomplete without comparative and cross-
cultural perspectives. Philosophical Transactions of the
Royal Society of London. Series B, Biological Sciences, 371,
20150071. https://doi.org/10.1098/rstb.2015.0071

Nielsen, M., Haun, D. B. M., Kaertner, J., & Legare, C. H.
(2017). The persistent sampling bias in developmental
psychology: A call to action. Journal of Experimental
Child Psychology, 162, 32–38.

Pagel, M. (2012). Evolution: Adapted to culture. Nature,
482, 297–299. https://doi.org/10.1038/482297a

Perry, S., Baker, M., Fedigan, L., Gros-Louis, J., Jack, K.,
MacKinnon, K. C., . . . Rose, L. (2003). Social conven-
tions in wild white-faced capuchin monkeys: Evidence
for traditions in a neotropical primate. Current Anthro-
pology, 44, 241–268. https://doi.org/10.1086/345825

Plotnik, J. M., Lair, R., Suphachoksahakun, W., & de Waal,
F. B. M. (2011). Elephants know when they need a help-
ing trunk in a cooperative task. Proceedings of the National
Academy of Sciences of the United States of America, 108,
5116–5121. https://doi.org/10.1073/pnas.1101765108

Pradhan, G. R., Tennie, C., & van Schaik, C. P. (2012).
Social organization and the evolution of cumulative
technology in apes and hominins. Journal of Human Evo-
lution, 63, 180–190. https://doi.org/10.1016/j.jhevol.
2012.04.008

Rosati, A. G., & Warneken, F. (2016). How comparative psy-
chology can shed light on human evolution: Response to
Beran et al.’s discussion of “Cognitive capacities for
cooking in chimpanzees.” Learning & Behavior, 44,
109–115. https://doi.org/10.3758/s13420-016-0220-7

Santos, L. R., & Rosati, A. G. (2015). The evolutionary
roots of human decision making. Annual Review of Psy-
chology, 66, 321–347. https://doi.org/10.1146/annurev-
psych-010814-015310

Short, S. D., & Hawley, P. H. (2015). The effects of evolu-
tion education: Examining attitudes toward and knowl-
edge of evolution in college courses. Evolutionary
Psychology, 13(1), 67–88. https://doi.org/10.1177/
147470491501300105

Shtulman, A., Neal, C., & Lindquist, G. (2016). Children’s
ability to learn evolutionary explanations for biological
adaptation. Early Education and Development, 27, 1222–
1236. https://doi.org/10.1080/10409289.2016.1154418

Terashima, H., & Hewlett, B. S. (2016). Social learning and
innovation in contemporary hunter-gatherers. Springer.
Retrieved from http://www.springer.com/series/11816

van Leeuwen, E. J. C., Call, J., & Haun, D. (2014). Human
children rely more on social information than chim-
panzees do. Biology Letters, 10, 20140487. https://doi.
org/10.1098/rsbl.2014.0487

van Schaik, C. P., & Burkart, J. M. (2011). Social learning
and evolution: the cultural intelligence hypothesis. Phi-
losophical Transactions of the Royal Society B: Biological
Sciences, 366, 1008–1016. http://doi.org/10.1098/rstb.
2010.0304

Warneken, F., & Tomasello, M. (2017). Altruistic helping
in human infants and young chimpanzees. Science,
311, 1301–1303. https://doi.org/10.1126/science.
1121448

Watson-Jones, R. E., & Legare, C. H. (2016). The social
functions of group rituals. Current Directions in Psycho-
logical Science, 25(1), 42–46. https://doi.org/10.1177/
0963721415618486

Watson-Jones, R. E., Legare, C. H., Whitehouse, H., &
Clegg, J. M. (2014). Task-specific effects of ostracism on
imitative fidelity in early childhood. Evolution and
Human Behavior, 35, 204–210. https://doi.org/10.1016/
j.evolhumbehav.2014.01.004

Wen, N. J., Herrmann, P. A., & Legare, C. H. (2016).
Ritual increases children’s affiliation with in-group
members. Evolution and Human Behavior, 37, 54–60.
https://doi.org/10.1016/j.evolhumbehav.2015.08.002

Wertz, A. E., & Wynn, K. (2014). Thyme to touch: Infants
possess strategies that protect them from dangers
posed by plants. Cognition, 130(1), 44–49. https://doi.
org/10.1016/j.cognition.2013.09.002

Whitehead, H., & Rendell, L. (2015). The cultural lives of
whales and dolphins. Chicago, IL: University of Chicago
Press.

Whiten, Andrew (2017). Culture extends the scope of evo-
lutionary biology in the great apes. Proceedings of the
National Academy of Sciences, 114(30), 7790–7797. http://
www.pnas.org/content/114/30/7790.

Whiten, A., & Erdal, D. (2012). The human socio-cogni-
tive niche and its evolutionary origins. Philosophical
Transactions of the Royal Society of London B: Biological
Sciences, 367, 2119–2129. https://doi.org/10.1098/rstb.
2012.0114

Evolutionary Developmental Psychology 2287

https://doi.org/10.1098/rstb.2015.0071

https://doi.org/10.1038/482297a

https://doi.org/10.1086/345825

https://doi.org/10.1073/pnas.1101765108

https://doi.org/10.1016/j.jhevol.2012.04.008

https://doi.org/10.3758/s13420-016-0220-7

https://doi.org/10.1146/annurev-psych-010814-015310

https://doi.org/10.1177/147470491501300105

https://doi.org/10.1080/10409289.2016.1154418

http://www.springer.com/series/11816

https://doi.org/10.1098/rsbl.2014.0487

http://doi.org/10.1098/rstb.2010.0304

https://doi.org/10.1126/science.1121448

https://doi.org/10.1177/0963721415618486

https://doi.org/10.1016/j.evolhumbehav.2014.01.004

https://doi.org/10.1016/j.evolhumbehav.2015.08.002

https://doi.org/10.1016/j.cognition.2013.09.002

http://www.pnas.org/content/114/30/7790

https://doi.org/10.1098/rstb.2012.0114

This document is a scanned copy of a printed document. No warranty is given about the
accuracy of the copy. Users should refer to the original published version of the material.

https://doi.org/10.1177/0022167817739732

Journal of Humanistic Psychology
2020, Vol. 60(6) 934 –958

sagepub.com/journals-permissions
DOI: 10.1177/0022167817739732

journals.sagepub.com/home/jhp

Regular Article

Maslow’s
Unacknowledged
Contributions to
Developmental
Psychology

Andrew M. Bland1 and Eugene M. DeRobertis2,3

Abstract
Few readily identify Maslow as a developmental psychologist. On the other
hand, Maslow’s call for holistic/systemic, phenomenological, and dynamic/
relational developmental perspectives in psychology (all being alternatives to
the limitations of the dominant natural science paradigm) anticipated what
emerged both as and in the subdiscipline of developmental psychology. In
this article, we propose that Maslow’s dynamic systems approach to healthy
human development served as a forerunner for classic and contemporary
theory and research on parallel constructs in developmental psychology
that provide empirical support for his ideas—particularly those affiliated
with characteristics of psychological health (i.e., self-actualization) and the
conditions that promote or inhibit it. We also explore Maslow’s adaptation
of Goldstein’s concept of self-actualization, in which he simultaneously: (a)
explicated a theory of safety versus growth that accounts for the two-steps-
forward-one-step-back contiguous dynamic that realistically characterizes
the ongoing processes of being-in-becoming and psychological integration
in human development/maturity and (b) emphasized being-in-the-world-with-
others with the intent of facilitating the development of an ideal society

1Millersville University, Millersville, PA, USA
2Brookdale College, Lincroft, NJ, USA
3Rutgers University–Newark, Newark, NJ, USA

Corresponding Author:
Eugene M. DeRobertis, Department of Psychology, Brookdale College, MAN 126c, Lincroft,
NJ 07738, USA.
E-mail: ederobertis@brookdalecc.edu

739732 JHPXXX10.1177/0022167817739732Bland and DeRobertisBland and DeRobertis
research-article2017

https://us.sagepub.com/en-us/journals-permissions

https://journals.sagepub.com/home/jhp

mailto:ederobertis@brookdalecc.edu

http://crossmark.crossref.org/dialog/?doi=10.1177%2F0022167817739732&domain=pdf&date_stamp=2017-11-08

Bland and DeRobertis 935

by promoting protective factors that illustrate Maslow’s safety, belonging,
and esteem needs. Finally, we dialogue with the extant literature to clarify
common misgivings about Maslow’s ideas.

Keywords
Maslow, self-actualization, developmental psychology, dynamic systems

Few readily identify Maslow as a developmental psychologist. Both he and
the humanistic movement are almost always excluded from developmental
textbooks (DeRobertis, 2008), and an EBSCO search in February 2017
yielded a dearth of relevant articles. In the rare instances in which Maslow is
included, his ideas are typically misrepresented. On the other hand, as we
have previously suggested (Bland & DeRobertis, 2017; DeRobertis, 2012),
Maslow and other founding humanistic psychologists’ calls for holistic/sys-
temic, phenomenological, and dynamic/relational developmental perspec-
tives in psychology (as alternatives to the dominant natural science paradigm)
anticipated what emerged both as and in the subdiscipline of developmental
psychology. Accordingly, herein, we propose that classic and contemporary
theory and research in developmental psychology provide empirical support
for Maslow’s ideas, particularly those affiliated with characteristics of psy-
chological health (i.e., self-actualization) and the factors that promote or
inhibit it.

Maslow (1999) observed that “from a developmental point of view,” self-
actualizing individuals “are more fully evolved” insofar as they are “not fix-
ated at immature or incomplete levels of growth” (p. 172). They strive toward
“unity of personality” and “spontaneous expressiveness” as well as “seeing
the truth rather than being blind,” “being creative,” and demonstrating “seren-
ity, kindness, courage, honesty, love, unselfishness, and goodness” (Maslow,
1999, p. 171). Using growth and health as his baseline, Maslow helped usher
in a focus on normative and transformative developmental processes in psy-
chology. At the same time, he acknowledged the role of regressive forces and
the potential for stagnation, often as the outcome of inadequate environmen-
tal conditions.

Maslow’s Developmentally Oriented Adaptation of
Goldstein’s

Self-Actualization

Maslow (1987) adapted the construct of self-actualization from Goldstein, an
organismic-oriented neurologist–psychiatrist. According to Whitehead (2017),

936 Journal of Humanistic Psychology 60(6)

Goldstein’s construct was built on three axioms. First, self-actualization refers
to a process of individuation (i.e., the ongoing emergence and regeneration of
a self as an active, creative authority distinct from other biochemical systems)
that, second, must be conceptualized holistically and not in isolation (i.e., it is
only through the organism–environment relationship that the meaning behind
behavior, pathology, personality, motivation, emotion, etc., can be under-
stood). Third, Goldstein proposed that behavior is invariantly motivated in
terms of self-actualization (i.e., is not synonymous with tension reduction or
mere self-preservation or survival).

Whereas Goldstein (1934/1995, 1963) primarily focused on self-actual-
ization vis-à-vis the resilient reorganization of a person’s capacities in
response to brain injury or psychopathology, Maslow further included over-
coming obstacles (real and perceived) and living authentically despite one’s
personal, environmental, and historical shortcomings as functions of healthy
development. Maslow (1999) explicated a theory of safety versus growth
that accounts for the two-steps-forward-one-step-back contiguous dynamic
that realistically characterizes the ongoing process of being-in-becoming
and of graded experiential awareness and psychological integration in
human maturity. Beginning in childhood and continuing throughout the
lifespan, individuals negotiate a dialectic between homeostasis (i.e., defen-
sively clinging to the familiar and predictable, irrespective of how stagnant,
disappointing, or precarious the outcome) and morphogenic enactment “of
all [their] capacities, toward confidence in the face of the external world at
the same time that [they] can accept [their] deepest, real, unconscious Self”
(Maslow, 1999, p. 55).

Maslow challenged the classical Freudian assumption of homeostasis as
an end state. Instead, like Erikson (1959/1994), he argued that “healthy chil-
dren enjoy growing and moving forward, gaining new skills, capacities and
powers” that evolve into “authentic selfhood, [i.e., knowing] what one really
wants and doesn’t want, what one is fit for and what one is not fit for”
(Maslow, 1999, pp. 30, 213). Taken together, Maslow’s focus on the dialecti-
cal relationship between a process of continuous improvement and ongoing
integration, organization, and self-consistency (see Frick, 1971) reflects
Goldstein’s aforementioned first axiom.

In addition, Maslow emphasized that self-actualization entails a sense of
being-in-the-world-with-others, interindividuality, community feeling, and
interest in making changes for an ideal society. These points are synonymous
with Adler’s (1931/1998) social interest and parallel Erikson’s (1959/1994)
emphasis on participating in (rather than struggling against) society as both
conducive to and reflective of healthy social and emotional development.
Maslow distinguished between uniqueness and distinctiveness in relation to

Bland and DeRobertis 937

others (Koydemir, Şimşek, & Demir, 2014), drawing from and making best
use of one’s potentials to benefit the collective:

Authentic or healthy [individuals] may be defined not . . . by [their] own
intrapsychic and non-environmental laws, not as different from the
environment, independent of it or opposed to it, but rather in environment-
centered terms. . . . Self-actualization . . . paradoxically makes more possible
the transcendence of . . . self-consciousness and of selfishness. It makes it
easier for [one] . . . to merge as a part in a larger whole. (Maslow, 1999, pp.
199, 231, italics added in first sentence)

This relational viewpoint is commensurate with Goldstein’s second axiom.
With regard to Goldstein’s third axiom, Maslow (1987) eschewed reduc-

tionistic explanations of behavior and emphasized that behavior is “overde-
termined or multimotivated,” reflecting combinations of needs in striving
toward self-actualization. As an organizing principle, Maslow (1987, 1999)
proposed a hierarchical structure from physiological to security to belonging
to self-esteem. Each set of needs is gratified on a continuum from more exter-
nalized (lower, more basic needs) to more intrinsic (higher, more idiosyn-
cratic needs). Furthermore, Maslow (1999) emphasized that one’s essential
“core” involves “potentialities, not final actualizations” that are “weak, sub-
tle, and delicate, very easily drowned out by learning, by cultural expecta-
tions, by fear, by disapproval, etc.” and can therefore become “forgotten,
[i.e.,] neglected, unused, overlooked, unverbalized, or suppressed” (pp. 212-
213). To illustrate:

[Children] who [are] insecure, basically thwarted, or threatened in [their] needs
for safety, love, belongingness, and self-esteem . . . will show more selfishness,
hatred, aggression, and destructiveness. . . . This implies a reactive, instrumental,
or defensive interpretation of hostility rather than an instinctive one. (Maslow,
1987, p. 86)

Maslow (1987) emphasized that fulfillment of the basic needs is neither a
lockstep progression nor confined to specific ages/phases of life, but rather is
a holistic process:

[The statement that] if one need is satisfied, then another emerges . . . might
give the false impression that a need must be satisfied 100% before the next
need emerges. In actual fact, most [individuals] are partially satisfied in all
their basic needs and partially unsatisfied in all their basic needs at the same
time. A more realistic description of the hierarchy would be in terms of
decreasing percentages of satisfaction as we go up the hierarchy of prepotency.

938 Journal of Humanistic Psychology 60(6)

. . . The emergence [of a new need] is not a sudden, saltatory phenomenon, but
rather a gradual emergence by slow degrees. (pp. 27-28, italics added)

Thus, Rowan (1999) used the analogy of Russian nesting dolls to illustrate
the idea that the lower needs are transcended but also included within the
higher ones, that is, they are never lost.

Dynamic Systems Developmental Orientation

Perhaps one reason that Maslow is not typically included in developmental
textbooks and research articles is that his quotidian vision of psychology as a
human science was not fully congruent with either discontinuous stage mod-
els or the continuous, quantitatively driven perspectives that constituted the
majority of the traditional developmental psychology literature during the
second half of the 20th century. Meantime, taken out of context, his emphasis
on self-actualization reeks of Western individualism and therefore generally
has been dismissed (or, at best, overlooked) by most sociocultural theorists in
the new millennium. On the other hand, during the past decade, dynamic
systems models—the paradigm with which Maslow’s (1987) “holistic-
dynamic” thinking aligned (p. 15)—have gained legitimacy in psychology
(see Bland & Roberts-Pittman, 2014; DeRobertis, 2011b; Gelo & Salvatore,
2016), and they were included as a theoretical category in Bergen’s (2008)
textbook on human development.

Dynamic systems models incorporate concepts of complexity, plasticity,
and recursive nested features (Bergen, 2008). Maslow (1971) emphasized
that self-actualizing should be conceptualized iteratively (i.e., as a verb) and
not as an achievement or trait (i.e., as a noun). Moreover, dynamic systems
models are built on the assumptions that (a) complex, chaotic systems (e.g.,
human beings) have the ability to self-organize into purposeful behaviors and
that (b) sensitive dependence on initial conditions—in which a small input in
a system may yield disparate results—can explain developmental change
(Bergen, 2008). Maslow (1987) accounted for the possibility of quantum
leaps in development, in which significant changes at one need level can
incite substantive changes at the subsequent levels.

Congruent with Maslow’s aforementioned safety versus growth principle
(two-steps-forward-one-step-back), Skalski and Hardy (2013) noted that
such quantum transformation is typically propagated by individuals’ under-
standings of themselves and the world becoming disintegrated by stress, rela-
tional difficulties, hopelessness, losing control/holding on, and psychological
turmoil and then enhanced by the presence of a trusted other who provides
corrective experiences (see Bland, 2014; Castonguay & Hill, 2012). To

Bland and DeRobertis 939

illustrate, DeRobertis’ (2016) study on children’s education implied that
quality teachers can serve not only as extensions of attachment relationships
(when they already exist) but also as surrogates thereof (when they do not).
In addition, whereas Graber, Turner, and Madill (2015) hypothesized that
during adolescence family support would moderate the significance of friend-
ships as a risk or protective mechanism, they discovered instead that, irre-
spective of family, having just one fulfilling friendship prevents relational,
emotional, and behavior problems.

Furthermore, dynamic systems models can be characterized as prototheo-
retical rather than fully developed, falsifiable theories and are supported by
research methods that involve collecting minute process data (Bergen, 2008).
Maslow’s aforementioned initial study on the characteristics of self-actualiz-
ing people (included in Maslow, 1987) and his research on peak experiences
(included in Maslow, 1999) employed iterative qualitative analyses (see Wertz
et al., 2011). These involved him extracting themes from biographies and
interviews with purposive samples to critically catalog and describe their
common attributes which he then triangulated with extant theory and empiri-
cal research in conjunction with quantitative and qualitative studies he had
conducted during his early career (see Hoffman, 1988; Maslow, 1973).
Maslow’s emphases on self-actualization and on values in psychology set the
stage for psychologists acknowledging the realities of plasticity and of multi-
dimensional, multidirectional developmental principles that value the whole
person in context and that are now underscored in developmental textbooks
(e.g., Capuzzi & Stauffer, 2016; Music, 2017). It is crucial to note that
Maslow’s theories were built as an outcome of his research (not the other way
around), that he was flexible, open to criticism, and constantly expanding and
revising his ideas, and that he emphasized the need for them to be empirically
tested and reworked as appropriate (see Frick, 1971; Maslow, 1971, 1999).

Finally, Maslow’s nonexclusive vision also paved the way for develop-
mental psychology’s resolution of long-held (stereotypically Western) con-
ceptual bifurcations (see Music, 2017). For example, with regard to nature
versus nurture, Maslow (1987) remarked as follows:

How can it be said that a complex set of reactions is either all determined by
heredity or not at all determined by heredity? There is no structure, however,
simple . . . that has genetic components alone. At the other extreme it is also
obvious that nothing is completely free of the influence of heredity, for humans
are a biological species. (p. 48)

Maslow (1966, 1971, 1987, 1999) also emphasized moving beyond the antin-
omies of free will versus determinism, continuity versus change, universality

940 Journal of Humanistic Psychology 60(6)

versus cultural specificity, and experimentalism versus experientialism in
understanding human development.

Classic and Contemporary Empirical Support

Physiological

Maslow’s working class upbringing as the eldest son of Russian Jewish
immigrants influenced his lifelong focus on social justice (Hoffman, 1988).
According to Anne Richards (personal communication, 2003), in his classes
during the 1960s, Maslow advocated for the development of reduced-price
meals in schools (now a given in most communities in the United States) as a
means of minimizing obstacles to impoverished children’s growth and
empowerment. Maslow’s suggestion brought awareness of how issues of
social policy and both availability and quality of resources at Bronfenbrenner’s
(1994) exosystemic, macrosystemic, and chronosystemic levels influence
individuals’ development, whereas the principal focus of psychology at mid-
20th century was almost exclusively at the individual and microsystemic
levels.

Accordingly, since Maslow’s day, developmental researchers have come
to emphasize the connections between malnutrition and children’s: (a) ability
to sustain attention (which in turn affects cognitive development and aca-
demic performance); (b) levels of irritability and self-regulation (which affect
social development); and (c) propensity to diagnosable mental health condi-
tions as well as susceptibility to infectious disease, obesity, and eventual dia-
betes and heart issues (as summarized in Arnett, 2016; Broderick & Blewitt,
2015). Congruent with the dynamic systems assumption that a small change
can spawn sustentative outcomes, Broderick and Blewitt (2015) commented,
“When we intervene to reduce one risk factor, such as malnutrition, we may
actually [also] reduce the impact of other negative influences” (p. 56). In
addition, Prince and Howard (2002) extended Maslow’s thinking on the
developmental implications of physiological needs to include access to ade-
quate health care, insurance, and living environments safe from toxicity (e.g.,
exposure to lead). Furthermore, Desmond’s (2016) ethnographic research
addressed the systemic challenges in tenants’, landlords’, and social service
agencies’ abilities to uphold sustainable living environments and the devel-
opmental impacts for both children and adults.

Maslow (1987; Maslow & Mittelmann, 1951) also noted that healthy
growth and development involves not only gratification of the basic needs
but also the ability to withstand reasonable deprivation. “Increased frustra-
tion tolerance through early gratification” enables individuals to “withstand

Bland and DeRobertis 941

food deprivation” because they “have been made secure and strong in the
earliest years,” which reciprocates into them remaining secure and strong
thereafter (Maslow, 1987, p. 27). As applied to the physiological needs,
Erikson (1959/1994) suggested that a developmental task of infancy is to
establish confidence in one’s caregivers to eventually attend to one’s needs
even if caregivers are unable to drop what they are doing the moment one
expresses a need. Accordingly, secure interactions between parent and child
moderate the relationship between low socioeconomic status and develop-
mental outcomes (Bronfenbrenner, 1994).

Safety

Maslow (1987) defined the safety needs as “security; stability; dependency;
protection; freedom from fear, anxiety, and chaos [and] need for structure,
order, law, and limits” (p. 18). In contrast with conventional wisdom in (par-
ticularly) American parenting practices that emphasize independence as
quickly as possible, models such as attachment parenting (Miller & Commons,
2010) promote the value of strong bonding early in life, congruent with less
ruggedly individualistic cultures around the globe (see Maté, 2011; Morelli &
Rauthbaum, as cited in Arnett, 2016). Researchers have noted that such
highly responsive caregiving practices: (a) mitigate potentially overwhelm-
ing negative emotional states (e.g., preventable fear, anger, distress) and
therefore propagate appropriate emotional regulation; (b) reduce exposure to
stressors that adversely affect brain development and self-regulation and that
contribute to eventual mental health problems; (c) are associated with fewer
expressions of distress; and (d) promote empathy, perspective-taking, social
competence, cooperative behavior, and engagement in school life (see
Broderick & Blewitt, 2015; Campa, 2013; Miller & Commons, 2010). In
contrast, executive functioning becomes impaired “when young children are
exposed to chronically stressful situations” insofar as:

the brain development of the lower portions of the brain, responsible for “fight
or flight” reactions, are strengthened while the development in the cortex
regions of the brain, which are responsible for functions such as abstract and
rational thinking, are weakened. (Prince & Howard, 2002, pp. 29-30)

Thus, paradoxically, a strong sense of attachment early on facilitates
appropriate levels of differentiation of self (Bowen, 1978; Firestone,
Firestone, & Catlett, 2013) and autonomy (Erikson, 1959/1994)—all of
which include mindful self-regulation and approaching unfamiliar situations
with curiosity and interest rather than as threatening. Accordingly, they are

942 Journal of Humanistic Psychology 60(6)

conducive to self-sufficiency (comfort in one’s skin and with one’s own
beliefs, attitudes, and preferences) and assertiveness (vs. aggression, passiv-
ity, or passive-aggression).

Attachment. Secure attachment (Ainsworth, as summarized and updated in
Siegel, 2012; also see Music, 2017), marked by caregivers’ sensitivity and
responsiveness to infants’ cues, is associated with curiosity and differentia-
tion of self by early childhood (i.e., preschool and kindergarten age), with
positive social interactions and stronger academic performance during school
age, and with appropriate self-esteem and a strong sense of identity as adults
(all prerequisites for, though not necessarily characteristics of, self-actualiz-
ing, Maslow, 1987). On the other hand, with respect to insecure attachment,
Maslow observed that when safety needs are not met, behavior and motiva-
tion are disposed to stagnation or regression:

Since others are so important and vital for the helpless baby and child, fear of
losing them (as providers of safety, food, love, respect, etc.) is a primal,
terrifying danger. Therefore [children], faced with a difficult choice between
[their] own delight experiences and the experience of approval from others,
must generally choose approval from others, and then handle [their] delight by
repression or letting it die, or not noticing it or controlling it by willpower. In
general, along with this will develop a disapproval of the delight experience, or
shame and embarrassment and secretiveness about it, with finally, the inability
to even experience it. (Maslow, 1999, pp. 59-60)

This can lend itself to rigidity; to efforts to distract oneself from inner experi-
ence (Frankl, 1978; Harris, 2006); to engagement in addictive and/or compul-
sive behaviors as surrogates for meaningful interaction (Maté, 2010); and/or
to involvement in (sometimes precarious) relationships (Campa, 2013) and/or
institutional affiliations (May, 1967) that offer the illusion of security.

Parenting Styles. Maccoby and Martin (1983) noted that authoritative parent-
ing (see Gordon, 1975; Shapiro & White, 2014)—characterized by a balance
of emotional warmth and high expectations (demandingness); associated
with secure attachment and, later, identity achievement (Erikson, 1959/1994;
Marcia, 1966)—promotes the development of assertiveness, competence and
self-confidence, social responsibility, healthy achievement orientation, adapt-
ability, and so on (all qualities of Maslovian self-actualizing people). In con-
trast, children of authoritarian parents (high demandingness, low warmth;
associated with avoidant attachment and, later, identity foreclosure) are prone
to conformity, dependency, perfectionism, resentful anxiety, and susceptibil-
ity to bullying. Children of permissive/indulgent parents (high warmth, low

Bland and DeRobertis 943

demandingness; associated with ambivalent attachment and, later, chronic
moratorium) are at risk for becoming impulsive, egocentric, low in self-reli-
ant decision making, underachieving, and easily frustrated by authority
(being unaccustomed to structure). Neglectful/uninvolved parenting (low
warmth and low demandingness; associated with disorganized attachment
and, later, identity diffusion) is predictive of delinquency and children devel-
oping a symptomatic presentation consisting of both externalizing (impulsiv-
ity, aggression) and internalizing (moodiness, low self-esteem) qualities.

Maslow (1999) alluded to authoritative parenting by saying that children
should “be directed . . . both toward cultivation of controls and cultivation of
spontaneity and expression” (p. 219) and noted that “youngsters need a world
that is just, fair, orderly, and predictable” and that “only strong parents can
supply these important qualities” (Maslow, 1996, p. 46). Maslow also cau-
tioned against both excessively authoritative and permissive parenting styles.
With regard to the former, Maslow (1999) suggested as follows:

It is necessary in order for children to grow well that adults have enough trust
in them and in the natural processes of growth, i.e., [to] not interfere too much,
not make them grow, or force them into predetermined designs, but rather let
them grow and help them grow in a Taoistic rather than an authoritarian way.
(p. 219)

With regard to the dangers of permissive parenting:

Children, especially younger ones, essentially need, want, and desire external
controls, decisiveness, discipline, and firmness . . . to avoid the anxiety of being
on their own and of being expected to be adultlike because they actually
mistrust their own immature powers. (Maslow, 1996, p. 45)

Maslow (1971) continued that this anxiety eventually manifests into the
defense mechanism of desacrilizing (i.e., mistrusting the possibility of values
and virtues associated with self-actualization) based on having felt “swindled
or thwarted in their lives” and therefore coming to “despise their elders” (p.
48). Similarly, Horney (1945) proposed that to deal with this anxiety, based
on their particular formative experiences, individual children develop means
of coping via moving toward others (compliance), against them (aggression),
or away from them (withdrawal).

Love and Belonging

Maslow (1987) conceptualized the love needs as “giving and receiving affec-
tion” without which one “will hunger for relations with people in general—for

944 Journal of Humanistic Psychology 60(6)

a place in the group or family—[because] the pangs of loneliness, ostracism,
rejection, friendlessness, and rootlessness are preeminent” (pp. 20-21). He
(Maslow & Mittelmann, 1951) continued that love needs include the abilities
to form sustainable emotional ties; to empathize, enjoy oneself, and laugh with
(vs. at) others; and to express resentment without losing control (i.e., one can
love others and be angry with them at the same time), as well as having valid
reasons for being unhappy (vs. harboring resentment). Concerning the place-
ment of love and belonging at the same hierarchical level, he stated, “It is clear
that, other things being equal, a person who is safe and belongs and is loved
will be healthier . . . than one who is safe and belongs, but who is rejected and
unloved” (Maslow, 1987, p. 38). For an example of the latter, consider gang or
cult membership.

Sociometric Status. Coie and Dodge (1988) and subsequent researchers explored
the relationship between how children are perceived by their peers (i.e., liked
vs. disliked) and their behavior. Popular children, most often rated as liked by
their peers, tend to be cooperative, friendly, sociable, and interpersonally sensi-
tive. Rejected children, typically boys, are most often rated as disliked and
rarely as liked by peers. They fall into one of two groups: (a) rejected-aggres-
sive children (most typical), who have reputations for bullying and disruptive-
ness and (b) rejected-withdrawn children (about 10% to 20% of cases), who are
perceived by others as depressed. Neglected children, typically girls, are rated
neither as liked nor disliked; however, their peers typically misremember them.
Average children are not rated at either extreme (they are neither popular nor
unpopular) but they are known for being socially skilled. Finally, controversial
children are rated as liked by some and disliked by others; they have reputa-
tions as class clowns and as leaders with disregard for social rules.

Ollendick, Weist, Borden, and Greene (1992) noted that teachers tend to
rate rejected children at highest risk of engaging in problematic behaviors
during ninth grade based on their sociometric status at fourth grade, followed
by, in order, controversial, neglected, and popular children—and average
children at minimal risk. With regard to actual engagement in behaviors that
led to suspension or legal issues, rejected children were highest. Perhaps
more notably, 20% of average children dropped out, whereas none of the
neglected children dropped out. Arguably, teachers’ reaching out to children
who had been neglected by their peers may have contributed to a sense of
belonging. In contrast, the average children, being overlooked by both teach-
ers and peers, were less likely to “identify with the establishments of schools”
and therefore drop out due to feeling “out of place” (Prince & Howard, 2002,
p. 30). Furthermore, Prinstein (2017) differentiated between popularity and
likability; Maslow would have regarded the former as deficiency (D-) love/

Bland and DeRobertis 945

belonging, and the latter as indicative of appropriately fulfilled (being, B-)
love/belonging.

Identity, Intimacy, and Generativity. Maslow’s love and belonging needs also are
implicated in Erikson’s (1959/1994) developmental tasks of adolescence and
adulthood. The mission of adolescence is to search for and settle on a sense
of stability and continuity in individuals’ personality amid confusion, change,
and uncertainty. One dimension of identity development is clarification of
their values and vocation—not only to earn money but also to strive for an
honest sense of accomplishment within the lens of their culture. Like Maslow
(1971, 1999), Erikson cautioned that American society’s overemphasis on
standardization and conformity places adolescents at risk of helplessness and
foreclosure, while its oversaturation of choices begets stagnation and avoid-
ance of responsibility. On the other hand, when the process goes well, adoles-
cents arrive at a sense of belonging and of congruence between their actual
self and the contributions they make to their society by employing their
potentials and abilities. Also, they become more at ease in multiple roles
across several life domains (e.g., work, family, community, etc.). Cordeiro,
Paixão, Lens, Lacante, and Luyckx (2016) noted that Portuguese adolescents’
perceived parental support (Maslow’s love/belonging) is a protective factor
in career decision making, while parental thwarting is a risk factor. Both are
mediated by adolescents’ subjective feelings of having their love/belonging
needs met, which result in either confidence in proactive exploration and
commitment making or in endless rumination over identity options.

As individuals enter adulthood, the development during childhood and
adolescence of a strong sense of self is necessary to merge identities with
another in a loving adult relationship without fear of losing their own identity,
autonomy, and integrity. Erikson (1959/1994) noted that disconnection and
repeated failed marriages arise out of failure to establish an intimate connec-
tion. On the other hand, when the process goes well, individuals are able to
engage in authentic relationships (vs. overly formal or stereotyped ones and/
or isolation). By middle adulthood, healthy development involves an increased
shift in focus from self toward other and toward guiding the next generation as
an expression of their belief in the species (not just their immediate social
network). On the other hand, if Maslow’s security and love/belonging needs
have not been adequately satisfied, Erikson (1959/1994) noted that people fall
into self-absorption and mechanical, unfulfilling routines.

Self-Esteem

“If . . . the person wins respect and admiration and because of this develops
self-respect, then he or she is still more healthy, self-actualizing, or fully

946 Journal of Humanistic Psychology 60(6)

human,” Maslow (1987) wrote; thus, “satisfaction of the self-esteem need
leads to feelings of self-confidence, worth, strength, capability, and adequacy,
of being useful and necessary in the world” (pp. 38, 21). Maslow (1987;
Maslow & Mittelmann, 1951) conceptualized self-esteem as a multifaceted
construct that includes (a) some originality, individuality, and independence
from group opinions—that is, real self instead of idealized pseudo-self
(Horney, Rogers, Winnicott, as cited in DeRobertis, 2008), differentiated self
instead of emotional cutoff (Bowen, 1978); (b) having achievable, realistic,
and compatible goals which involve some good to society as well as reason-
able persistence of effort to achieve them; (c) absence of excessive need for
reassurance and approval; (d) desire for adequacy, mastery, competence, and
achievement; (e) a sense of confidence in the face of the world—which, like
Adler (1927/2010; see also DeRobertis, 2011a), Maslow (1987) distinguished
from sheer willpower and determination; (f) positive (vs. negative) freedom;
(g) desire for dignity, appreciation, and deserved respect from others—which
Maslow distinguished from external fame, celebrity, and unwarranted adula-
tion; (h) appreciation of cultural differences; and (i) realistic appraisal of per-
sonal strengths, limitations, motivations, desires, goals, ambitions, inhibitions,
defenses, compensations, and so on.

With regard to acceptance of one’s imperfections and defenses as well as
one’s strengths, Neff (2011) proposed the construct of self-compassion as an
alternative to both the hubristic and fleeting images of self-esteem propa-
gated by American culture and psychology in the interest of self-enhance-
ment—which Maslow would have classified as D-esteem. Rather,
self-compassion emphasizes nonjudgmental, mindful self-awareness as a
means of overcoming self-consciousness and improving self-efficacy and
well-being. Maslow (1999) noted that “fear of knowledge of oneself is very
often isomorphic with, and parallel with, fear of . . . any knowledge that could
cause us to despise ourselves or to make us feel inferior, weak, worthless,
evil, shameful” (p. 71). Cultivating self-compassion can result in lower self-
condemnation and higher self-forgiveness (Cornish & Wade, 2015) as well as
in decreased maladaptive dependency and increased sense of connectedness
(Chui, Zilcha-Mano, Dinger, Barrett, & Barber, 2016).

Self-Actualization

When conditions are favorable and the intrinsic self is heeded, the possibility
of self-actualizing comes into focus for the developing person. Maslow
(1971) noted that “self-actualization means experiencing fully, vividly, self-
lessly, with full concentration and total absorption” and therefore “being
more easily [oneself]” and “expressing rather than coping,” that is, directing
one’s energies toward the best uses of one’s potentials and abilities and

Bland and DeRobertis 947

feeling discontent and restless when one is not doing what one was uniquely
fitted for (pp. 44, 290). Maslow (1999) identified several interrelated quali-
ties of self-actualizing people:

Clearer, more efficient perception of reality; more openness to experience;
increased integration, wholeness, and unity of the person; increased spontaneity,
expressiveness, full functioning, aliveness; a real self, firm identity, autonomy,
uniqueness; increased objectivity, detachment, transcendence of self; recovery
of creativeness; ability to fuse concreteness and abstractness; democratic
character structure; ability to love, etc. (pp. 172-173; see also Maslow, 1987)

Maslow’s description of self-actualizing people is consistent with R. Walsh’s
(2015) conceptualization of wisdom, which involves the following: (a) peo-
ple’s abilities to “more deeply and accurately . . . see into themselves, reality,
and [their] existential challenges and limitations” and to embrace “ethicality
and benevolence [as] appropriate ways to live”; (b) the motivation to benefit
others; and (c) operating on the awareness that “the deeper the kind of bene-
fits they can offer, . . . the more skillfully they may offer them” (p. 289).

Propriate Striving. As noted earlier, self-actualizing is an outcome of healthy
personality development, which entails an ongoing process of striving for
still greater improvement and growth as opposed to an end state, as synony-
mous with Allport’s (1955) propriate striving. Self-actualizing people assume
the courage and freedom to create/recreate aspects of their personality based
on new life experiences and interactions with others—especially those that
test their ordinary ways of thinking, being, and relating and which liberate
and integrate their intellect, emotions, and body—rather than remain homeo-
statically fixated in their comfort zones. This paves the way for self-transcen-
dence (Maslow, 1971). Likewise, McAdams (2015) proposed that personality
development involves a tripartite emerging process of social actor (disposi-
tional traits, temperament), motivated agent (personal goals, projects, plans,
values), and autobiographical author (narrative identity).

Social Interest. Self-actualizing involves a greater sense of identification with
humanity and therefore compassion and altruism, devoting one’s “energies
and thoughts to socially meaningful interests and problems” beyond one’s
own self-interest and/or need gratification (Maslow, 1999, p. 22). Because
healthier people “need less to receive love [and] are more capable to give
love, [they] are more loving people” (Maslow, 1999, p. 47). Therefore, they
demonstrate increased comfort being alone and enhanced self-discipline ver-
sus gregariously exuberant disposition (Maslow, 1987). At the same time,

948 Journal of Humanistic Psychology 60(6)

they are more democratic, interdependent, and problem focused; have better
interpersonal relations; are more accepting and forgiving of others; and are
able to extend these capacities to a variety of relationships (Maslow, 1999;
Maslow & Mittelmann, 1951). Toumbourou (2016) outlined a framework for
identifying and evaluating beneficial action (i.e., altruistic and prosocial
behavior) based on developmental and contextual influences that resemble
Maslow’s needs theory.

Resilience. Maslow (1999) wrote, “Self-actualization does not mean a tran-
scendence of all human problems. Conflict, anxiety, frustration, sadness,
hurt, and guilt all can be found in healthy human beings”; on the other hand,
“with increasing maturity,” one’s focus shifts “from neurotic pseudo-prob-
lems to the real, unavoidable existential problems” (p. 230). Maslow (Maslow
& Mittelmann, 1951) emphasized the abilities to constructively adapt to cir-
cumstances beyond one’s control, to sustainably and nondefensively remain
collected in the face of crisis, and to withstand setbacks as opportunities for
growth (instead of as threatening). “The child with a good basis of safety,
love, and respect-need-gratification is able to profit from . . . frustrations and
become stronger thereby” (Maslow, 1999, p. 220). Maslow (1996) also
accentuated that tragedy is conducive to growth insofar as it “confronts [indi-
viduals] with the ultimate values, questions, and problems that [they] ordi-
narily forget about in everyday existence” (p. 56). Likewise, F. Walsh (2016)
defined resilience as follows:

“Struggling well,” experiencing both suffering and courage, effectively
working through difficulties both internally and interpersonally, . . . [striving]
to integrate the fullness of the experience of . . . life challenges into the fabric
of [one’s] individual and collective identity, influencing how we go on with our
lives. (p. 5)

Aldwin (2007) cited cognitive skills (insight, creativity, humor, morality),
temperament (independence and initiative), and social integration (all remi-
niscent of self-actualizing people) as factors that characterize resilient chil-
dren irrespective of social class or ethnicity. Furthermore, Masten (2014)
identified attributes and outcomes of a supportive, accepting, and enriching
but also appropriately challenging family, school, and community environ-
ment (i.e., Maslow’s safety, belonging, and esteem needs) as protective fac-
tors that promote resilience.

Postconventional Morality. In self-actualizing people, locus of control shifts from
externalized to intrinsic, and both motivation and ethics follow suit. They are

Bland and DeRobertis 949

“not only or merely [their institutional and/or national affiliation] but also mem-
bers at large of the human species” and “[looking] within for the guiding values
and rules to live by” (Maslow, 1999, p. 201). Being strongly focused on prob-
lems outside themselves, their focus broadens to include matters reflecting a
desire for truth, justice, beauty, and so on. In addition, being comfortable in their
skin, they are inclined to do what is right versus what is easy even if it goes
against the tide (i.e., resistance to enculturation and transcendence of one’s envi-
ronment). Using Kohlberg’s (1984) model of moral development, postconven-
tional morality is characterized first by right action based on compromise and
reciprocity. The letter of the law is considered insufficient to uphold a society;
rather, rules are broken and/or revised when one is faced with situations in which
the rules interfere with human rights or needs. Second, right/wrong is based on
universal ethical principles of fairness and equality, and individuals turn to their
inner conscience with respect for diversity, dignity, and human welfare and for
balancing individual and social concerns. Similarly, Gilligan (1982) proposed a
parallel concept, a morality of nonviolence (i.e., preventing harm to self and oth-
ers), as the telos of her feminist moral development model.

Postformal Cognition and Psychological Flexibility. Maslow (1999) emphasized that
cognition associated with self-actualizing people is marked by “[sharpened]
awareness of the limitations of purely abstract thinking, of verbal thinking, and
of analytic thinking” and by “dichotomies [becoming] resolved, opposites . . .
seen to be unities, and the whole dichotomous way of thinking . . . recognized
to be immature” (pp. 227-228). Post-Piagetian psychologists (e.g., Basseches,
Kitchener, Labouvie-Vief, Perry, Sinnot, etc., as cited in Arnett, 2016; Broder-
ick & Blewitt, 2015) emphasized that, when conditions are favorable, the for-
mal operational thought of adolescence gives way to more flexible, complex,
and integrated postformal cognition characterized by pragmatism (adapting
idealistic, logical thinking to the practical constraints of real-life situations),
dialectical thought (awareness that problems often have no clear solution), and
reflective judgment, relativism, and postskeptical rationalism. Decisions are
based on situational circumstances, and emotion is integrated with logic to
form context-dependent principles. Accordingly, the legitimacy of competing
points of view and of psychological flexibility (Wilson, Bordieri, & Whiteman,
2012) is recognized and favored over making arguments for the justification of
only one true/accurate perspective at the exclusion of others (Schneider, 2013).
Maslow (1966) discussed how these principles could be applied to develop a
more humanistic approach to science.

Emotional Intelligence. Maslow (1999) observed that “the ability to be aggres-
sive and angry is found in all self-actualizing people, who are able to let it

950 Journal of Humanistic Psychology 60(6)

flow forth freely when the external situation ‘calls for’ it” (p. 216). They are
accepting of the full range of human impulses without rejecting them in the
interest of reducing tension. Like postformal cognition, Goleman’s (1995;
see also Dalai Lama, 2012) emotional intelligence theory emphasizes mov-
ing away from Western dualistic assumptions about emotions as inherently
positive (approach, pleasant) or negative (withdrawal, unpleasant) and
instead recognizing that each emotion has both beneficial (constructive) and
afflictive (destructive) elements. For example, fear can signal legitimate
threats and promote survival, and righteous anger is necessary and appropri-
ate for confronting injustice. The ability to accept emotions as they are rather
than deny, repress, or project them also promotes empathy and compassion
(toward both self and others), consistent with Maslow’s simultaneous focus
on propriate striving and social interest.

Creativity. Maslow (1999) recognized creativity as the dialectic integration of
primary (childlike, Dionysian) and secondary (rational, Apollonian) pro-
cesses, a conceptualization that was elaborated by Arieti (1976) and explored
in a qualitative inquiry by Bland (2003). Specifically, Maslow focused on the
nonduality between young and old (i.e., a sense of playfulness and the ability
to integrate imagination with practical wisdom). In addition, he emphasized
that creativity (a) is not limited to production of products (i.e., art, music,
literature, scientific work) but also includes the propriate process of individu-
als’ growth and development and (b) serves to benefit society by providing
alternatives to the limitations of convention. Sternberg (2016) proposed a
triangular theory of creativity that involves defying the crowd (i.e., the beliefs,
values, and practices of one’s field despite the short-term interpersonal risks),
defying oneself (self-challenging and self-transcending by moving beyond
one’s own earlier values, practices, and beliefs), and defying the zeitgeist (i.e.,
the unconsciously accepted presuppositions and paradigms in a field). In
addition, consistent with Maslow’s suggestions for social conditions that are
conducive to self-actualization (i.e., a consistent and nurturing environment
that enables one to express oneself rather than cope and conform), Ren, Li,
and Zhang (2017) noted that while Chinese adolescents’ creativity is enhanced
by behavioral control from their parents, it is stifled by parents’ psychologi-
cal control over them.

Dialogue With the Extant Literature

Maslow’s work has been met with ongoing criticism and confusion since he
initially introduced his ideas at mid-20th century. His association with the
worst of 1960s counterculture (about which he publicly expressed

Bland and DeRobertis 951

frustration; see Maslow, 1964/1970; 1984; 1987) arguably contributed to his
work being dismissed (or at best ignored) today by many conventional psy-
chologists as a historical relic. In addition, since Maslow’s death in 1970, the
more complex and nuanced aspects of his thinking have become distorted or
lost due to oversimplified and/or inaccurate portrayals of his work in second-
ary sources that resemble an academic game of “telephone” (Bland &
DeRobertis, 2017).

Applied to this article, perhaps the most troubling misrepresentation of
Maslow’s work has been the attempt by developmentally oriented research-
ers to reformulate his dynamic systems approach as a discontinuous stage
model with clearly defined categorical phases. For example, some have
attempted to equate each level of his needs hierarchy with specific stages in
extant models (e.g., Bauer, Schwab, & McAdams, 2011; D’Souza & Gurin,
2016; Harrigan & Commons, 2015), and others with factors on assessment
measures (e.g., Reiss & Haverkamp, 2005). We find these efforts problem-
atic, as they fail to uphold Maslow’s emphasis on holistic conceptualization
and his cautioning against misunderstanding fulfillment of the basic needs as
a simplistic, lockstep progression (“not a sudden, saltatory phenomenon,”
Maslow, 1987, p. 27) but rather as a dynamic process in which fulfillment of
the higher needs is proportional to fulfillment of the lower needs. Accordingly,
we agree with Rowan’s (1998) call to “[do] away with the triangle!” (p. 88).
First, Maslow never actually represented his theory with a pyramid (Eaton,
2012)—at least in the way that it is commonly presented in textbooks (see
Bland, 2013). More important, while such a visual image is convenient for
instructional purposes, it implies that maturation has an end point, which
belies Maslow’s foci on propriate striving and on self-transcendence (Rowan,
1998). As an alternative, we propose the aforementioned image of Russian
nesting dolls, an expanding spiral or helix, or a lightning bolt, all of which
better convey the two-steps-forward-one-step-back, contiguous dynamic of
maturation as an ongoing process (see Kegan, 1982).

Another criticism leveled at Maslow (e.g., see Hanley & Abell, 2002) is
his emphasis on hedonistic values and on culture-biased notions of self-
esteem and self-actualization. However, numerous international studies have
directly (e.g., Koydemir et al., 2014; Winston, Maher, & Easvaradoss, 2017)
or indirectly (see citations throughout the previous section of this article)
demonstrated the cross-cultural validity of Maslow’s theorizing.

Furthermore, others have (a) made pleas for a more dynamic interactional
self as an alternative to Maslow’s proposition of an instinctoid self in his
adaptation of Goldstein (Frick, 1982; Morley, 1995) and (b) accused Maslow
of “[emphasizing] the importance of maintaining a unified, coherent self,”
whereas “the self-concept differentiates with maturity, [incorporating] both

952 Journal of Humanistic Psychology 60(6)

the private and the more public sides of our nature, accommodating our abil-
ity to keep our own counsel and still be known to others by virtue of our
interactions with them” (Broderick & Blewitt, 2015, pp. 169-170). These cri-
tiques tend to focus on the individuating aspects of self-actualization (i.e.,
Goldstein’s first and third axioms) without adequately acknowledging
Maslow’s emphasis on maturity within a social–environmental context
(Goldstein’s second axiom), which has been more properly acknowledged by
Sassoon (2015). Maslow (1987, 1999) accentuated that the difference
between merely healthy individuals and self-actualizing ones who genuinely
embody social interest is mediated in part by adequate cultural–societal con-
ditions. Likewise, he insinuated that, paradoxically, individuals are simulta-
neously both more externalized and ego-centered at the lower end of his
needs hierarchy, whereas at the higher end they are guided by more idiosyn-
cratic/intrinsic aims while also becoming more self-transcendent.

Conclusion

In this article, we have employed Maslow’s needs hierarchy as a dynamic
systems process framework for situating parallel developmental constructs
that serve as empirical support for his ideas at multiple ages and in various
contexts, and we have sought to clarify common misgivings about his ideas on
psychological health (i.e., self-actualization) and the factors that promote or
inhibit it. Our intent has been to legitimize Maslow’s unacknowledged contri-
butions to developmental psychology in an effort to overcome the “recurrent
Maslow bashing that one finds in the literature” (Winston et al., 2017, p. 309).
We further reach the conclusion that Maslow ought to be counted as a forerun-
ner of contemporary existential–humanistic developmental thought (see
DeRobertis, 2008, 2012, 2015; DeRobertis & McIntyre, 2016).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publi-
cation of this article.

References

Adler, A. (1998). What life could mean to you. Center City, MN: Hazelden. (Original
work published 1931)

Bland and DeRobertis 953

Adler, A. (2010). Understanding human nature. Eastford, CT: Martino. (Original
work published 1927)

Aldwin, C. M. (2007). Stress, coping, and development: An integrative perspective
(2nd ed.). New York, NY: Guilford.

Allport, G. W. (1955). Becoming: Basic considerations for a psychology of personal-
ity. New Haven, CT: Yale University Press.

Arieti, S. (1976). Creativity: The magic synthesis. New York, NY: Basic.
Arnett, J. J. (2016). Human development: A cultural approach (2nd ed.). Boston, MA:

Pearson.
Bauer, J. J., Schwab, J. R., & McAdams, D. P. (2011). Self-actualizing: Where ego

development finally feels good? The Humanistic Psychologist, 39, 121-136. doi:
10.1080/08873267.2011.564978

Bergen, D. (2008). Human development: Traditional and contemporary theories.
Upper Saddle River, NJ: Prentice Hall.

Bland, A. (2003). The tree of life: A theory of musical creativity and its role in per-
sonal and social development (Unpublished master’s thesis). University of West
Georgia, Carrollton.

Bland, A. (2013, April-May). Toward a definition of psychological health:
Appreciating Maslow’s conceptual vision. AHP Perspective, 6-11.

Bland, A. M. (2014). Corrective experiences in corrections counseling. Journal of
Theoretical & Philosophical Criminology, 6, 46-74.

Bland, A. M., & DeRobertis, E. M. (2017). The humanistic perspective. In V. Zeigler-
Hill & T. K. Shackelford (Eds.), Encyclopedia of personality and individual dif-
ferences. doi:10.1007/978-3-319-28099-8_1484-1

Bland, A. M., & Roberts-Pittman, B. J. (2014). Existential and chaos theory: “Calling”
for adaptability in career decision-making. Journal of Career Development, 41,
382-401. doi:10.1177/0894845313498303

Bowen, M. (1978). Family therapy in clinical practice. New York, NY: Aronson.
Broderick, P. C., & Blewitt, P. (2015). The life span: Human development for helping

professionals (4th ed.). Boston, MA: Pearson.
Bronfenbrenner, U. (1994). Ecological models of human development. In T. Husen &

T. N. Postlethwaite (Eds.), International encyclopedia of education (2nd ed., Vol.
3, pp. 1643-1647). Oxford, England: Pergamon.

Campa, M. I. (2013). Developmental trends and bonding milestones: From parents
to partners. In C. Hazan & M. I. Campa (Eds.), Human bonding: The science of
affectional ties (pp. 74-100). New York, NY: Guilford.

Capuzzi, D, & Stauffer, D. (Eds.). (2016). Human growth and development across the
lifespan: Applications for counselors. Hoboken, NJ: Wiley.

Castonguay, L. G, & Hill, C. E. (Eds.). (2012). Transformation in psychotherapy:
Corrective experiences across cognitive-behavioral, humanistic, and psychody-
namic approaches. Washington, DC: American Psychological Association.

Chui, H., Zilcha-Mano, S., Dinger, U., Barrett, M. S., & Barber, J. P. (2016).
Dependency and self-criticism in treatments for depression. Journal of Counseling
Psychology, 63, 452-459. doi:10.1037/cou0000142

954 Journal of Humanistic Psychology 60(6)

Coie, J. D., & Dodge, K. A. (1988). Multiple sources of data on social behavior and
social status in the school: A cross-age comparison. Child Development, 59, 815-
829. doi:10.2307/1130578

Cordeiro, P. M. G., Paixão, M. P., Lens, W., Lacante, M., & Luyckx, K. (2016).
Parenting styles, identity development, and adjustment in career transitions:
The mediating role of psychological needs. Journal of Career Development, 45,
83-97. doi:10.1177/0894845316672742

Cornish, M. A., & Wade, N. G. (2015). Working through past wrongdoing:
Examination of a self-forgiveness counseling intervention. Journal of Counseling
Psychology, 62, 521-528. doi:10.1037/cou0000080

Dalai Lama. (2012). Beyond religion: Ethics for a whole world. New York, NY:
Houghton Mifflin.

DeRobertis, E. M. (2008). Humanizing child developmental theory: A holistic
approach. Bloomington, IN: iUniverse.

DeRobertis, E. M. (2011a). Deriving a third force approach to child development
from the works of Alfred Adler. Journal of Humanistic Psychology, 51, 492-515.
doi:10.1177/0022167810386960

DeRobertis, E. M. (2011b). Existential-humanistic and dynamic systems approaches
to child development in mutual encounter. The Humanistic Psychologist, 39, 3-
23. doi:10.1080/08873267.2011.539934

DeRobertis, E. M. (2012). The whole child: Selected papers on existential-humanistic
child psychology. Charleston, SC: CreateSpace.

DeRobertis, E. M. (2015). Toward a humanistic-multicultural model of develop-
ment. In K. J. Schneider, J. F. Pierson, & J. F. T. Bugental, (Eds.), Handbook of
humanistic psychology: Theory, research, and practice (2nd ed., pp. 227-242).
Thousand Oaks, CA: Sage.

DeRobertis, E. M. (2016). Becoming enthusiastic about learning for the first time
as a child. Journal of Humanistic Psychology, 56, 394-413. doi:10.1177/
0022167815574430

DeRobertis, E. M., & McIntyre, S. (2016). Development through a humanistic lens.
In R. Bargdill & R. Broomé (Eds.), Humanistic contributions for Psychology
101: Growth, choice, and responsibility (pp. 117-132). Colorado Springs, CO:
University Professors Press.

Desmond, M. (2016). Evicted: Poverty and profit in the American city. New York,
NY: Crown.

D’Souza, J., & Gurin, M. (2016). The universal significance of Maslow’s concept
of self-actualization. The Humanistic Psychologist, 44, 210-214. doi:10.1037/
hum0000027

Eaton, S. E. (2012). Maslow’s hierarchy of needs: Is the pyramid a hoax? Retrieved from
https://drsaraheaton.wordpress.com/2012/08/04/maslows-hierarchy-of-needs/

Erikson, E. H. (1994). Identity and the life cycle. New York, NY: Norton. (Original
work published 1959)

Firestone, R. W., Firestone, L., & Catlett, J. (2013). The self under siege: A therapeu-
tic model for differentiation. New York, NY: Routledge.

https://drsaraheaton.wordpress.com/2012/08/04/maslows-hierarchy-of-needs/

Bland and DeRobertis 955

Frankl, V. E. (1978). The unheard cry for meaning: Psychotherapy and humanism.
New York, NY: Washington Square.

Frick, W. (1971). Humanistic psychology: Interviews with Maslow, Murphy, and
Rogers. Columbus, OH: Merill.

Frick, W. (1982). Conceptual foundations of self-actualization: A contribution to
motivation theory. Journal of Humanistic Psychology, 22, 33-52.

Gelo, O. C. G., & Salvatore, S. (2016). A dynamic systems approach to psychotherapy:
A meta-theoretical framework for explaining psychotherapy change processes.
Journal of Counseling Psychology, 63, 379-395. doi:10.1037/cou0000150

Gilligan, C. (1982). In a different voice: Psychological theory and women’s develop-
ment. Cambridge, MA: Harvard University Press.

Goldstein, K. (1963). Human nature in the light of psychopathology. New York, NY:
Shocken.

Goldstein, K. (1995). The organism: A holistic approach to biology derived from
pathological data in man. New York, NY: Zone. (Original work published 1934)

Goleman, D. (1995). Emotional intelligence. New York, NY: Bantam.
Gordon, T. (1975). Parent effectiveness training. New York, NY: Plume.
Graber, R., Turner, R., & Madill, A. (2015). Best friends and better coping: Facilitating

psychological resilience through boys’ and girls’ closest friendships. British
Journal of Psychology, 107, 338-358. doi:10.1111/bjop.12135

Hanley, S. J., & Abell, S. C. (2002). Maslow and relatedness: Creating an interper-
sonal model of self-actualization. Journal of Humanistic Psychology, 42, 37-57.
doi:10.1177/002216702237123

Harrigan, W. J., & Commons, M. L. (2015). Replacing Maslow’s needs hierarchy
with an account based on stage and value. Behavioral Development Bulletin, 20,
24-31. doi:10.1037/h0101036

Harris, R. (2006). Embracing your demons: An overview of acceptance and commit-
ment therapy. Psychotherapy in Australia, 12, 70-76.

Hoffman, E. (1988). The right to be human: A biography of Abraham Maslow. Los
Angeles, CA: Tarcher.

Horney, K. (1945). Our inner conflicts. New York, NY: Norton.
Kegan, R. (1982). The evolving self: Problem and process in human development.

Cambridge, MA: Harvard University Press.
Kohlberg, L. (1984). The psychology of moral development: The nature and validity

of moral stages (Essays on moral development, Vol. 2). New York, NY: Harper
& Row.

Koydemir, S., Şimşek, Ő. F., & Demir, M. (2014). Pathways from personality to hap-
piness: Sense of uniqueness as a mediator. Journal of Humanistic Psychology,
54, 314-335. doi:10.1177/0022167813501226

Maccoby, E. E., & Martin, J. A. (1983). Socialization in the context of the family:
Parent-child interaction. In E. M. Hetherington (Ed.), Handbook of child psy-
chology, Vol. IV: Socialization, personality, and social development (4th ed., pp.
1-102). New York, NY: Wiley.

956 Journal of Humanistic Psychology 60(6)

Marcia, J. E. (1966). Development and validation of ego identity status. Journal of
Personality and Social Psychology, 3, 551-558. doi:10.1037/h0023281

Maslow, A. H. (1966). The psychology of science: A reconnaissance. New York, NY:
Gateway.

Maslow, A. H. (1970). Religions, values, peak experiences. New York, NY: Penguin.
(Original work published 1964)

Maslow, A. H. (1971). The farther reaches of human nature. New York, NY: Penguin.
Maslow, A. H. (1973). Dominance, self-esteem, self-actualization: Germinal papers

of A. H. Maslow. Monterey, CA: Brooks/Cole.
Maslow, A. H. (1984). Politics 3. In T. Greening (Ed.), American politics and human-

istic psychology (pp. 80-96). Dallas, TX: Saybrook Institute Press.
Maslow, A. H. (1987). Motivation and personality (3rd ed.). New York, NY:

HarperCollins.
Maslow, A. H. (1996). Future visions: The unpublished papers of Abraham Maslow.

Thousand Oaks, CA: Sage.
Maslow, A. H. (1999). Toward a psychology of being (3rd ed.). New York, NY: Wiley.
Maslow, A. H., & Mittelmann, B. (1951). Principles of abnormal psychology: The

dynamics of psychic illness (2nd ed.). New York, NY: Cotler.
Masten, A. S. (2014). Ordinary magic: Resilience in development. New York, NY:

Guilford.
Maté, G. (2010). In the realm of hungry ghosts: Close encounters with addiction.

Berkeley, CA: North Atlantic.
Maté, G. (2011). Dr. Gabor Maté on attachment and conscious parenting. Retrieved

from https://www.youtube.com/watch?v=_tdljIW86e8
May, R. (1967). Psychology and the human dilemma. Princeton, NJ: Von

Nostrand.
McAdams, D. P. (2015). The art and science of personality development. New York,

NY: Guilford.
Miller, P. M., & Commons, M. L. (2010). The benefits of attachment parenting for

infants and children: A behavioral developmental view. Behavioral Development
Bulletin, 16, 1-14. doi:10.1037/h0100514

Morley, J. (1995). Holistic biology and the organismic foundations of humanistic
psychology. The Humanistic Psychologist, 23, 358-364. doi:10.1080/08873267
.1995.9986836

Music, G. (2017). Nurturing natures: Attachment and children’s emotional, sociocul-
tural, and brain development. New York, NY: Routledge.

Neff, K. (2011). Self-compassion. New York, NY: Morrow.
Ollendick, T. H., Weist, M. D., Borden, M. C., & Greene, R. W. (1992). Sociometric

status and academic, behavioral, and psychological adjustment: A five-year
longitudinal study. Journal of Consulting and Clinical Psychology, 60, 80-87.
doi:10.1037/0022-006x.60.1.80

Prince, D. L., & Howard, E. H. (2002). Children and their basic needs. Early
Childhood Education Journal, 30, 27-31. doi:10.1023/a:1016589814683

Bland and DeRobertis 957

Prinstein, M. (2017). Popular: The power of likability in a status-obsessed world.
New York, NY: Viking.

Reiss, S., & Haverkamp, S. M. (2005). Motivation in developmental context: A new
method for studying self-actualization. Journal of Humanistic Psychology, 45,
41-53. doi:10.1177/0022167804269133

Ren, F., Li, Y., & Zhang, J. (2017). Perceived parental control and Chinese mid-
dle school adolescents’ creativity: The mediating role of autonomous motiva-
tion. Psychology of Aesthetics, Creativity, and the Arts, 11, 34-42. doi:10.1037/
aca0000078

Rowan, J. (1998). Maslow amended. Journal of Humanistic Psychology, 38, 81-92.
doi:10.1177/00221678980381008

Rowan, J. (1999). Ascent and descent in Maslow’s theory. Journal of Humanistic
Psychology, 39, 125-133. doi:10.1177/0022167899393010

Sassoon, J. (2015). The humanist society: The social blueprint for self-actualization.
Bloomington, IN: iUniverse.

Schneider, K. J. (2013). The polarized mind: Why it’s killing us and what we can do
about it. Colorado Springs, CO: University Professors Press.

Shapiro, S., & White, C. (2014). Mindful discipline: A loving approach to set-
ting limits and raising an emotionally intelligent child. Oakland, CA: New
Harbinger.

Siegel, D. J. (2012). The developing mind: How relationships and the brain interact
to shape who we are (2nd ed.). New York, NY: Guilford.

Skalski, J. E., & Hardy, S. A. (2013). Disintegration, new consciousness, and dis-
continuous transformation: A qualitative investigation of quantum change. The
Humanistic Psychologist, 41, 159-177. doi:10.1080/08873267.2012.724271

Sternberg, R. J. (2016). A triangular theory of creativity. Psychology of Aesthetics,
Creativity, and the Arts, 12, 50. doi:10.1037/aca0000095

Toumbourou, J. W. (2016). Beneficial action within altruistic and prosocial behavior.
Review of General Psychology, 20, 245-258. doi:10.1037/gpr0000081

Walsh, F. (2016). Strengthening family resilience (3rd ed.). New York, NY: Guilford.
Walsh, R. (2015). What is wisdom? Cross-cultural and cross-disciplinary syntheses.

Review of General Psychology, 19, 278-293. doi:10.1037/gpr0000045
Wertz, F. J., Charmaz, K., McMullen, L. M., Josselson, R., Anderson, R., &

McSpadden, E. (2011). Five ways of doing qualitative analysis. New York, NY:
Guilford.

Whitehead, P. M. (2017). Goldstein’s self-actualization: A biosemiotic view. The
Humanistic Psychologist, 45, 71-83. doi:10.1037/hum0000047

Wilson, K. G., Bordieri, M., & Whiteman, K. (2012). The self and mindfulness. In
L. McHugh & I. Stewart (Eds.), The self and perspective taking: Contributions
and applications from modern behavioral science (pp. 181-197). Oakland, CA:
New Harbinger.

Winston, C. N., Maher, H., & Easvaradoss, V. (2017). Needs and values: An explora-
tion. The Humanistic Psychologist, 45, 295-311. doi:10.1037/hum0000054

958 Journal of Humanistic Psychology 60(6)

Author Biographies

Andrew M. Bland is a member of the graduate clinical psy-
chology faculty at Millersville University in Lancaster County,
PA. He received a master’s degree from the University of West
Georgia’s humanistic-existential-transpersonal psychology pro-
gram in 2003 and a PhD in counseling psychology from Indiana
State University in 2013. He is a licensed psychologist; since
2004, he has provided therapeutic services in a variety of set-
tings in four states, currently at Samaritan Counseling Center in
Lancaster, PA. His research interests include the practical appli-
cation of themes from contemporary existential-humanistic

psychology in the domains of love, work, social justice, the processes of therapy and
education, creativity, spirituality, and human development. His passions include lis-
tening to and creating music, gardening, traveling, and spending time with his wife
and their two young children.

Eugene M. DeRobertis is a professor of psychology at
Brookdale College and a Lecturer at Rutgers University–
Newark in New Jersey. He holds a BA in philosophy from St.
Peter’s University and a PhD in psychology from Duquesne
University. He is the author of Humanizing Child Developmental
Theory: A Holistic Approach (2008), The Whole Child: Selected
Papers on Existential-Humanistic Child Psychology (2012),
Existential-Phenomenological Psychology: A Brief Introduction
(2012), Profiles of Personality: An Approach-Based Companion
(2013), and The Phenomenology of Learning and Becoming:
Enthusiasm, Creativity, and Self-Development (2017).

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 1

REVIEW
published: 19 October 2018

doi: 10.3389/fpsyg.2018.01992

Edited by:
Jill Popp,

The LEGO Foundation, Denmark

Reviewed by:
Nicolas Cuperlier,

Université de Cergy-Pontoise, France
Gautier Durantin,

The University of Queensland,
Australia

Eiji Uchibe,
Advanced Telecommunications

Research Institute International (ATR),
Japan

*Correspondence:
Hélène Cochet

helene.cochet@univ-tlse2.fr

Specialty section:
This article was submitted to
Developmental Psychology,

a section of the journal
Frontiers in Psychology

Received: 13 October 2017
Accepted: 28 September 2018

Published: 19 October 2018

Citation:
Cochet H and Guidetti M (2018)

Contribution of Developmental
Psychology to the Study of Social

Interactions: Some Factors in Play,
Joint Attention and Joint Action

and Implications for Robotics.
Front. Psychol. 9:1992.

doi: 10.3389/fpsyg.2018.01992

Contribution of Developmental
Psychology to the Study of Social
Interactions: Some Factors in Play,
Joint Attention and Joint Action and
Implications for Robotics
Hélène Cochet* and Michèle Guidetti

CLLE, Université de Toulouse, CNRS, UT2J, Toulouse, France

Children exchange information through multiple modalities, including verbal
communication, gestures and social gaze and they gradually learn to plan their
behavior and coordinate successfully with their partners. The development of joint
attention and joint action, especially in the context of social play, provides rich
opportunities for describing the characteristics of interactions that can lead to shared
outcomes. In the present work, we argue that human–robot interactions (HRI) can
benefit from these developmental studies, through influencing the human’s perception
and interpretation of the robot’s behavior. We thus endeavor to describe some
components that could be implemented in the robot to strengthen the feeling of dealing
with a social agent, and therefore improve the success of collaborative tasks. Focusing
in particular on motor precision, coordination, and anticipatory planning, we discuss the
question of complexity in HRI. In the context of joint activities, we highlight the necessity
of (1) considering multiple speech acts involving multimodal communication (both
verbal and non-verbal signals), and (2) analyzing separately the forms and functions of
communication. Finally, we examine some challenges related to robot competencies,
such as the issue of language and symbol grounding, which might be tackled by
bringing together expertise of researchers in developmental psychology and robotics.

Keywords: human–robot interaction, human development, joint attention, joint action, coordination, complexity,
gestures

INTRODUCTION

Developmental psychologists aim at describing and explaining changes across the life span in a
wide range of areas such as social, emotional, and cognitive abilities. Focusing on childhood is
a way of grasping numerous changes, especially in terms of communication: infants gradually
learn to identify the common ground they have with others and engage in social interactions. The
development of such abilities relies on the personal experiences shared between partners in specific
contexts (Liebal et al., 2013), among which social play may offer particularly rich opportunities
for children to acquire joint action and joint attention skills. Studying the different forms and
functions of communication in this context paves the way for identifying the necessary ingredients

Frontiers in Psychology | www.frontiersin.org 1 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/journals/psychology#editorial-board

https://doi.org/10.3389/fpsyg.2018.01992

http://creativecommons.org/licenses/by/4.0/

https://doi.org/10.3389/fpsyg.2018.01992

http://crossmark.crossref.org/dialog/?doi=10.3389/fpsyg.2018.01992&domain=pdf&date_stamp=2018-10-19

https://www.frontiersin.org/articles/10.3389/fpsyg.2018.01992/full

http://loop.frontiersin.org/people/214150/overview

http://loop.frontiersin.org/people/313268/overview

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 2

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

for effective joint activities and therefore better understanding
the architecture of human–social interactions. Even though
the concept of effectiveness may cover different theoretical
frameworks, the latter objectives have several applications, for
example in supporting children with atypical development,
especially when they have difficulty communicating both verbally
and non-verbally (e.g., children with autism spectrum disorders,
ASD), but also in the field of artificial intelligence. The role of
robots in society raises indeed a lot of debates and challenges,
as they share more and more space and tasks with humans, for
instance in service robotics to assist elderly people. The robots’
ability to initiate and respond to social interactions is one of the
key factors that will shape their integration in our everyday life
in the future. Researchers in social robotics have been working
on the question of joint action for over two decades now,
sometimes in collaboration with developmental psychologists
(e.g., Scassellati, 2000), in order to improve robots’ motor
and communicative skills. Developmental models of human
communicative behavior can indeed help define the components
to implement in human–robot interactions (HRI), so as to build
rich and natural joint activities (Breazeal et al., 2004; Lemaignan
et al., 2017).

The objective of this paper is twofold. First, we intend to
present the point of view and some research perspectives of
developmental psychologists on joint attention and joint action,
in particular in the context of social play. To this end, we
will also define, starting from studies on non-human primates,
what can be regarded as complex (or rich) and natural (or
effective) interactions in both human communication and HRI.
Second, we aim to show the extent to which the above-mentioned
issues may be of interest to roboticists, in helping conceptualize
and implement some variables associated with joint attention
and joint action in the context of HRI. Collaborative tasks
involving robot and human partners, regarded as tantamount to
children’s social play, will thus be considered through the prism
of pragmatic communication, allowing researchers to dissociate
the forms and the functions of communication.

HOW DOES COMMUNICATION
DEVELOP IN THE CONTEXT OF SOCIAL
PLAY?

The definitions of play include a wide range of activities,
which makes it difficult to determine where play begins and
where it ends, even though it is traditionally associated with
positive affective valence (Garvey, 1990). Play, which occurs in
several animal species (most notably in mammals), has been
argued to allow “practice of real-world skills in a relatively safe
environment” (Byrne, 2015). We will focus here on social play
in human children, which may also enable them, as highlighted
by Bruner (1973), to “learn by doing” as they interact with
one or several partners. At the individual level, children can
indeed explore and enhance specific skills like motor control and
creativity, while developing for example cooperation abilities at
the social level. The concepts of artifact-mediated and object-
oriented action, originally formulated by Vygotsky (1999), are

particularly relevant to describe these situations: the relationship
between the child and the surrounding objects is indeed mediated
by cultural means, tools, and signs. Studying the development
of play can therefore reveal how children come to represent and
think about their environment.

Social attention is a crucial capacity for the emergence of
these play situations, allowing children to focus on some of
the other’s characteristics such as the facial expressions, gaze
direction, gestures, and vocalizations. When the direction of
another’s attention has been identified (for example through gaze
following or point following), we can shift our own attention to
focus at the same time on the same external object or event as our
partner. This process of joint attention is usually inferred from
behavioral cues, including mainly gaze alternation between one’s
partner and a specific referent (Bourjade, 2017). Joint attention
seems therefore necessary for individuals to perform joint action,
i.e., to coordinate their actions in space and time to produce a
joint outcome, whether it involves here symbolic play (with or
without objects), construction toys, board games or any other
forms of play.

Joint attention and joint action begin to appear at the end
of the first year in human development (Carpenter et al., 1998),
gradually allowing children to integrate the notion of common
ground and engage in social interactions. The development of
gaze understanding, which has been widely studied, plays a
key role in this regard. It was for example shown in a study
using habituation-of-looking-time procedure that infants start to
understand ecologically valid instances of social gaze between
two adults interacting, and to have expectations concerning gaze
target at 10 months of age (Beier and Spelke, 2012). Besides,
responsive joint attention skills (e.g., gaze following and point
following) have been reported to emerge before initiative joint
attention skills, from 8 months of age (Corkum and Moore, 1998;
Beuker et al., 2013).

However, depending on the authors, the definitions of these
social-cognitive skills can be more or less demanding, the main
difference lying in whether or not individuals have mutual
understanding of their shared focus of attention. The ability to
“know together” that we are attending to the same thing as
our partner has sometimes been referred to as shared attention
(Emery, 2000; Shteynberg, 2015), which would develop in parallel
with shared intentionality (Tomasello and Carpenter, 2007). The
latter involves the motivation to share goals and intentions
with the other, as well as forms of cognitive representation for
doing so. This ability has been argued to constitute a hallmark
of the human species (Tomasello et al., 2005), even though it
is particularly difficult to assess when verbal language is not
available as a clue to these representations (in pre-linguistic
children or non-human primates). Similarly, joint action may
rely solely on the learning of the cues that appear significant
(e.g., gestures and eye contact) to coordinate actions in space and
time with a partner, or it may also involve, in a more demanding
perspective, the common and explicit knowledge of the objectives
of the activity and of the way to achieve them (Tomasello and
Carpenter, 2007).

Joint attention and joint action, whether they are accompanied
or not with shared and explicit intentions, thus allow children

Frontiers in Psychology | www.frontiersin.org 2 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 3

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

to participate with others in collaborative activities in which
each partner benefits from the joint outcome and/or from
the interaction in itself. In a series of experiments, the ability
to coordinate with a partner in social games was shown to
significantly improve between 18 and 24 months of age, whether
the games involved complementary or similar roles (Warneken
et al., 2006). In the first game of this study, one person had to
send a wooden block down one of a tube mounted on a box
on a 20 degrees incline, while the other person had to catch
it at the other end with a tin can that made a rattling sound.
Two tubes were mounted in parallel so that individuals could
perform in turn the different roles. In the second game, two
persons had to make a wooden block jump on a small trampoline
(67 cm diameter ring covered with cloth) by holding the rim
on opposite sides. The trampoline collapsed when being held on
only one side. Children successfully participated in both games,
although the 24 month-olds were more proficient than the 18
month-olds, and they all produced at least one communicative
attempt to reengage the adult partner when the latter stopped
participating in the activity. Children for example pointed at the
object, and/or vocalized while looking at the adult, which was
regarded as evidence for a uniquely human form of cooperation,
involving shared intentionality (Warneken et al., 2006). A less
“mentalistic” interpretation could be proposed (D’Entremont and
Seamans, 2007), but these results nevertheless highlight children’s
motivation for reinstating joint action toward a shared goal. The
development of this capacity has received much attention from
researchers, as the initiation of joint attention appears to be
strongly related to language comprehension and production in
the second and third year of life (Colonnesi et al., 2010; Cochet
and Byrne, 2016), as well as to theory of mind ability (e.g.,
Charman et al., 2000; Milward et al., 2017) in both typical and
atypical development (e.g., Adamson et al., 2017).

In addition, the observation of children’s behavior during
collaborative activities may lead to a thorough description
of multimodal communication (e.g., gaze, facial expressions,
gestures, and verbal language) and of the way its components
become coordinated. For example, the production of gestures
gradually coordinates with gaze in the course of development.
Children start to produce pointing gestures to orient the attention
of another person around 12 months of age; an object, a person
or an event can become the shared focus of attention but then
children do not usually look at their partner while they point
(Franco and Butterworth, 1996). A couple of months later, they
are able to alternate their gaze between their partner and the
object of interest, which represents a key feature of intentional
triadic interactions (Cochet and Vauclair, 2010). At 16 months of
age, gaze toward the adult can precede the production of pointing
(Franco and Butterworth, 1996), suggesting that children may
thus take into account the partner’s attentional state before
initiating communication (Lamaury et al., 2017).

Children also gradually learn to take account of their partner’s
facial expressions to infer their emotional state and adjust their
response accordingly. Infants are sensitive to the characteristics
of faces from very early on; newborns look for example
significantly longer at happy expressions than at fearful ones,
demonstrating some discrimination skills (Farroni et al., 2007).

The still-face paradigm, initially designed by Tronick et al. (1978)
also suggests that infants have expectations about interactional
reciprocity from a few months of age, partly relying on emotional
expression. This sensitivity manifests itself in specific behavioral
and physiological responses (e.g., reduced positive affect and
gazing at the parent, increased negative affect, rise in facial
skin temperature) when the mother puts on a neutral and
unresponsive face, after a period of spontaneous play with his/her
infant (Aureli et al., 2015). The ability to recognize and identify
facial expressions of basic emotions further develops in preschool
children, before they can understand a few months later the
external causes of emotions and then, around 5 years of age, the
role of other’s desires or beliefs in emotional expression (Pons
et al., 2004).

During play interactions, being attentive to the other’s facial
expressions allows each partner to consider the emotional
nature of the signals (e.g., joy, surprise, and frustration) and
to possibly modify his/her own behavior to change or maintain
this emotional state. The development of facial expression
perception thus plays a key role in the emergence of joint actions,
in coordination with other communicative modalities. Facial
expressions are indeed usually synchronized with vocalizations
and/or gestures, and this from infancy.

The vocal and the gestural modalities also become more
and more coordinated as children grow older, which represents
a key feature of human communication as we use gestures
as we speak throughout our life. Communicative gestures are
first complemented by vocalizations, whose prosodic patterns
may already code for semantic and pragmatic functions (Leroy
et al., 2009). In the second year of life, children then produce
their first gesture-word combinations, which have an important
role in the transition to the two-word stage (e.g., Butcher and
Goldin-Meadow, 2000). Pointing and conventional gestures (e.g.,
waving goodbye, gestural agreement, and refusal: Guidetti, 2002,
2005) remain in the child repertory after the two-word stage,
but other forms of gestural-vocal coordination are observed
from 3 years of age with the emergence of co-speech gestures.
Although we are usually not aware of producing or perceiving
them, co-speech gestures can lend rhythm, emphasize speech
and sometimes serve deictic or iconic functions. The deictic
presentation of pointing gesture can for example be combined
with vocal pointing, performed through syntactic or prosodic
means (Lœvenbruck et al., 2008). Such coordination between the
vocal and gestural modalities is omnipresent in adults and play a
crucial role in face-to-face communication for both speaker and
listener (e.g., McNeill, 2000; Kendon, 2004).

Moreover, the characteristics of gaze, gestures, and
vocalizations and their coordination may vary according to
the communicative function of the signal. A gesture can indeed
serve different purposes, starting with the traditional distinction
between imperative and declarative functions (Bates et al., 1975).
Imperative gestures are used to request a specific object or action
from a partner whereas declarative gestures are used to share
interest with the other about some referent or provide him/her
with information that might be useful. Imperative and declarative
pointing, which both represent powerful means of establishing
joint attention, have been extensively studied and compared:

Frontiers in Psychology | www.frontiersin.org 3 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 4

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

hand shape and body posture were shown to differ according
to the communicative function of the pointing gesture (Cochet
et al., 2014), as well as the frequency of gaze alternation between
the partner and the referent and the frequency of vocalizations
(Cochet and Vauclair, 2010). These comparisons (see section
“Pragmatics in HRI: Which Ingredients Are Necessary for
Effective Interactions?” for more detailed results) thus highlight
the strong relationship between the form of the gestures (in the
broad sense, i.e., including visual and vocal behavior in addition
to movement kinematics and hand shapes) and pragmatic
features in children, even semantic ones in adults (Cochet and
Vauclair, 2014).

To sum up, when two children are playing together or when
a child is playing with an adult, they do so in the framework
of joint action; they attend to a common situation and use
multimodal communication to initiate, maintain, or respond to
the interaction. These three different roles in the interaction
can be assessed with the Early Social Communication Scales,
in particular with the French version (Guidetti and Tourrette,
2017). In an evaluation situation, giving the child the opportunity
to initiate the interaction is particularly crucial in atypical
development, for example in children with ASD. The initiation
of shared attention is a key ability in this context as it allows joint
action coordination (Vesper et al., 2016) and has also significant
consequences on the development of cognitive and emotional
processes (Shteynberg, 2015). Whether this coordination relies
on the representation and the understanding of the other’s
intentions or only on behavioral cues is a challenging question,
as we do not have any direct access to the other’s subjectivity. In
the field of HRI, an objective that appears sufficiently ambitious
for now, or at least the one we chose to focus on in the present
review, is to design robots able to identify the observable changes
in the human’s behavior, in order to make the right inferences and
thus the appropriate decisions in the interaction. This appears as
an essential condition for a successful exchange between a robot
and a human, which can depend on the joint outcome (has the
common goal been reached?), but also on the way the interaction
has been perceived by each individual, for example in terms of
coordination between gaze and gesture and fluidity of movement
(Hough and Schlangen, 2016). The richness of communication
here lies indeed in the ability of each partner to integrate multiple
communicative cues in a way that what will seem natural to the
humans, i.e., that will be close to peer interaction in everyday life.

This appears as a complex ability and probably the most
challenging one to replicate in HRI. In pursuit of this objective,
we now need to further describe the concept of appropriateness
and propose a frame to determine the relative importance and
the relative complexity of the different behaviors observed during
joint activities such as social play.

TO WHAT EXTENT CAN INTERACTIONS
BE CHARACTERIZED AS COMPLEX?

Smith (2015) has argued that “development, like evolution and
culture, is a process that creates complexity by accumulating
change.” This perspective applies to the development of

social interactions, from the emergence of joint attention to
coordinated and multimodal communication that enable joint
action. Several attempts have been made in developmental
robotics to explore the cognitive, social, and motivational
dynamics of human interactions (Oudeyer, 2017); algorithmic
and robotic models can then be used to study the developmental
processes involved for instance in imitation (Demiris and
Meltzoff, 2008) or language (Cangelosi et al., 2010). In this
context, roboticists aim at designing systems allowing for self-
organized and “progressive increase in the complexity” of the
robot’s behavior (Oudeyer et al., 2007).

To benefit further from their exchanges, developmentalists
and roboticists may therefore need to frame the study of
HRI by disambiguating the concept of complexity. Because
“complicated systems will be best understood at the lowest
possible level” (Smith, 2015), we aim to differentiate different
levels of complexity depending on the nature of the elements to
take into account for decision making. This analysis will allow us
to go forward in the study of joint attention and joint action and
define what is implied by the qualifying terms “complex” (or rich)
and “appropriate” (or effective) when referring to interactions.

To this end, we used a categorization recently proposed in
research on animal behavior, including human and non-human
primates, to define the concept of complexity (Cochet and Byrne,
2015). Three dimensions have been described: motor precision,
coordination, and anticipatory planning, which can relate to
both individual and social activities. The authors argue that “the
complexity of a given mechanism/behavior can be assessed by
distinguishing which of these three dimensions are involved and
to what degree,” which may “clarify our understanding of animal
behavior and cognition.” Such analysis applied to joint attention
and joint action, although there may be other ways of untangling
the question of complexity, may here allow researchers to dissect
the different factors involved in social interactions for each
dimension, and thus help them assess the “manipulability” of
these factors in HRI.

In order to make appropriate decisions in a collaborative
task, i.e., decisions leading to the desired joint outcome and/or
decisions that approach the characteristics of human interactions,
the robot first needs to recognize specific patterns in his/her
partners’ behavior, without asking for agreement or information
for all actions. The robot can for example rely on gaze
direction, manual movements or body posture to identify the
human’s attentional and intentional states and thus define the
most useful role it can play in the interaction. By way of
illustration, if a human and a robot share the common goal
of building a pile with four cubes in a definite order and
putting a triangle at the top, each of them can perform different
actions: they can grasp an object (a cube or a triangle) on
the table, grasp an object on the pile, give an object to the
partner, support the pile while the partner places a cube on
it, etc. Other actions can emerge, for example if the pile
collapses or if one agent does not pile the cubes in the correct
order (Clodic et al., 2014). Individuals can then blame each
other, or give each other some instructions. In addition to
the perception of its own environment, the robot thus has to
observe the activity of the human and take his/her perspective

Frontiers in Psychology | www.frontiersin.org 4 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 5

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

(e.g., to determine whether an object is reachable for the
other).

Motor precision is therefore necessary in this context to
obtain flexible and human-aware shared plan execution (Devin
and Alami, 2016), as it enables a selective shift of attention
toward aspects of the environment that will become shared
knowledge, which has also been described as the accuracy of
shared attention states (Shteynberg, 2015). First, the emergence
of joint attention requires to properly use gaze and/or pointing
gesture to localize the object or event referred to. Verbal cues also
demand particularly fine motor skills through speech articulators.
Second, joint action necessitates some motor control to reach
the expected outcome, hence the importance of evaluating
beforehand human motor skills, especially during development,
as well as the technical capabilities of the robot. Following on
from the previous example, children’s grasping skills in relation
to the size of the cubes as well as the characteristics of robotic
gripper to handle objects have to be finely described.

Moreover, recent experimental findings have shown that
the execution of object-oriented actions is influenced by the
social context such as the relative position of another person
and the degree of familiarity with this person (Gianelli et al.,
2013). Individuals perform for example more fluent reach-to-
grasp movements, with lower acceleration peaks and longer
reaction time when a partner is located close enough to be
able to intervene on the same object than when he/she is
farther away (Quesque et al., 2013). In addition, there is a
significant relationship between the kinematic features of the
actions and the actor’s explicit social intention: movements
have longer durations, higher elevations and longer reaction
times when individuals place an object on a table for
another person than when they place the object for a later
personal use (Quesque and Coello, 2015). These variations,
although they do not seem to be intentionally produced,
have been suggested to facilitate the partner’s detection
of planned actions, thus enhancing potential interactions.
These kinematic effects were indeed shown to influence
the subsequent motor productions of an observer (Quesque
et al., 2015). The motor characteristics of actions performed
in a social context may therefore prime the perceiver to
prepare and anticipate appropriate motor responses in the
interaction.

The second dimension that can allow us to understand the
complexity of joint activities pertains to the coordination between
several communicative modalities and between interacting
individuals. Whether joint action involves complementary or
similar roles, it can be performed through several coordination
processes, which can determine the efficiency of shared
attention states (Shteynberg, 2015). Efficiency requires here a
representational shift from the first-person singular to the first-
person plural, as the partners attend to the same referent at
the same time. The ability to monitor each other’s attention
and action, using behavioral cues such as gaze direction, facial
expressions, gestures, and speech is essential for successful
coordination. The intentional production of communicative
signals, representing hints for one’s partner, is also an efficient way
of achieving joint outcomes.

Coordination is therefore necessary first at the individual level,
so that the different communicative modalities such as gestures
and gaze synchronize or follow one another in a natural order, i.e.,
acceptable with regard to human interaction patterns (see above).
Each agent can then make decisions based on these signals,
moderate their behavior accordingly and thus coordinate at the
social level to reach a common objective. The ability to adjust
one’s behavior to others’ actions during collaborative activities
(including play) has been argued to “reach a higher degree of
complexity when intentional and referential signals are directly
addressed to specific individuals” (Cochet and Byrne, 2015). In
order to build the pile of cubes, interacting partners can then
for example point toward a specific cube or ask the other to wait
before placing another cube.

In those cases, coordination processes can be enhanced by
predicting the effects of each other’s actions on joint outcomes
and by distributing tasks effectively (Vesper et al., 2016). This
ability involves the third dimension characterizing the question
of complexity, namely the dimension of anticipatory planning
(Cochet and Byrne, 2015). It requires to go beyond the immediate
perception of the environment and represent the relationship
between a sequence of actions and a precise goal. At the
individual level, planning ability implies to mentally review an
action sequence in anticipation of a future need (e.g., selecting
a specific cube in a first room in order to build a pile of cubes
in another room). At the social level, planning ability allows
individuals to predict the other’s behavior and adjust one’s own
sequence of actions, leading to a better coordination. Whether
the ability to make such inferences necessitates to mentalize about
others’ inner states (e.g., beliefs and preferences) is still subject of
debate, but again, this question may not be central in the context
of joint attention and joint action between a robot and a human.

The above-described categorization can therefore provide
a common ground between ethologists, psychologists, and
roboticists that may clarify which dimensions need to be
considered in an attempt to implement the characteristics of
motor precision, coordination and anticipatory planning in
human–robot joint activities (see Table 1 for an overview). The
objective is to approach the complexity (or richness) of human
interactions and obtain appropriate (or effective) responses from
robots with regard to these different dimensions.

PRAGMATICS IN HRI: WHICH
INGREDIENTS ARE NECESSARY FOR
EFFECTIVE INTERACTIONS?

The increasing complexity of communicative abilities
(complexity that involves the three above-mentioned
dimensions) in the course of human development leads to
a rich potential of interactions. Children actively go through
different stages allowing them to engage successfully in joint
activities, i.e., to operate within their physical environment,
coordinate with other people, plan their own behavior and
anticipate their partners’. Intending to model, at least partially,
human developmental pathway seems a fruitful way of designing
robots that can effectively initiate and respond to communicative

Frontiers in Psychology | www.frontiersin.org 5 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 6

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

TABLE 1 | Complexity in HRI: illustration of three dimensions at the individual and
social levels (adapted from Cochet and Byrne, 2015).

Individual Social

(1) Motor precision Joint attention: ability to
properly use gaze or
pointing to identify the
object or event referred to.
Joint action: human motor
skills/technical capabilities
of the robot to reach the
expected outcome

Influence of the social
context (e.g., relative
position of the individuals,
intention of the actor:
moving an object for
oneself or for another
person) on the kinematics
features of the actions
performed

(2) Coordination Coordination (including
synchronization) between
different modalities of one’s
communicative signal
(gaze, gesture,
vocalizations, etc.)

Ability to take into account
the multimodal behavioral
cues produced by a partner
to adjust one’s own
behavior

(3) Anticipatory
planning

Representation of a
sequence of actions to
anticipate a personal future
need

Ability to predict the effects
of the other’s actions on
joint outcomes to plan
one’s own behavior

situations. Such enterprise, although still recent, has given rise
to a substantial amount of literature in robotics, especially
from the 2000s, covering several sub-fields such as for example
developmental and epigenetic robotics, cognitive systems
and social robotics. Several journals, including both HRI
experimental studies and computational modeling, focus entirely
on these questions (e.g., IEEE Transactions on Cognitive and
Developmental Systems, Journal of Human-Robot Interaction,
Journal of Social Robotics), and numerous conferences also take
place every year, whose proceedings are usually available online1.

The data from developmental psychology described in the
first section, coupled with the framework proposed in the
second section to help researchers define complex and effective
HRI, may contribute to this growing body of work. To this
effect, it seems necessary (1) to consider the multimodality of
interactions and (2) to adopt a pragmatic perspective to be
based upon an accurate representation of human communicative
behaviors. Indeed, children learn to communicate through joint
activities with adults who combine various forms of expressions,
serving various functions. In the course of development, children
gradually integrate the dissociation between the form and the
function of language – they become more and more flexible in
understanding that a single form can serve different functions
and reciprocally, that a single function can be expressed through
several forms. Language is here regarded as more than a medium
to convey an information, in agreement with a proposition that
was developed in the speech act theory (Austin, 1962; Searle
and Vanderveken, 1985). Language would be way of acting on
the environment, of “doing things with words,” independently
of its structural properties. Initially aiming at describing the
relationships between the forms and functions of linguistic
utterances, this theory defines several speech acts, depending on
whether one intends to assert, comment, warn, request, deplore,

1 For example, http://www.lucs.lu.se/epirob/

etc. This theory has later been adapted to non-verbal behavior
(e.g., McNeill, 1998; Guidetti, 2002). The form still refers to
the message structure, but applies to the whole body, including
the posture, the structure of communicative gestures (kinematic
features and hand shape), gaze and facial expressions. These non-
verbal signals can be used in complementarity with speech or be
used alone for example in the case of conventional gestures (see
Guidetti, 2002). The function refers to the illocutionary force of
the speech act (what one achieves by speaking), in other words
here to the effect of these communicative acts in a specific context,
thus giving some insight into the signaller’s intention. Gestures,
and especially the conventional gestures produced by children
during the prelinguistic period, are thus regarded as genuine
communicative acts, with a propositional content that can equal
the one expressed by words. For instance, agreeing and refusing
can be expressed gesturally by nodding or shaking one’s head. The
separate analysis of the forms and functions of communication,
as well as the description of the different modalities involved
during interactions, therefore provide a key framework to help
define what capacities the robot should be equipped with to
ensure efficient collaboration with humans.

In this perspective, Mavridis (2015) has proposed a list of
“ten desiderata that human–robot systems should fulfill” to
maximize communication effectiveness. One of the guiding
lines relates to the importance of considering multiple speech
acts, for both verbal and non-verbal communication, and not
restrict the robot competencies to “motor command requests.”
In the same way as imperative gestures (see section “How
Does Communication Develop in the Context of Social Play?”)
are generally understood and produced later than declarative
gestures in human development (Camaioni et al., 2004), robotic
systems initially aimed to assign the robot a servant role, with the
human driving the interaction. Devising wider robots’ pragmatic
abilities is a first step toward the conception of human–robot
shared plans. The robot may for example comment on the pile of
cubes as it is being built (see example section “To What Extent
Can Interactions Be Characterized as complex?”) to support
or correct the human’s action, rather than just producing a
motor response to the human request. The dimension of social
coordination is thus added to that of motor precision (see
Table 1).

Similarly, flexibility in HRI also requires “mixed initiative
dialog” (Mavridis, 2015), so that the robot can both initiate and
respond to the interaction. Integrating models based on human
adaptation and probabilistic decision processes, Nikolaidis et al.
(2017) have indeed shown that the performance of human–robot
teams in collaborative tasks is improved when the robot guides
the human toward an effective strategy, compared to the common
approach of having the robot strictly adapting to the human.
The human’s trust in the robot was also facilitated by a greater
symmetry in role distribution and adaptation between the robot
and the human, which might in turn lead to greater acceptability
of HRI.

Designing such “socially intelligent and cooperative robots”
(Breazeal et al., 2004) requires specific temporal dynamics of the
interaction, which represents a considerable challenge especially
at a computational level. These dynamics convey social meanings

Frontiers in Psychology | www.frontiersin.org 6 October 2018 | Volume 9 | Article 1992

http://www.lucs.lu.se/epirob/

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 7

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

to such an extent that any delay in the interaction can sometimes
question its effectiveness. Researchers here face a dilemma that
seem to bring into opposition interaction complexity (which
requires to take account of numerous parameters) and interaction
timing. The implementation of fast timescales (on the order of
100 ms) is usually considered necessary for robots to integrate
(i.e., detect, interpret, and predict) and react to social stimuli in
a timely manner through interactions (Durantin et al., 2017).
Researchers developing a storytelling robot interacting with
children aged 4–5 years have confirmed the importance of
temporal features in the pragmatics of interactions. Contingent
responses from the robot, in relation to the attentional and social
cues signaled by the children, were indeed found to facilitate
engagement of the latter (Heath et al., 2017).

The variation in some characteristics of the robot’s behaviors
according to the action performed may also illustrate further
the question of pragmatics in HRI, moving us one step closer
toward human-like interactions. For example, the morphological
differences that have been reported in young children between
pointing and reaching (Cochet et al., 2014) could be applied
to the robot. First, regarding body posture, we might expect
robots to lean closer to a given object when they intend to
grasp it than when they want to communicate about that object.
Second, depending on the robot technical possibilities (e.g., two-
or three-finger grippers, biomimetic anthropomorphic hands),
differences in the form of manual gestures produced should
be observed between imperative and declarative pointing. The
former is typically characterized by whole-hand gestures (all
the fingers are extended in the direction of the referent), while
the latter is mostly associated with index-finger gestures (the
index finger is extended toward the referent and the other
fingers are curled inside the hand) (Cochet and Vauclair, 2010;
Liszkowski and Tomasello, 2011). Hand shape is also influenced
by precision constraints: imperative gestures are likely to shift
from whole-hand pointing to index-finger pointing when the
target is surrounded by distractors (Cochet et al., 2014), which
can be the case when the robot has to identify a specific object
among several (e.g., the human can ask the robot to give him/her
the red cube). Here, the notion of iconicity, which plays a role in
both oral and sign languages, may help researchers to precisely
analyze the structure of gestures and better understand the
interface between gestures and signs (Guidetti and Morgenstern,
2017). The importance of motor precision is here directly related
to the dimensions of coordination and anticipatory planning,
therefore providing a comprehensive framework to assess the
complexity and effectiveness of HRI.

Moreover, the importance of implementing responsive social
gaze in robots has previously been highlighted (e.g., Yoshikawa
et al., 2006), but this response might also vary depending
on the communicative function involved. To mirror child
development, gaze alternation between the partner and the
referent should indeed be more frequent in declarative situations
than in imperative ones (Cochet and Vauclair, 2010). The
coordination between gestures and gaze (see also section “How
Does Communication Develop in the Context of Social Play?”) is
also an important factor, which can help the robot to estimate
the state of goals, plans, and actions from human point of

view, and allow the human to feel that he/she is involved in
fluid interactions with the robot, both facilitating the emergence
of joint outcomes. If a robot alternates its gaze between an
object and its partner before initiating a pointing gesture, the
human may for example interpret this behavior as the robot’s
willingness to take into account his/her attentional state before
gesturing, thus favoring the exchange of information. Broadly
speaking, coordinated gaze behavior could be considered as the
most fundamental modality for effective HRI, or at least as a key
prerequisite in collaborative tasks.

The consideration of facial expressions may also facilitate
turn-taking dynamics and limit miscommunication, by allowing
some inferences about the other’s affective state. Integrating the
emotional component into HRI gives each partner additional
cues to decide what is the most appropriate response in a given
situation. The development of methods for facial expression
analysis raises several issues though (e.g., Kanade et al., 2000).
Even if there have been some attempts to design facial expression
mechanism in humanoid robots (e.g., Hashimoto et al., 2006; Gao
et al., 2010), most of current robots’ facial features are still far
from the extremely rich motor possibilities of the human face.
In parallel, the development of real time coding of emotional
expressions seems to be an achievable goal (Bartlett et al., 2003),
allowing robots to directly perceive some changes in the human
facial expressions.

In addition to visual information, the auditory modality can
also play a role in influencing robots’ and humans’ decisions and
coordination processes. In children at around 2 years of age,
vocalizations accompany more frequently declarative gestures
than imperative ones (Cochet and Vauclair, 2010). More recently,
the prosody of these vocalizations was shown to gradually match
the function of pointing during the second year of life (Tiziana
et al., 2017), allowing to differentiate imperative from declarative
gestures (Grünloh and Liszkowski, 2015). Other features such
as the positioning of the object and the attentional state of the
partner have also been suggested to influence the rising and
falling tones in the vocal productions simultaneous to gestures
(Leroy et al., 2009). Prosody can therefore serve pragmatic
purposes, and changes in pitch, intensity, or duration of speech
or vocalizations can in this regard be considered as a full-fledged
component of multimodal communication.

Beyond prosody, language content may be the most effective
way for human–robot teams to coordinate. However, the design
of robots with language comprehension and production abilities
that could lead to fluid conversations with humans raises several
issues. Verbal language requires indeed symbolic representations,
which need to be connected not only to the robot’s sensory
system, but also to “mental models” of the world internalized
within its cognitive system. Mavridis (2015) has highlighted here
the question of “situated language and symbol grounding.” For
example, the relation between the verbal label “cube” uttered by
the human and the physical cube that it refers to in front of
the robot can be mediated through sensory data, but the use of
conventional signs should allow the robots to go beyond the here-
and-now and extend symbol grounding to abstract entities in
addition to objects, people, or events. To implement architecture
that can be compared to human interactions, this relation should

Frontiers in Psychology | www.frontiersin.org 7 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 8

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

be bidirectional: the visual perception of a cube should activate
the right symbol in the robot’s cognitive system, leading to the
production of the word “cube”; reciprocally, a request addressed
to the robot to give the human the cube should create a precise
representation, allowing the robot to identify the right object.

Moreover, the identification of emotion labels in the verbal
modality could also contribute, in addition to the recognition of
emotional facial expressions and acoustic properties of speech
(see Breazeal, 2004 for a complete review on emotion systems
in robots), to a better coordination between each partner of
the interaction. The haptic modality, playing an important role
in social interactions, is also regarded as a valuable medium
for expressing emotion (Yohanan and MacLean, 2012). By
developing motion capturing system and tactile sensors, the
robot may use its human partner’s positions and such “affective
touch” to estimate human intentions (Miyashita et al., 2005).
This modality, essential in human development, may be a
particularly good candidate to study complexity of HRI, involving
simultaneously motor precision, coordination and planning (see
section “To What Extent Can Interactions Be Characterized as
complex?”).

Finally, in addition to the coordination dimension, the verbal
dialog between a robot and a human would ideally imply
purposeful speech and planning (Mavridis, 2015), in order to
avoid fixed mapping between stimuli and responses. Anticipatory
planning abilities, as described in Section “To What Extent Can
Interactions Be Characterized as complex?”, would enable the
robot to make the most appropriate or efficient decisions in a
given shared activity, in conjunction with its perspective-taking
skills and the goal of the activity. If the robot can represent which
information are needed by the human to perform a specific action
(and therefore identify which information the human misses),
it can decide to express a verbal request or comment on the
situation, and/or plan a sequence of actions to coordinate with
its partner.

This last example raises the question of intrinsic motivation
in interactions: why is each partner engaged in this multimodal
coordination, and to what extent does it influence the
characteristics of the interaction? Studies in developmental
robotics have shown that intrinsic motivation systems based
on curiosity can directly impact learning skills and lead to
autonomous mental development in robots (Oudeyer et al.,
2007). Such mechanism is obviously involved in human
development and in social play in particular: children discover
and create new possibilities by exploring their physical and social
environment. Through the development of social referencing,
self-consciousness or cooperation, human social interactions may
even sometimes constitute a motivated goal per se (Tomasello,
2009), which provides some perspectives to shape robots’
intrinsic motivation with a “social reward” function.

We can see here that the relationships between theories
in developmental psychology and robotics offer bidirectional
benefits. To put it in a nutshell, some models in developmental
robotics are based on psychological theories, which are then
formalized and implemented in robots, while developmental
robotics allows researchers in psychology to go further in the
elaboration of their theories through thorough experimentations

and hypothesis testing. This applies to a variety of questions
addressed in this review, from the conditions that influence
learning process during interactions (Boucenna et al., 2014) to
the description of stages in language development (Morse and
Cangelosi, 2017). Advances in developmental robotics may thus
provide previous help in the analysis and implementation of the
processes involved in interactions.

CONCLUSION AND PERSPECTIVES

The question at stake in the present work was to improve the
effectiveness of human–robot interactions in collaborative tasks,
first in terms of joint outcomes – has the task been completed? –
but also with regard to the human’s perception and interpretation
of the interaction. Is the robot’s behavior appropriate, i.e.,
acceptable, considering the frame of human communication?
We argue here that the observation of the development and
the structure of interactions between the child and the adult,
especially in the context of social play, can help answer this
question. To shape a shared common space between the human
and the robot that could reflect the complexity of human
interactions, we have also proposed to focus on three dimensions:
motor precision, coordination, and anticipatory planning. The
specific examples developed in Section “Pragmatics in HRI:
Which Ingredients Are Necessary for Effective Interactions?”
suggest that the more robots use human-like communicative
modalities (e.g., facial expressions, gestures, and language) in
respect to these three dimensions, the more they invite interactive
behaviors that are natural to people. The interpretation of dealing
with a social agent is strengthened, which facilitates in turn
the interaction with robots. In this sense, and to paraphrase
Cangelosi et al. (2010), the integration of action and language
may constitute a roadmap to better frame and assess HRI from
a developmental point of view and with a pragmatic perspective.

However, there are still numerous obstacles before
achieving the level of details pictured in the present article,
involving mainly technological challenges, given the motor
and cognitive correlates of the above-mentioned behaviors.
To put it bluntly, developmental psychologists cannot expect
roboticists to implement in robots all the subtleties of multimodal
communication that occur in human children. There may also be
some conceptual difficulties as the attempts to approach human
realism, aiming at maintaining the human’s trust in the robot,
can sometimes be confronted with an uneasy feeling of viewing
and/or hearing a robot that looks imperfectly human. This
uncanny valley effect (Mitchell et al., 2011; Mori, 1970, 2012),
which was shown to emerge in middle childhood in relation to
developing expectations about humans and machines (Brink
et al., 2017), may complicate the design of socially interactive
robots, both in terms of appearance and behavior. Empirical
evidence for the uncanny valley seems nevertheless inconsistent
or restricted to specific conditions (Kätsyri et al., 2015), with the
definition of human-likeness mostly involving physical realism.

By contrast, anthropomorphic behavior (see Duffy, 2003), in
addition to its facilitating role in the interaction with humans (see

Frontiers in Psychology | www.frontiersin.org 8 October 2018 | Volume 9 | Article 1992

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 9

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

above), also results in better and faster learning by the
robots. For example, in a task in which they have to
learn the meaning of words, the robots’ performances are
enhanced when they provide humans with social cues to
communicate a learning preference, as these cues influence
the tutoring of the human teacher (de Greeff and Belpaeme,
2015). We observe the same phenomena when human
children start to learn new concepts: according to Bruner’s
constructivist theory, children need scaffolding from adults
(or from children who have already acquired the concept)
in the form of active support, which may represent at
first a reduction in the choices a child might face. Such
learning processes play obviously an important role in
human development, and may also enable quick and effective
application of robotic systems. Multi-level learning may indeed
constitute a key line of research for HRI (Mavridis, 2015),
which might again benefit from research in developmental
psychology.

Reciprocally, the field of robotics provides interesting
perspectives for psychologists, especially for research on atypical
development. Atypical development might be a direct window
on typical development and vice versa: “development is the
key to understanding developmental disorders” (Karmiloff-
Smith, 1998). Joint action and joint attention are for example
usually impaired in children with ASD; the comparison
with typical development has revealed different use of social
gaze and often a lack of the declarative function, both
for verbal and non-verbal communication. The exchanges
between robotics and developmental psychology could help
conceptualize the stages of joint attention in order to better
understand how children develop joint attention and get
through the whole sequence of declarative pointing. This
will have an impact on elaborating intervention programs
for children with neurodevelopmental disorders. Moreover,
numerous intervention programs have recently been proposed

showing the added value of therapy robot for the development
of communication, play, or emotional skills (e.g., Robins et al.,
2009; Huijnen et al., 2016).

In conclusion, the combination of insights and methods in
robotics and developmental psychology allows researchers to
conceive models of HRI in which the robots can come to develop
motor, social, and cognitive skills. These models may benefit
fundamental research on joint attention and joint action in
typical development, but also early evaluation and intervention
programs for atypical development (e.g., Dautenhahn, 2007). The
continuation of these interdisciplinary discussions, which may
possibly integrate some of the elements proposed in the present
article, will undoubtedly lead to more and more solid HRI models
in the next decades.

AUTHOR CONTRIBUTIONS

HC and MG devised the conceptual ideas presented in the article.
HC drafted the manuscript. MG revised it critically and gave final
approval of the version to be submitted.

FUNDING

This article is part of the project JointAction4HRI, funded by the
French National Agency for Research (n◦16-CE33-0017).

ACKNOWLEDGMENTS

Many ideas presented in this paper stem from fruitful discussions
with R. Alami, A. Clodic, and E. Pacherie, all involved in the Joint
Action for Human-Robot Interaction project funded by French
National Agency for Research (Project No. 16-CE33-0017-01).

REFERENCES
Adamson, L. B., Bakeman, R., Suma, K., and Robins, D. L. (2017). An expanded

view of joint attention: Skill, engagement and language in typical development
and autism. Child Dev. doi: 10.1111/cdev.12973 [Epub ahead of print].

Aureli, T., Grazia, A., Cardone, D., and Merla, A. (2015). Behavioral
and facial thermal variations in 3-to 4- month-old infants during the
Still-Face Paradigm. Front. Psychol. 6:1586. doi: 10.3389/fpsyg.2015.
01586

Austin, J. (1962). How to do Things with Words. Oxford: Clarendon Press.
Bartlett, M. S., Littlewort, G., Fasel, I., and Movellan, J. R. (2003). Real time

face detection and expression recognition: development and application to
human-computer interaction. CVPR Workshop Comput. Vis. Patt. Recogn.
Hum. Comput. Interact. 5, 53–58.

Bates, E., Camaioni, L., and Volterra, V. (1975). The acquisition of performatives
prior to speech. Merrill Palmer Q. 21, 205–226.

Beier, J. S., and Spelke, E. S. (2012). Infants’ developing understanding
of social gaze. Child. Dev. 83, 486–496. doi: 10.1111/j.1467-8624.2011.
01702.x

Beuker, K. T., Rommelse, N. J., Donders, R., and Buitelaar, J. K. (2013).
Development of early communication skills in the first two years of life. Infant
Behav. Dev. 36, 71–83. doi: 10.1016/j.infbeh.2012.11.001

Boucenna, S., Anzalone, S., Tilmont, E., Cohen, D., and Chetouani, M. (2014).
Learning of social signatures through imitation game between a robot and

a human partner. IEEE Trans. Auton. Mental Dev. 6, 213–225. doi: 10.1109/
TAMD.2014.2319861

Bourjade, M. (2017). “Social attention,” in The International Encyclopedia of
Primatology, ed. A. Fuentes (New York, NY: John Wiley and Sons).

Breazeal, C. (2004). Function meets style: Insights from emotion theory applied to
HRI. IEEE Trans. Syst. Man Cybern. 34, 187–194. doi: 10.1109/TSMCC.2004.
826270

Breazeal, C., Brooks, A., Gray, J., Hoffman, G., Kidd, C., Lee, H., et al. (2004).
Humanoid robots as cooperative partners for people. Int. J. Hum. Rob. 1, 1–34.
doi: 10.3389/fnbot.2012.00003

Brink, K. A., Gray, K., and Wellman, H. M. (2017). Creepiness creeps in: Uncanny
valley feelings are acquired in childhood. Child Dev. doi: 10.1111/cdev.12999
[Epub ahead of print].

Bruner, J. S. (1973). Organization of early skilled action. Child Dev. 44, 1–11.
doi: 10.2307/1127671

Butcher, C., and Goldin-Meadow, S. (2000). “Gesture and the transition from
one- to two-word speech: when hand and mouth come together,” in Language
and Gesture, ed. D. McNeill (Cambridge: Cambridge University Press),
235–257.

Byrne, R. W. (2015). The what as well as the why of animal fun. Curr. Biol. 25,
R2–R4. doi: 10.1016/j.cub.2014.09.008

Camaioni, L., Perucchini, P., Bellagamba, F., and Colonnesi, C. (2004). The role
of declarative pointing in developing a theory of mind. Infancy 5, 291–308.
doi: 10.1207/s15327078in0503_3

Frontiers in Psychology | www.frontiersin.org 9 October 2018 | Volume 9 | Article 1992

https://doi.org/10.1111/cdev.12973

https://doi.org/10.3389/fpsyg.2015.01586

https://doi.org/10.1111/j.1467-8624.2011.01702.x

https://doi.org/10.1016/j.infbeh.2012.11.001

https://doi.org/10.1109/TAMD.2014.2319861

https://doi.org/10.1109/TSMCC.2004.826270

https://doi.org/10.3389/fnbot.2012.00003

https://doi.org/10.1111/cdev.12999

https://doi.org/10.2307/1127671

https://doi.org/10.1016/j.cub.2014.09.008

https://doi.org/10.1207/s15327078in0503_3

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 10

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

Cangelosi, A., Metta, G., Sagerer, G., Nolfi, S., Nehaniv, C., Fischer, K., et al. (2010).
Integration of action and language knowledge: a roadmap for developmental
robotics. IEEE Trans. Autonom. Mental Dev. 2, 167–195. doi: 10.1109/TAMD.
2010.2053034

Carpenter, M., Nagell, K., and Tomasello, M. (1998). Social cognition, joint
attention, and communicative competence from 9 to 15 months of age. Monogr.
Soc. Res. Child Dev. 63:255. doi: 10.2307/1166214

Charman, T., Baron-Cohen, S., Swettenham, J., Baird, G., Cox, A., and Drew, A.
(2000). Testing joint attention, imitation, and play as infancy precursors to
language and theory of mind. Cogn. Dev. 15, 481–498. doi: 10.1016/S0885-
2014(01)00037-5

Clodic, A., Alami, R., and Chatila, R. (2014). Key elements for human-robot joint
action. Front. Artif. Intell. Appl. 273, 23–33.

Cochet, H., and Byrne, R. W. (2015). Complexity in animal behaviour:
towards common ground. Acta Ethol. 18, 237–241. doi: 10.1007/s10211-014-
0205-5

Cochet, H., and Byrne, R. W. (2016). Communication in the second and third year
of life: relationships between nonverbal social skills and language. Infant Behav.
Dev. 44, 189–198. doi: 10.1016/j.infbeh.2016.07.003

Cochet, H., Jover, M., Oger, L., and Vauclair, J. (2014). Morphological differences
between imperative and declarative pointing: hand shape, arm extension
and body posture. J. Motor Behav. 46, 223–232. doi: 10.1080/00222895.2014.
889066

Cochet, H., and Vauclair, J. (2010). Pointing gestures produced by toddlers
from 15 to 30 months: different functions, hand shapes and laterality
patterns. Infant Behav. Dev. 33, 432–442. doi: 10.1016/j.infbeh.2010.
04.009

Cochet, H., and Vauclair, J. (2014). Deictic gestures and symbolic gestures produced
by adults in an experimental context: hand shapes and hand preferences.
Laterality 19, 278–301. doi: 10.1080/1357650X.2013.804079

Colonnesi, C., Stams, G., Koster, I., and Noom, M. J. (2010). The relation between
pointing and language development: a meta-analysis. Dev. Rev. 30, 352–366.
doi: 10.1016/j.dr.2010.10.001

Corkum, V., and Moore, C. (1998). Origins of joint visual attention in infants. Dev.
Psychol. 34, 28–38. doi: 10.1037/0012-1649.34.1.28

Dautenhahn, K. (2007). Socially intelligent robots: dimensions of human–robot
interaction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 679–704. doi: 10.1098/
rstb.2006.2004

de Greeff, J., and Belpaeme, T. (2015). Why robots should be social:
enhancing machine learning through social human-robot interaction. PLoS One
10:e0138061. doi: 10.1371/journal.pone.0138061

Demiris, Y., and Meltzoff, A. (2008). The robot in the crib: a developmental
analysis of imitation skills in infants and robots. Inf. Child Dev. 17, 43–53.
doi: 10.1002/icd.543

D’Entremont, B., and Seamans, E. (2007). Do infants need social cognition to
act socially? An alternative look at infant pointing. Child Dev. 78, 723–728.
doi: 10.1111/j.1467-8624.2007.01026.x

Devin, S., and Alami, R. (2016). “An implemented theory of mind to improve
human-robot shared plans execution,” in Proceedings of the 11th ACM/IEEE
International Conference HRI, New York, NY, 319–326.

Duffy, B. R. (2003). Anthropomorphism and the social robot. Rob. Auton. Syst. 42,
177–190. doi: 10.1016/S0921-8890(02)00374-3

Durantin, G., Heath, S., and Wiles, J. (2017). Social moments: a perspective on
interaction for social robotics. Front. Robot. AI 4:24. doi: 10.3389/frobt.2017.
00024

Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of
social gaze. Neurosci. Biobehav. Rev. 24, 581–604. doi: 10.1016/S0149-7634(00)
00025-7

Farroni, T., Menon, E., Rigato, S., and Johnson, M. H. (2007). The perception
of facial expressions in newborns. Eur. J. Dev. Psychol. 4, 2–13. doi: 10.1080/
17405620601046832

Franco, F., and Butterworth, G. (1996). Pointing and social awareness: declaring
and requesting in the second year. J. Child Lang. 23, 307–336. doi: 10.1017/
S0305000900008813

Gao, J., Huang, Q., Yu, Z., Chen, X., Xu, W., and Zhang, Y. (2010). Design of the
facial expression mechanism for humanoid robots. ROMANSY 18 Rob. Design
Dynam. Control 524, 433–440.

Garvey, C. (1990). Play. Cambridge, MA: Harvard University Press.

Gianelli, C., Scorolli, C., and Borghi, A. M. (2013). Acting in perspective: the role of
body and language as social tools. Psychol. Res. 77, 40–52. doi: 10.1007/s00426-
011-0401-0

Grünloh, T., and Liszkowski, U. (2015). Prelinguistic vocalizations distinguishing
pointing acts. J. Child Lang. 42, 1312–1336. doi: 10.1017/S030500091400
0816

Guidetti, M. (2002). The emergence of pragmatics: forms and functions of
conventional gestures in young French children. First Lang. 22, 265–285. doi:
10.1177/014272370202206603

Guidetti, M. (2005). Yes or no? How do young children combine gestures
and words to agree and refuse. J. Child Lang. 32, 911–924. doi: 10.1017/
S0305000905007038

Guidetti, M., and Morgenstern, A. (2017). The gesture sign interface in language
acquisition. Lang. Interact. Acquis. 8, 1–12. doi: 10.1075/lia.8.1.01gui

Guidetti, M., and Tourrette, C. (2017). Echelle d’Evaluation de la Communication
Sociale Précoce, 1st Edn. Firenze: Giunti OS.

Hashimoto, T., Hitramatsu, S., Tsuji, T., and Kobayashi, H. (2006). Development of
the face robot Saya for rich facial expressions. Proc. IEEE SICE-ICASE Int. Joint
Conf. 5423–5428. doi: 10.1109/SICE.2006.315537

Heath, S., Durantin, G., Boden, M., Hensby, K., Taufatofua, J., Olsson, O., et al.
(2017). Spatiotemporal aspects of engagement during dialogic storytelling
child-robot interaction. Front. Robot. AI 4:27. doi: 10.3389/frobt.2017.
00027

Hough, J., and Schlangen, D. (2016). “Investigating fluidity for human-robot
interaction with real-time, real-world grounding strategies,” in Proceedings of
the 17th Annual SIGdial Meeting on Discourse and Dialogue, Los Angeles, CA,
288–298. doi: 10.18653/v1/W16-3637

Huijnen, C. A. G. J., Lexis, M. A. S., and de Witte, L. P. (2016). Matching robot
KASPAR to autism spectrum disorder (ASD): therapy and educational goals.
Int. J. Soc. Rob. 8, 445–455. doi: 10.1007/s12369-016-0369-4

Kanade, T., Cohn, J., and Tian, Y. (2000). “Comprehensive database for facial
expression analysis,” in Proceedings of the 4th International Conference Face
Gesture Recognition, London, 46–53.

Karmiloff-Smith, A. (1998). Development itself is the key to understanding
developmental disorders. Trends Cogn. Sci. 2, 389–398. doi: 10.1016/S1364-
6613(98)01230-3

Kätsyri, J., Förger, K., Mäkäräinen, M., and Takala, T. (2015). A review of
empirical evidence on different uncanny valley hypotheses: support for
perceptual mismatch as one road to the valley of eeriness. Front. Psychol. 6:390.
doi: 10.3389/fpsyg.2015.00390

Kendon, A. (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge
University Press. doi: 10.1017/CBO9780511807572

Lamaury, A., Cochet, H., and Bourjade, M. (2017). Acquisition of joint attention
by olive baboons gesturing toward humans. Anim. Cogn. doi: 10.1007/s10071-
017-1111-9 [Epub ahead of print].

Lemaignan, S., Warnier, M., Clodic, A., and Alami, R. (2017). Artificial cognition
for social human-robot interaction: an implementation. Artif. Intell. 247, 45–69.
doi: 10.1016/j.artint.2016.07.002

Leroy, M., Mathiot, E., and Morgenstern, A. (2009). “Pointing gestures,
vocalizations and gaze: two case studies,” in Studies in Language and Cognition,
eds J. Zlatev, M. Andren, M. J. Falck, and C. Lundmark (Newcastle upon Tyne:
Cambridge Scholars Publishing), 261–275.

Liebal, K., Carpenter, M., and Tomasello, M. (2013). Young children’s
understanding of cultural common ground. Br. J. Dev. Psychol. 31, 88–96.
doi: 10.1111/j.2044-835X.2012.02080.x

Liszkowski, U., and Tomasello, M. (2011). Individual differences in social,
cognitive, and morphological aspects of infant pointing. Cogn. Dev. 26, 16–29.
doi: 10.1016/j.cogdev.2010.10.001

Lœvenbruck, H., Vilain, C., and Dohen, M. (2008). From gestural pointing to vocal
pointing in the brain. Rev. Fr. Ling. Appl. 13, 23–33.

Mavridis, N. (2015). A review of verbal and non-verbal human-robot interactive
communication. J. Rob. Auton. Syst. 63, 22–35. doi: 10.1016/j.robot.2014.
09.031

McNeill, D. (1998). “Speech and gesture integration,” in The Nature and Functions
of Gesture in Children’s Communication, eds J. M. Iverson and S. Goldin-
Meadow (San Francisco, CA: Jossey-Bass).

McNeill, D. (2000). Language and Gesture: Window into Thought and Action.
Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511620850

Frontiers in Psychology | www.frontiersin.org 10 October 2018 | Volume 9 | Article 1992

https://doi.org/10.1109/TAMD.2010.2053034

https://doi.org/10.2307/1166214

https://doi.org/10.1016/S0885-2014(01)00037-5

https://doi.org/10.1007/s10211-014-0205-5

https://doi.org/10.1016/j.infbeh.2016.07.003

https://doi.org/10.1080/00222895.2014.889066

https://doi.org/10.1016/j.infbeh.2010.04.009

https://doi.org/10.1080/1357650X.2013.804079

https://doi.org/10.1016/j.dr.2010.10.001

https://doi.org/10.1037/0012-1649.34.1.28

https://doi.org/10.1098/rstb.2006.2004

https://doi.org/10.1371/journal.pone.0138061

https://doi.org/10.1002/icd.543

https://doi.org/10.1111/j.1467-8624.2007.01026.x

https://doi.org/10.1016/S0921-8890(02)00374-3

https://doi.org/10.3389/frobt.2017.00024

https://doi.org/10.1016/S0149-7634(00)00025-7

https://doi.org/10.1080/17405620601046832

https://doi.org/10.1017/S0305000900008813

https://doi.org/10.1007/s00426-011-0401-0

https://doi.org/10.1017/S0305000914000816

https://doi.org/10.1177/014272370202206603

https://doi.org/10.1017/S0305000905007038

https://doi.org/10.1075/lia.8.1.01gui

https://doi.org/10.1109/SICE.2006.315537

https://doi.org/10.3389/frobt.2017.00027

https://doi.org/10.18653/v1/W16-3637

https://doi.org/10.1007/s12369-016-0369-4

https://doi.org/10.1016/S1364-6613(98)01230-3

https://doi.org/10.3389/fpsyg.2015.00390

https://doi.org/10.1017/CBO9780511807572

https://doi.org/10.1007/s10071-017-1111-9

https://doi.org/10.1016/j.artint.2016.07.002

https://doi.org/10.1111/j.2044-835X.2012.02080.x

https://doi.org/10.1016/j.cogdev.2010.10.001

https://doi.org/10.1016/j.robot.2014.09.031

https://doi.org/10.1017/CBO9780511620850

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

fpsyg-09-01992 October 19, 2018 Time: 12:18 # 11

Cochet and Guidetti Joint Action in Developmental Psychology and Robotics

Milward, S. J., Kita, S., and Apperly, I. A. (2017). Individual differences in children’s
corepresentation of self and other in joint action. Child Dev. 8, 964–978.
doi: 10.1111/cdev.12693

Mitchell, W. J., Szerszen, K. A., Lu, A. S., Schermerhorn, P. W., Scheutz, M., and
MacDorman, K. F. (2011). A mismatch in the human realism of face and voice
produces an uncanny valley. I-Perception 2, 10–12. doi: 10.1068/i0415

Miyashita, T., Tajika, T., Ishiguro, H., Kogure, K., and Hagita, N. (2005). “Haptic
communication between humans and robots,” in Proceedings of the 12th
International Symposium Robot Research, Berlin, 525–536.

Mori, M. (1970). Bukimi no tani [The uncanny valley]. Energy 7, 33–35.
Mori, M. (2012). The uncanny valley. IEEE Robot. Autom. Mag. 19, 98–100. doi:

10.1109/Mra.2012.2192811
Morse, A. F., and Cangelosi, A. (2017). Why are there developmental stages in

language learning? A developmental robotics model of language development.
Cogn. Sci. 41, 32–51. doi: 10.1111/cogs.12390

Nikolaidis, S., Hsu, D., and Srinivasa, S. (2017). Human-robot mutual adaptation
in collaborative tasks: models and experiments. Int. J. Robot. Res. 36, 618–634.
doi: 10.1177/0278364917690593

Oudeyer, P. Y. (2017). What do we learn about development from baby robots?
Wiley Interdiscip. Rev. Cogn. Sci. 8:e1395.

Oudeyer, P. Y., Kaplan, F., and Hafner, V. V. (2007). Intrinsic motivation systems
for autonomous mental development. IEEE Trans. Evol. Comput. 11, 265–286.
doi: 10.1109/TEVC.2006.890271

Pons, F., Harris, P. L., and de Rosnay, M. (2004). Emotion comprehension between
3 and 11 years: developmental periods and hierarchical organization. Eur. J.
Dev. Psychol. 1, 127–152. doi: 10.1080/17405620344000022

Quesque, F., and Coello, Y. (2015). Perceiving what you intend to do from what
you do: evidence for embodiment in social interactions. Socioaffect. Neurosci.
Psychol. 5:28602. doi: 10.3402/snp.v5.28602

Quesque, F., Delevoye-Turrell, Y., and Coello, Y. (2015). Facilitation effect of
observed motor deviants in a cooperative motor task: evidence for direct
perception of social intention in action. Q. J. Exp. Psychol. 69, 1451–1463.
doi: 10.1080/17470218.2015.1083596

Quesque, F., Lewkowicz, D., Delevoye-Turrell, Y. N., and Coello, Y. (2013). Effects
of social intention on movement kinematics in cooperative actions. Front.
Neurorobot. 7:14. doi: 10.3389/fnbot.2013.00014

Robins, B., Dautenhahn, K., and Dickerson, P. (2009). “From isolation to
communication: a case study evaluation of robot assisted play for children
with autism with a minimally expressive humanoid robot,” in Proceedings
of the 2nd International Conference Advance Computing-Human Interaction,
New York, NY.

Scassellati, B. (2000). “How robotics and developmental psychology complement
each other,” in Proceedings of the NSF/DARPA Workshop Development Learning
Michigan State University, Lansing, MI.

Searle, J. R., and Vanderveken, D. (1985). Foundations of Illocutionary Logic.
Cambridge: Cambridge University Press.

Shteynberg, G. (2015). Shared attention. Perspect. Psychol. Sci. 5, 579–590.
doi: 10.1177/1745691615589104

Smith, L. B. (2015). “Foreword,” in Developmental robotics – From babies to
robots. Cambridge, eds A. Cangelosi and M. Schlesinger (London: The MIT
Press).

Tiziana, A., Spinelli, M., Fasolo, M., Garito, M. C., Perucchini, P., and
D’Odorico, L. (2017). The pointing-vocal coupling progression in the first
half of the second year of life. Infancy 22, 801–818. doi: 10.1111/infa.
12181

Tomasello, M. (2009). Why We Cooperate. Cambridge, MA: MIT Press.
Tomasello, M., and Carpenter, M. (2007). Shared intentionality. Dev. Sci. 10,

121–125. doi: 10.1111/j.1467-7687.2007.00573.x
Tomasello, M., Carpenter, M., Call, J., Behne, T., and Moll, H. (2005).

Understanding and sharing intentions: the origins of cultural cognition. Behav.
Brain Sci. 28, 675–691. doi: 10.1017/S0140525X05000129

Tronick, E. Z., Als, H., Adamson, L., Wise, S., and Brazelton, T. B. (1978). The
infant’s response to entrapment between contradictory messages in face-to-face
interaction. J. Am. Acad. Child Psychol. 17, 1–13. doi: 10.1016/S0002-7138(09)
62273-1

Vesper, C., Schmitz, L., Safra, L., Sebanz, N., and Knoblich, G. (2016). The role
of shared visual information for joint action coordination. Cognition 153,
118–123. doi: 10.1016/j.cognition.2016.05.002

Vygotsky, L. S. (1999). “Tool and sign in the development of the child,” in The
Collected Works of L. S. Vygotsky, Vol. 6, ed. R. W. Rieber (New York, NY:
Kluwer Academic/Plenum), 3–68.

Warneken, F., Chen, F., and Tomasello, M. (2006). Cooperative activities in young
children and chimpanzees. Child Dev. 77, 640–663. doi: 10.1111/j.1467-8624.
2006.00895.x

Yohanan, S., and MacLean, K. E. (2012). The role of affective touch in
human-robot interaction: Human intent and expectations in touching the
haptic creature. Int. J. Soc. Robot. 4, 163–180. doi: 10.1007/s12369-011-
0126-7

Yoshikawa, Y., Shinozawa, K., Ishiguro, H., Hagita, N., and Miyamoto, T.
(2006). Responsive robot gaze to interaction partner. Conf. Robot. Sci. Syst. 2,
287–293.

Conflict of Interest Statement: The authors declare that the research was
conducted in the absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.

Copyright © 2018 Cochet and Guidetti. This is an open-access article distributed
under the terms of the Creative Commons Attribution License (CC BY). The use,
distribution or reproduction in other forums is permitted, provided the original
author(s) and the copyright owner(s) are credited and that the original publication
in this journal is cited, in accordance with accepted academic practice. No use,
distribution or reproduction is permitted which does not comply with these terms.

Frontiers in Psychology | www.frontiersin.org 11 October 2018 | Volume 9 | Article 1992

https://doi.org/10.1111/cdev.12693

https://doi.org/10.1068/i0415

https://doi.org/10.1109/Mra.2012.2192811

https://doi.org/10.1111/cogs.12390

https://doi.org/10.1177/0278364917690593

https://doi.org/10.1109/TEVC.2006.890271

https://doi.org/10.1080/17405620344000022

https://doi.org/10.3402/snp.v5.28602

https://doi.org/10.1080/17470218.2015.1083596

https://doi.org/10.3389/fnbot.2013.00014

https://doi.org/10.1177/1745691615589104

https://doi.org/10.1111/infa.12181

https://doi.org/10.1111/j.1467-7687.2007.00573.x

https://doi.org/10.1017/S0140525X05000129

https://doi.org/10.1016/S0002-7138(09)62273-1

https://doi.org/10.1016/j.cognition.2016.05.002

https://doi.org/10.1111/j.1467-8624.2006.00895.x

https://doi.org/10.1007/s12369-011-0126-7

http://creativecommons.org/licenses/by/4.0/

https://www.frontiersin.org/journals/psychology/

https://www.frontiersin.org/

https://www.frontiersin.org/journals/psychology#articles

Contribution of Developmental Psychology to the Study of Social Interactions: Some Factors in Play, Joint Attention and Joint Action and Implications for Robotics

Introduction
How Does Communication Develop in the Context of Social Play?
To What Extent Can Interactions Be Characterized as Complex?
Pragmatics in Hri: Which Ingredients Are Necessary for Effective Interactions?
Conclusion and Perspectives
Author Contributions
Funding
Acknowledgments
References

EuropEan Journal of DEvElopmEntal psychology, 2017
vol. 14, no. 6, 629–646
https://doi.org/10.1080/17405629.2017.1382344

INTRODUCTION

Developmental psychology without positivistic
pretentions: An introduction to the special issue on
historical developmental psychology

Willem Koopsa and Frank Kesselb

autrecht university, utrecht, the netherlands; buniversity of new mexico, albuquerque, nm, usa

ABSTRACT
Emphasizing the importance of understanding children and child development as
‘cultural inventions’, William Kessen urged developmental psychologists to forego
‘positivistic dreaming’. The first section of this paper summarizes Kessen’s central
ideas. In the second section the pretensions of positivism (classical nineteenth
century positivism as well as twentieth century neo-positivism) are analyzed.
The core critique of positivism is based on Poppers falsificationism and the so-
called Positivismusstreit within the Frankfurter Schule. Despite those and related
fundamental critiques, anti-positivism (such as Kessen’s) does not imply anti-
empiricism. One corollary – Although contemporary developmental psychology
is dominated by empirical-quantitative approaches, a wider range of philosophical
and methodological approaches are called for if the failings of lingering positivism
are to avoided. In particular, twenty-first century developmental psychology
requires critical thinking about the discipline’s foundations and history, along
with deep analyses of how childhood and child development, and the field itself,
are historically and culturally embedded (as Kessen asserted). Section 4 concludes
with several critical notes regarding, e.g., the predominantly Western orientation of
historical studies of child development and the need to recognize the unavoidable
normative, moral dimension in the study of human development. The final section
provides a brief overview of the papers that comprise this special issue on historical
developmental psychology.

ARTICLE HISTORY received 17 september 2017; accepted 17 september 2017

KEYWORDS historical developmental psychology; positivistic psychology; analytical empirical
approaches; developmental science; srcD

1. Introduction

The often-cited developmental psychologist William Kessen (1925–1999) con-
sidered the American child a ‘cultural invention’ (Kessen, 1979). Inter alia, this

CONTACT Willem Koops W.Koops@fss.uu.nl

mailto: W.Koops@fss.uu.nl

http://www.tandfonline.com

http://crossmark.crossref.org/dialog/?doi=10.1080/17405629.2017.1382344&domain=pdf

630 W. KOOPS AND F. KESSEL

implies that developmental psychology cannot function fruitfully without his-
torical analysis. And that is what this special issue seeks to demonstrate.

The first section of this introduction focuses on Kessen’s ideas. The second
section takes a closer look at the meaning of ‘positivism’, a concept and move-
ment Kessen often considered. In particular, we will assert that it is possible to
be an anti-positivist while simultaneously believing that theoretical conceptions
must be approached systematically and assessed empirically-analytically as rig-
orously as possible. In other words, while recognizing – as Kessen did later in his
career – that scientific knowledge is always contingent on time and place, schol-
arly concepts and claims must remain open to empirical inquiry. Conversely,
section three elaborates on the view that scientific thinking not only depends
on empirical-analytical research, but also requires self-reflection, in particular,
critical thinking about a discipline’s foundations and history. Section four con-
cludes with several critical notes as a bridge to brief descriptions of the papers
that comprise this special issue.

2. Kessen’s plea

During the (only) ‘International Year of the Child’ (1979), Kessen wrote an influ-
ential essay on ‘The American Child and Other Cultural Inventions’ (Kessen, 1979;
also Kessel & Siegel, 1983). Among several other central observations Kessen
pointed out that:

No other animal species has been cataloged by responsible scholars in so many
wildly discrepant forms, forms that a perceptive extraterrestrial could never see
as reflecting the same beast. (Kessen, 1983, p. 27)

Understandably scientists who study children wish to continue to pursue what
Kessen referred to as a ‘positivistic dream’, in which such multiple variations in
the definition of the child are considered the ‘removable [correctible] error of
an [as-yet-]incomplete science’ (l.c.). Kessen’s view, however, was that develop-
mental psychologists needed to finally attempt to bridge what he considered
the abyss of the positivistic nightmare. This requires them to recognize that the
upbringing and development of children, as well as the sciences of developmen-
tal psychology and pedagogy, are culturally-historically influenced in significant
ways:

For not only are American children shaped and marked by the larger cultural forces
of political maneuverings, practical economics, and implicit ideological commit-
ments (a new enough recognition); child psychology is itself a peculiar invention
that moves with the tidal sweeps of the larger culture in ways that we understand
at best dimly and often ignore. (Kessen, l.c.)

Grounding his position in historical analysis, Kessen suggested that, in the
mid-nineteenth century, the United States of America was being prepared,
socio-culturally, for the birth of what came to be known as child psychol-
ogy. Against the background of the industrial revolution, he discussed three

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 631

cultural-historical changes that have shaped the fundamental principles of
developmental psychology through to the present. And while Kessen conse-
quently spoke about American child and developmental psychology, his critique
applies to Western culture as a whole, if only because of the dominant influence
of North American developmental psychology in the twentieth and even twen-
ty-first century. He did, therefore, delete the adjective ‘American’ (from the title
‘The American Child and Other Cultural Inventions’) in his revisited version in
1983; hence: ‘The Child and Other Cultural Inventions’ (Kessen, 1983).

In Kessen’s analysis, the first cultural-historical change that led to the inven-
tion of the modern-day child was the gradual division between the domains
of work and family. When women in America between 1830 and 1840 were
excluded from the industrial workforce, this division, marked by the walls of the
family home, became increasingly impregnable (Kessen, l.c., p. 31). Work was
carried out in specialist workplaces (factories) by specialist people (men), and
home became a place where one did not work.

Second, masculinity and femininity were so strongly separated that two
different worlds arose: the ‘ugly aggressive, corrupting, chaotic, sinful and irre-
ligious’ world of men and the ‘sweet, chaste, calm, cultured, loving, protective
and godly’ world of women (Kessen, l.c.). This division made women ‘naturally’
and exclusively responsible for the upbringing of and caring for children.

A third change followed from the other two: As children no longer had access
to the grown-up professional world and home ‘took on the coloration of mother,
hearth and heaven’ (Kessen, l.c.), children became sentimentalized. They were
seen as pure, unspoiled, and even ‘heavenly’. Wordsworth’s ‘Heaven lies about
us in our infancy’ succinctly expresses this view (Stassijns & van Strijten, 2004,
pp. 138, 139). Moreover, the critical importance of early childhood for lifetime
development was canonized. Again, Wordsworth is (too) often quoted: ‘The
Child is father of the Man’ (l.c. pp. 128, 129).

Kessen’s central message is that these cultural foundations of ‘developmental
science’ are generally not recognized as such, i.e., as culturally-historically con-
tingent, and are often even regarded as fundamental laws of nature. As Kessen
argued, in developmental psychology the importance of a harmonious family,
the significant role of mothers, and the decisive role of early experience in the
development of the child are conventionally considered as principles anchored
in the laws of nature, for which researchers seek and find empirical evidence.
And the now commonplace use of ‘developmental science’ can be seen as a
final integration of developmental psychology with ‘science’, and not with the
humanities. As a key qualifying corollary, this view overlooks, or at least tends
to overlook, any and all alternative conceptions of the child and of develop-
ment in different historical epochs and socio-cultural contexts. To Kessen, this
also means that progress in developmental psychology, i.e., seeking deeper
understanding of human development, cannot be achieved by collecting more
and more empirical data but, also and as important, has to entail analyzing the

632 W. KOOPS AND F. KESSEL

basic principles and processes of the field from a cultural-historical perspective
(again, as the title of his essay so precisely states: ‘The [American] Child and
Other Cultural Inventions’.

To provide a preview of the final sections below – Working within such a phil-
osophical framework, the authors of the papers in this special issue are focused
on a particular task: Providing various insightful illustrations of a critical cultur-
al-historical approach to the (‘fundamentals’ of ) developmental psychology.

3. Positivistic and empirical-analytical approaches

As signaled above, in Kessen’s view the positivist program was and is funda-
mentally misguided; he therefore criticized ‘positivistic dreaming’, viz., the
assumption that empirical-experimental (human and social) science will yield
fundamental facts and universal, timeless principles. Given that the terms
positivism and positivistic have long been used by critics of ‘normal science’,
examining them further is warranted. In the next subsection, the origin and
meaning of the concept of positivism will be discussed, as well as a number of
its characteristics that have been subject to criticism and that, therefore connect
to the core of Kessen’s position.

As a prefatory note, it is important to underline that the assumptions and
goals of positivism can be rejected without implying an opposition to empir-
ical research (as some movements in psychology and pedagogy antithetical
to empirical science have asserted). Thus neither Kessen nor we are opposed
to systematic and reflective empirical research in (developmental) psychology
and pedagogy. On the contrary, whenever possible, and while acknowledging
limitations on the possibility of data-collection and certainty of related interpre-
tation, researchers should analyze and reflect on their theoretical assumptions
in the context of the most systematic available empirical data. Such a principled
stance of vulnerability, or humility, regarding the limits of scientific knowledge
– which can be traced back, among others, to the philosopher of science Karl
Popper (1902–1994) – will be referred to as an empirical-analytical approach.

3.1. The positivist approach

Positivism is the notion that only the empirical sciences can yield valid knowl-
edge. Positivism asserts that science is solely based on empirical facts and rejects
all metaphysical assertions and assumptions. The classical positivism from the
nineteenth century merges with the belief in the progress of the Enlightenment,
i.e., progress in science will eventually provide solutions to all possible problems
(Steel, 1989, p. 99). Positivism emerged from the confrontation of philosophy
with the successful modern (physical) sciences and the consequential view
that the certain knowledge of empirical science could not be matched by phi-
losophy (or the humanities in general). Already by the end of the eighteenth

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 633

century philosophy had begun to cede the field of nature to positivistic modern
physics and chemistry. And in the nineteenth century the humanities were also
gradually redefined as empirical sciences. Positivistic sociology appeared first,
promoted by Auguste Comte (1798–1857), considered the ‘father of positivism’
(Bourdeau, 2011); he referred to sociology as ‘social physics’ (Steel, 1989, p. 98).
Comte’s positivistic program for sociology was elaborated by his pupil Émile
Durkheim (1858–1917).

In the case of psychology, the establishment of the first psychology labo-
ratory by Wilhelm Wundt (1832–1920) in Leipzig (in 1879) has conventionally
been identified as the beginning of positivistic, i.e., scientific, experimental psy-
chology (Boring, 1950; as the canonical source). But here it is especially worth
noting that critical historians have demonstrated how such accounts com-
pletely neglected the other deep dimension of Wundt’s work (1900–1920), viz.,
his Völkerpsychologie studies of phenomena such as language, art, myths and
religions, law, culture in historical perspective, and more. (See, e.g., Blumenthal,
1977; Danziger, 1979; Leary, 1979) And it is plausible to see the spirit of the ‘other
Wundt’ expressed in the various, emerging-in-the-1990s and now-vibrant forms
of ‘Cultural (Developmental) Psychology’, for example, in the rich, paradigm-de-
fining writings and research of Michael Cole (1996), Barbara Rogoff (2003) and
Richard Shweder (1994). (See also Goodnow, Miller, & Kessel, 1995).

Comte’s positivistic program of principles was described in his writing about
the positive mind (Comte, 1844). That program also demonstrates the strong
belief in progress that was intertwined with Comte’s positivism. He believed, for
instance, that the rise of the modern sciences was a slow but inevitable process.
He formulated the law of the three stages: First, a theoretical stage when the
explanation for phenomena is sought in supernatural powers. This is followed by
humanity entering a metaphysical stage when the world is explained by referring
to abstract principles or essences. Eventually humanity enters a positive stage
when it becomes clear that only empirical science can yield real knowledge. At
this stage knowledge should be considered as accessible to all and relevant for
all daily requirements and needs. Therefore, according to Comte, positivism is
inevitable. The law of the three stages is founded on the presumed positivistic
law of progress and evolution of human thinking. Delanty and Strydom (2003,
p. 14) have extensively described positivism. Assuming access to their writing,
we will now further concentrate here on some criticisms of positivism by way
of a discussion of Popper’s ideas and the debate within the Frankfurter Schule.1

Positivists believe that scientific knowledge originates primarily via induc-
tion: By systematically observing specific perceptible phenomena one induces
general and abstract laws. This, however, is not how it appears to unfold in
scientific reality. A fundamental criticism of such a position came in the form of

1a detailed discussion of all these ideas, and those in the previous section, is contained in Koops (2016),
notably chapter 1.

634 W. KOOPS AND F. KESSEL

Popper’s falsificationism (Popper, 1935, 1959) and his classic ‘Even if one has seen
a hundred white swans, the next one could still be black’. According to Popper,
one can only falsify or refute, not ever confirm or prove a posited hypothesis.
So ‘all swans are white’, can only be refuted by encountering a black swan. It
follows then that one must first posit a certain assumption (hypothesis or H1)
and then try to refute it by rejecting the contrary hypothesis (H0). And even if
this succeeds, the basic hypothesis (H1) can only be maintained provisionally,
and never be lifted to the level of absolute knowledge through observation
alone. For Popper, then, there is no pure induction from observation to certain
knowledge.

Popper’s description of this logical asymmetry between verification and
falsification – hence Conjectures and Refutations (1963) – is a key element in
his philosophy of science. It led him to choose falsifiability as the criterion to
distinguish science from non- or pseudo-science: A theory can only be genu-
inely scientific if it is falsifiable. Falsifiability as a demarcation criterion for the
distinction between science and un-science therefore led him reject the claims
to scientific status of both Marxism and psychoanalysis, given that both theories
are not falsifiable.

In summary, Popper made it clear that induction from observation cannot
lead to true, universal knowledge, and that all scientific knowledge is tempo-
rary and provisional. This is a significant qualification of original positivist pre-
tensions. Popper’s analysis also seriously undermined the notion of inevitable
progress. As a consequence, those following Popper’s analysis have little reason
to believe that scientific knowledge automatically advances and improves; they
also do not have any reason to expect that we will ever be able to solve all social
problems purely through knowledge derived via positivistic science. In addition,
classical positivism’s assumption that knowledge can be exhaustive and that
induction one day will have yielded all important certainties about the universe
is, for Popper, an indefensible optimism.

A complementary set of ideas emerged in the 1960s, in the form of a debate
in Germany that became known as the ‘Positivismusstreit’ (the Positivism bat-
tle); this exchange deepened the critique on positivism. (See Adorno, Albert, &
Dahrendorf, 1993; Dahms, 1994) Even though it focused primarily on the meth-
odology and epistemology of sociology, the debate was particularly relevant
to positivist claim that science is value-free. The discussion between Theodor
Adorno (1903–1969) and Popper, in particular, focused on this topic. It is note-
worthy that both agree that the scientific practitioners are always embedded
in cultural history and that their minds, therefore, are pervaded or shaped by
this context. According to Popper, however, the ensuing scientific research is
meant precisely to determine empirical-analytically the tenability of the claims
of the embedded scientific researcher.

For their part, the philosophers of the so-called Frankfurter Schule, with key
representatives such as Adorno and Jürgen Habermas (born in 1929), believed

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 635

that although society as a whole could be analyzed scientifically, value-freedom
was an illusion in all respects. The members of the Frankfurter Schule built on the
theories of Marx, Hegel, and Freud (as noted, all unscientific in Popper’s view).
The fundamental assumption of the was that they could understand the struc-
ture of society as a whole and, in principle, identify the conditions to change
or entirely alter this structure consistent with certain value assumptions. This
approach was thus called Critical Theory. (In the 1960s and 1970s, widespread
social criticism from the democratization movements, particularly at universities,
was inspired by the Frankfurter Schule publications.)

The discussions that emerged during the Positivismusstreit made it clear that
the value-freedom as claimed by positivism should at least be qualified. Such
an analysis underlines that scientific researchers are members of a community
whose work is shaped by the wider society’s values. As a key corollary, these
values play a role in their theoretical and methodological choices and com-
mitments. For his part, Popper thought that falsificationism would provide the
critical means to subject the tenability of theoretical claims to empirical-ana-
lytical test; and that, in turn, could always lead to the refutation of the claims.
And the Frankfurter Schule believed that they could understand and modify
social value patterns and dynamics. However, contrary to classical positivism’s
assumption that objective observations and logical induction guarantee value
freedom, both did not deny – indeed, acknowledged – that values play a role
in science. While Popper aimed to subject value-laden theoretical notions to
scrutiny via falsificationism, the Frankfurther Schule sought to respond ‘critically’
to these values.

As a final note in this section, it is worth emphasizing that the account above
is an abbreviated, selective account of the critique of positivism. Among other
strands (emerging especially in the 1960s): Thomas Kuhn’s analysis of scien-
tific ‘paradigms’ and ‘revolutions’, and Michael Polanyi’s discussion of ‘personal
knowledge’ and ‘the tacit dimension’ (See Kessel, 1969).

3.2. The empirical-analytical approach

Whereas Adorno (1993) invented the term ‘Positivismusstreit’, Popper objected
to the term as he did not want to call himself a positivist. Or, more accurately,
he objected being considered (even) a neo-positivist.

Neo-positivism had originated during the period of the Wiener Kreis. The
Wiener Kreis (1920–1938) referred to a group of philosophers and scientists who
gathered around Moritz Schlick (1862–1936). Key figures included the econ-
omist Otto Neurath (1882–1945), and the philosophers Friedrich Waismann
(1896–1959) and Rudolf Carnap (1891–1970). Often present but not a formal
member of the group, Popper deviated from key points of the logical positiv-
ism or logical empiricism promoted by the group. Rejecting metaphysics and
epistemology as useless, the Wiener Kreis sought to unify science by making

636 W. KOOPS AND F. KESSEL

use of a common scientific language, symbolic logic. Eschewing such a common
core language, Popper described his own approach as critical rationalism. The
term ‘critical’ is rather ambiguous: The Frankfurter Schule uses it to refer to social
criticism; Popper wished to (critically) determine which theories were tenable
and which were not.

Thus Popper criticized both classical positivism and the neo-positivism of the
Wiener Kreis. Contrary to the positivistic tradition, he believed that all science is
partial (and always incomplete); that scientific knowledge does not automati-
cally advance but often has to take apparent detours; and that science cannot
ever produce complete understanding and the solution of all social issues. He
also agreed with the Frankfurter Schule that the theories and hypotheses of a
scientist are can never be value-free in the sense that they are unconnected with
the everyday lived environment (of the scientific community). But contrary to
the Frankfurter Schule, Popper deemed it possible to refute untenable assump-
tions and hypotheses via falsification, not as in the tradition of positivism via
induction. In essence, his falsificationism is an empirical-analytical tool: It aims
to examine and refute hypotheses and predictions by making use of empirical
data.2

Conducting path-breaking empirical-analytical research on infants, Kessen
in fact established his reputation in the American behaviorist tradition, a par-
adigm that more than any other institutionalized positivist assumptions in
mainstream psychology.3 He was thus certainly aware of the importance of the
empirical-analytical tradition. Later in his scholarly career, however, and based
in part on his own study of historical sources (Kessen, 1965), Kessen sought to
understand how the practice of developmental psychology and of theorizing
in general was linked to the cultural history in which it was, and is, embedded.
For him, and us, such self-critical understanding is the only way to preclude
assumptions and views about children that are, as Kessen underlined, variable
across cultural space and historical time being mistakenly considered natural,
universal phenomena.

In a related vein, we suggest that scientific questions can only be answered in
a meaningful way at the level of organization of the phenomenon being studied.
We borrow this conception of ‘level of organization’ from the Dutch psychologist,
Johannes Linschoten (1964). In a distinctive way, he made clear that questions
at one level of organization cannot be answered by data at a different level of
organization, at least not without losing relevance and understanding. As a

2such an empirical-analytical approach remains a core element of the methodology of contemporary psy-
chology. to give an example, in the second half of the twentieth century Dutch (and European) psychol-
ogy was totally redefined by De groot’s classic book (1961; English translation 1969), whose prescriptive
methodology was largely based on the ideas of popper. (see Busato, 2014 for a description of the effect
of De groot’s work.).

3Kessen was not alone in forcefully rejecting psychology’s positivist-behaviorist paradigm in which he made
his early, widely-recognized contributions. sigmund Koch serves as another compelling, still-relevant
example. (see finkelman & Kessel, 1999; leary, 2001).

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 637

corollary, Kessen’s plea for historical understanding of the child and of child
development implies that gathering and analysing only quantifiable data, we
will miss essential understanding of the historical and cultural embeddedness of
child development. Because the complexity and the time scale of historical phe-
nomena call for a different level of organization than that of the individual child
in the here-and-now (to be studied exclusively via experimental approaches),
the study of cultural historical embeddedness of human development requires
different methods and analyses.

In a similar spirit: The anthropologically informed, ethnographically sophisti-
cated developmental psychologist whose research entails making observations
of and conducting conversations with children (and their families) in diverse
cultures seeks to capture those in field notes that, in the best case, serve as
the basis for rich, meaningful narratives about the meanings of child behavior
and experience in such contexts. Such plausible stories cannot be replaced by
exclusive quantitative analysis of isolated variables. This illustrates Lischoten’s
caution: To gain understanding of local knowledge and dynamics at the level of
cultural complexity, research calls for subtle stories and not purely quantitative
analyses and models based on a certain conception of the natural sciences.

The empirical-analytically minded researcher thus tries to find the most pre-
cise answer possible at the level of complexity that defines or represents the
problem focus of the research. Even if that most precise answer is in the form
of a verbal explanation, it is still possible and even necessary to try to test the
tenability of a hypothesis by examining opposing hypotheses (as a non-numer-
ical, verbal form of Popperian falsification).

What we are advocating is best considered ‘methodological liberalism’, which
implies that, in adopting an empirical-analytical approach, the researcher seeks
to collect and fit data at the level of the organization of the object of study, and
that these data are analyzed as accurately as possible. If possible, and where
appropriate, researchers derive and analyse data using numbers/statistics; but
if this is not possible at the given level of organization, or meaningful in the
context of the problem being studied, then the researcher should turn positively
to narratives. As suggested above: At some levels of organisation mathematical
models are not feasible or even desirable, so narrative accounts are preferable.
But even then it is possible to profit from Popper’s falsification ideal: The goal
should always be to search systematically for a convincing, possibly conclusive
argument that is in conflict with the conclusion. It is, for example, acceptable
within modern psychology to test statistically whether a mathematical model
is ‘fitting’. Popper might have regretted this, for such an approach is the oppo-
site of his falsificationism. However, even within such a ‘model-fitting’ research
approach, respect could be paid to Popper’s falsificationism by systematically
exploring alternative, contrasting models.

In summary, even though – humbly echoing Kessen – we are anti-positivists,
we too are not opposed to empirically-oriented developmental psychology.

638 W. KOOPS AND F. KESSEL

Our view is that developmental psychology should be empirical-analytical
in overall methodology while recognizing and accepting that empirical data
cannot, indeed should not, be more exact than appropriate for the level of
organization of the studied phenomenon. Among other things, this means that
research methods can be qualitative or quantitative; that while data collection
and analysis seek to be as exact and detailed as possible, sometimes they will –
indeed should – consist of the systematic interpretation of texts and citations,
other times of statistical data derived via measurements. Indeed, over the past
couple of decades clear signs have emerged that qualitative methods, of various
kinds, are being accepted as both legitimate and important forms of develop-
mental inquiry (Jessor, Colby, & Shweder, 1996; Kessel, 2013; Weisner, 2005) and
psychology more broadly (Bevan & Kessel, 1994; Josselson, 2017; Packer, 2004;
Willig & Rogers, 2008).

4. Principles and history

In the previous sections, drawing on an admittedly shorthand review of some
previous ideas from the philosophy of science that undermined positivism, we
have attempted to delineate some of our ‘pluralistic’ methodological and epis-
temological convictions. As a corollary, we suggest it is still all too simplifying
and seductive to take classical, Newtonian physics as a model for the study of
human and social phenomena. Given different levels of organization (that is,
different levels of complexity of phenomena studied), we need to continue to
develop, adapt, and accept a variety of different methodologies, i.e., many forms
of descriptive-analytic narrative approaches as well as ‘conventional’ experimen-
tal-statistical methods. Nor can there be a simple formula for making insightful
decisions about which method is best suited to the topic, issue, and level of
organization at hand. And more: A critical scientific discipline will strive to be
knowledgeable about its own history and engaged in self-critical reflection on
its foundations.

It is therefore of great and immediate importance that developmental psy-
chology focuses on questions that deal with the foundations and history of the
field. This is more urgent than gathering ever more ‘empirical’ data and adding
ever more ‘empirical’ papers to myriad journals. It is particularly essential to
establish understanding of the discipline’s intellectual and institutional con-
text, rather than only accumulating further fragmented and complicating, or
worse, simplifying knowledge. To that end, in our view critical historical analysis
is essential. As the papers in this special issue make clear, past and present (and
future) empirical research on child behavior and experience is intricately con-
nected with the history of theories and assumptions about children and their
development. Moreover, self-critical knowledge of the history of the discipline
itself can help prevent the reinvention of the wheel or, to adapt another met-
aphor, simply (and misleadingly) placing old wine in shiny new bottles. While

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 639

research practice within the discipline requires defined empirical-analytical
methodology (both qualitative and quantitative), study of the structure and
nature of the discipline requires extended contemplation, analysis, and critical
thinking, especially about its past as embedded in the present and future. (Below
we address the central question of how, where, and by whom such analysis and
thinking are best conducted.)

5. Concluding discussion

The fact that Kessen referred to the child as a ‘cultural invention’ might be taken
to mean he was suggesting that children are not (also) biological beings. We
should realize, however, that, via his title and related analysis, Kessen sought
to provoke self-critical reflection. In effect, he wanted to wake up those he saw
as mainstream and misguided ‘positivistic dreamers’. The child is also a cultural
invention, a product of the ‘Zeitgeist’. Kessen challenges developmental psy-
chologists to be critically aware of their (normative) concepts of the child, con-
cepts which they absorb, reinforce, and reify in their own theories and methods,
influenced by the socio-cultural zeitgeist, and which they are inclined to regard
and present as empirically established, universal, value-free laws of nature.

Consider the example of the decisive role of early experience, an assumption
often adhered to in the nineteenth century, consistent with the sentimentalized
child image of that century. It is at least debatable how decisive early experi-
ences are; but it is comforting to adhere to this view, in part because, as Kagan
(1984) asserted, it dovetails with and reinforces the convictions of the wider
public. He argued that developmental psychologists find it hard to consider
contra-indications seriously and prefer to keep adhering to the notion of early
determinism. For him, defending this absolute early determinism is as intelligent
as Ptolemy ‘proving’ that the earth is the stable center of the universe (Kagan,
1984, p. XI). Of course, in light of many findings since 1984, early experience and
education are undoubtedly important. But Kagan objected, in our view appro-
priately, to overgeneralizing and overvaluing early development as uniquely
causal in determining life course outcomes. In essence, Kagan wanted research-
ers to refrain from uncritically adopting general culturally-shaped assumptions,
concluding that ‘We celebrate empirical science because it corrects pleasing,
but not always accurate, intuitions.’ (l.c., p. X).

It is important that the celebration of empirical science does not lapse into
a positivistic tendency in the sense that we have described here. So, although
this may be a primarily rhetorical point, why is it that it has recently become de
rigueur to emphasize that we are engaged in developmental science? For one
thing, what does that imply about the (ir)relevance of multi-dimensional disci-
plines such as anthropology and history for shedding light on the complexities
of human development in rich and diverse socio-cultural contexts? And on the
same theme: When child developmental inquiry was institutionalized (in North

640 W. KOOPS AND F. KESSEL

America) in the 1920s. It was explicitly seen as a multi- or even inter-disciplinary
endeavor. Only later (primarily in the post-World-War II era) did it come to be
regarded and practiced as developmental psychology; moreover, one specific
conception and form of the discipline, viz., ‘experimental’ and, in essence, posi-
tivistic (Kessel, 2009). Is there now meaningful movement away from that restric-
tive, uni-disciplinary perspective? Perhaps. (See our concluding comments.)

Our view of what empirical-analytical research should be seems to dovetail
with what Kagan proposed when he wrote that ‘The Vienna philosophers went
too far in their accommodation to the new discoveries in physics . . .’ (l.c., p. XIII).
Consistent with what we suggested earlier about the Wiener Kreis, Kagan serves
to illustrate that Kessen’s critical, post-positivist chords are being echoed and
amplified by many (developmental) psychologists. (See Bronfenbrenner, Kessel,
Kessen, & White, 1986; Brown & Cole, 2001; Kessel, 1983) And that includes the
authors of the papers in this issue (as we hope will be evident).

An important closing question is whether Kessen’s critique and the contribu-
tions in this special issue are unduly oriented towards the West, or global North.
While these papers, for practical reasons of manageability, are primarily focused
on trends in Europe and the United States, we suggest that the analyses and
understanding emerging from the study of the history of childhood – exem-
plified by the contributions here – can be generalized to other continents and
regions.

Stearns’ book about Childhood in World History (2006) is an instructive
example. Because, however reluctant and careful he is in generalizing Western
developments to the rest of the world, Stearns nevertheless makes it clear that
contemporary globalization reflects general trends regarding the changing
contexts of child and family life. In particular, the transition from agricultural to
modern societies – such as first occurred after the Enlightenment in Europe and
North America – offers a number of discernible general patterns. For example,
children who no longer contribute to the workforce but go to school to learn
what is now considered relevant for their future functioning in the community
or society; also, children who have fewer siblings and will die at an early age
much less frequently than previously. And while the order and causal relation-
ships between these dynamic changes may differ at different locations in the
world and at different points in time, such patterns do indeed occur worldwide.

As another example, Stearns believes that the emphasis on the striving
for children’s happiness has also become a global tendency which is subtly
interlinked with these other patterns. Thus, the decrease of child mortality and
increase in health care, as well as increased welfare, allows more room for con-
cerns about ‘happiness’ (Stearns, 2010, 2011).

Thus, while the majority of the papers of this special issue are primarily
focused on elements of Western culture, we should emphasize that this is not an
unintended expression of a form of ethnocentrism but, rather, because historical
trends in the West are also unfolding in other parts of the world as a function

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 641

of modernization processes (Koops & de Winter, 2011). Moreover, as powerfully
illustrated by the Keller and Vicedo papers in this issue, a reciprocal process is
underway, where both historical and culturally-informed scholarship prompt a
critical perspective on core, deeply-entrenched mainstream assumptions and
theories. In their case, the focus is on attachment theory and research; a parallel
process is now underway regarding the presumed ‘word gap’, i.e., supposed con-
sequential deficiencies in the verbal environments of poor children, including
those in contexts outside the North America and Europe, compared to their
more affluent peers. (See Avineri & Johnson, 2015; Miller & Sperry, 2012)

The articles collected in this special issue will, we hope, encourage fur-
ther (self-)critical historical research, further contributions to what might
be called Historical Developmental Psychology (Koops & Elder, 1996). That,
in turn, reflects two related and fundamental features of what a generative
post-positivist paradigm entails: First, affirmative recognition that (develop-
mental) psychology is, at root, a ‘moral science’ (White, 1983a), where normative
assumptions are embedded, more or less explicitly, in both theory and practice,
and most fundamentally in the very notion(s) of ‘development’ itself (White,
1983b). And second:

An historical [developmental] psychology involves, … in the first place, an aware-
ness of the historicity of the very norms that are dominant in a given culture or
within a given science at a given time. In the second place, an historical [develop-
mental] psychology takes seriously the variety of culturally and socially operative
factors that go into the very constitution of such norms, whether of childhood,
or of cognitive development [or of social development], and at the same time
an historical [developmental] psychology is aware or critically self-aware of the
status of its own operative norms and methods of inquiry … An historical [devel-
opmental] psychology is thus, necessarily, a normative psychology, not only in
the descriptive sense of studying prevalent or historical norms, but also in the
critical sense of rejecting and proposing norms. (Wartofsky, 1983, p. 189; see also,
Bronfenbrenner et al., 1986, p. 1227)

And this follow-up passage powerfully underlines our overall theme (on the
critical role of historical developmental psychology, and psychologists):

[None of this means] we can’t get started until we all agree on the norms. But it
puts the determination and critique of norms right in the ballpark as a concern
of actual psychological theory and psycho logical practice. [This] doesn’t mean
every psychologist has to do so every day or say it as a little prayer every morning.
But it means that it [the determination and critique of norms becomes an integral
part of the field … and not some after-hours, cracker-barrel stuff you do when the
real psychologists aren’t around … which is its usual status. That is, consideration
of norms has to become integral to the field such that those who are doing it are
in touch with those who aren’t, making them aware of it and so percolating the
field. (op cit., p. 219)

All of which, we believe, both reinforces and honors Bill Kessen’s plea for a cul-
tural-historical approach to the study of child and human development (Kessel,
1991).

642 W. KOOPS AND F. KESSEL

6. Brief overview of the papers in this issue

We are pleased to record that this special issue originated in Ann Arbor,
Michigan on 12–14 May 2016 in a symposium on ‘Historical Perspectives on
Child Development: Implications for Future Research’ organized by then-still-
in-place History Committee of the Society for Research in Child Development
(SRCD). In essence the following papers are the written, final versions of the
symposium presentations by members of the Committee and other guests.
Appreciative of the Society’s support for that event, we conclude this introduc-
tion with the following Socratically-intended comments.

At least in its current stated strategic goals (Sherrod, 2016), SRCD – perhaps
the most influential organization of its kind – is signaling commitments to some
of the suggestions we have made here. As a noteworthy example, it has declared
the goal of returning to its, and the field’s, multi/inter-disciplinary roots. Similarly,
there is announced emphasis on seeking understanding of the cultural and con-
textual dimensions of human development, with a presumed greater receptivity
to qualitative forms of inquiry still more widely practiced in disciplines such
as cultural and linguistic anthropology and history; also, a recent, loosely-re-
lated focus on ‘social justice’. And there are some signs of a recognition that
relationships, both institutional and intellectual, between the global north and
south need to become more reciprocal, where theories, methods and findings
are open to critique and fundamental revision from outside–the-mainstream
perspectives. (See, e.g., Dawes, 2016; Kessel & Lukowski, 2016; Verma, 2016)

How will these commitments be institutionally enacted, most notably in the
planning and review processes for both SRCD’s highly sought-after biennial
meetings and its visible, high-impact journals? For a range of reasons, including
the understandable inertial dynamics of large professional-scientific organi-
zations, our view is that only time will tell how deep and consequential such
projected philosophical and substantive shifts will be.

In particular, and again for a combination of reasons, we are skeptical –
though hopeful – that the Society and the field as a widespread whole will create
significant space for the kind of critical historical-developmental scholarship that
these papers so persuasively represent. Such space would signal recognition, for
example, that twenty-first Century ‘history’ is far from being a single, stand-alone
‘discipline’ but, instead, variously engages issues at the intersection of culture,
society, politics and both epistemology and moral philosophy.

So will we get to the stage when, in Wartofsky’s perennially challenging
words, ‘[the determination and critique] of norms [are] integral to the field such
that those who are doing it are in touch with those who aren’t, making them
aware of it and so percolating the field’? Seeing such self-critical engagement
with normative and moral (and political) questions as akin to a final post-pos-
itivist frontier for developmental inquiry (at least in our professional lifetimes),
and inspired by these papers, we will strive to keep hope alive!

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 643

Consistent with the main theme of this collection, Steven Mintz (Why history
matters: Placing infant and child development in historical perspective) under-
scores the importance of systematic knowledge about how conceptions of
childhood vary across social time and space. In the process, he demonstrates
how such understanding helps rebut myths, undercuts linear views of progress
(and thus ‘development’), and sheds light on often misunderstood long-term
trends and processes.

With a focus on changing social views and standards regarding children and
emotions, Peter Stearns (Children and emotions history) provides a detailed case
study of the overall themes underlined by Mintz. He examines, in socio-cul-
tural context, two major changes in American approaches (around 1800 and
in the 1920s), as well as possible explanations for shifts in specific emotions
(e.g., happiness and shame). In a more ‘meta’ mode, he considers complexities
in discussing changes or continuities in children’s emotions, and reflects on the
possibilities for connections between historical and psychological approaches.

As a complement to Stearns’ paper, Paul Harris (Emotion, imagination and
the world’s furniture) considers how a particular conception of emotions that
emerged in the late nineteenth century was uncritically embedded in the
work of subsequent generations of psychologists (at least in Europe and North
America). Highlighting that conception’s heavily evolutionary-biological under-
pinnings, he then examines two species-specific qualities of human emotions,
where culture and imagination are central. Finally, he suggests that such a frame-
work would be generative in considering connections between the history of
emotions and their development in children.

Focusing on how John Bowlby’s and Mary Ainsworth’s ethological theory of
attachment was received in different disciplinary communities, Marga Vicedo
(Putting attachment in its place: Disciplinary and cultural contexts) makes two essen-
tial points: From Margaret Mead’s 1950s anthropological-critique onwards, cultural
challenges to the theory’s central assumptions (e.g., of universality) have been
essentially ignored. And second, that such a (dys)functional dynamic can be best
explained in terms of different disciplinary paradigms – philosophical and meth-
odological, positivist and not. She thus provides an insightful illustration of some of
the themes presented above, not least how fine-grained historical-archival research
can shed illuminating light on major areas of ‘mainstream developmental science’.

Heidi Keller (Cultural and historical diversity in early relationship formation)
complements both Vicedo’s paper and, again, parts of the ‘post-positivist’ view
presented above. Drawing on the work of cultural psychologists and anthropolo-
gists, as well as context-attuned developmental psychologists (herself included),
she reviews how caregiving/socialization beliefs and patterns vary, consequen-
tially, across changing sociocultural environments and historical time. She
concludes with reflections on how the often-assumed contradiction between
cultural-historical specificity and universality, e.g., in the realm of attachment
and overall early relationship formation, can be overcome. As a corollary, she

644 W. KOOPS AND F. KESSEL

too illustrates how historical and developmental analysis can and should be
two sides of the same analytic coin.

Lassonde’s focus (Authority, disciplinary intimacy & parenting in middle-class
America) is on historical shifts, from the mid-19th-century through to the pres-
ent, in beliefs and childcare advice regarding appropriate styles of parenting.
Examining such topics as views regarding corporal punishment and its pre-
sumed link to authoritarianism, his analysis emphasizes the importance of
understanding changing dynamics in wider socio-cultural-political contexts,
not least for critically locating the contingent views of developmental ‘experts’
(during various periods). And although the patterns he discerns are based in
the United States, that sort of analytic principle and goal is no less important for
work elsewhere, i.e., historical-developmental research (on ‘parenting’) that can
yield a picture of differences and similarities across time and space(s).

As a broad complement to Lassonde’s paper, Sandin (The parent: A cultural
invention. The politics of parenting) explores how – in Sweden in the twentieth
century – beliefs, practices, and policies regarding the relative roles of parents
and social institutions in fostering and protecting children were significantly
shaped by the philosophy of the wider welfare state. He analyzes how govern-
ment policies and (interventionist) practices both reflected and reified assump-
tions, for example, about limited parental responsibility, especially in relation
to children seen as competent, individual agents with adult-like rights; such
assumptions were, in turn, tied to certain notions of ‘developmental well-being’
and reinforced by certain international conventions.

In the end, then, the harmonious sounding of some distinctly Bill Kessen-like
themes!

Disclosure statement

No potential conflict of interest was reported by the authors.

References

Adorno, T. W., Albert, H., & Dahrendorf, R. (1993). Der Positivismusstreit in der deutschen
Soziologie. München: DTV.

Avineri, N., & Johnson, E. (2015). Introduction to “Bridging the Word Gap”. Linguistic
Anthropology, 25, 67–68.

Bevan, W., & Kessel, F. S. (1994). Plain truths and home cooking: Notes on the making and
remaking of psychology. American Psychologist, 49, 505–509.

Blumenthal, A. L. (1977). Wilhelm Wundt and early American psychology” A clash of two
cultures. Annals of the New York Academy of Sciences, 291, 13–20.

Boring, E. G. (1950). A history of experimental psychology. New York, NY: Appleton.
Bourdeau, M. (2011). Auguste comte. In E. N. Zalta (Ed.), The Stanford encyclopedia of

philosophy. Stanford: Stanford University.
Bronfenbrenner, U., Kessel, F., Kessen, W., & White, S. (1986). Toward a critical social history

of developmental psychology: A propaedeutic discussion. American Psychologist, 41,
1218–1230.

EUROPEAN JOURNAL OF DEVELOPMENTAL PSYCHOLOGY 645

Brown, K., & Cole, M. (2001). A Utopian methodology as a tool for cultural and critical
psychologies: Toward a positive critical theory. In M. J. Packer, & M. B. Tappan (Eds.),
Cultural and critical perspectives on human development (pp. 41–66). New York, NY:
SUNY Press.

Busato, V. (2014). A.D. de Groot(1914-2006): Meester in de psychologie. In V. Busato, M.
van Essen, & W. Koops (Red.), Van fenomenologie naar empirisch analytische psychologie
(249–314). Amsterdam: Bert Bakker.

Cole, M. (1996). Cultural psychology: A once and future discipline. Cambridge: Harvard
University Press.

Comte, A. (1844). Discours sur l’esprit positif. Paris: Pain et Thunot.
Dahms, H.-J. (1994). Positivismusstreit. Die Auseinandersetzungen der Frankfurter Schule

mit dem logischen Positivismus, dem amerikanischen Pragmatismus und dem Kritíschen
Rationalismus. Frankfurt am Main: Suhrkamp.

Danziger, K. (1979). The positivist repudiation of Wundt. Journal of the History of the
Behavioral Sciences, 15, 205–230.

Dawes, A. (2016). The interface of rights and knowledge? Developments (SRCD Newsletter).,
59, 4–5.

Delanty, G., & Strydom, P. (2003). Philosophies of social science: The classic and contemporary
readings. London: Open University.

Finkelman, D., & Kessel, F. S. (Eds.). (1999). Sigmund Koch: Psychology in human context
– Essays in dissidence and reconstruction. Chicago, IL: The University of Chicago Press.

Goodnow, J., Miller, P., & Kessel, F. S. (Eds.). (1995). Cultural practices as contexts for
development. New directions in child development, No. 67. San Francisco, CA: Jossey Bass.

de Groot, A. D. (1961). Methodologie. Grondslagen van onderoek en denken in de
gedragswetenschappen. Amsterdam: Noordhollandse Uitgeversmaatschappij.

de Groot, A. D. (1969). Methodology. Foundations of inference and research in the behavioral
sciences. Den Haag: Mouton.

Jessor, R., Colby, A., & Shweder, R. A. (Eds.). (1996). Ethnography and human development:
Context and meaning in social inquiry. Chicago, IL: The University of Chicago Press.

Josselson, R. (2017). Editorial. Qualitative Psychology, 4(1), 1.
Kagan, J. (1984). The nature of the child. New York, NY: Basic Books.
Kessel, F. S. (1969). The philosophy of science as proclaimed and science as practiced:

“Identity” or “dualism”? American Psychologist, 24, 999–1005.
Kessel, F. S. (1983). On cultural construction and reconstruction in psychology: Voices in

conversation. In F. S. Kessel & A. W. Siegel (Eds.), The child and other cultural inventions.
New York, NY: Praeger.

Kessel, F. S. (1991). Vision, scholarship, and the child. In F. S. Kessel, M. H. Bornstein, & A.
J. Sameroff (Eds.), Contemporary constructions of the child: Essays in honor of William
Kessen. Hillsdale, NJ: Erlbaum.

Kessel, F. S. (2009). Research on child development: Historical perspectives. In R. A.
Shweder, et al. (Eds.), The child: An encyclopedic companion. Chicago, IL: The University
of Chicago Press.

Kessel, F. S. (2013). On the qualities, and quality, of qualitative (developmental) research:
Some semi-random reflections. ISSBD Bulletin, Serial No., 64, 6–8.

Kessel, F. S., & Lukowski, A. (2016). Global perspectives. Developments (SRCD Newsletter).,
59(1), 1.

Kessel, F. S., & Siegel, A. W. (Eds.). (1983). The child and other cultural inventions. New York,
NY: Praeger.

Kessen, W. (1965). The child. New York, NY: Wiley.
Kessen, W. (1979). The American child and other cultural inventions. American Psychologist,

34, 815–820.

646 W. KOOPS AND F. KESSEL

Kessen, W. (1983). The child and other cultural inventions. In F. S. Kessel, & A. W. Siegel
(Eds.), The child and other cultural inventions (pp. 26–39). New York, NY: Praeger.

Koops, W. (2016). Een beeld van een kind. Amsterdam: Boom (Images of children).
Koops, W., & de Winter, M. (Red.). (2011). Wereldwijd opvoeden (Global education).

Amsterdam: SWP.
Koops, W., & Elder, G. H., Jr. (1996). Historical developmental psychology: some

introductory remarks. International Journal of Behavioral Development, 19, 369–371.
Leary, D. E. (1979). Wundt and after: Psychology’s shifting relations with the natural

sciences, social sciences, and philosophy. Journal of the History of the Behavioral
Sciences, 15, 231–241.

Leary, D. E. (2001). One big idea, one ultimate concern: Sigmund Koch’s critique of
psychology and hope for the future. American Psychologist, 56, 425–432.

Linschoten, J. (1964). Idolen van de psycholoog (Idols of Psychologists). Utrecht: Bijleveld.
Miller, P. J., & Sperry, D. E. (2012). Déjà vu: The continuing misrecognition of low-income

children’s verbal abilities. In H. R. Markus, & S. Fiske (Eds.), Facing social class: How
societal rank influences interaction. New York, NY: Russell Sage Foundation.

Packer, M. (2004). Editorial. Qualitative Research in Psychology, 1, 265.
Popper, K. (1935). Logik der Forschung. Wenen: Julius Springer Verlag.
Popper, K. (1959). The logic of scientific discovery. London: Hutchinson.
Popper, K. (1963). Conjectures and refutations: The growth of scientific knowledge. London:

Routledge.
Rogoff, B. (2003). The cultural. Nature of human development. New York, NY: Oxford

University Press.
Sherrod, L. (2016). SRCD in the 21st century. Developments (SRCD Newsletter)., 59(3), 1–2.
Shweder, R. A. (1994). Thinking through cultures: Expeditions in cultural psychology.

Cambridge: Harvard University Press.
Stassijns, K., & van Strijten, J. (Red.). (2004). De mooiste van William Wordsworth [The most

beautiful poems of William Wordsworth]. Amsterdam: Lannoo en Atlas.
Stearns, P. N. (2006). Childhood in world history. New York, NY: Routledge.
Stearns, P. N. (2010). Defining happy childhoods: Assessing a recent change. The Journal

of the History of Childhood and Youth, 3, 165–186.
Steel, C. (1989). Historische inleiding tot de Wijsbegeerte [Historical introduction to

Philosophy]. Leuven: Universitaire Pers Leuven.
Verma, S. (2016). A coalition with a truly international agenda. Developments (SRCD

Newsletter)., 59, 2–3.
Wartofsky, M. (1983). The child’s construction of the world and the world’s construction

of the child: From historical epistemology to historical psychology. In F. S. Kessel, & A.
W. Siegel (Eds.). (1983). The child and other cultural inventions. New York, NY: Praeger.

Weisner, T. S. (Ed.). (2005). Discovering successful pathways in children’s development: Mixed
methods in the study of childhood and family life. Chicago, IL: The University of Chicago
Press.

White, S. H. (1983a). Psychology as a moral science. In F. S. Kessel, & A. W. Siegel (Eds.),
The child and other cultural inventions. New York, NY: Praeger.

White, S. H. (1983b). The idea of development in developmental psychology. In R. M.
Lerner (Ed.), Developmental psychology: Historical and philosophical perspectives.
Hillsdale, NJ: Erlbaum.

Willig, C., & Rogers, W. S. (2008). Handbook of qualitative methods in psychology. Thousand
Oaks, CA: Sage.

Wundt, W. M. (1900-1920). Völkerpsychologie. Bnd. 1-2: Die Sprache(1900); Bnd 3: Die Kunst
(1908); Bnd. 4-5-6: Mythos und Religion (1910-1914); Bnd. 7-8: Die Gesellschaft (1917);
Bnd. 9: Das Recht (1918); Bnd. 10: Kultur in der Geschichte (1920). Leipzig: Engelmann.

Copyright of European Journal of Developmental Psychology is the property of Taylor &
Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder’s express written permission. However, users may print,
download, or email articles for individual use.

Abstract

1. Introduction
2. Kessen’s plea
3. Positivistic and empirical-analytical approaches
3.1. The positivist approach
3.2. The empirical-analytical approach
4. Principles and history
5. Concluding discussion
6. Brief overview of the papers in this issue
Disclosure statement
References

D
ow

nl
oa

de
d

fr
om

h
tt

ps
:/

/r
oy

al
so

ci
et

yp
ub

li
sh

in
g.

or
g/

o
n

09
F

eb
ru

ar
y

20
2

rstb.royalsocietypublishing.org

Research
Cite this article: Kline MA, Shamsudheen R,
Broesch T. 2018 Variation is the universal:

making cultural evolution work in

developmental psychology. Phil. Trans. R. Soc.

B 373: 20170059.
http://dx.doi.org/10.1098/rstb.2017.0059

Accepted: 12 December 201

One contribution of 16 to a theme issue

‘Bridging cultural gaps: interdisciplinary studies

in human cultural evolution’.

Subject Areas:
cognition, evolution, developmental biology,

behaviour

Keywords:
cultural evolution, developmental psychology,

cross-cultural psychology, ethnocentrism,

evolution and human behaviour

Author for correspondence:
Michelle Ann Kline

e-mail: michelle.ann.kline@gmail.com

†The second and third authors contributed

equally to this manuscript.

Variation is the universal: making cultural
evolution work in developmental
psychology

Michelle Ann Kline1,2, Rubeena Shamsudheen3,† and Tanya Broesch1,†

1Department of Psychology, Simon Fraser University, Burnaby, BC, Canada V5A 1S

2Institute of Human Origins, Arizona State University, Tempe, AZ 85287-4101, USA
3Department of Cognitive Science, Central European University, Nador u. 9, 1051 Budapest, Hungary

MAK, 0000-0002-1998-692

Culture is a human universal, yet it is a source of variation in human psy-

chology, behaviour and development. Developmental researchers are now

expanding the geographical scope of research to include populations beyond

relatively wealthy Western communities. However, culture and context still

play a secondary role in the theoretical grounding of developmental psychol-

ogy research, far too often. In this paper, we highlight four false assumptions

that are common in psychology, and that detract from the quality of both stan-

dard and cross-cultural research in development. These assumptions are: (i) the
universality assumption, that empirical uniformity is evidence for universality,
while any variation is evidence for culturally derived variation; (ii) the Western
centrality assumption, that Western populations represent a normal and/or
healthy standard against which development in all societies can be compared;

(iii) the deficit assumption, that population-level differences in developmental
timing or outcomes are necessarily due to something lacking among non-Wes-

tern populations; and (iv) the equivalency assumption, that using identical
research methods will necessarily produce equivalent and externally valid

data, across disparate cultural contexts. For each assumption, we draw on cul-

tural evolutionary theory to critique and replace the assumption with a

theoretically grounded approach to culture in development. We support

these suggestions with positive examples drawn from research in development.

Finally, we conclude with a call for researchers to take reasonable steps towards

more fully incorporating culture and context into studies of development, by

expanding their participant pools in strategic ways. This will lead to a more

inclusive and therefore more accurate description of human development.

This article is part of the theme issue ‘Bridging cultural gaps: interdisci-

plinary studies in human cultural evolution’.

1. Human development requires culture
Humans stand out among other animals because we adapt to new environments

both by being clever innovators [1] and through the accumulation of cultural

knowledge across generations [2,3]. Social learning, including intensive forms

such as teaching [4 – 6], can facilitate cumulative cultural evolution. In fact, low-

cost social learning mechanisms, as well as sources of innovation, are prerequisites

for the evolution of cumulative culture. For this reason, social learning mechan-

isms are central to the understanding of cultural evolution—and cultural

evolution is key to explaining why and how human ontogeny is so very flexible.

Culture is a human universal: all societies have shared knowledge, practices,

beliefs and rituals that are transmitted socially. At the same time, culture is also a

source of psychological and behavioural variation both within and across popu-

lations. Developmental processes that are sensitive to socio-environmental

influences are one way that flexibility can evolve [7,8], and evolution can produce

developmental processes that vary in adaptive ways in terms of the degree and

nature of their flexibility [9]. Elaborating on the relationship between culture

http://crossmark.crossref.org/dialog/?doi=10.1098/rstb.2017.0059&domain=pdf&date_stamp=2018-02-12

http://dx.doi.org/10.1098/rstb/373/174

http://dx.doi.org/10.1098/rstb/373/1743

mailto:michelle.ann.kline@gmail.com

http://orcid.org/

http://orcid.org/0000-0002-1998-6928

rstb.royalso

D
ow

nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y

20
22

and development first requires recognizing that evolution and

development are not mutually exclusive, then building on that

insight to explore how evolved developmental mechanisms

that are sensitive to cultural influence can create psychological

and behavioural variation across and within societies [8].

cietypublishing.org
Phil.Trans.R.Soc.B

373:20170059

2. Developmental psychology requires a
culturally diverse sample

Despite the importance of culture to development, develop-

mental psychology as a field retains a near-absolute focus on

development in relatively wealthy Western, English-speaking

populations. Henrich et al. [9] term general psychology’s par-
ticipant pool ‘WEIRD:’ Western, educated, industrialized, rich

and democratic. A recent review provides evidence that this

is also the case in leading developmental psychology journals:

more than 90% of study populations represented there are from

the USA, Europe and/or are English-speaking [10]. The rest of

the world is vastly underrepresented, with only approximately

7% of participant populations coming from non-Western

human populations (the remainder are non-human animal

populations). In this context, developmental psychologists

who pursue cross-cultural research are wisely expanding

the scope of research to include participants beyond predomi-

nantly Western, upper middle class and often ethnically

white participants [9,11,12]. We applaud these efforts—

anything less would only perpetuate an incomplete and

inaccurate picture of human development.

Poor sampling, however, is not the only problem in the

field. Arnett [11], and Meadon & Spurrett [13] address a lack

of inclusivity in the broader practice of psychology: theories,

studies and publications in the American Psychological Associ-

ation journals are all overwhelmingly created, reviewed and

edited by this same subset of the world’s population. This is

one reason why the sampling problem in developmental

psychology is not likely to be solved by laboratory-based

researchers making the decision to take on cross-cultural

work unilaterally, in the short term. Dropping in on commu-

nities with unfamiliar cultures to run brief, one-off studies

without a long-term reciprocal relationship with the commu-

nity can be ethically dubious [14], especially where there is a

power differential. Further, interpreting results in isolation

from a population’s daily cultural context can produce more

confusion than answers [15]. And yet avoiding these pitfalls

requires investing what can be a prohibitive amount of time,

effort and funding to start and maintain a field site. A more

plausible way to ameliorate psychology’s WEIRD problem is

to recruit, support, include and collaborate with more scientists

from beyond the WEIRD populations that have created the bias

in the first place [11,13]. Alternatively, researchers can work

with non-university populations nearby, to explore variation

among people in their own local context [14]. More generally,

researchers who study WEIRD populations must also recog-

nize that their populations are also influenced by culture and

should consider carefully how to define the specific population

from which they recruit participants. Both these strategies fit

with a broader, theoretically motivated approach to expand

the inclusiveness of sampling in developmental psychology.

This paper aims to show why developmental psychology

needs this change, and establish some guidelines for how to

study culture’s role in development, no matter how near or

far from home the study site may be.

3. Cultural evolution can motivate a better
science of developmental psychology

Cross-cultural data are expensive to get, but valuable to have.

Their rarity in developmental psychology is due to more

than a lack of interest in cross-cultural sampling, and we

cannot dissolve those very real barriers in this paper. Instead,

our goals in this paper are twofold. First, we aim to convince

researchers in the field of developmental psychology that con-

siderations of culture are relevant to their work, even if they

do not do far-flung fieldwork themselves. Second, for cross-

cultural developmental psychologists, we aim to leverage

cultural evolutionary theory to enrich the central role of

cross-cultural data to developmental psychology as a field.

To achieve these aims, we highlight four common but false

assumptions in present-day approaches to cultural variation

in developmental psychology, and critique each in turn by

drawing on cultural evolutionary theory and empirical find-

ings. This step of identifying and refuting these assumptions

will help to integrate the ‘cross-cultural’ niche within develop-

mental psychology, in general, by demonstrating how culture

and culture-based assumptions underlie some of the basic

ideas that motivate research in developmental psychology.

Those assumptions are that: (i) universality and uniformity

are equivalent: that what is universal must necessarily follow

a uniform pattern of development; (ii) Western populations

are central in human psychology; (iii) differences among

populations in development are always indicative of deficits;

(iv) methods can automatically be transported across cultural

contexts and yet maintain validity. We critique each assump-

tion in turn, by drawing both on cultural evolutionary theory

and on positive examples from the developmental psychology

literature. In our conclusion section (§8), we summarize a gen-

eral strategy for research that eschews these assumptions, and

argue that this approach can pave the way for an improved

science of developmental psychology by placing the cultural

nature of humans at its centre.

4. Problem no. 1: the universality as uniformity
assumption

The universality assumption is the belief that observed uniformity
is evidence for species-wide, biologically based universality. By

contrast, any variation is regarded as evidence for culturally

derived differences. By ‘universal’, we mean core mental or

behavioural attributes shared by humans everywhere [16].

This assumption sometimes takes the form of an explicit

claim that uniformity implies genetic underpinnings (often mis-

categorized as ‘biological’ or ‘evolutionary’), while variation

necessarily indicates ‘cultural’ influences [17]. In all its forms,

this assumption rests on the false nature/nurture dichotomy,

that culture and biology are separate, opposite and competing

explanations. In reality, human cultural capacities are part of

our biology [18,19]. Equating psychological or behavioural vari-

ation with cultural influence precludes a deeper understanding

of human behaviour, because a universally shared develop-

mental process can function to produce behavioural or

psychological variation. Instead, developmental flexibility and

culture are both parts of the biology of human development,

not alternative explanations—culture is a part of human

biology and development [8].

rstb.royalsocietypublishing.org
Phil.Trans.R.Soc.B

373:20170059

3
D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

This false dichotomy between nature and nurture pro-

duces two versions of the universality as uniformity

assumption: (a) that variation is equivalent to a lack of uni-

versality, and that (b) psychological/behavioural similarity

is equivalent to universality. For the sake of clarity, we

address each in turn.

(a) Variation equals cultural influence and lack
of universality

This assumption is often implicit in data analysis and study

interpretation. For example, researchers conduct cross-site com-

parisons and conclude that any between-site difference is

‘cultural’, without explaining how culture produces differences

in psychology and behaviour. In addition, researchers often

treat whole cultures as if they are a single experimental con-

dition, without considering the influence of environmental

factors, such as resource availability, wealth or differences in

the interpretation of the method (see §6 below). For example,

directly comparing norms for anonymous sharing among

wealthy Americans with those among poor, food-insecure

Polynesian populations may result in differences—but those

differences may be due to circumstances specific to resource

scarcity, rather than some underspecified aspect of culture.

This line of reasoning is not considered sufficient for studies

of culture in other animals, and leads to energetic debates

about sources of behavioural variation even in our closest

living relatives (e.g. [20 – 22]). However, the same logic is

rarely questioned in cross-cultural comparisons of human

psychology. While cross-cultural comparisons do contribute

to our knowledge of the range of variation in human behaviour,

most fall short of understanding the sources and the scale of

variation that can emerge via developmental processes—the

real question at hand.

(b) Uniformity equals genetic roots and lack of cultural
influence

The other side of the universality assumption consists of a belief

that uniformity in behaviour and psychology is indicative of

universally ‘innate’ traits that develop without cultural inputs.

When developmental psychologists ask whether a feature

is innate, and then seek to show that it emerges early and

reliably across human populations, they rely upon assump-

tions that equate sameness, universality and innateness. By

contrast, biologists have recognized notions of innateness as

useless in ecology, biology and behaviour since the early

1990s [19]. This rests on a recognition, as Barrett [8, p.157]

writes, that ‘. . .[t]here are not two kinds of things, the innate

and the non-innate, but only one, the developmental process

itself.’ Put simply, genes rely upon the environment in order

to create an organism, and vice versa. In humans, culture is

part of that ever-present environment.

describe a general human psychology in these terms, have

long been a driving philosophy in American psychology

[11,16,23]. While valuable as a first pass, documenting simi-

larities across sociocultural contexts is a subpar strategy for

data collection when the goal is to understand culture’s role

in shaping development, or vice versa. Cultural evolutionary

theory offers an alternative perspective for shaping research

questions: that genes and culture have co-evolved in humans.

Because of this ‘dual-inheritance’ system, both genetic and cul-

tural information are essential ingredients in any explanation

of human biology. Most developmental psychologists would

not argue with this stance, but putting it into action in a

research programme is still a challenge. Cultural evolutionary

theory is useful in this practical sense, because it provides a

working definition of culture that can inform quantitative

work: ‘[c]ulture is information capable of affecting individuals’

behaviour, that they acquire from other members of their

species through teaching, imitation, and other forms of social

transmission’ [19, p. 5].

Cultural evolution’s distinction of culture as socially

learned information is useful as a research tool because it

means developmental psychologists need not ask whether

any particular trait is universal, biological and innate, versus

cultural. When biology and culture are not opposites, this

either/or is a meaningless, and therefore unanswerable,

question. Instead, developmental psychology can embrace a

transformed question: what is the relative influence of environ-

mental, cultural and other contextual factors on shaping

development of specific traits, in particular population? In

other words, how variable and flexible is the development of

this trait? Answering this context-rich question through studies

that theorize about the functional role of variation will produce

a body of evidence on how human psychological development

varies. From this, researchers can build a more complete map

of human psychological development.

This view, rooted in cultural evolutionary theory, places

flexibility at the centre of understanding what is universal

about human psychological development. This provides a

theoretically motivated way to predict when and how culture

ought to impact development, rather than simply checking

Western-based work against non-Western populations and

lumping traits that are the ‘same’ as universal, and those that

are ‘different’ as cultural.

(d) Developmental research case study
Studies of human language acquisition and socialization pro-

vide evidence for both variation in a cultural context, and

shared developmental processes. Geographically and culturally

disparate populations typically speak different languages, and

in some cases even show variation in the neurological under-

pinnings necessary to master and use different languages [24].

The cultural expectations for children as language learners

are shaped by their cultural contexts, and in some ways are inse-

parable from socialization more generally [25]. Language

acquisition processes illustrate that developmental processes

themselves—such as statistical learning [26]—can constitute

universal learning mechanisms, which in turn generate behav-

ioural and psychological variation. The same can be said for

children’s early learning environments: there are both shared

and variable features, cross-culturally. For example, Broesch &

Bryant [27,28] find that mothers and fathers across disparate

societies routinely modify the properties of their speech when

addressing young infants compared to when they address

adults, yet they do so in different ways [28]. Despite identifying

the existence of infant-directed speech by caregivers in North

America, Kenya, Fiji and Vanuatu, they also find that parents

vary cross-culturally in the form their infant-directed speech

takes. Mothers across diverse societies and rural Vanuatu

rstb.royalsocietypublis

D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

fathers modified their speech by adjusting features of the per-

ceived pitch of their speech to infants. However, fathers in

North America only slowed down the rate of their speech, with-

out adjusting the perceived pitch [27]. The results of this study

demonstrate why researchers cannot simply search for univers-

ality by equating it with similarity: it is too broad a question,

and would lead us to ignore key details about the flexible

nature of developmental processes.

hing.org
Phil.Trans.R.Soc.B
373:20170059

5. Problem no. 2: the Western centrality
assumption

The Western centrality assumption is the belief that Western
populations represent a normal and/or healthy standard

against which development in all societies can and should be

compared. This assumption literally fits the original definition

of ethnocentric [29], in that it divides global populations into
two rough categories, ‘the West’ and ‘the Rest,’ with Western

societies at the centre of everything. This assumption is rarely

if ever made explicit in print, but it is worked into the foun-

dation of much developmental research, including the

cognitive and medical milestones that serve as guidelines for

both Western parents and international health agencies.

(a) Improvements
From a cultural evolutionary perspective, lumping Western and

non-Western societies into two broad categories of analysis is

simply throwing data away. The study of cultural evolution is

necessarily built on the study of the cultural history of societies

all over the world, because explaining cultural variation

requires a breadth of data across socioecological environments

([19]; see e.g. the range of sites included in Mace et al.’s edited
volume [30]). From this perspective, every cultural context is

an equally valid study site, and the importance of a particular

site is down to its specific cultural features and their relevance

to the research question. For example, Polynesia’s history of

step-wise settlement by ocean-faring canoe and its estimable

rates of contact among societies make its cultural history an

excellent case study on how population interconnectedness

can influence the accumulation of complex material culture

[31,32]. The key message from cultural evolutionary theory

here is that these studies stand alone, and do not require a

Western comparison sample to lend them value.

(b) Developmental research case study
The Western centrality assumption directly damages the accu-

racy and usefulness of developmental research. For example,

Karasik et al. [33] review how developmental textbooks and
medical guidelines employ standards for motor development

that are built exclusively on American middle-class samples

as proscriptive milestones. Karasik et al.’s data, drawn from
six different societies, document within- and between-

population variation in both the timing of the motor develop-

ment of sitting, as well as the social and material contexts that

contribute to those differences. This establishes a causal link

between context and developmental trajectories. Karasik et al.
conclude that using American-centric guidelines as if they

are universal has ‘led to a gross misrepresentation of motor

development’ ( p. 1033). Treating Western samples as a univer-

sal measuring stick for development is, unfortunately, a

pervasive practice. Greenfield et al. [34] review evidence that

developmental trajectories derived from the study of Western

populations, with their focus on independence, are unlikely

to match how children learn and grow in sociocultural contexts

where interdependence is prioritized. This is particularly true

for social development. For example, while adolescence may

be a transition to autonomy in independence-focused societies,

in an interdependent society it is instead a relational shift that

makes sense only in the context of kinship and community [34].

Likewise, classic theories of attachment [35] presuppose

that the end goal of child development is independence and

autonomy, rather than locally appropriate integration into

kinship- and community-based interdependent relationships.

In a review, Keller [36] questions whether these theories hold

up when used to explain behaviour in cultural contexts

beyond Western societies, and argues that incorporating data

from additional populations requires revising existing theory

along lines suggested by cultural and evolutionary theories

of development.

6. Problem no. 3: the deficit assumption
The deficit assumption is that population-level differences in
developmental timing or outcomes are necessarily caused by

something lacking, typically in parenting or educational

systems. This line of reasoning allows for no flexibility, and

assumes a single, inflexible developmental outcome. The

assumption rides the coattails of the Western centrality assump-

tion, in that the timeline that establishes ‘normal’ development

from ‘delayed’ development is typically anchored on data from

Western populations. However, the deficit assumption can also

apply to Western populations or subpopulations therein. For

example, Lancy [37] argues that excessive levels of teaching in

Western societies may impinge on the development of a

child’s autonomy, The deficit assumption is also sometimes

applied to subpopulations within Western societies, and so

has recently become an important domain for self-critique in

the field of developmental psychology (see [38]). However,

the deficit assumption differs from the Western centrality

assumption in two important ways. First, the deficit assump-

tion carries an extra layer of interpretation in comparison

to the Western centrality assumption. By this we mean that

researchers simultaneously judge a given pattern in deve-

lopment as deviant and also attribute that difference to

something that is lacking or missing from a family’s or a popu-

lation’s way of raising children. This carries with it a value

judgement that goes beyond a scientific approach to describing

and explaining variation, and in doing so obscures the science

itself. Second, the Western centrality assumption functions

only in one direction. By contrast, the deficit assumption can

lead researchers to claim that Western children are somehow

worse off than non-Western ones. Often this takes the form

of arguing that Western children are coddled, spoiled or

excessively dependent on direct parent intervention.

In assuming that group-level developmental differences

are due to what is lacking in schooling or parenting, research-

ers frequently fail to (a) give any evidence for this mechanism

beyond handwaving that ‘culture’ is the cause, and (b) in

doing so, fail to consider the many specific axes of varia-

tion that comprise between-population differences. When

researchers fail to give a specific cultural mechanism yet attri-

bute differences to ‘culture,’ some of the variation may be due

to situation (e.g. resource insecurity) rather than culturally

rstb.royalsocietypublishing.org
Phil.Trans.R.Soc.B
373:20170059

D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

inherited differences (e.g. collective ownership norms). Where

this is the case, it is a serious challenge to the validity of cross-

cultural comparisons, in that it fails to account for potential

confounding variables. Recognizing and controlling for poten-

tial confounds are accepted as a crucial components of high-

quality research in developmental psychology, with particular

attention to detail in experimental studies. The same standard

should be applied at the level of cross-cultural comparisons.

The risk of neglecting to recognize a confounding variable

decreases with a research team’s expertise in the local context

at their study site. Finally, the deficit assumption reinforces a

deeper-seated assumption, (c) that there is one shared, correct

outcome for various stages of development, and that this does

not vary across populations or across societies.

(a) Improvements
Cultural evolutionary theory instead presents a functionalist

perspective. This means that the focus is on how different

domains of development fit into both physical maturity and

context-dependent social, emotional and relational factors.

This emphasis on function in context is shared with dynamic

systems theories [39], but an evolutionary approach is further

motivated by understanding how developmental processes

have emerged over an evolutionary timespan and in compari-

son to other species. From this perspective, developmental

flexibility, including social learning, is part of what allows

human culture to evolve faster than the human gene pool

[40], and this in turn makes humans adaptable over short time-

scales [2]. (In contrast with dynamic systems theory, the term

‘adapt’ is almost never used in cultural evolutionary theory

to refer to the timescale of a single individual behaving flexibly,

but rather it is a population-level concept.) As a result, psycho-

logical development is pluralistic by design, and this evolved

because flexibility is incredibly useful for a wide-ranging, inva-

sive species like Homo sapiens. Barrett [8] has coined the term
‘designed emergence’ to capture the idea that developmental

processes are flexible as a result of evolution by natural selection.

Simply put, this means there is a range of healthy, functional

outcomes that emerge from developmental processes. Outside

of that range, pathology is still possible, especially in cases of

extreme abuse or neglect that fall outside the breadth of typical

human experience. Specific outcomes are not predetermined by

genes, but are instead shaped by the interaction between genes

and environment in ways that have been manufactured by

natural selection. For developmental psychologists, the take-

home message here is that shared processes of human dev-

elopment have a variety of outcomes, and this flexibility in
outcomes is a feature rather than a bug. Developmental

researchers can leverage this insight to create and evaluate

hypotheses about how the form and developmental timing of

psychological phenomena fit in functional ways with children’s

roles in varying sociocultural contexts.

(b) Developmental case study
For example, psychologists have long assumed that direct,

active teaching (often characterized by the verbal communi-

cation of abstract ideas) is the most efficient way to scaffold

learning, and that therefore it must be present in all human

societies (for review see [6]). By contrast, some anthropologists

have often conflated direct instruction with involuntary, forced

transmission, which replaces more enjoyable and (by this

account) effective forms of learning by participation

([37,41,42]; see [6] for review). For both accounts, at least

some societies have got the wrong answer to how children

learn best—and children in those societies are at a deficit.

Kline [6,43] uses cultural evolutionary theory as a foun-

dation to argue that there are many functionally distinct

types of teaching, which can be mixed and matched with learn-

ing problems. From this perspective, no single type always

provides a ‘best’ outcome for the learner, because it depends

on the learning problem at hand. This approach treats develop-

ment as an integral working part of evolutionary processes,

and prioritizes functional and causal explanations of variation.

This is in contrast with other evolutionary accounts that

explain why humans, and only humans, teach by referring to

constraints in other animals. When successful, a cultural

evolutionary approach uses the rich and culturally specific

interpretations offered by ethnographic research as insights

that can inform broader claims about the evolution and

nature of human developmental psychology. Taking a func-

tionalist, cultural evolutionary perspective offers power for

generating and testing hypotheses in developmental psychol-

ogy by incorporating the full range of human variation into

what developmental psychologists term ‘typical’ development.

7. Problem no. 4: the equivalency assumption
The equivalency assumption is that using identical research
methods, scales or questions will automatically produce equiv-

alent and externally valid data, even across disparate cultural

contexts. Arnett [11] elaborates on this rationale as the predo-

minant philosophy of science in experimental American

psychology: that in the laboratory, it does not matter who the

participants are, or where or how they live—it matters only

that the procedures within the experiment itself are sufficiently

controlled. The equivalency assumption is demonstrably false

when taken to the extreme: written methods must be trans-

lated, and translation inevitably brings up questions of

whether or not there are shared concepts and meanings,

across sociolinguistic contexts. Non-linguistic methods may

avoid the problem of translation, but the question of whether

methods and stimuli map to shared concepts, social context

and expected behaviour across cultural groups is still an impor-

tant one. Such comparisons are only useful when the meaning

of the protocol is comparable across societies [44 – 46]. Further,

assuming equivalency also means that researchers may fail to

account for culturally specific environmental factors in devel-

opment that are either present in WEIRD contexts but not at

their study site, or that are absent in WEIRD contexts and there-

fore may be unrecognized as important factors at their study

sites. For example, while direct verbal instruction may be rare

in many non-Western societies, ethnographic studies of devel-

opment in these contexts reveal a rich, interactive social context

in which learning happens via participant observation and

inclusion of children in everyday activities [37,41,47]. The

social learning mechanisms vary but learning and develop-

mental change happen in all cultural contexts.

(a) Improvements
Cultural evolutionary theory treats the human brain, mind

and behaviour as having evolved in the context of human

interaction with the world, rich with social and cultural

context. Ignoring that this cultural context affects how parti-

cipants understand and respond to methods is particularly

rstb.royalsocietypublishing.org
Phil.Trans.R.Soc.B
373:20170059
6
D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

problematic when transporting methodologies across sociocul-

tural contexts that differ in broad ways [16,44,48,49]. This is a

problem even for developmental psychologists who do not

venture to do cross-cultural work, because it means their

methods and their results may be culture-bound and therefore

limited in ways they have not explored.

The equivalency assumption raises a particularly difficult

challenge for cross-cultural comparisons in developmental

psychology. The standards for experimental control are strin-

gent and technically demanding. For example, effect sizes

and statistical significance for studies with infants can

depend on looking times that differ in terms of milliseconds.

These tasks often require electricity, delicate equipment,

trained personnel and quiet laboratory space to run effec-

tively. However, even a perfectly replicated and controlled

methodology cannot guarantee that participants from two

different sociocultural contexts are interpreting the situation

in similar ways and therefore the behaviours observed may

not be comparable.

As Heine and co-workers [44,50] conclude, there is no

straightforward solution for this broad problem of context-

specific methodological validity. Instead, establishing real

comparability across populations requires more context, not
less—and this means bringing ethnography into the picture

as a standard resource to inform the design and interpretation

of studies in developmental psychology. Cultural evolutionary

research may seem an unlikely resource for addressing this

methodological challenge because the field has no signature

methodology of its own: for example, its studies of learning

biases draw upon established psychological methods, and its

studies of behaviour build on human behavioural ecology

and animal behaviour. The formal mathematical models that

established the field are themselves built on established

models in epidemiology and genetics. The field is so

thoroughly interdisciplinary that some cultural evolutionists

have even proposed a division of labour within cultural evol-

utionary studies that subsumes existing disciplines [51]. We

advocate instead for a mixed-methods approach, deploying

methods in combinations that strategically compensate for

the particular shortcomings of each method, and that are

suitable for the research problem at hand. This is standard

practice in some areas of social science, including the anthropo-

logical sciences, where both qualitative and quantitative data

and analyses are used as needed [52].

(b) Developmental case study
For example, researchers often treat mutual eye gaze between

infant and caretaker as a reliable and stand-alone indicator of

joint attention in the study of infant cognition. However,

Akhtar & Gernsbacher [53] point out that the social role of

eye gaze is variable across cultural contexts, and hence is not

always a reliable indicator of joint attention. North Americans

typically privilege eye contact and verbal interaction as a key

part of parenting [54], but Gusii mothers in Kenya avert their

eyes in response to mutual eye gaze with an excited infant, in

part to keep their babies calm [55]. According to LeVine &

LeVine [55], gaze avoidance by mothers is consistent with

polite behaviour by Gusii adults, where excessive eye contact

is considered rude and sometimes even aggressive. Gaze

avoidance does not mean Gusii mothers are inattentive to

their infants, but rather that they do not use mutual gaze as a

means of establishing joint attention. Instead, they may use

more physical types of interaction—a typical Gusii mother

cosleeps with her infant, breastfeeds on demand and responds

quickly to her infant’s distress. Based on Lancy’s review of the

ethnographic literature on children and childhood [54], the

Gusii approach of using more tactile contact and gestural com-

munication may be more typical around the world than the

North American approach, which emphasizes eye contact

and verbal communication. An excessive focus on eye gaze

as the key element in joint attention (e.g. [56]) may twist the

scientific understanding of joint attention by underestimating

its prevalence in societies where eye gaze is less important

than in North American contexts.

Rather than the narrowly Western-centric cue of eye gaze,

vocal and postural behaviours may represent a more culturally

generalizable set of cues for the study of infant social cognition

[53]. In fact, gestural, postural and vocal cues may play an

important role in Western contexts, but one that is de-empha-

sized in developmental psychology as a reflection of North

American culture. However, the plurality of methodological

approaches suggested by cultural evolutionary theory means

there is another option besides searching for single (or a set

of ) cues that always indicate joint attention, across sociocul-

tural contexts. Instead, researchers should use an array of

cues, designed for particular sociocultural contexts, to compare

the prevalence and behavioural form of joint attention across

human populations. Using identical methods based on cultu-

rally specific cues will produce only superficially comparable

data, and will produce a misleading picture of the ways in

which populations vary.

8. Conclusion
For each assumption above, we offer a shift in perspective that

uses cultural evolutionary theory to pry those assumptions

loose from present-day developmental psychological research.

For standard developmental psychology, this means seeing the

culture-bound nature of the questions, methods and results,

and appropriately characterizing the generalizability of the

research given the limited samples. For cross-cultural develop-

mental psychology, this means guarding against some of the

assumptions that are common in psychology more generally,

and employing cultural evolutionary theory to improve how

cross-cultural research is designed, conducted and interpreted.

Using this approach, researchers can take some small steps to

remediate the sampling problem in developmental psychology.

Researchers working at institutions in WEIRD societies can step

off campus to create more inclusive study by sampling popu-

lations in their towns but beyond campus, and in doing so can

increase the inclusivity of their samples with a moderate level

of investment in community engagement. They can also collab-

orate with and learn from colleagues at institutions outside of

North America and Western Europe, to work with scholars

who are both highly trained academics as well as regional

experts in the societies in which they work and live. We do not

argue that researchers should avoid studying or drawing

comparisons between WEIRD populations and additional

populations around the world. Instead, we argue that carefully

specifying the meanings of cross-cultural studies, using cultural

evolutionary theory, may open up a rich avenue for compara-

tive research. This includes comparisons both within and

between populations, to look for robust relationships between

cultural variation and corresponding psychological, behavioural

rstb.royalsocietypublishing.org
Phil.Trans.R.Soc.B
373:20170059
7
D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

and developmental variation. This kind of data will allow

researchers to study just how flexible human psychological

development may be, because it allows us to ask whether the

same causal relationships hold for development across popu-

lations, or whether the relationships and processes themselves

are flexible. In essence, this approach ties the form of develop-

mental flexibility to the sociocultural and ecological contexts in

which human psychology functions over the lifespan.

Researchers before us have tackled the question of appro-

priate cross-cultural comparisons, with a similar emphasis on

the need for strategic selection of field sites and research

problems (see e.g. [9,16]). In addition to these existing rec-

ommendations, we caution against any approach that treats

entire ‘cultures’ or nations as indivisible wholes that are cultu-

rally, psychologically or behaviourally homogeneous. Rather

than comparing whole ‘cultures,’ researchers should aim to

map variation both within and across populations, along mea-

surable axes of variation. This is especially applicable to broad

cross-site surveys, which often include only coarse measures

of cultural variation (e.g. gross domestic product, Gini

coefficient or years of education), treat single sites as represen-

tative of entire countries, and further conflate those countries

with ‘cultures.’ However, it is equally applicable to studies

restricted to Western populations, where researchers can both

expand the inclusivity of their samples, and be more explicit

about the degrees of variation included in those samples.

Both these practices will lead to better science in developmen-

tal psychology. By placing cultural context—and the flexibility

that it entails—at the centre of this work, researchers will gain a

deeper understanding of the developmental processes that

build human cultural variation.

The overarching message from a cultural evolutionary

perspective is that developmental trajectories and endpoints

can vary due to the human ability to learn flexibly, acquire

information from others, and to recombine socially and

individually learned information in creative ways. Using

this as a springboard, developmental psychologists are well

positioned to explore the developmental mechanisms and

processes by which human children adapt to their local socio-

cultural and environmental contexts. Doing so will mean

shedding light on one of the broadest human universals of

all: variability.

Data accessibility. This article has no additional data.
Authors’ contributions. M.A.K. conceived of and drafted the manuscript.
R.S. and T.B. both made intellectual contributions prior to the manu-
script’s first draft, and made edits and contributions to manuscript
drafts. R.S. and T.B. contributed equally. All the authors approved
the final version of this manuscript.

Competing interests. We declare we have no competing interests.
Funding. This research was made possible through the support of a
grant from the John Templeton Foundation to the Institute of
Human Origins at Arizona State University (no. 14020515). The
opinions expressed in this publication are those of the authors and
do not necessarily reflect the views of the John Templeton
Foundation.

Acknowledgements. We would like to thank Central European Univer-
sity’s Department of Cognitive Science, for inviting the authors to a
Social Mind Institute Workshop, which led to the formation of
some of the early ideas for this paper.

References

1. Pinker S. 2010 The cognitive niche: coevolution of
intelligence, sociality, and language. Proc. Natl
Acad. Sci. USA 107(Suppl. 2), 8993 – 8999. (doi:10.
1073/pnas.0914630107)

2. Boyd R, Richerson PJ, Henrich J. 2011 Colloquium
Paper: The cultural niche: why social learning is
essential for human adaptation. Proc. Natl Acad. Sci.
USA 108(Suppl. 2), 10 918 – 10 925. (doi:10.1073/
pnas.1100290108)

3. Henrich J. 2015 The secret of our success: how
culture is driving human evolution, domesticating
our species, and making us smarter. Princeton, NJ:
Princeton University Press.

4. Tennie C, Call J, Tomasello M. 2009 Ratcheting up
the ratchet: on the evolution of cumulative culture.
Phil. Trans. R. Soc. B 364, 2405 – 2415. (doi:10.
1098/rstb.2009.0052)

5. Dean LG, Vale GL, Laland KN, Flynn E, Kendal RL.
2013 Human cumulative culture: a comparative
perspective. Biol. Rev. 89, 284 – 301. (doi:10.1111/
brv.12053)

6. Kline MA. 2015 How to learn about teaching: an
evolutionary framework for the study of teaching
behavior in humans and other animals. Behav. Brain
Sci. 38, 1 – 70. (doi:10.1017/S0140525X14001071)

7. Jablonka E, Lamb MJ. 2014 Evolution in four
dimensions, 2nd edn. Cambridge, MA: MIT press.

8. Barrett HC. 2014 The shape of thought. Oxford, UK:
Oxford University Press.

9. Henrich J, Heine SJ, Norenzayan A. 2010 The
weirdest people in the world? Behav. Brain Sci. 33,
61 – 83. (doi:10.1017/S0140525X0999152X)

10. Nielsen M, Haun D, Kartner J, Legare CH. 2017 The
persistent sampling bias in developmental
psychology: a call to action. J. Exp. Child Psychol.
162, 31 – 38. (doi:10.1016/j.jecp.2017.04.017)

11. Arnett JJ. 2008 The neglected 95%. Am. Psychol.
63, 602 – 614. (doi:10.1037/0003-066X.63.7.602)

12. Nielsen M, Haun D. 2015 Why developmental
psychology is incomplete without comparative and
cross-cultural perspectives. Phil. Trans. R. Soc. B
371, 20150071. (doi:10.1098/rstb.2015.0071)

13. Meadon M, Spurret D. 2010 It’s not just the
subjects—there are too many WEIRD researchers.
Behav. Brain Sci. 33, 104 – 115. (doi:10.1017/
S0140525X10000208)

14. Fernald A. 2010 Getting beyond the ‘convenience
sample’ in research on early cognitive development.
Behav. Brain Sci. 33, 91 – 92. (doi:10.1017/
S0140525X10000294)

15. Rai TS, Fiske A. 2010 ODD (observation-and
description-deprived) psychological research. Behav.
Brain Sci. 33, 106 – 107. (doi:10.1017/
S0140525X10000221)

16. Norenzayan A, Heine SJ. 2005 Psychological
universals: what are they and how can we know?
Psychol. Bull. 131, 763 – 784. (doi:10.1037/0033-
2909.131.5.763)

17. Apicella CL, Barrett HC. 2016 Cross-cultural
evolutionary psychology. Curr. Opin. Psychol. 7,
92 – 97. (doi:10.1016/j.copsyc.2015.08.015)

18. Boyd R, Richerson PJ. 1985 Culture and the
evolutionary process. Chicago, IL: University of
Chicago Press.

19. Richerson PJ, Boyd R. 2005 Not by genes alone: how
culture transformed human evolution. Chicago, IL:
University of Chicago Press.

20. Whiten A, Horner V, Marshall-Pescini S. 2003
Cultural panthropology. Evol. Anthropol. 12,
92 – 105. (doi:10.1002/evan.10107)

21. Langergraber KE, Vigilant L. 2011 Genetic differences
cannot be excluded from generating behavioural
differences among chimpanzee groups. Proc. R. Soc. B
278, 2094 – 2095. (doi:10.1098/rspb.2011.0391)

22. Langergraber K, Schubert G, Rowney C,
Wrangham R, Zommers Z, Vigilant L. 2011
Genetic differentiation and the evolution of
cooperation in chimpanzees and humans.
Proc. R. Soc. B 278, 2546 – 2552. (doi:10.1098/rspb.
2010.2592)

23. Shweder RA. 1999 Why cultural psychology? Ethos
27(1), 62 – 73. (doi:10.1525/eth.1999.27.1.62)

24. Gea J, Peng G, Lyu B, Wang Y, Zhuoe Y, Niuf Z, Tang
LH. 2015 Cross-language differences in the brain
network subserving intelligible speech. Proc. Natl
Acad. Sci. USA 112, 2972 – 29777. (doi:10.1073/
pnas.1416000112)

http://dx.doi.org/10.1073/pnas.0914630107

http://dx.doi.org/10.1073/pnas.1100290108

http://dx.doi.org/10.1098/rstb.2009.0052

http://dx.doi.org/10.1111/brv.12053

http://dx.doi.org/10.1017/S0140525X14001071

http://dx.doi.org/10.1017/S0140525X0999152X

http://dx.doi.org/10.1016/j.jecp.2017.04.017

http://dx.doi.org/10.1037/0003-066X.63.7.602

http://dx.doi.org/10.1098/rstb.2015.0071

http://dx.doi.org/10.1017/S0140525X10000208

http://dx.doi.org/10.1017/S0140525X10000294

http://dx.doi.org/10.1017/S0140525X10000221

http://dx.doi.org/10.1037/0033-2909.131.5.763

http://dx.doi.org/10.1016/j.copsyc.2015.08.015

http://dx.doi.org/10.1002/evan.10107

http://dx.doi.org/10.1098/rspb.2011.0391

http://dx.doi.org/10.1098/rspb.2010.2592

http://dx.doi.org/doi:10.1525/eth.1999.27.1.62

http://dx.doi.org/10.1073/pnas.1416000112

rstb.royalsocietypublishing.org
Phil.Trans.R.Soc.B
373:20170059
8
D
ow
nl
oa
de
d
fr
om
h
tt
ps
:/
/r
oy
al
so
ci
et
yp
ub
li
sh
in
g.
or
g/
o
n
09
F
eb
ru
ar
y
20
22

25. Schieffelin B, Ochs E. 1986 Language socialization.
Annu. Rev. Anthropol. 15, 163 – 191. (doi:10.1146/
annurev.an.15.100186.001115)

26. Saffran JR, Aslin RN, Newport EL. 1996
Statistical learning by 8-month-old infants.
Science 274, 1926 – 1928. (doi:10.1126/science.274.
5294.1926)

27. Broesch T, Bryant GA. 2017 Fathers’ infant-directed
speech in a small-scale society. Child Dev. (doi:10.
1111/cdev.12768)

28. Broesch TL, Bryant GA. 2015 Prosody in infant-
directed speech is similar across western and
traditional cultures. J. Cogn. Dev. 16, 31 – 43.
(doi:10.1080/15248372.2013.833923)

29. LeVine RA. 2001 Ethnocentrism. In International
encyclopedia of the social and behavioral sciences
(eds NJ Smelser, PB Baltes), pp. 4852 – 4854.
Oxford, UK: Oxford University Press.

30. Mace R, Holden C, Shennan S (eds). 2005 The
evolution of cultural diversity: a phylogenetic
approach. Walnut Creek, CA: Leftcoast Press.

31. Kline MA, Boyd R. 2010 Population size
predicts technological complexity in Oceania.
Proc. R. Soc. B 277, 2559 – 2564. (doi:10.1098/rspb.
2010.0452)

32. Henrich J et al. 2016 Understanding
cumulative cultural evolution. Proc. Natl Acad. Sci.
USA. 113, E6724 – E6725. (doi:10.1073/pnas.
1610005113)

33. Karasik LB, Tamis-LeMonda CS, Adolph KE,
Bornstein MH. 2015 Places and postures. J. Cross
Cult. Psychol. 46, 1023 – 1038. (doi:10.1177/
0022022115593803)

34. Greenfield PM, Keller H, Fuligni A, Maynard A. 2003
Cultural pathways through universal development.
Annu. Rev. Psychol. 54, 461 – 490. (doi:10.1146/
annurev.psych.54.101601.145221)

35. Bowlby J. 1989 Attachment theory. Los Angeles, CA:
Lifespan Learning Institute.

36. Keller H. 2013 Attachment and culture. J. Cross Cult.
Psychol 44, 175 – 194. (doi:10.1177/0022022
112472253)

37. Lancy DF. 2010 Learning ‘from nobody’: the limited
role of teaching in folk models of children’s
development. Childhood Past. 3.1, 79 – 106. (doi:10.
1179/cip.2010.3.1.79)

38. Akhtar N, Jaswal VK. 2013 Deficit or difference?
Interpreting diverse developmental paths: an
introduction to the special section. Dev. Psychol. 49,
1 – 3. (doi:10.1037/a0029851)

39. Smith LB. 1993 A dynamic systems approach to
development: applications. Cambridge, MA: The MIT
Press.

40. Perreault C. 2012 The pace of cultural evolution.
PLoS ONE 7, e45150. (doi:10.1371/journal.pone.
0045150)

41. Paradise R, Rogoff B. 2009 Side by side: learning by
observing and pitching in. Ethos 37, 102 – 138.
(doi:10.1111/j.1548-1352.2009.01033.x)

42. Rogoff B, Matusov E, White C. 1996 Models of
teaching and learning: participation in a
community of learners. In The handbook of
education and human development: New models
of learning, teaching and schooling (eds DR
Olson, N Torrance), pp. 388 – 414. Oxford, UK:
Blackwell.

43. Kline MA. 2016 TEACH: an ethogram-based method
to observe and record teaching behavior. Field
Methods 29, 205 – 220. (doi:10.1177/
1525822X16669282)

44. Heine SJ, Norenzayan A. 2006 Toward a
psychological science for a cultural species. Perspect.
Psychol. Sci. 1, 251 – 269. (doi:10.1111/j.1745-6916.
2006.00015.x)

45. Pepitone A, Triandis HC. 1987 On the universality
of social psychological theories. J. Cross Cult.
Psychol 18, 471 – 498. (doi:10.1177/0022002
187018004003)

46. Poortinga YH. 1989 Equivalence of cross-cultural
data: an overview of basic issues. Int. J. Psychol. 24,
737 – 756. (doi:10.1080/00207598908246809)

47. Rogoff B, Paradise R, Arauz R, Correa-Chávez M,
Angelillo C. 2003 Firsthand learning through intent
participation. Annu. Rev. Psychol. 54, 175 – 203.
(doi:10.1146/annurev.psych.54.101601.145118)

48. Cohen D. 2007 Methods in cultural psychology.
In Handbook of cultural psychology, pp. 196 – 236.
London, UK: The Guilford Press.

49. Greenfield PM. 1997 Culture as process: empirical
methods for cultural psychology. In Handbook of
cross-cultural psychology: theory and method (eds
JW Berry, YH Poortinga, J Pandey), pp. 301 – 346.
Boston, MA: Allyn & Bacon.

50. Heine SJ, Lehman DR, Peng K, Greenholtz J. 2002
What’s wrong with cross-cultural comparisons of
subjective Likert scales?: The reference-group effect.
J. Pers. Soc. Psychol. 82, 903 – 918. (doi:10.1037/
0022-3514.82.6.903)

51. Mesoudi A, Whiten A, Laland KN. 2006 Toward a
unified science of cultural evolution. Behav. Brain Sci.
29, 329 – 383. (doi:10.1017/S0140525X06009083)

52. Bernard HR. 2011 Research methods in
anthropology: qualitative and quantitative
approaches, 5th edn. New York, NY: Altamira Press.

53. Akhtar N, Gernsbacher MA. 2008 On privileging the
role of gaze in infant social cognition. Child Dev.
Perspect. 2, 59 – 65. (doi:10.1111/j.1750-8606.2008.
00044.x)

54. Lancy DF. 2008 The anthropology of childhood.
Cambridge, UK: Cambridge University Press.

55. LeVine RA, Levine S. 1996 Child care and culture:
lessons from Africa. Cambridge, UK: Cambridge
University Press.

56. Tomasello M, Carpenter M, Call J, Behne T, Moll H.
2005 Understanding sharing intentions: the origins
of cultural cognition. Behav. Brain Sci. 28,
675 – 735. (doi:10.1017/S0140525X05000129)

http://dx.doi.org/10.1146/annurev.an.15.100186.001115

http://dx.doi.org/10.1126/science.274.5294.1926

http://dx.doi.org/doi:10.1111/cdev.12768

http://dx.doi.org/10.1080/15248372.2013.833923

http://dx.doi.org/10.1098/rspb.2010.0452

http://dx.doi.org/10.1073/pnas.1610005113

http://dx.doi.org/10.1177/0022022115593803

http://dx.doi.org/10.1146/annurev.psych.54.101601.145221

http://dx.doi.org/10.1177/0022022112472253

http://dx.doi.org/10.1179/cip.2010.3.1.79

http://dx.doi.org/10.1037/a0029851

http://dx.doi.org/10.1371/journal.pone.0045150

http://dx.doi.org/10.1111/j.1548-1352.2009.01033.x

http://dx.doi.org/10.1177/1525822X16669282

http://dx.doi.org/10.1111/j.1745-6916.2006.00015.x

http://dx.doi.org/10.1177/0022002187018004003

http://dx.doi.org/10.1080/00207598908246809

http://dx.doi.org/10.1146/annurev.psych.54.101601.145118

http://dx.doi.org/10.1037/0022-3514.82.6.903

http://dx.doi.org/10.1017/S0140525X06009083

http://dx.doi.org/10.1111/j.1750-8606.2008.00044.x

http://dx.doi.org/10.1017/S0140525X05000129

Variation is the universal: making cultural evolution work in developmental psychology

Human development requires culture
Developmental psychology requires a culturally diverse sample
Cultural evolution can motivate a better science of developmental psychology
Problem no. 1: the universality as uniformity assumption
Variation equals cultural influence and lack of universality
Uniformity equals genetic roots and lack of cultural influence
Improvements
Developmental research case study
Problem no. 2: the Western centrality assumption
Improvements
Developmental research case study
Problem no. 3: the deficit assumption
Improvements
Developmental case study
Problem no. 4: the equivalency assumption
Improvements
Developmental case study
Conclusion
Data accessibility
Authors’ contributions
Competing interests
Funding
Acknowledgements
References

Vol.:(0123456789)

Advances in Health Sciences Education (2020) 25:1025–1043
https://doi.org/10.1007/s10459-020-10011-0

1 3

I N V I T E D PA P E R

How cognitive psychology changed the face of medical
education research

Henk G. Schmidt1 · Silvia Mamede1

Received: 12 September 2020 / Accepted: 27 October 2020 / Published online: 26 November 2020
© Springer Nature B.V. 2020

Abstract
In this article, the contributions of cognitive psychology to research and development of
medical education are assessed. The cognitive psychology of learning consists of activa-
tion of prior knowledge while processing new information and elaboration on the resulting
new knowledge to facilitate storing in long-term memory. This process is limited by the
size of working memory. Six interventions based on cognitive theory that facilitate learn-
ing and expertise development are discussed: (1) Fostering self-explanation, (2) elaborative
discussion, and (3) distributed practice; (4) help with decreasing cognitive load, (5) pro-
moting retrieval practice, and (6) supporting interleaving practice. These interventions con-
tribute in different measure to various instructional methods in use in medical education:
problem-based learning, team-based learning, worked examples, mixed practice, serial-cue
presentation, and deliberate reflection. The article concludes that systematic research into
the applicability of these ideas to the practice of medical education presently is limited and
should be intensified.

Keywords Knowledge acquisition · Self-explanation · Elaborative discussion · Distributed
practice · Cognitive load · Retrieval practice · Interleaving practice · Medical expertise

Introduction

Research into medical education began to attract serious attention with the publication of
the Journal of Medical Education (now Academic Medicine) in 1951. Not surprisingly,
from its very beginning it has been influenced by what was current in the psychology of
learning and instruction and always reflected its ongoing concerns. In the fifties and sixties
the language of behaviorism was dominant in the medical education literature. Learning
was seen as the result of repetition and reward, with its application to so called ‘learn-
ing machines’ (Owen et al. 1965, 1964), to programmed instruction (Lysaught et al. 1964;
Weiss and Green 1962), and with its emphasis on ‘behavioral’ objectives (Varagunam
1971). Cognitive-psychology concepts such as ‘memory,’ ‘retention,’ and ‘reasoning’

* Henk G. Schmidt
schmidt@fsw.eur.nl

1 Department of Psychology, Erasmus University, P.O. Box 1738, 3000, DR, Rotterdam,
the Netherlands

http://orcid.org/0000-0001-8706-0978

http://crossmark.crossref.org/dialog/?doi=10.1007/s10459-020-10011-0&domain=pdf

1026 H. G. Schmidt, S. Mamede

1 3

started to appear only in the early seventies (Elstein et al. 1972; Klachko and Reid 1975;
Levine and Forman 1973), and found an early synthesis in the groundbreaking work of
Elstein and colleagues on medical problem solving (Elstein et al. 1978). The purpose of
the present article is to assess the role of cognitive psychology in the study of medical
education (and by extension health professions education). We will focus here on how cog-
nitive conceptualizations of learning and instruction have assisted in an understanding of
knowledge acquisition and expertise development in medicine. Of course, these two top-
ics, knowledge acquisition and expertise development, are closely intertwined. However,
the study of clinical reasoning is so vital to medical education and has seized upon its own
niche within the research community, that we will discuss it separately. Since this article
was written to contribute to the celebration of the 25th anniversary of Advances in Health
Sciences Education, references are to articles published by this journal whenever possi-
ble. First however we present a crash course in the cognitive psychology of knowledge
acquisition.

A brief introduction to the cognitive psychology of knowledge
acquisition

When first-year medical students are confronted with information new to them from a
chapter of Guyton and Hall’s textbook of medical physiology, they activate prior knowl-
edge from high-school or college biology to help them interpret the new information; they
use existing knowledge to construct new knowledge. This new understanding, if sufficient
thorough, is stored in long-term memory to be used for subsequent learning or application
(Anderson et al. 2017). What can be learned however is also dependent on limitations of
working memory, the part of memory where knowledge is consciously processed (Badde-
ley and Hitch 1974; Mayer 2010). Finally, knowledge needs to be biologically consolidated
in memory in order to survive (Lee 2008; McGaugh 2000). This consolidation is biochemi-
cal in nature first, then synaptic. These processes take several hours to stabilize. It is well-
known that memory for things learned is much better after a good night sleep. A third and
final process is systems consolidation in which memories are moved from the hippocampal
area to the cortex and become indestructible—although not necessarily retrievable (Wino-
cur and Moscovitch 2011). This process takes years. Retrievability is influenced by the
extent to which students apply their knowledge in contexts of sufficient variability and the
extent to which these contexts resemble the context in which it was learned initially (Eva
et al. 1998; Norman 2009).

Instructional interventions that foster learning

The cognitive processes described above, delineating what the mind, engaged in learning,
does naturally, can be boosted by instructional interventions. We will first describe these
interventions here, focusing on the most important ones. Some of these interventions aim
at strengthening the relationship between prior knowledge and new information. Others
attempt to facilitate processing of information. A third category aims to strengthen long-
term memory. In a subsequent section we will relate these interventions to some of the
most prevalent instructional approaches to medical education developed since the early
seventies.

1027How cognitive psychology changed the face of medical education…

1 3

Interventions aimed at strengthening the relationship with prior knowledge

Encouraging self‑explanation

Self-explanation is a form of elaboration upon what is learned. The students do this by
relating new information to knowledge previously acquired or repeat the information ver-
bally in their own words (Chi et al. 1989, 1994). Elaboration is known to be more helpful
than simple repetition of new material (Craik and Lockhart 1972). Chi et al. (1994) found
that students who were asked to self-explain after reading each line of a passage on the
human circulatory system had a significantly greater knowledge gain from pre- to posttest
than students who read the text twice. In an experiment of van Blankenstein et al. (2011)
students either listened to an explanation provided for a particular problem or had to gener-
ate an explanation themselves, before studying an appropriate text. There were no immedi-
ate effects on retention of the text. However, one month later, participants who had actively
engaged in self-explanation remembered 25% more from the text.

Facilitating elaborative discussion

If students are allowed to discuss subject matter with peers or are being prompted by a
teacher, learning improves considerably. In a meta-analysis of small-group learning in sci-
ence, mathematics, engineering, and technology (Springer et al. 1999) found effects on
learning considerably more sizable than those of most other educational interventions. Ver-
steeg et al. (2019) studied how elaborative discussion among peers would foster under-
standing of physiology concepts compared with individual self-explanation and a control
condition. They found that the elaborative-discussion group outperformed the self-explana-
tion group, while both outperformed the control group. Interestingly, students with initially
wrong concepts profited even when discussing them with a peer who also had an initial
wrong understanding.

Promoting distributed practice

If one spreads learning and retrieval activities over time, returning to the same contents a
couple of times, knowledge become better consolidated. Distributed-study opportunities
usually produce better memory than massed-study opportunities (Delaney et al. 2010).
It turned out difficult however to find a suitable example of the effects of massed versus
spaced practice in medical education. Kerfoot et al. (2007) conducted a number of studies
in which they sent to residents at regular intervals emails on four urology topics. These
emails consisted of a short clinically relevant question or clinical case scenario in multiple-
choice question format, followed by the answer, teaching point summary, and explanations
of the answers. Students were randomized to receive weekly e-mailed case scenarios in
only 2 of the 4 urology topics. At the end of the academic year, residents outperformed
their peers on the questions related to the emails they had received. However, this effect
could also be explained by mere exposure since the residents apparently had not received
the same information in massed form.

1028 H. G. Schmidt, S. Mamede

1 3

Interventions aimed at facilitating processing of new information

Help in decreasing cognitive load

As indicated above, working memory allows for only limited information to be processed at
the same time. If the cognitive load of information exceeds what can be processed, learning
is hampered (van Merrienboer and Sweller 2010). Much research has gone into the ques-
tion how cognitive load could be optimized by instruction. One successful strategy is the use
of worked examples. Rather than require students to solve problems in a particular domain
by themselves, the teacher presents worked-out examples of these problems for study (Chen
et al. 2015). The assumption here is that by seeing all elements required to solve a problem,
decreases cognitive load. Students with limited knowledge seem to profit from such approach,
whereas students with enough knowledge are sometimes hampered (Kalyuga et al. 2001).

Interventions aimed at strengthening long‑term memory

Fostering retrieval practice

When you ask students to retrieve information previously learned from memory, for
instance by providing them with regular quizzes, knowledge reactivated this way becomes
more entrenched in memory. Dobson and Linderholm (2015) for instance, had students
reading anatomy and physiology texts either three times, two times with the possibility of
making notes, or two times interspersed by an attempt to retrieve as much information as
possible. After a one-week retention interval, those who engaged in retrieval practice dem-
onstrated superior performance compared to the other two groups.

Fostering interleaving practice

Offering cases with different diagnoses in a clinical reasoning exercise boosts learning because
students learning to distinguish between cases that look the same but have different diagnoses,
and cases that look different but have the same diagnosis. Interleaving may slow initial learning
but, in the end, leads to better retention and application. An illustrative example is provided by
Hatala et al. (2003). They presented students with electrocardiograms with the aim to learning to
diagnose such ECGs. In one of their experiments, students were randomly allocated to one of two
practice phases, either “contrastive” where examples from various categories are mixed together,
or “non-contrastive” where all the examples in a single category are practiced in a single block.
Students in the mixed-examples condition outperformed those in the blocked-practice condition
while diagnosing a set of new ECGs. See for another example Kulasegaram et al. (2015).

To what extent are these interventions applied to the practice
of medical education?

No doubt, these interventions are sometimes applied by teachers in their courses on an
individual basis. Teachers allow students to discuss subject matter in small groups or pro-
vide quizzes during their lectures. However, there have been attempts, most of them only

1029How cognitive psychology changed the face of medical education…

1 3

during the last twenty years, to develop instructional models explicitly based on cognitive
principles as discussed above. We will outline four of these: Problem-based learning, team-
based learning, worked examples, and mixed practice.

Problem-based learning. (PBL) was actually an early innovation. It was developed at
McMaster University, Canada where in 1969 a first group of 20 students entered medi-
cal school. PBL has the following six defining characteristics: (i) Biomedical or clinical
problems are used as a starting point for learning; (ii) students collaborate in small groups
for part of the time; (iii) under the flexible guidance of a tutor. Because problems are the
trigger for learning (iv) the curriculum includes only a limited number of lectures; (v)
learning is student-initiated, and (vi) the curriculum includes ample time for self-study. For
the founding staff PBL was merely a combination of good educational practices aimed at
increasing motivation among students (Servant-Miklos 2019a). However, by the end of the
seventies, and due to work done at Maastricht University, the Netherlands, PBL underwent
a reinterpretation in line with cognitive psychology findings (Schmidt 1983; Servant-Mik-
los 2019b). Table 1 contains the authors’ labelling of cognitive processes and interventions
underlying PBL (Schmidt et al. 2011).

Team-based learning (TBL) was developed in 1997 by Larry Michaelsen at the Uni-
versity of Central Missouri, US, when increasing class sizes prevented him from teaching
in the Socratic fashion (Michaelsen et al. 2002). The idea emerged for the first time in the
medical education literature in 2005 (Koles et al. 2005). TBL consists of three phases: (i)
A preparatory phase, in which students study individually preassigned materials often con-
veyed through video; (ii) an in-class readiness assurance phase, consisting of an individual
test, a subsequent retest taken after discussion of the answers to the individual test are dis-
cussed in a team, and teacher feedback; (iii) an in-class application phase in which stu-
dents through facilitated interteam discussion solve new problems and answer new ques-
tions derived from the initial learning materials. Schmidt et al. (2019) and colleagues have
recently provided the cognitive account of what happens to the learner in TBL as outlined
in Table 1.

Worked examples are common in text books on physics, mathematics and chemistry. It
was probably Sweller and Cooper (1985) who saw their potential for reducing cognitive
load while problem solving. In the previous section we have already provided a successful
example of the application of cognitive load theory in the health professions field (Chen
et al. 2015). However, the number of studies on worked examples reported in that litera-
ture is still limited. A search into the three most-cited journals in health professions educa-
tion, Academic Medicine, Medical Education, and Advances in Health Sciences Educa-
tion unearthed 15 articles, the oldest being from 2002. The use of worked examples would
potentially be a fruitful addition to the arsenal of methods used to teach clinical reasoning,
but we definitively need more studies.

Mixed practice or interleaving has large potential for medical education, in particular
because one of its important functions is the teaching of diagnostic problem solving (Rich-
land et al. 2005; Rohrer 2012). Cases that superficially look the same may have different
causes. Alternatively, cases demonstrating a quite different array of symptoms, may have
the same underlying pathology. Training student to compare and contrast such cases would
be optimal using this instructional approach. However, only six illustrative examples could
be found in the extant health professions literature, interestingly most of them provided by
Geoffrey Norman, and his associates from McMaster University.

Table 1 summarizes the extent to which each of the cognitive principle discussed in the
previous section are actualized in these four instructional approaches.

1030 H. G. Schmidt, S. Mamede

1 3

The study of medical expertise

Medical expertise is an attractive domain of study for cognitive psychologists. This is so not
only because the quality of our care as patients depends on the performance of our physi-
cians but also because of peculiar features of the medical practice. Physicians operate upon an
extremely broad and complex knowledge basis, and clinical problem-solving involves a large
spectrum of cognitive processes, ranging from attention and perception to decision-making.
Not surprisingly, medical expertise has drawn researchers’ attention over four decades (Norman
2005). This research has focused on clinical reasoning, particularly the diagnostic process. One
of major goals of medical education is to develop students’ clinical reasoning and helping stu-
dents become good diagnosticians is much valued. Medical expertise research has contributed
substantially to our understanding of how this goal can be achieved (or at least how it should be
pursued). The following session summarizes the main contributions of this research to what we
know about, first, the nature of clinical reasoning and, second, how it develops in medical stu-
dents. Subsequently, we will discuss the impact of this research on medical education, particu-
larly how its contributions have interacted with conceptualizations of learning and instruction
discussed earlier in this article to inform the teaching of clinical reasoning.

The nature of clinical reasoning

The major findings that have shed light on the nature of clinical reasoning can be grouped
into three subheadings that parallels the history of the research on the subject.

The ‘hypothetico‑deductive’ method as a general model of clinical problem‑solving

Early in a clinical encounter, physicians generate one or a few diagnostic hypotheses and
subsequently gather additional information to either confirm or refute these hypotheses.

Table 1 Extent to which cognitive principles are actualized in four instructional models

+ + means that according to literature the principle is explicitly operationalized in the instructional
model. + means that it can be expected to play a role although not explicitly assumed.—means that it does
not play a role

Problem-based
learning

Team-based
learning

Worked
examples

Mixed practice

Activation of prior knowledge + + + + + +
Consolidation − + + − −
Appropriate context + + + + + + +
Self-explanation + + + + − −
Elaborative discussion + + + + − −
Decreasing cognitive load − − + + −
Retrieval practice + + + − −
Distributed practice − + − + +
Interleaving practice − − − + +

1031How cognitive psychology changed the face of medical education…

1 3

This ‘hypothetico-deductive’ method was revealed by pioneering studies conducted in the
1970s using traditional methods of cognitive psychology research, such as observing phy-
sicians and students interacting with standardized patients while thinking aloud (Elstein
et al. 1978, 2009). These studies attempted to uncover the reasoning process that char-
acterizes experts’ reasoning, which could then be taught to students. However, although
the hypothetico-deductive method provides a general representation of diagnostic reason-
ing, subsequent studies soon showed that it does not explain expert performance (Elstein
et al. 1978; Neufeld et al. 1981). Medical students also employed the same approach, and
what differentiated expert and novice diagnosticians was not a particular reasoning process
but rather the quality of their diagnostic hypotheses (Barrows et al. 1982). An additional
crucial finding of the same period was that diagnostic performance on one clinical case
did not predict performance on another case. The phenomenon, labeled by Elstein ‘content
specificity’ (Elstein et al. 1978), was proved to happen even when the cases were within the
same specialty (Eva et al. 1998; Norman et al. 1985).

How medical knowledge is structured in memory and used in diagnostic reasoning

It is not a particular process that determines expert performance, but rather the content of
reasoning, i.e. knowledge itself (Norman 2005). This conclusion came from a new era of
studies conducted when researchers, faced with the aforementioned findings, turned atten-
tion to the kinds of medical knowledge, how knowledge is structured in memory and used
to diagnose clinical problems. These studies relied heavily on methods from cognitive psy-
chology research to carefully search from differences in knowledge structures of expert and
non-expert diagnosticians. For example, many of these studies requested medical students
at different years of training and (more or less) experienced physicians to diagnose clini-
cal cases and subsequently explain the patient’s signs and symptoms or, alternatively, to
solve the case while thinking-aloud. The resulting protocols were analyzed to identify the
kinds and amount of knowledge used during diagnostic reasoning (Patel and Groen 1986;
Schmidt et al. 1990). Several knowledge structures have been proposed, suggesting that
diseases would be represented in memory, for example, as prototypes (Bordage and Zacks
1984), or as instances of previously seen patients (Norman et al. 2007), or yet as sche-
mas and scripts (Schmidt et al. 1990). Some of these proposals, such as prototype models,
consisted of application of representation models long existing in psychology to medical
knowledge. Other authors however developed formats specifically for representing medical
knowledge, such as the concept of illness scripts. Illness scripts are mental scenarios of the
conditions under which a disease emerges, the disease process itself, and its consequences
in terms of possible signs, symptoms, and management alternatives (Feltovich and Bar-
rows 1984). Some empirical support exists for several proposals, and it is likely that (some
of) these different knowledge structures coexist in physicians’ memory to be mobilized
when needed (Custers et al. 1996; Schmidt and Rikers 2007).

These conceptualizations have framed our understanding of diagnostic reasoning.
Notice that, despite their differences, they share the basic idea that diseases are associated
in memory with a set of observable clinical manifestations. Briefly, the presence of some
of these manifestations in a patient activates in the physician’s memory the mental repre-
sentation of the disease, generating a diagnostic hypothesis. Search for additional informa-
tion follows to verify whether other manifestations associated with the disease are actually
present. When this search reveals findings that contradict the initial diagnosis and rather
suggest others, new hypotheses may be activated and tested against the patient findings.

1032 H. G. Schmidt, S. Mamede

1 3

The dual nature of diagnostic reasoning

Dual-process theories of reasoning, long studied in psychology, represent another approach
to understanding and conceptualizing diagnostic reasoning. They assume that two different
forms of reasoning exist, one that is associative, based on pattern-recognition, fast, effort-
less and largely unconscious (usually named System 1 or Type 1) and another that depends
on applying rules, is slow, effortful and takes place under conscious control (System 2 or
Type 2) (Evans 2008, 2006; Kahneman 2003). While Type 1 processes accounts for intui-
tive judgments, Type 2 processes have to take place when these judgments are verified.
Appling this model to medical diagnosis, Type 1 reasoning would explain the generation of
diagnostic hypotheses whose subsequent verification depends on Type 2 processes. Indeed,
studies within the medical expertise research tradition seem in line with dual-process mod-
els. There is substantial evidence that physicians use non-analytical reasoning to arrive
at diagnoses (Norman and Brooks 1997). Radiologists, for example, were able to detect
abnormalities in medical images with around 70% accuracy in 200 ms (Evans et al. 2013;
Kundel and Nodine 1975). Studies on the role of similarity in diagnosis also provide addi-
tional evidence: diagnostic accuracy increased when a dermatological case was preceded
by a similar one (Brooks et al. 1991), and similarity affected the diagnosis even when what
was similar in two cases was a diagnostically irrelevant feature (e.g. the patient occupation)
(Hatala et al. 1999). There is also substantial evidence that physicians adopt both intuitive
and analytical reasoning modes in different degrees depending on the circumstances such
as the level of complexity of the case or perception of how problematic a case might be
(Mamede et al. 2007, 2008).

Dual-process representations of diagnostic reasoning have become prominent in the
medical literature (Croskerry 2009). A research tradition has grown triggered by increasing
concerns with the problem of diagnostic error. Flaws in the physician’s cognitive processes
have been detected in the majority of diagnostic errors (Graber 2005), and the sources of
cognitive errors have been much discussed in the medical literature (Norman 2009; Nor-
man et al. 2017). Several authors have attributed flaws in reasoning, and consequently
errors, to cognitive biases induced by heuristics, shortcuts in reasoning frequent in Type 1
processes (Croskerry 2009; Redelmeier 2005). Conversely, other authors argue that heuris-
tics are usually efficient and point to specific knowledge deficits rather than particular rea-
soning processes as the explanation for reasoning flaws (Eva and Norman 2005; McLaugh-
lin et al. 2014; Norman et al. 2017). This controversy should not be seen as a theoretical
discussion only, because it has direct consequences for medical education. While the first
position demands educational interventions aimed at increasing trainees’ and practicing
physicians’ ability to recognize biases and counteracting them, the second points to inter-
ventions that enhance knowledge acquisition and restructuring. We will return to this point
when discussing the teaching of clinical reasoning. To discuss teaching, we need first to
understand how clinical reasoning develops in medical students.

The development of clinical reasoning in medical students

In the course towards becoming an expert, medical students move through different stages
characterized by qualitatively different knowledge structures that underlie their perfor-
mance (Schmidt et al. 1990; Schmidt and Rikers 2007). This restructuring theory of medi-
cal expertise development has come out of a research program focused on understanding

1033How cognitive psychology changed the face of medical education…

1 3

how knowledge was organized in memory and used to solve clinical problems as students
progress through education. In the first years of their training, students rapidly develop
mental structures representing causal networks that explain the origins and consequences
of diseases on the basis of their pathophysiological mechanisms (Schmidt et al. 1990;
Schmidt and Rikers 2007). Studies that asked students at this stage to diagnose clinical
problems showed that, because students still do not recognize patterns of connected symp-
toms, they try to explain isolated symptoms based on their causal mechanisms. This pro-
cessing is effortful and detailed, with much use of basic sciences knowledge. This trans-
lated, for example, in the finding that students recalled more from a case than experts,
which has become known as the ‘intermediate effect’ (Schmidt and Boshuizen 1993).

A first qualitative shift in knowledge structure occurs when students start to apply the
knowledge that they have acquired to solve clinical problems. Gradually, the detailed
knowledge of the chain of events that leads to a symptom is ‘encapsulated’ in more generic
explanatory models or diagnostic labels that stands for the detailed explanation (Schmidt
et al. 1990; Schmidt and Rikers 2007). Through this process, a small number of abstract,
higher-order concepts, representing for example a syndrome or a simplified causal mecha-
nism, ‘summarize’ a larger number of lower-levels concepts. For example, when students
were requested to explain the clinical manifestations in a patient presenting with bacte-
rial endocarditis and sepsis, they reasoned step-by-step through the chain of events that
starts with the use of contaminated syringes until their consequences, i.e. the symptoms.
Conversely, experts used the concept of ‘sepsis’ as a label that ‘encapsulates’ much of
the chain of events, without the need to use this knowledge in their diagnostic reasoning
(Schmidt et al. 1988). Many studies have shown experts to make much use of this type of
‘encapsulated’ concepts when reasoning through a case, leading to think aloud or recall
protocols that contain less reference to basic sciences concepts or underlying mechanisms
than the students’ ones (Boshuizen and Schmidt 1992; Rikers et al. 2004, 2000). However,
basic sciences knowledge remains available and is indeed ‘unconsciously’ used during the
diagnosis as studies with indirect measures of reasoning have shown (Schmidt and Rikers
2007).

A second shift in knowledge structures occurs as exposure to patients increases. Encap-
sulated knowledge is gradually reorganized into narrative structures that ‘represent’ a
patient with a particular disease (Feltovich and Barrows 1984; Schmidt et al. 1990). These
‘illness scripts’ contain little knowledge of the causal mechanisms of the disease, because
of encapsulation, but are rich in clinical knowledge about the enabling conditions of the
disease and its clinical manifestations (Custers et al. 1998). Knowledge of enabling condi-
tions tends to increase with experience and play a crucial role in expert physicians’ reason-
ing (Hobus et al. 1987). As exposure to actual patients increases, traces of previously seen
patients are also stored in memory. Illness scripts exist therefore at different levels of gen-
erality, ranging from representations of disease prototypes to representations of previously
seen patients (Schmidt and Rikers 2007).

Successful diagnostic reasoning seems to depend critically on developing rich, coher-
ent mental representations of diseases (Cheung et al. 2018). For instance, a series of stud-
ies attempting to investigating the role of biomedical knowledge in diagnostic reasoning
had students learning the clinical features associated with a disease either together with
explanations of how they are produced or without explanation (Woods et al. 2007). Learn-
ing how the clinical features are connected by causal mechanisms led to higher diagnostic
accuracy when diagnosing cases of the disease after a delay. Besides bringing additional
evidence of the knowledge encapsulation process, these studies suggest that understand-
ing their underlying mechanisms help ‘glue’ the clinical features together, leading to more

1034 H. G. Schmidt, S. Mamede

1 3

coherent and stable mental representations of the diseases, which make it easier to recog-
nize them when diagnosing similar cases in the future.

This body of research contributed to our understanding of how students develop the
ability to diagnose clinical problems in the course of medical education and to set a for the
design of interventions for the teaching of clinical reasoning.

The teaching of clinical reasoning

The research described above provides substantial evidence that expert physicians do not
employ any peculiar reasoning mode and there is no such thing as general reasoning skills
that can be taught to students. Nevertheless, proposals for teaching students how to rea-
son, common in the 1990s, are still very frequent in the literature (Schmidt and Mamede
2015). Indeed, more recently, as dual-process theories have gained attention, these propos-
als have also gained the form of interventions such as courses on clinical reasoning and
cognitive bias (Norman et al. 2017). Not surprisingly, whenever trainees’ actual diagnostic
performance was evaluated, the effect of these process-oriented interventions has been null
or minimal (Norman et al. 2017; Schmidt and Mamede 2015). Conversely, interventions
directed towards acquisition and restructuring of disease knowledge, which seems more in
line with what we know about the nature of clinical reasoning and how it develops, looked
much more promising. For example, an intervention directed at increasing knowledge of
features that discriminate between similar-looking diseases successfully ‘immunized’ phy-
sicians against bias in reasoning (Mamede et al. 2020).

We try here to give a brief account of interventions that have been proposed for the
teaching of clinical reasoning, focusing on those that have been empirically investigated
and trying to relate them with the research discussed so far. Interventions that appear prom-
ising, consistently with evidence on the knowledge structures underlying diagnostic rea-
soning and the role of exposure to clinical problems in the development of such structures,
share two basic features: they are directed at refinement of diseases knowledge and consist
of exercises with clinical cases.

The serial-cue approach with simulation of the hypothetico-deductive model appeared
in a recent review of the literature as the most prevalent intervention proposed for the
teaching of clinical reasoning (Schmidt and Mamede 2015). In this approach informa-
tion of the case is disclosed step-by-step, and students required in each step to generate
diagnostic hypotheses and identify which additional information is needed to arrive at a
diagnostic decision. The approach has rarely been investigated. While two studies showed
the approach to have no effect on students’ diagnostic accuracy relative to a control group
(Windish 2000; Windish et al. 2005), a recent study showed a slight advantage of using
serial-cue during a learning session over employing self-explanation (Al Rumayyan et al.
2018). Its similarity to real practice may explain the widespread use of the serial cue
approach, but it has been argued that it may be overwhelming for students who do not have
yet developed illness scripts to guide the search for information.

Self-explanation as an instructional approach for the teaching of clinical reasoning has
been tested in a series of studies conducted by Chamberland and colleagues (Chamberland
et al. 2013, 2015, 2011) in recent years. Basically, these studies involved a learning ses-
sion, in which students diagnosed clinical cases either with self-explanation, i.e., explain-
ing aloud how the clinical features were produced, or without self-explanation, and a one-
week later test. Students who used self-explanation better diagnosed similar cases in the

1035How cognitive psychology changed the face of medical education…

1 3

test than their peers who had practiced without self-explanation. Students only benefitted
from self-explanation on cases with which they were less familiar and which required them
to extensively use biomedical knowledge, a finding that reaffirms the value of such knowl-
edge in diagnostic reasoning. Together with deliberate reflection (see below), self-expla-
nation has been adopted in a longitudinal curricular program at the Sherbrooke Medical
school, an experience which has been recently reported (Chamberland et al. 2020).

Instructional interventions that, differently from self-explanation, focus on clinical
rather than biomedical knowledge have also been proposed. These interventions foster
retrieval of previous acquired clinical knowledge and elaboration on the information at
hand during practice with clinical problems. Despite the different formats they may take,
these interventions share the basic idea of providing students with guidance to compare
and contrast different alternative diagnoses for the problem at hand. One example is con-
cept mapping, which has been employed in various formats (Montpetit-Tourangeau et al.
2017; Torre et al. 2019) to foster students’ clinical reasoning. One of the most investigated
of this type of interventions is deliberate reflection, which presents students with clinical
cases that look similar but have different diagnoses (e.g. diseases that have chest pain as
chief complaint) and requests students to generate, for each case, plausible diagnoses, com-
paring and contrasting them in light of the case features (Mamede et al. 2019, 2012, 2014).
In several studies, students who engaged in deliberate reflection during practice with clini-
cal cases provided better diagnoses for new cases of the same (or related) diseases in future
tests than students who adopted a more conventional approach such as making differen-
tial diagnosis. An intervention that used deliberate reflection to strengthening knowledge
of features that discriminate between similar-looking diseases has been recently shown
to increase internal medicine residents’ ability to counteract bias in diagnostic reasoning
(Mamede et al. 2020).

Interleaving practice, usually referred to in medical education as ‘mixed practice’, is
a requirement for the abovementioned interventions. It is only possible to compare and
contrast the features of clinical problems that may look similar but have in fact different
diagnoses when problems of different diseases that look alike are presented together in
the same exercise. The benefits of mixed practice relative to blocked practice, which pre-
sents examples of the same diagnosis together, have been demonstrated in studies compar-
ing students’ performance when interpreting EKG after being trained either with mixed or
blocked practice (Ark et al. 2007; Hatala et al. 2003).

Decreasing processing through the use of worked examples in the teaching of clinical
reasoning has been more scarcely investigated. Nevertheless, indication that this interven-
tion deserves further attention has come from a few studies exploring the influence of using
erroneous examples and different types of feedback on learning diagnostic knowledge
(Kopp et al. 2008, 2009) or the benefits of studying worked examples of reflective reason-
ing for diagnostic competence (Ibiapina et al. 2014).

Table 2 presents an attempt to summarize the extent to which these interventions for
the teaching of clinical reasoning allows for the realization of the cognitive principles dis-
cussed in the first sections of this paper.

Summing up, cognitive psychology research has provided crucial contributions to guide
teaching of clinical reasoning. Many of these contributions have translated into instruc-
tional interventions that have had their effectiveness empirically evaluated, with promis-
ing results. Nevertheless, as a recent review of these interventions highlighted, the existing
empirical research is still scarce considering the importance of clinical reasoning in medi-
cal education. More interventions based on the conceptualizations of learning and instruc-
tion offered by cognitive psychology and more theory-driven research are much needed.

1036 H. G. Schmidt, S. Mamede

1 3

How often do manuscripts delineating these ideas appear in advances
in health sciences education?

Twenty-five years ago, the founding editors of the journal, both cognitive psychologists,
and among them the first author of this article, found it necessary to create a journal in
which these new approaches to medical education would feature explicitly. To what extent
did they succeed? Table 2 contains the results of a search for appropriate articles in
Advances in Health Sciences Education, published between 1995 and 2020. The total num-
ber of articles published in that period was 1249.

Twenty-five percent of the manuscripts published in Advances in Health Sciences Edu-
cation discussed or studied the role of cognition in medical education. One could say that
the initial motivation for establishing the journal has not yet entirely been fulfilled. There
is clearly still room for more research into the application of these important principles of
learning, expertise development, and instruction to our field.

The future of cognition in medical education: Cognitive science

New areas hitherto not so much explored will probably attract increasing attention within
medical education development and research. We refer here to artificial intelligence and to
the neurosciences, both incorporated with cognitive psychology under the heading cogni-
tive science. We discuss two examples here. First, developments in clinical practice that
have strong implications for education have brought new research demands. One of these
developments is the digitalization of health care, including the incorporation of artificial
intelligence (Wartman and Combs 2018). Computer-based algorithms, whether derived
from expert knowledge or machine learning, are expected to dramatically improve diag-
nostic and prognosis decisions (Obermeyer and Emanuel 2016). However, “side effects”
have long been identified. For example, “automation bias” resulting from overreliance on
automation systems tends to make clinicians less prone to review their initial impressions,
eventually causing errors (Bond et al. 2018; Lyell and Coiera 2017). Future research should
explore how clinicians can be better prepared to incorporate these developments in their
practice, aiming also at better understanding the mechanisms underlying such biases and
how to make trainees less susceptible to them. Moreover, the digitalization of health care

Table 2 Numbers of studies published in Advances in Health Sciences Education between 1995 and 2020
applying cognitive principles and instructional models

Cognitive principles No of articles Instructional models No of articles

Activation of prior knowledge 29 Problem-based learning 121
Consolidation 2 Team-based learning 4
Appropriate context 16 Worked examples 3
Self-explanation 7 Mixed practice 4
Elaborative discussion 21 Teaching of clinical reasoning 17
Decreasing cognitive load 17
Retrieval practice 4
Distributed practice 0
Clinical reasoning 62

1037How cognitive psychology changed the face of medical education…

1 3

has brought changes to the clinical setting that affect what students can learn from their
experiences there. Think, for example, of clinical decision support systems, often asso-
ciated with electronic health records (EHR), now widely adopted (Keenan et al. 2006).
Patient care has been substantially altered by the widespread presence of computers, with
clinical encounters now involving the ‘provider-computer-patient triangulation’ and staff
rooms changed into rows of students and residents staring at computer screens. On the one
hand, EHRs can be powerful educational tools. Many of them offer instant access to online
learning resources at point of care. Trainees can, for example, ‘pull’ clinical guidelines or
recommendations about care management during the clinical encounter. This would allow
for new knowledge to be learned in a context very similar to the one in which it would be
used in the future, a basic principle to facilitate retrievability. EHRs also gives trainees the
possibility to easily go back to review a case and facilitates keeping track of one’s clini-
cal experiences (Keenan et al. 2006; Tierney et al. 2013). On the other hand, potentially
adverse effects have been discussed. For example, the volume of online information may
be overwhelming, and trainees’ attention may be diverted from the patient to the data-
entering process. More subtly, EHRs give trainees the possibility to easily convey the raw
patient data to supervisors, without being compelled to interpret findings and build a nar-
rative out of them. Incentive for the student or resident to reflect upon the problem there-
fore decreases, and so does the opportunity for discussion with attending physicians (Peled
et al. 2009; Wald et al. 2014). How EHRs and CDDS affect trainees learning and which
specific characteristics of the system itself or of its use can be optimized to foster learning
are examples of areas that are likely to call attention within cognitive science research.

A second expanding research area involves the use of neurosciences tools to get insights
on the processes in the brain associated with learning and expertise development. Although
the complexity and cost of some of the approaches for capturing brain activity make their
use less attractive, non-invasive, lower-cost tools have emerged that seem promising. Elec-
troencephalography (EEG) signals arising from neural activities have been used to estimate
students’ learning states, including within e-learning environments (Lin and Kao 2018).
For example, a device that showed to be wearable proved EEG-based technology to accu-
rately assess mental overload while surgeons performed procedures of different levels of
complexity (Morales et al. 2019). Detecting mental overload in surgeons is crucial to guide
the design of training programs so that situations that may bring threats to the patient or
the resident can be avoided. Near Infra-Red Spectroscopy (NIRS) is another promising
tool that has recently started to be employed in medical education. By measuring the level
of blood oxygenation of the prefrontal cortex, NIRS provides a cost-effective alternative
to other techniques such as functional Magnetic Resonance Imaging to look at the brain
while students and clinicians solve problems. For example, by using NIRS in a study which
trained medical students in diagnosing chest X-ray, Rotgans et al. showed that activation of
the prefrontal cortex decreases with experience with a case, supporting the idea that exper-
tise development is associated with a pattern-recognition based reasoning mode (Rotgans
et al. 2019).

Trying to predict the future is always a risky endeavor, but these two areas have great
potential to draw the attention of cognitive research in the coming years. If our bet is cor-
rect, we will see the products of this attention in the anniversary issue of Advances in
Health Sciences Education twenty-five years from now.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as
you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons

1038 H. G. Schmidt, S. Mamede

1 3

licence, and indicate if changes were made. The images or other third party material in this article are
included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the mate-
rial. If material is not included in the article’s Creative Commons licence and your intended use is not per-
mitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.

References

Al Rumayyan, A., Ahmed, N., Al Subait, R., Al Ghamdi, G., Mohammed Mahzari, M., Awad Mohamed,
T., et al. (2018). Teaching clinical reasoning through hypothetico-deduction is (slightly) better than
self-explanation in tutorial groups: An experimental study. Perspectives in Medical Education, 7(2),
93–99. https ://doi.org/10.1007/s4003 7-018-0409-x

Anderson, R. C., Spiro, R. J., & Montague, W. E. (2017). Schooling and the acquisition of knowledge. Lon-
don, UK: Routledge. https ://doi.org/10.4324/97813 15271 644

Ark, T. K., Brooks, L. R., & Eva, K. W. (2007). The benefits of flexibility: The pedagogical value of instruc-
tions to adopt multifaceted diagnostic reasoning strategies. Medical Education, 41(3), 281–287. https
://doi.org/10.1111/j.1365-2929.2007.02688 .x

Baddeley, A. D., & Hitch, G. (1974). Working Memory. In G. Bower (Ed.), The psychology of learning and
motivation (pp. 47–89). Cambridge: Academic Press.

Barrows, H. S., Norman, G. R., Neufeld, V. R., & Feightner, J. W. (1982). The clinical reasoning of ran-
domly selected physicians in general medical practice. Clinical Investigative Medicine, 5(1), 49–55.
https ://www.ncbi.nlm.nih.gov/pubme d/71167 14

Bond, R. R., Novotny, T., Andrsova, I., Koc, L., Sisakova, M., Finlay, D., et al. (2018). Automation bias in
medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when read-
ing electrocardiograms. Journal of Electrocardiology, 51(6S), S6–S11. https ://doi.org/10.1016/j.jelec
troca rd.2018.08.007

Bordage, G., & Zacks, R. (1984). The structure of medical knowledge in the memories of medical students
and general practitioners: Categories and prototypes. Medical Education, 18(6), 406–416. https ://doi.
org/10.1111/j.1365-2923.1984.tb012 95.x

Boshuizen, H. P. A., & Schmidt, H. G. (1992). The role of biomedical knowledge in clinical reasoning by
experts, intermediates and novices. Cognitive Science, 16, 153–184.

Brooks, L. R., Norman, G. R., & Allen, S. W. (1991). Role of specific similarity in a medical diag-
nostic task. Journal of Experimental Psychology General, 120(3), 278–287. https ://doi.
org/10.1037//0096-3445.120.3.278

Chamberland, M., Mamede, S., Bergeron, L., & Varpio, L. (2020). A layered analysis of self-explanation
and structured reflection to support clinical reasoning in medical students. Perspectives in Medical
Education. /. https ://doi.org/10.1007/s4003 7-020-00603 -2

Chamberland, M., Mamede, S., St-Onge, C., Rivard, M. A., Setrakian, J., Levesque, A., et al. (2013). Stu-
dents’ self-explanations while solving unfamiliar cases: The role of biomedical knowledge. Medical
Education, 47(11), 1109–1116. https ://doi.org/10.1111/medu.12253

Chamberland, M., Mamede, S., St-Onge, C., Setrakian, J., & Schmidt, H. G. (2015). Does medical stu-
dents’ diagnostic performance improve by observing examples of self-explanation provided by peers
or experts? Advances in Health Sciences Education, 20(4), 981–993. https ://doi.org/10.1007/s1045
9-014-9576-7

Chamberland, M., St-Onge, C., Setrakian, J., Lanthier, L., Bergeron, L., Bourget, A., et al. (2011). The influ-
ence of medical students’ self-explanations on diagnostic performance. Medical Education, 45(7),
688–695. https ://doi.org/10.1111/j.1365-2923.2011.03933 .x

Chen, R., Grierson, L., & Norman, G. (2015). Manipulation of cognitive load variables and impact on
auscultation test performance. Advances in Health Sciences Education, 20(4), 935–952. https ://doi.
org/10.1007/s1045 9-014-9573-x

Cheung, J. J. H., Kulasegaram, K. M., Woods, N. N., Moulton, C. A., Ringsted, C. V., & Brydges, R.
(2018). Knowing how and knowing why: Testing the effect of instruction designed for cognitive inte-
gration on procedural skills transfer. Advances in Health Sciences Education, 23(1), 61–74. https ://
doi.org/10.1007/s1045 9-017-9774-1

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations – How stu-
dents study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182. https
://doi.org/10.1207/s1551 6709c og130 2_1

http://creativecommons.org/licenses/by/4.0/

https://doi.org/10.1007/s40037-018-0409-x

https://doi.org/10.4324/9781315271644

https://doi.org/10.1111/j.1365-2929.2007.02688.x

https://www.ncbi.nlm.nih.gov/pubmed/7116714

https://doi.org/10.1016/j.jelectrocard.2018.08.007

https://doi.org/10.1111/j.1365-2923.1984.tb01295.x

https://doi.org/10.1037//0096-3445.120.3.278

https://doi.org/10.1007/s40037-020-00603-2

https://doi.org/10.1111/medu.12253

https://doi.org/10.1007/s10459-014-9576-7

https://doi.org/10.1111/j.1365-2923.2011.03933.x

https://doi.org/10.1007/s10459-014-9573-x

https://doi.org/10.1007/s10459-017-9774-1

https://doi.org/10.1207/s15516709cog1302_1

1039How cognitive psychology changed the face of medical education…

1 3

Chi, M. T. H., Deleeuw, N., Chiu, M. H., & Lavancher, C. (1994). Eliciting self-explanations improves
understanding. Cognitive Science, 18(3), 439–477. https ://doi.org/10.1016/0364-0213(94)90016 -7

Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of
verbal learning and verbal behavior, 11(6), 671–684.

Croskerry, P. (2009). Clinical cognition and diagnostic error: applications of a dual process model of rea-
soning. Advances in Health Sciences Education, 14(Suppl 1), 27–35. https ://doi.org/10.1007/s1045
9-009-9182-2

Custers, E. J., Regehr, G., & Norman, G. R. (1996). Mental representations of medical diagnostic knowl-
edge: A review. Academic Medicine, 71(10 Suppl), S55-61. https ://doi.org/10.1097/00001 888-19961
0000-00044

Custers, E. J. F. M., Boshuizen, H. P. A., & Schmidt, H. G. (1998). The role of illness scripts in the devel-
opment of medical diagnostic expertise: Results from an interview study. Cognition and instruction,
16(4), 367–398.

Delaney, P. F., Verkoeijen, P. P., & Spirgel, A. (2010). Spacing and testing effects: A deeply critical, lengthy,
and at times discursive review of the literature. In B. Ross (Ed.), Psychology of learning and motiva-
tion (Vol. 53, pp. 63–147). Cambridge, MA: Academic Press.

Dobson, J. L., & Linderholm, T. (2015). Self-testing promotes superior retention of anatomy and physiology
information. Advances in Health Sciences Education, 20(1), 149–161. https ://doi.org/10.1007/s1045
9-014-9514-8

Elstein, A. S. (2009). Thinking about diagnostic thinking: A 30-year perspective. Advances in Health Sci-
ences Education, 14(Suppl 1), 7–18. https ://doi.org/10.1007/s1045 9-009-9184-0

Elstein, A. S., Kagan, N., Shulman, L. S., Jason, H., & Loupe, M. J. (1972). Methods and theory in the study
of medical inquiry. Academic Medicine, 47(2), 85–92.

Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medical problem solving: An Analysis of clini-
cal reasoning. Cambridge, MA: Harvard University Press. https ://doi.org/10.1177/01622 43978 00300
337.

Eva, K. W., Neville, A. J., & Norman, G. R. (1998). Exploring the etiology of content specificity: Factors
influencing analogic transfer and problem solving. Academic Medicine, 73, S1-5.

Eva, K. W., & Norman, G. R. (2005). Heuristics and biases–a biased perspective on clinical reasoning.
Medical Education, 39(9), 870–872. https ://doi.org/10.1111/j.1365-2929.2005.02258 .x

Evans, J. S. B. T. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual
Review of Psychology, 59, 255–278. https ://doi.org/10.1146/annur ev.psych .59.10300 6.09362 9

Evans, J. S. T. (2006). The heuristic-analytic theory of reasoning: Extension and evaluation. Psychonomic
Bulletin and Review, 13(3), 378–395. https ://doi.org/10.3758/Bf031 93858

Evans, K. K., Georgian-Smith, D., Tambouret, R., Birdwell, R. L., & Wolfe, J. M. (2013). The gist of the
abnormal: Above-chance medical decision making in the blink of an eye. Psychonomic Bulletin and
Review, 20(6), 1170–1175. https ://doi.org/10.3758/s1342 3-013-0459-3

Feltovich, P. J., & Barrows, H. S. (1984). Issues of generality in medical problem solving. In H. G. Schmidt
& M. L. De Volder (Eds.), Tutorials in problem-based learning (pp. 128–142). Assen, the Nether-
lands: Van Gorcum.

Graber, M. (2005, Feb). Diagnostic errors in medicine: A case of neglect. The joint commission journal on
quality and patient safety, 31(2), 106–113. https ://www.ncbi.nlm.nih.gov/pubme d/15791 770

Hatala, R., Norman, G. R., & Brooks, L. (1999). Influence of a single example on subsequent electrocardio-
gram interpretation. Teaching and Learning in Medicine, 11(2), 110–117.

Hatala, R. M., Brooks, L. R., & Norman, G. R. (2003). Practice makes perfect: The critical role of mixed
practice in the acquisition of ECG interpretation skills. Advances in Health Sciences Education, 8(1),
17–26. https ://doi.org/10.1023/a:10226 87404 380

Hobus, P. P., Schmidt, H. G., Boshuizen, H. P., & Patel, V. L. (1987). Contextual factors in the activation of
first diagnostic hypotheses: Expert-novice differences. Medical Education, 21(6), 471–476. https ://
doi.org/10.1111/j.1365-2923.1987.tb014 05.x

Ibiapina, C., Mamede, S., Moura, A., Eloi-Santos, S., & van Gog, T. (2014). Effects of free, cued and mod-
elled reflection on medical students’ diagnostic competence. Medical Education, 48(8), 796–805.
https ://doi.org/10.1111/medu.12435

Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American psy-
chologist, 58(9), 697–720.

Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to
studying worked examples. Journal of Educational Psychology, 93(3), 579–588. https ://doi.
org/10.1037/0022-0663.93.3.579

https://doi.org/10.1016/0364-0213(94)90016-7

https://doi.org/10.1007/s10459-009-9182-2

https://doi.org/10.1097/00001888-199610000-00044

https://doi.org/10.1007/s10459-014-9514-8

https://doi.org/10.1007/s10459-009-9184-0

https://doi.org/10.1177/016224397800300337

https://doi.org/10.1111/j.1365-2929.2005.02258.x

https://doi.org/10.1146/annurev.psych.59.103006.093629

https://doi.org/10.3758/Bf03193858

https://doi.org/10.3758/s13423-013-0459-3

http://www.ncbi.nlm.nih.gov/pubmed/15791770

https://doi.org/10.1023/a:1022687404380

https://doi.org/10.1111/j.1365-2923.1987.tb01405.x

https://doi.org/10.1111/medu.12435

https://doi.org/10.1037/0022-0663.93.3.579

1040 H. G. Schmidt, S. Mamede

1 3

Keenan, C. R., Nguyen, H. H., & Srinivasan, M. (2006). Electronic medical records and their impact on res-
ident and medical student education. Academic Psychiatry, 30(6), 522–527. https ://doi.org/10.1176/
appi.ap.30.6.522

Kerfoot, B. P., DeWolf, W. C., Masser, B. A., Church, P. A., & Federman, D. D. (2007). Spaced educa-
tion improves the retention of clinical knowledge by medical students: A randomised controlled
trial. Medical Education, 41(1), 23–31. https ://doi.org/10.1111/j.1365-2929.2006.02644 .x

Klachko, D. M., & Reid, J. C. (1975). The effect on medical students of memorizing a physical examina-
tion routine. Academic Medicine, 50(6), 628–630.

Koles, P., Nelson, S., Stolfi, A., Parmelee, D., & DeStephen, D. (2005). Active learning in a year 2
pathology curriculum. Medical Education, 39(10), 1045–1055.

Kopp, V., Stark, R., & Fischer, M. R. (2008). Fostering diagnostic knowledge through computer-sup-
ported, case-based worked examples: Effects of erroneous examples and feedback. Medical Edu-
cation, 42(8), 823–829. https ://doi.org/10.1111/j.1365-2923.2008.03122 .x

Kopp, V., Stark, R., Kuhne-Eversmann, L., & Fischer, M. R. (2009). Do worked examples foster medical
students’ diagnostic knowledge of hyperthyroidism? Medical Education, 43(12), 1210–1217. https
://doi.org/10.1111/j.1365-2923.2009.03531 .x

Kulasegaram, K., Min, C., Howey, E., Neville, A., Woods, N., Dore, K., & Norman, G. (2015). The
mediating effect of context variation in mixed practice for transfer of basic science. Advances in
Health Sciences Education, 20(4), 953–968. https ://doi.org/10.1007/s1045 9-014-9574-9

Kundel, H. L., & Nodine, C. F. (1975). Interpreting chest radiographs without visual search. Radiology,
116(3), 527–532. https ://doi.org/10.1148/116.3.527

Lee, J. L. C. (2008). Memory reconsolidation mediates the strengthening of memories by additional
learning. Nature Neuroscience, 11(11), 1264–1266. https ://doi.org/10.1038/nn.2205

Levine, H., & Forman, P. (1973). A study of retention of knowledge of neurosciences information. Aca-
demic Medicine, 48(9), 867–869.

Lin, F. R., & Kao, C. M. (2018). Mental effort detection using EEG data in E-learning contexts. Comput-
ers and Education, 122, 63–79.

Lyell, D., & Coiera, E. (2017). Automation bias and verification complexity: A systematic review. Jour-
nal of the American Medical Informatics Association, 24(2), 423–431. https ://doi.org/10.1093/
jamia /ocw10 5

Lysaught, J. P., Sherman, C. D., & Williams, C. M. (1964). Programmed learning: Potential values for
medical instruction. JAMA, 189(11), 803–807.

Mamede, S., de Carvalho-Filho, M. A., de Faria, R. M. D., Franci, D., Nunes, M., Ribeiro, L. M. C.,
et al. (2020). “Immunising” physicians against availability bias in diagnostic reasoning: A ran-
domised controlled experiment. BMJ Quality and Safety. https ://doi.org/10.1136/bmjqs -2019-
01007 9

Mamede, S., Figueiredo-Soares, T., Eloi Santos, S. M., de Faria, R. M. D., Schmidt, H. G., & van Gog,
T. (2019). Fostering novice students’ diagnostic ability: The value of guiding deliberate reflection.
Medical Education, 53(6), 628–637. https ://doi.org/10.1111/medu.13829

Mamede, S., Schmidt, H. G., Rikers, R. M., Penaforte, J. C., & Coelho-Filho, J. M. (2007). Breaking
down automaticity: Case ambiguity and the shift to reflective approaches in clinical reasoning.
Medical Education, 41(12), 1185–1192. https ://doi.org/10.1111/j.1365-2923.2007.02921 .x

Mamede, S., Schmidt, H. G., Rikers, R. M., Penaforte, J. C., & Coelho-Filho, J. M. (2008). Influence
of perceived difficulty of cases on physicians’ diagnostic reasoning. Academic Medicine, 83(12),
1210–1216. https ://doi.org/10.1097/ACM.0b013 e3181 8c71d 7

Mamede, S., van Gog, T., Moura, A. S., de Faria, R. M., Peixoto, J. M., Rikers, R. M., & Schmidt, H. G.
(2012). Reflection as a strategy to foster medical students’ acquisition of diagnostic competence.
Medical Education, 46(5), 464–472. https ://doi.org/10.1111/j.1365-2923.2012.04217 .x

Mamede, S., van Gog, T., Sampaio, A. M., de Faria, R. M., Maria, J. P., & Schmidt, H. G. (2014). How
can students’ diagnostic competence benefit most from practice with clinical cases? The effects
of structured reflection on future diagnosis of the same and novel diseases. Academic Medicine,
89(1), 121–127. https ://doi.org/10.1097/ACM.00000 00000 00007 6

Mayer, R. E. (2010). Applying the science of learning to medical education. Medical Education, 44(6),
543–549.

McGaugh, J. L. (2000). Memory–a century of consolidation. Science, 287(5451), 248–251.
McLaughlin, K., Eva, K. W., & Norman, G. R. (2014). Reexamining our bias against heuristics. Advances

in Health Sciences Education, 19(3), 457–464. https ://doi.org/10.1007/s1045 9-014-9518-4
Michaelsen, L. K., Knight, A. B., & Fink, L. D. (Eds.). (2002). Team-based learning: A transformative

use of small groups. Westport, CT: Greenwood publishing group.

https://doi.org/10.1176/appi.ap.30.6.522

https://doi.org/10.1111/j.1365-2929.2006.02644.x

https://doi.org/10.1111/j.1365-2923.2008.03122.x

https://doi.org/10.1111/j.1365-2923.2009.03531.x

https://doi.org/10.1007/s10459-014-9574-9

https://doi.org/10.1148/116.3.527

https://doi.org/10.1038/nn.2205

https://doi.org/10.1093/jamia/ocw105

https://doi.org/10.1136/bmjqs-2019-010079

https://doi.org/10.1111/medu.13829

https://doi.org/10.1111/j.1365-2923.2007.02921.x

https://doi.org/10.1097/ACM.0b013e31818c71d7

https://doi.org/10.1111/j.1365-2923.2012.04217.x

https://doi.org/10.1097/ACM.0000000000000076

https://doi.org/10.1007/s10459-014-9518-4

1041How cognitive psychology changed the face of medical education…

1 3

Montpetit-Tourangeau, K., Dyer, J. O., Hudon, A., Windsor, M., Charlin, B., Mamede, S., & van Gog, T.
(2017). Fostering clinical reasoning in physiotherapy: Comparing the effects of concept map study
and concept map completion after example study in novice and advanced learners. BMC Medical
Education, 17(1), 238. https ://doi.org/10.1186/s1290 9-017-1076-z

Morales, J. M., Ruiz-Rabelo, J. F., Diaz-Piedra, C., & Di Stasi, L. L. (2019). Detecting mental workload in
surgical teams using a wearable single-channel electroencephalographic device. J Surg Educ, 76(4),
1107–1115. https ://doi.org/10.1016/j.jsurg .2019.01.005

Neufeld, V. R., Norman, G. R., Feightner, J. W., & Barrows, H. S. (1981). Clinical problem-solving by
medical students: A cross-sectional and longitudinal analysis. Medical Education, 15(5), 315–322.
https ://doi.org/10.1111/j.1365-2923.1981.tb024 95.x

Norman, G. (2005). Research in clinical reasoning: Past history and current trends. Medical Education,
39(4), 418–427.

Norman, G. (2009). Teaching basic science to optimize transfer. Medical teacher, 31(9), 807–811.
Norman, G., Young, M., & Brooks, L. (2007). Non-analytical models of clinical reasoning: The role of

experience. Medical Education, 41(12), 1140–1145. https ://doi.org/10.1111/j.1365-2923.2007.02914
.x

Norman, G. R., & Brooks, L. R. (1997). The non-analytical basis of clinical reasoning. Advances in Health
Sciences Education, 2(2), 173–184.

Norman, G. R., Monteiro, S. D., Sherbino, J., Ilgen, J. S., Schmidt, H. G., & Mamede, S. (2017). The causes
of errors in clinical reasoning: Cognitive biases, knowledge deficits, and dual process thinking. Aca-
demic Medicine, 92(1), 23–30. https ://doi.org/10.1097/ACM.00000 00000 00142 1

Norman, G. R., Tugwell, P., Feightner, J. W., Muzzin, L. J., & Jacoby, L. L. (1985). Knowledge and clini-
cal problem-solving. Medical Education, 19(5), 344–356. https ://doi.org/10.1111/j.1365-2923.1985.
tb013 36.x

Obermeyer, Z., & Emanuel, E. J. (2016). Predicting the future – big data, machine learning, and clinical
medicine. New England Journal of Medicine, 375(13), 1216–1219. https ://doi.org/10.1056/NEJMp
16061 81

Owen, S., Hall, R., Anderson, J., & Smart, G. (1965). Programmed learning in medical education. An
experimental comparison of programmed instruction by teaching machine with conventional lectur-
ing in the teaching of electrocardiography to final year medical students. Postgraduate medical jour-
nal, 41(474), 201.

Owen, S., Hall, R., & Waller, I. (1964). Use of a teaching machine in medical education; preliminary experi-
ence with a programme in electrocardiography. Postgraduate medical journal, 40(460), 59.

Patel, V. L., & Groen, G. J. (1986). Knowledge-based solution strategies in medical reasoning. Cognitive
Science, 10, 91–116.

Peled, J. U., Sagher, O., Morrow, J. B., & Dobbie, A. E. (2009). Do electronic health records help or hinder
medical education? PLoS Med, 6(5), e1000069. https ://doi.org/10.1371/journ al.pmed.10000 69

Redelmeier, D. A. (2005). Improving patient care. The cognitive psychology of missed diagnoses. Ann
Intern Med, 142(2), 115–120. https ://www.ncbi.nlm.nih.gov/pubme d/15657 159

Richland, L. E., Bjork, R. A., Finley, J. R., & Linn, M. C. (2005). Linking cognitive science to educa-
tion: Generation and interleaving effects. In B. G. Bara, M. Bucciarelli, & L. Barsalou (Eds.). In Pro-
ceedings of the twenty-seventh annual conference of the Cognitive Science Society (pp. 1850–1855).
Mahwa, NJ: Lawrence Erlbaum.

Rikers, R. M., Loyens, S. M., & Schmidt, H. G. (2004). The role of encapsulated knowledge in clinical case
representations of medical students and family doctors. Medical Education, 38(10), 1035–1043. https
://doi.org/10.1111/j.1365-2929.2004.01955 .x

Rikers, R. M. J. P., Schmidt, H. G., & Boshuizen, H. P. A. (2000). Knowledge encapsulation and the inter-
mediate effect. Contemporary Educational Psychology, 25(2), 150–166. https ://doi.org/10.1006/
ceps.1998.1000

Rohrer, D. (2012). Interleaving helps students distinguish among similar concepts. Educational Psychology
Review, 24(3), 355–367.

Rotgans, J. I., Schmidt, H. G., Rosby, L. V., Tan, G. J. S., Mamede, S., Zwaan, L., & Low-Beer, N. (2019).
Evidence supporting dual-process theory of medical diagnosis: A functional near-infrared spectros-
copy study. Medical Education, 53(2), 143–152. https ://doi.org/10.1111/medu.13681

Schmidt, H. G. (1983). Problem-based learning – rationale and description. Medical Education, 17(1),
11–16.

Schmidt, H. G., & Boshuizen, H. P. A. (1993). On the origin of intermediate effects in clinical case recall.
Memory and Cognition, 21(3), 338–351. https ://doi.org/10.3758/Bf032 08266

Schmidt, H. G., Boshuizen, H. P. A., & Hobus, P. P. M. (1988). Transitory stages in the development of
medical expertise: The “intermediate effect” in clinical case representation studies. In Proceedings of

https://doi.org/10.1186/s12909-017-1076-z

https://doi.org/10.1016/j.jsurg.2019.01.005

https://doi.org/10.1111/j.1365-2923.1981.tb02495.x

https://doi.org/10.1111/j.1365-2923.2007.02914.x

https://doi.org/10.1097/ACM.0000000000001421

https://doi.org/10.1111/j.1365-2923.1985.tb01336.x

https://doi.org/10.1056/NEJMp1606181

https://doi.org/10.1371/journal.pmed.1000069

http://www.ncbi.nlm.nih.gov/pubmed/15657159

https://doi.org/10.1111/j.1365-2929.2004.01955.x

https://doi.org/10.1006/ceps.1998.1000

https://doi.org/10.1111/medu.13681

https://doi.org/10.3758/Bf03208266

1042 H. G. Schmidt, S. Mamede

1 3

the tenth annual conference of the cognitive science society (pp. 139–145). Hillsdale, NJ: Lawrence
Erlbaum.

Schmidt, H. G., & Mamede, S. (2015). How to improve the teaching of clinical reasoning: A narrative
review and a proposal. Medical Education, 49(10), 961–973. https ://doi.org/10.1111/medu.12775

Schmidt, H. G., Norman, G. R., & Boshuizen, H. P. A. (1990). A cognitive perspective on medical expertise
– theory and implications. Academic Medicine, 65(10), 611–621.

Schmidt, H. G., & Rikers, R. M. (2007). How expertise develops in medicine: Knowledge encapsulation and
illness script formation. Medical Education, 41(12), 1133–1139.

Schmidt, H. G., Rotgans, J. I., Rajalingam, P., & Low-Beer, N. (2019). A Psychological foundation for
team-based learning: knowledge reconsolidation. Academic Medicine, 94(12), 1878–1883. https ://doi.
org/10.1097/acm.00000 00000 00281 0

Schmidt, H. G., Rotgans, J. I., & Yew, E. H. J. (2011). The process of problem-based learning: What works
and why. Medical Education, 45(8), 792–806. https ://doi.org/10.1111/j.1365-2923.2011.04035 .x

Servant-Miklos, V. F. (2019a). Fifty years on: A retrospective on the world’s first problem-based learning
programme at McMaster university medical school. Health Professions Education, 5(1), 3–12.

Servant-Miklos, V. F. (2019b). Problem solving skills versus knowledge acquisition: The historical dispute
that split problem-based learning into two camps. Advances in Health Sciences Education, 24(3),
619–635.

Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergradu-
ates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational
Research, 69(1), 21–51.

Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in
learning algebra. Cognition and instruction, 2(1), 59–89.

Tierney, M. J., Pageler, N. M., Kahana, M., Pantaleoni, J. L., & Longhurst, C. A. (2013). Medical education
in the electronic medical record (EMR) era: Benefits, challenges, and future directions. Academic
Medicine, 88(6), 748–752. https ://doi.org/10.1097/ACM.0b013 e3182 905ce b

Torre, D. M., Hernandez, C. A., Castiglioni, A., Durning, S. J., Daley, B. J., Hemmer, P. A., & LaRochelle,
J. (2019). The Clinical reasoning mapping exercise (CResME): a new tool for exploring clinical rea-
soning. Perspectives in Medical Education, 8(1), 47–51. https ://doi.org/10.1007/s4003 7-018-0493-y

van Blankenstein, F. M., Dolmans, D. H. J. M., van der Vleuten, C. P. M., & Schmidt, H. G. (2011). Which
cognitive processes support learning during small-group discussion? The role of providing explana-
tions and listening to others. Instructional Science, 39(2), 189–204. https ://doi.org/10.1007/s1125
1-009-9124-7

van Merrienboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional educa-
tion: Design principles and strategies. Medical Education, 44(1), 85–93. https ://doi.org/10.111
1/j.1365-2923.2009.03498 .x

Varagunam, T. (1971). Student awareness of behavioural objectives: The effect on learning. Medical Educa-
tion, 5(3), 213–216.

Versteeg, M., van Blankenstein, F. M., Putter, H., & Steendijk, P. (2019). Peer instruction improves com-
prehension and transfer of physiological concepts: a randomized comparison with self-explanation.
Advances in Health Sciences Education, 24(1), 151–165. https ://doi.org/10.1007/s1045 9-018-9858-6

Wald, H. S., George, P., Reis, S. P., & Taylor, J. S. (2014). Electronic health record training in undergradu-
ate medical education: bridging theory to practice with curricula for empowering patient- and rela-
tionship-centered care in the computerized setting. Academic Medicine, 89(3), 380–386. https ://doi.
org/10.1097/ACM.00000 00000 00013 1

Wartman, S. A., & Combs, C. D. (2018). Medical education must move from the information age to the age
of artificial intelligence. Academic Medicine, 93(8), 1107–1109. https ://doi.org/10.1097/ACM.00000
00000 00204 4

Weiss, R. J., & Green, E. J. (1962). The applicability of programmed instruction in a medical school cur-
riculum. Academic Medicine, 37(8), 760–766.

Windish, D. M. (2000). Teaching medical students clinical reasoning skills. Academic Medicine, 75(1),
90–90. https ://doi.org/10.1097/00001 888-20000 1000-00022

Windish, D. M., Price, E. G., Clever, S. L., Magaziner, J. L., & Thomas, P. A. (2005). Teaching medical
students the important connection between communication and clinical reasoning. Journal of General
Internal Medicine, 20(12), 1108–1113. https ://doi.org/10.1111/j.1525-1497.2005.0244.x

Winocur, G., & Moscovitch, M. (2011). Memory transformation and systems consolidation. Journal of the
International Neuropsychological Society: JINS, 17(5), 766.

Woods, N. N., Brooks, L. R., & Norman, G. R. (2007). It all make sense: Biomedical knowledge, causal
connections and memory in the novice diagnostician. Advances in Health Sciences Education, 12(4),
405–415. https ://doi.org/10.1007/s1045 9-006-9055-x

https://doi.org/10.1111/medu.12775

https://doi.org/10.1097/acm.0000000000002810

https://doi.org/10.1111/j.1365-2923.2011.04035.x

https://doi.org/10.1097/ACM.0b013e3182905ceb

https://doi.org/10.1007/s40037-018-0493-y

https://doi.org/10.1007/s11251-009-9124-7

https://doi.org/10.1111/j.1365-2923.2009.03498.x

https://doi.org/10.1007/s10459-018-9858-6

https://doi.org/10.1097/ACM.0000000000000131

https://doi.org/10.1097/ACM.0000000000002044

https://doi.org/10.1097/00001888-200001000-00022

https://doi.org/10.1111/j.1525-1497.2005.0244.x

https://doi.org/10.1007/s10459-006-9055-x

1043How cognitive psychology changed the face of medical education…

1 3

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

How cognitive psychology changed the face of medical education research

Abstract
Introduction
A brief introduction to the cognitive psychology of knowledge acquisition
Instructional interventions that foster learning
Interventions aimed at strengthening the relationship with prior knowledge
Encouraging self-explanation
Facilitating elaborative discussion
Promoting distributed practice
Interventions aimed at facilitating processing of new information
Help in decreasing cognitive load
Interventions aimed at strengthening long-term memory
Fostering retrieval practice
Fostering interleaving practice

To what extent are these interventions applied to the practice of medical education?
The study of medical expertise
The nature of clinical reasoning
The ‘hypothetico-deductive’ method as a general model of clinical problem-solving
How medical knowledge is structured in memory and used in diagnostic reasoning
The dual nature of diagnostic reasoning
The development of clinical reasoning in medical students
The teaching of clinical reasoning
How often do manuscripts delineating these ideas appear in advances in health sciences education?
The future of cognition in medical education: Cognitive science
References

Scholarship of Teaching and Learning
in Psychology
Sometimes a Demo Is Not Just a Demo: When
Demonstrating Cognitive Psychology Means Confronting
Assumptions
Marianne E. Lloyd
Online First Publication, April

6, 2020. http://dx.doi.org/10.1037/stl0000192

CITATION
Lloyd, M. E. (2020, April 16). Sometimes a Demo Is Not Just a Demo: When Demonstrating
Cognitive Psychology Means Confronting Assumptions. Scholarship of Teaching and Learning in
Psycholog

Advance online publication. http://dx.doi.org/10.1037/stl0000192

PEDAGOGICAL POINTS TO PONDER

Sometimes a Demo Is Not Just a Demo: When Demonstrating
Cognitive Psychology Means Confronting Assumptions

Marianne E. Lloyd
Seton Hall University

Recent cross talk between cognitive psychology and education has yielded an onslaught
of articles, books, and demonstrations to improve application of basic cognitive
principles to educational settings. This essay will describe an example of how one of
these demonstrations can also illustrate potentially incorrect assumptions about stu-
dents. Specifically, a highly effective demonstration on the negative effects of task
switching on performance also revealed assumptions about language automaticity and
neurotypicality. Rather than avoiding these concerns, such demonstrations, in addition
to highlighting cognitive phenomena, can also be a springboard to explicit discussions
of issues of assumptions that may have implications for inclusion.

Keywords: demonstrations, assumptions, task switching

Several years ago, I started using a classroom
demonstration from The Learning Scientists (Wein-
stein, 2018) to demonstrate the impact of task
switching on performance. In this demonstration,
participants pair up and then time themselves per-
forming three tasks. First, each person recites the
alphabet from A to Z. Then each counts from 1 to 26.
For the final task, the two previous tasks are com-
bined (i.e., 1-A, 2-B, 3-C . . . 26-Z). The single tasks
of reciting the alphabet and counting typically takes
less than 10 s, whereas combing counting and the
alphabet often takes 1–2 min. There is usually laugh-
ter at people’s ability to speak with lightning pace for
just letters or numbers, but there is often a very
different, much quieter tone to the room for the
interleaving task. During this third task, one can
usually see the students using a variety of strategies
to manage the difficulty of combining the two lists—
closing eyes, plugging ears, using fingers as spatial
markers, getting encouragement from their partner,
and sometimes even giving up. In this way, it is a

very successful demonstration of the difficulties of
task switching versus engaging in a single task.

I have implemented this demonstration in my
one-off lectures for first-year students on im-
proving college performance as an activity on
the first day of many of my other classes to
justify my no– cell phone policy and in faculty
development events to encourage bringing cog-
nitive psychology findings to pedagogy. It
works beautifully every time. Although the stu-
dents show some level of frustration, faculty
especially groan at the task switching condition,
and I remind them this can help build empathy
to the difference between how we feel about
material (crystal clear and fun) and a student’s
experience (muddy and bleak). Students are
readily able to understand that things that are
easy on their own because of automaticity be-
come difficult when combined. Overall, it was a
perfect demonstration as far as I was concerned
to show that easy is not always as such.

However, this summer, when I again pre-
sented the demonstration as part of a program
for boosting first-year student success in col-
lege, I realized that some of my assumptions
about it might not always be correct. First, I had
always assumed that the first two tasks are easy
because of their automaticity. As an English-

Correspondence concerning this article should be ad-
dressed to X Marianne E. Lloyd, Department of Psychol-
ogy, Seton Hall University, 400 South Orange Avenue,
South Orange, NJ 07079. E-mail: marianne.lloyd@shu.edu

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

Scholarship of Teaching and Learning in Psychology
© 2020 American Psychological Association 2020, Vol. 2, No. 999, 000
ISSN: 2332-2101 http://dx.doi.org/10.1037/stl0000192

https://orcid.org/0000-0003-0210-9555

mailto:marianne.lloyd@shu.edu

http://dx.doi.org/10.1037/stl0000192

speaking monolingual person from Ohio, yes,
this is true for me. However, it might not be true
for students who did not grow up learning the
same A-Z alphabet or counting from 1 to 26 in
English. Consequently, these might not be au-
tomatic tasks for all participants. Second, clas-
sifying these tasks as easy or difficult assumed
some level of neurotypicality among partici-
pants. Presumably, this task would not be as
easy for any of my students who have language
or numeracy difficulties or working memory
impairments. Lastly, I assumed the demo would
be easy because it was a low-stakes task (i.e.,
done in pairs instead of in front of the entire
class). However, performing for a partner could
add additional difficulty to the task for a student
with a lot of social anxiety or one with a speech
impairment who does not feel comfortable re-
citing the material under time pressure.

When the demonstration is finished and I
collect the data to present to the class, we typ-
ically start by discussing the main idea—that
even tasks that are usually accomplished
quickly can take more time when completing
multiple tasks at once. I use this as a plea for
keeping phones away during class. After all,
what is happening in class and on the phones are
both more complicated than reciting something
likely already memorized, so following both
lecture and one’s phone distractions at the same
time should be even more difficult than com-
bining numbers and letters. Once you add in
that the phone is pretty much guaranteed to be
of greater interest than what is happening in
class, this kind of multitasking becomes an even
bigger risk of becoming a barrier to learning. At
least for that day, the students seemed con-
vinced that task switching is not an ideal way to
spend class time.

Now when I included this demonstration in
my class, I follow the discussion on the main
idea that switching tasks is problematic with
pointing out all the assumptions that I had made
about the tasks they had just performed. I am
explicit that this failure to consider my own
assumptions was likely not limited to this dem-
onstration and asked the students to help me
realize when I am making incorrect assump-
tions. I also highlight that this does not, how-
ever, change the key point of the demonstration
that learning will be more effective with less
divided attention. Despite my potentially incor-
rect assumptions about the task being easy for

everyone, the demonstration does reliably yield
data in which the task-switching condition has a
higher total completion time by a significant
margin than the total completion time of the two
single tasks.

Being aware of the assumptions we make
about the students in our classes is perhaps a
regular part of teaching reflection for many pro-
fessors. However, I must be honest that I was
considering these assumptions for the first time.
When I revise my courses, I take many factors
into consideration—student feedback, test per-
formance, difficulty of material, and insights
from teaching conferences and journal articles.
However, I was not trained to contemplate in-
dividual differences may increase or decrease
the suitability of blanket statements I make
about how a demonstration works. This does
not mean these helpful demonstrations should
be discarded but rather that I need to be more
thoughtful about how I frame them. This spring,
I have continued to include the demonstration in
my courses. The students enjoy it and it gives
me to opportunity to nudge them toward better
classroom choices.

I expect that now that I have found these
assumptions inherent in this demonstration, I
am only just beginning to see the way that
assumptions about my students might influence
my teaching. I have also begun to recognize
other assumptions that I use in my lectures on
increasing academic success. The demonstra-
tion of the difficulty in picking out the correct
penny from a set of distractors (Nickerson &
Adams, 1979) and the Moses Illusion (“How
many animals of each type did Moses bring on
the ark?”; Erickson & Mattson, 1981) both also
come with assumptions. The former assumes
experience with American currency and the lat-
ter expects some familiarity with characters and
stories associated with Abrahamic faiths. These
are still acceptable choices to demonstrate key
teaching points in class, but now I try to balance
their use with some context about how they
work only when one has consistent background
experience and that for some students in my
classes this may not be the case.

As I work to improve the teaching of psy-
chology, I think we need to be aware of other
places in which assumptions such as these may
lurk. A quick review of the ancillary materials
for an introductory psychology textbook sug-
gests other potentially less inclusive demos, in-

2 LLOYD

cluding those that relied on visual materials for
demonstrations of perceptual illusions or
schema memory examples that presumed a
level of familiarity with the stimuli. There has
been some progress toward acknowledging
these concerns with the recent distribution of a
manual for including issues of disability in in-
troductory psychology courses (Rosa, Bogart,
& Dunn, 2018). This resource contains demon-
strations such as a tactile version of Gestalt
principles, which would be suitable for vision-
impaired students in ways the more conven-
tional pictorial stimuli would not. A similar
resource would be a welcome addition for fac-
ulty wanting to consider additional factors re-
lated to diversity and inclusion. In the mean-
time, I am not going to try to find only those
demonstrations that are universally appropriate
because this would likely be impossible. In-
stead, I will model my willingness to learn by
continuing to share many kinds of demonstra-
tions while adding in discussions of the assump-
tions about the participants that are inherent in
each.

References

Erickson, T. D., & Mattson, M. E. (1981). From words to
meaning: A semantic illusion. Journal of Verbal Learn-
ing and Verbal Behavior, 20, 540–551. http://dx.doi
.org/10.1016/S0022-5371(81)90165-1

Nickerson, R. S., & Adams, M. J. (1979). Long-term
memory for a common object. Cognitive Psychol-
ogy, 11, 287–307. http://dx.doi.org/10.1016/0010-
0285(79)90013-6

Rosa, N. M., Bogart, K., & Dunn, D. S. (2018).
Increasing inclusiveness and awareness: Disability
in introductory psychology. Retrieved from http://
teachpsych.org/resources/Documents/otrp/resources/
Disability%20in%20Intro%20Psych%20Revision
%20042419%20-%20Google%20Docs

Weinstein, Y. (2018). The cost of task switching: A
simple yet very powerful demonstration. Retrieved
fromhttps://www.learningscientists.org/blog/2017/
7/28-1.

Received October 10, 2019
Revision received February 19, 2020

Accepted March 14, 2020 �

3SOMETIMES A DEMO

http://dx.doi.org/10.1016/S0022-5371%2881%2990165-1

http://dx.doi.org/10.1016/0010-0285%2879%2990013-6

http://teachpsych.org/resources/Documents/otrp/resources/Disability%20in%20Intro%20Psych%20Revision%20042419%20-%20Google%20Docs

https://www.learningscientists.org/blog/2017/7/28-1

Sometimes a Demo Is Not Just a Demo: When Demonstrating Cognitive Psychology Means Confronting A …

References

Revista Argentina de Clínica Psicológica

2020, Vol. XXIX, N°2, 871-877

DOI: 10.24205/03276716.2020.324

2020, Vol. XXIX, N°2, 871-877

REVISTA ARGENTINA

DE CLÍNICA PSICOLÓGICA

871

APPLICATION OF GAME ACTIVITIES IN MENTAL HEALTH EDUCATION
OF KINDERGARTENS BASED ON COGNITIVE PSYCHOLOGY

Na Yao, Liping Wang*

Abstract

Game activities are the most effective means of mental health education in kindergartens. This paper
mainly explores how to design game activities based on cognitive psychology, and apply them in mental health
education of kindergartens. First, the psychological characteristics of young children were analyzed from the
perspective of cognitive psychology. Then, game activities were designed based on these characteristics, and
verified through a case study. The children’s behaviors before and after the game activities were compared in
details. The results show that the game activities designed based on cognitive psychology accord with the
psychological characteristics of kindergarten children of all ages, meeting their psychological needs and
mental health demands; moreover, the designed game activities can correct the problematic behaviors of
young children and promote the children’s cognition of behavior. The research results provide a reference for
mental health education in kindergartens.

Key words: Cognitive Psychology, Game Activities, Mental Health Education,

Psychological

Characteristics.

Received: 18-05-19 | Accepted: 12-08-19

INTRODUCTION

When studying the psychology of criminals,
Professor Li Meijin of People’s Public Security
University of China divides the criminal
population into “dangerous personality” type
and “dangerous heart knot” type. “Dangerous
personality” type criminals are mostly caused by
problems in the process of mental cognitive
structure and personality development in their
congenital or early years. Hence, at the most
important stage of the formation of
psychological cognitive structure and personality
in infant aged 0-6 years (Renaud & Suissa, 1989),
it is mainly dependent on family education and
kindergarten school education, Cognitive
knowledge and experience acquired during this
period lay the foundation for life -long learning
(Donaldson Vollmer, Krous et al., 2011). The

Tangshan Normal University, Tangshan,063000, China.
E-Mail: menadw34woe@163.com

physical and mental development of infants aged
0-3 years mainly depends on family education,
while the physical and mental development of
infants aged 3-6 years is greatly influenced by
the kindergarten environment. Their strong
physique, coordinated actions, emotional self –
control, good habits and independence are the
main factors for judging children’s physical and
mental health (Nacheret, Garcia -Sanjuan, &
Jaen, 2016).

Therefore, mental health education in
kindergarten aims at actively responding to the
children’s inner psychological needs, setting up
the corresponding mental health education
curriculum according to the mental and physical
cognitive characteristics of children aged 3 -6
years, and cultivating the preschool children’s
mental and physical health development, so that
they have positive emotional regulation, strong
quality of consciousness, positive behavior and
good social communication skills (Larson , Russ,
Nelson et al., 2015). The quality of mental health
education directly affects the development of

NA YAO, LIPING WANG

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

872

children’s mental health and plays an important
role in the formation of 3-6-year-old children’s
personality.

China’s kindergarten education concepts and
management methods are relatively backward
and traditional. In addition, the shortage of
kindergarten teachers and professional mental
health teachers leads to the current situation of
a teacher-student ratio of 1:15 in kindergarten.
Compared with a teacher-student ratio of 1:5 in
foreign kindergartens, kindergarten teachers in
China have to undertake too many tasks in daily
learning activities. In order to unify the
management for time saving, early kindergarten
teachers often label “problem children” on
children whose personality behavior is special.
At the same time, these children are classified as
the focus of mental health education (Jones ,
Greenberg, & Crowley, 2015).

Due to the lack of professional mental health
teachers, kindergarten teachers are unable to
carry out mental health education from the
perspective of psychology. In order to better
discipline young children and highlight the
effectiveness of mental health education or to
cope with the supervision of higher education
departments, many children who don’t really
have mental problems are included in the
correction scope of mental health problems. For
example, slow eating, dietary bias, willfulness
and cowardice, or children’s

behavior

characteristics that will naturally disappear in
the psychological development with age are
taken as children’s psychological problems to be
corrected, which will only deepen the children’s
impression on the so-called “problem behavior”,
and even reinforces the behavior into
internalized personality. For this kind of
personality problem that will disappear
naturally, the education focus shall be changed
to correct the problem behavior by the way of
guiding and encouraging (for example, game
activities). Constructive mental health education
is carried out from the perspective of children’s
inner psychological cognition.

This study constructs game activities from the
professional angle of cognitive psychology,
analyzes children’s real psychological problems
and behaviors, carries out

psychological

intervention and correction pertinently, and
promotes the children to acquire the
psychological self-confidence, satisfaction and
security through happy game activities. At the
same time, it can promote children’s

interpersonal communication ability, team
cooperation ability, language expression ability,
logical thinking ability and emotion regulation
ability through collective game activities, so that
young children develop in a better direction
from the mutual coordination of physiological
and psychological interaction.

RELATIONSHIP BETWEEN COGNITIVE

PSYCHOLOGY AND THE MENTAL OF
KINDERGARTEN CHILDREN

Starting from the children’s cognitive
psychology, it is found that children have a
specific sensitivity to a certain thing at a specific
period (Feshbach & Price, 1984). During this
period, targeted guidance shall be conducted

(Melhuish, 2011). Children’s body and mind

are satisfied and supported with their curiosity,
desire, exploration, self-confidence, satisfaction,
and safe feeling, which will further st imulate
deeper exploration and cognition. At the same
time, they can gain the learning effect with twice
the effort. First, we should understand the
psychological characteristics of kindergarten
children of different ages, as shown in Figure 1,
which aims at constructing game activities,
correcting so-called problem behaviors through
game activities imperceptibly, and promoting
the cognition of correct behaviors.

Figure 1. The exclusive psychological
characteristics of different ages

Strong curiosity psychology

Prominent group psychology

Unconscious imitation psychology

Emotion control psychology

Attachment psychology

Lively and active love to

play psychology

Image thinking

Intentional behavior

Love to learn questions

Abstract thinking

The expression of the story

Intentional behavior increased

The initial formation of personality

Year

old

5-6

Year
old

3-4

Year
old
Psychological

characteristics

APPLICATION OF GAME ACTIVITIES IN MENTAL HEALTH EDUCATION OF KINDERGARTENS BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

873

The psychological cognitive characteristics of
3-4-year-old preschool children have strong
curiosity psychology, attachment psychology,
prominent group psychology, unconscious
imitation psychology and emotion -dominated
psychology. The behavior of children in this age
group is not controlled by brain rationality, and
it belongs to the behavior dominated by
primitive psychological needs and is greatly
influenced by the external environment (Freitag ,
Feineis-Matthews, Valerian et a l., 2012). For
example, 3-year-old children who just go to the
kindergarten will cry to find their mothers when
seeing other children crying to look for their
mothers. This emotional control psychology will
cause group cry, which is the most annoying for
caregivers. Attachment psychology is the
attachment to fosters. This kind of attachment
psychology is obvious in the separation on the
morning for 3-year-old children who just go to
kindergarten. The unconscious imitation
psychology, group psychology, the m utual
infection of anxiety emotion of 3 -4-year-old
children control the collective insecurity of the
whole small-class children, which increases the
workload and work intensity of kindergarten
teachers. Under the condition of abnormal
teacher-student ratio, kindergarten teachers
may make cognitive judgment or behavior which
is unfavorable to children’s mental health when
they are subjectively tired.

Preschoolers aged 4-5 years, compared with
the preschoolers aged 3-4 year, have changed a
lot. Their inner potentials are released and their
psychological cognitive characteristics are lively
and active, image thinking and intentional
behavior (Kraybill & Bell, 2013). Young children
of this age are energetic, lively and active. Their
cognition of things is no longer aimless wait-and-
see, but actively participation in guiding the
game process like a director to guide and
practice the characters in the game. The 4 -5-
year-old preschoolers have accumulated a
certain amount of things cognitive experience,
and the individual’s output behaviors begin to be
controlled by the brain, so that they can rely on
the existing mental cognitive structure of the
brain to carry out image-specific thinking
activities, and be able to listen to adult requests
and suggestions. This is reflected in intentional
behaviors in attention, memory and imagination,
more intentional concentration in listening to
stories and remembering the characteristics of
things, and more imaginative activities and game

activities.
The psychological cognitive characteristics of

preschoolers aged 5-6 years are mainly
manifested in their interest in learning and
asking, their abstract thinking ability and their
strong desire to express their stories, increased
intentional behaviors and initial personality
formation (Warman & Cohen, 2000). Unlike the
thirst for knowledge of 3-4-year-old children
under the psychological effect of curiosity, 5 -6-
year-old children are no longer satisfied with the
superficial knowledge when learning and asking.
They will spontaneously observe and explore
related knowledge related knowledge of things
at a deeper level under the joint action of active
thinking (Jobe, 2003). The abstract logic thinking
of 5-6-year-old children begin to develop, and
they can carry out thinking activities of
generalization thinking and logic abstraction. For
example, they can classify objects according to
their function and scope (Dijksterhuis & Aarts,
2009).

MENTAL HEALTH CURRICULUM SETTING BASED

ON THE GUIDANCE OF COGNITIVE PSYCHOLOGY

Direction of constructing game activities
based on children’s cognitive psychology

According to the psychological cognitive
characteristics of 3-year-old children, the game
activities developing children’s cognitive ability
are constructed. For example, based on the
children’s curiosity about the surrounding
things, the daily objects of interest are printed
on the drawings and cut into two, three or four
pieces. Then the children are asked to combine
freely to obtain the complete puzzle, and say the
name of the thing. Then we use the group
psychology to construct game activities to
develop the children’s interpersonal
communication ability. The children are asked to
exchange the puzzles or things in their hands, to
name the things in their hands, or to describe
and ask questions about the features of the
things, or to describe the color, shape, softness,
transparency of the things. Besides, we can ask
heuristic questions about the function and use of
the objects. Combining the unconscious
imitation psychology, we can cons truct game
activities of developing children’s living habits
and rules by organizing children to wash hands
and feed fruits for dolls, and let children imitate
adults to dress dolls, wear shoes and hats, and
fasten buttons for dolls.

NA YAO, LIPING WANG

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

874

Table 1. Construct the game according to the psychological characteristics of 3 -4 children

Age
Psychological

characteristics

Game setting

3-4 years
old

Strong curiosity
psychology

New thing puzzles, light clay making (hand clay painting, animal clay model,
etc.), leaf puzzle and other cognitive game activities

Prominent group
psychology

The eagle catches the chicken (children take turns to be the eagle and the
chicken, the teacher does the chicken mother)

Unconscious imitation
psychology

Act out a picture book story

Table 2. Construct the game according to the psychological characteristics of 4 -5 children

Age Psychological characteristics Game setting

4-5 years

old

Lively and active love to play
psychology and image thinking

Go out to play in order to know nature, the garden in the
campus, insects, flowers and trees things cognition

Intentional behavior
Role-playing game activities (e.g., playing doctor to see a patient,

playing cook to cook, etc.)

Table 3. The exclusive psychological characteristics of different ages

Age
Psychological
characteristics
Game setting

5-6 years
old

Knowledge discovery Books on “a hundred thousand whys”

Abstract logic
Mathematical game activities such as classification of things, number

graphics game activities
Expressive desire for

language
Picture book story description

According to the psychological cognitive

characteristics of 4-5-year-old children, they are
guided to observe their life through game
activities or multiple scenes in their
development stage of intentional behaviors.
They can constantly get growth in insig ht in the
observation and enrich psychological cognitive
structure with a large number of specific images
experience while developing focus, memory,
thinking organization, imagination and language
skills. The curiosity psychology of 4 -5-year-old
children is expressed as “Hundred Thousand
Whys” questions in lively and active activities.
The cognitive psychology with high thirst for
knowledge and imagination at this stage is
utilized to construct game activities to develop
the expression and creativity of chi ldren.
Moderate help, spiritual encouragement and
affirmation will be given to increase children’s
self-confidence, satisfaction, pleasure and
stimulate their desire to recreate, so as to
stimulate children’s positive and healthy mental
conditions.

Although the logical thinking ability of 5 -6-
year-old children is relatively superficial, their
thinking mode and thinking ability have
undergone qualitative changes, providing
important psychological cogni tive ability for

entering primary school learning activities.
According to the psychological characteristics of
5-6-year-old children with a strong desire to
express their stories and combining the
development of logical thinking and the
accumulation of language words, we can
construct story-telling game activities to satisfy
the children’s desire to express their stories,
thereby helping them to build up self -confidence
and develop ability of telling stories in a
coherent language, providing the basis for
entering primary school to learn written
language.

In accordance with children’s cognitive
psychology, we construct game activities that
are easy to be accepted by children. On the one
hand, the children’s advantageous behaviors
such as focus and creativity are optimized. On
the other hand, the children’s problem behaviors
considered by the adult are improved, and
children’s internal stress and anxiety are
released from game activities, and they gain
pleasure and self-confidence from game
activities, which enables children to reconstruct
the psychological cognitive structure in the game
activities with autonomy and participation, and
develop the output behavior in a positive
direction. Figure 2 shows a flowchart of

APPLICATION OF GAME ACTIVITIES IN MENTAL HEALTH EDUCATION OF KINDERGARTENS BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

875

constructing game activities based on the
children’s cognitive psychology.

Figure 2. Construct the flow chart of game
activities based on children’s cognitive
psychology

Psychological
characteristics

of children

Game

Behavior

advantages

Behavior

disadvantages
improve

Optimize

Healthy

psychological

feelings

Normative

behavior

output

3-6-year-old preschool children own the

absorptive mind so that they can absorb a large
amount of input information from the
surrounding environment through the sense
organs in a conscious state. Children have
different physical and mental development
characteristic in different age groups, that is,
each age group has a specific psychological
cognitive structure. We should carry out mental
health education according to the specific
psychological cognitive structure, and construct
appropriate game activities to guide the children
to explore and learn freely and stimulate the
children’s internal potentials, so as to acquire
self-confidence, security and concentration,
creativity and good interpersonal skills.

Guide the direction of mental health

education according to the cognitive
psychological model of children

Every child’s cognition of the world is based
on his/her acquired knowledge structure, which
constructs the world he/she sees, and guides the
output of his/her own behavior. The abnormal
behaviors behind the problem children depend
on the knowledge structure acquired from the
living environment. The purpose of mental
health education curriculum in kindergarten is to
reconstruct the children’s cognitive structure
through game activities and make them de velop
into positive behaviors. It is significant to let the
children have the autonomy and the
participation feeling. Positive absorption of
active participation is greater than the passive
discipline teaching. The trial and error learning
game setting is advantageous to the mental
development.

Children’s thinking mode and behavior mode

are influenced by the original psychological
cognition mode and children of different age
groups in kindergarten have different
acceptance and absorption abilities for the s ame
game activities, so the construction of game
activities should be in accordance with children’s
psychological cognitive structure.

As shown in Figure 3, the orange pentagram
represents the psychological cognitive structure
of 3-4-year-old children, i.e., cognitive models
such as emotion, unconscious imitation, and
prominent group psychology. The orange
snowflake square represents the psychological
cognitive structure of 4-5-year-old children,
namely, the psychological structure of active
exploration, intentional behavior and image
thinking. The orange hexagon represents the
psychological cognitive structure of 5 -6-year-old
children, namely, the psychological structure of
interest in learning and asking, abstract logical
thinking, the desire of language expression, and
increased intentional behavior. Under the same
input of game activities, children of different age
groups have different emphasis and ability
development and exercise directions, and
children of the same age group also have
different cognitive results according to their own
psychological cognitive structure. Therefore,
kindergarten teachers only need to give children
a more relaxed, free and respectful learning
environment, so that children can maximize the
development of their internal potentials without
interference.

Figure 3. Cognitive models of children of
different ages

3-4 year old

5-6 year old

4-5 year old

Input model Cognitive model Restoring model

NA YAO, LIPING WANG

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

876

The same input information is compiled
through different psychological cognitive
structures, and each person reprocesses the
input information according to his/her own
existing knowledge structure. The obtained
result depends on the original experience and
knowledge inventory. Children of different age
groups reprocess the same input model, and the
obtained cognitive remodeling model has the
characteristics common to each age group.
Therefore, it is necessary to construct game
activities pertinently for chi ldren of different
ages.

The important period for the development of
children’s ability to understand psychological
emotion is 4-5 years old in the middle class of
kindergarten. Thus, constructing game activities
in mental health in kindergarten should aim at
the cognitive psychology of 4 -5 years old
children in the sensitive period of emotion
understanding. It is easier to promote the
development of young children’s emotional
understanding by setting up game activities with
role exchange and emotional alignment, and it is
easier to establish friendly communication
modes among young children, and to understand
other people’s behaviors and make appropriate
responses, thus reducing the number and
frequency of attacks.

CASE STUDY ON THE CONSTRUCTION OF

GAME ACTIVITIES IN COGNITIVE PSYCHOLOGY

From the perspective of children’s own
cognitive psychology, the psychological
problems of children are mainly manifested as
lack of sense of security and self -confidence,
which leads to shyness, self-abasement and even
depression. Adults usually analyze children’s
behavior problems from the perspective of
cognitive psychology of children’s ability
because adults think that children’s ability is
insufficient. Children’s learning exploration
behavior, independent behavior and trial and
error behavior are deprived, which results in
children’s weak ability to resist setbacks, and
weak self-confidence and independent learning
ability. The problem behavior of emotional
regulation disorder comes from the improper
educational method. Excessive doting education
will cause the children to be willful and self –
respecting. Emotional disorder and aggressive
behavior appear when their desire is not
satisfied.

Lack of sense of security will lead to anxiety,
crying, cowardice and self-abasement. Artificial
hindrance deprives young children of their
desire and curiosity, fear of failure, and
aggression caused by abnormal emotional
regulation which come from the deviation of
children’s psychological cognitive structure.
When the children’s inner needs are concerned
and satisfied, their behaviors are enthusiastic,
loving and positive. On the contrary, when the
children’s inner needs are not concerned or
ignored, their emotional color is black and
negative and their behaviors are prone to appear
anger, attack, and destruction.

This study observes and records the
frequency of occurrence of anxiety behavior and
the duration of attentiveness, and the period of
data record is four months, one semester. A total
of 80 groups of sample values are obtained and
the trend map of the psychol ogical behavior
improved by game activities is shown in Figure 4.
It is found that constructing game activities in
accordance with cognitive psychology meets the
psychological needs of the children of the
corresponding age group, and the abilities
required for game activities accord with the
mental ability range of the children. Children
obtain sense of satisfaction and security through
pleasant game activities, thus reducing their
inner anxiety. When just going to kindergarten,
children frequently appear anxiety behaviors.
They gradually obtain emotional stability as
shown by a solid pink line in Figure 4. Later, their
learning focus gradually increases as shown by
the blue dotted line in Figure 4.

Figure 4. A chart of mental behavior
improved by game activity

0 10 20 30 40 50 60 70 8

0
5

Concentration

F
r
e
q

u
e
n

c
y
/D

a
y

Test times

Anxiety behavior

C
o
n
c
e
n
tr

a
ti

o
n
/T

im
e

0
40
35
30
25
20
15
10
5

APPLICATION OF GAME ACTIVITIES IN MENTAL HEALTH EDUCATION OF KINDERGARTENS BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°2, 871-877
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

877

CONCLUSIONS

This study constructs game activities from the
professional angle of cognitive psychology, and
explores the relationship between mental health
education in kindergarten and the effect of game
activities on children’s mental health, and draws
conclusions as follows:

(1) Based on the psychological cognitive
characteristics of children, this study
summarizes the psychological needs of
kindergarten children of all ages, and
understands the real mental health demands
behind the problem behaviors.

(2) Based on children’s cognitive psychology,
this study constructs appropriate game activities
pertinently according to the psychological
cognitive characteristics of different ages, to
stimulate the children’s inner potentials
purposefully.

(3) According to the children’s cognit ive
psychology model, the mental health education
curriculum in kindergarten is set up, and the
children’s cognitive structure is reconstructed
through game activities, so as to make them
develop towards positive behavior.

The empirical study shows that th e
construction of game activities based on
cognitive psychology has positive effect on
children’s psychology and behavior.

Acknowledgement

Social science fund project of Hebei province,
Study on the gamification model of construction
activity curricula in kindergartens, No: HB18JY054.

REFERENCES

Dijksterhuis, A., & Aarts, H. (2009). Goals, attention,
and (un)consciousness. Annual Review of
Psychology, 61(1), 467-490.

Donaldson, J. M., Vollmer, T. R., Krous, T., Downs, S.,
& Berard, K. P. (2011). An evaluation of the good

behavior game in kindergarten classrooms.
Journal of Applied Behavior Analysis, 44(3), 605-
609.

Feshbach, S., & Price, J. (1984). Cognitive
competencies and aggressive behavior: a
developmental study. Aggressive Behavior, 10(3),
185-200.

Freitag, C. M., Feineis-Matthews, S., Valerian, J.,
Teufel, K., & Wilker, C. (2012). The frankfurt early
intervention program ffip for preschool aged
children with autism spectrum disorder: a pilot
study. Journal of Neural Transmission, 119(9),
1011-1021.

Jobe, J. B. (2003). Cognitive psychology and self-
reports: models and methods. Quality of Life
Research, 12(3), 219-227.

Jones, D. E., Greenberg, M., & Crowley, M. (2015).
Early social-emotional functioning and public
health: the relationship between kindergarten
social competence and future wellness.
American Journal of Public Health, 105(11), e1-
e8.

Kraybill, J. H., & Bell, M. A. (2013). Infancy predictors
of preschool and post-kindergarten executive
function. Developmental Psychobiology,55(5),
530-538.

Larson, K., Russ, S. A., Nelson, B. B., Olson, L. M., &
Halfon, N. (2015). Cognitive ability at
kindergarten entry and socioeconomic status.
PEDIATRICS, 135(2), e440-e448.

Melhuish, E. C. (2011). Preschool matters. Science,
333(6040), 299-300.

Nacher, V., Garcia-Sanjuan, F., & Jaen, J. (2016).
Interactive technologies for preschool game-
based instruction: experiences and future
challenges. Entertainment Computing,
S1875952116300210.

Renaud, L., & Suissa, S. (1989). Evaluation of the
efficacy of simulation game activities in traffic
safety education of kindergarten children.
American Journal of Public Health, 153(3), 307-9.

Warman, D. M., & Cohen, R. (2000). Stability of
aggressive behaviors and children’s peer
relationships. Aggressive Behavior, 26(4), 277-
290.

Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.

Revista Argentina de Clínica Psicológica

2020, Vol. XXIX, N°2, 854-859

DOI: 10.24205/03276716.2020.321

2020, Vol. XXIX, N°2, 854-859

REVISTA ARGENTINA

DE CLÍNICA PSICOLÓGICA

854

APPLICATION OF PSYCHOLOGICAL PREFERENCE AND EMOTIONAL
GUIDANCE IN THE PERFORMANCE OF FILM AND TELEVISION ART: AN

ANALYSIS BASED ON BASED ON COGNITIVE PSYCHOLOGY

Dunru Xie

Abstract

The performance of film and television art enriches our life. The psychological preference and
connoisseurship of the audience for film and television art may vary with age, gender, region and cultural
background. Based on cognitive psychology, this study investigates the status quo of film and television art in
the Internet era through a questionnaire survey, and explores the application of psychological preference and
emotional guidance in the performance of film and television art. The results show that, from the perspective
of cognitive psychology, the appreciation and evaluation of the audience for the performance of film and
television art is a form of artistic accomplishment; the psychology and emotion of the audience is influenced
by the plot, visual rhythm, special effects, color and animation of film and television art; the audience’s
preference for film and television art is determined by the expressive force, infection, and the theme of
performance. The research lays a theoretical basis for the quality and proliferation of film and television art.

Key words: Film and Television Art, Psychological Preference, Connoisseurship, Cognitive
Psychology, Emotional Guidance.

Received: 18-05-19 | Accepted: 12-08-19

INTRODUCTION

Film and television have been dominant
throughout the film and television art industry.
As a tool of cultural industry or ideology, they
can’t be separated from the support of the
audience (Haslam Parsons, Omylinska-Thurston
et al., 2019). The social conditions of China’s
rapid development provide convenience for the
change of film and television art. With the
audience’s preference for the mainstream
culture and the mainstream ideology, mass
culture and consumer culture constantly
conflict, collide and cons pire under the
background of the rapid development of China’s
film and television industry, constituting a
diverse culture of film and television art (Halpern

Department of art design, Sichuan Film and Television
University, Chengdu 61000, China.
E-Mail: x670245799@163.com

& O”Connor, 2013). Taking cognitive psychology
as the basis for evaluating the aesthetic
psychology of film and television embodies the
complexity of the aesthetic system of film and
television art. The aesthetic psychology
displayed by people according to psychological
preference and psychological emotion is active
and comprehensive, including psychological
factors such as emotional color, thinking
association, aesthetic perception and plot
understanding (Stambulova, & Wylle man, 2018;
Krentz & Earl, 2013). Psychological emotion is
the expression and evaluation of the relationship
between people and the surrounding world,
which is a complex psychological reaction and an
attitude of the subject to the object (Dubey,
Ropar, & Hamilton, 2016).

The performance of film and television art is
related to film and television color, film and
television special effects and film and television
animation. Color, special effects and animation

DUNRU XIE

2020, Vol. XXIX, N°2, 854-859
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

855

greatly enrich the performance means of film,
and guide the audience’s emotion and
preference through intense visual impact and
auditory shock (Buchheim & Kolaska, 2016). The
producers and designers

of film and television

art will follow people’s psychological preference,
and will also consider guiding the audience’s
emotion through the visual and auditory aspects
of film and television art. Therefore,
psychological preference and emotional
guidance are the focus of film and television art
design (Balietti Goldstone, & Helbing, 2016).
Along with the development of the Internet, film
and television works are widely produced with
different levels of quality due to inadequate
network supervision, leading to low quality of
film and television art works. Thus, the current
film and television art does n ot only refer to
television and films. Short videos played with the
help of new media are also classified as film and
television art, with greater impact on people’s
emotion and psychology (Levine-Madori &
Bendel, 2013; Karasik, 2014). Based on the
principle of cognitive psychology, this study
investigates the status quo of film and television
art under the Internet, and explores the
application of psychological preference and
emotional guidance in the performance of film
and television art. This study prov ides a
theoretical basis for the high quality
development and transmission of film and
television art.

PSYCHOLOGICAL MECHANISM OF AUDIENCE

ON THE PERFORMANCE OF FILM AND TELEVISION
ART

The cultural types of film and television
works are mainly divided into dominant cultural
type, mass cultural type and elite cultural type.
The audience of film and television art will vary
with region, gender, cultural level and age, and
their psychological preference is different (Smith
& Reffin, 2006). The audience’s aesthetic level,
cultural level and connoisseurship of film and
television art are related to their psychological
preference. When meeting their own preferred
film and television works, the audience will
consciously take advantage of the positive
effects expressed by the film and television art
works to improve them and build complete
values of life (Quigley, Westall, Wade et al.,
2014). College students are taken as the
audience group to explore their preference for

themes of film and television art works. Figure 1
shows the statistics of theme type of films of
college students, which clearly shows that male
college students prefer science fiction films and
action films, while female college students
prefer comedy films and love films.

The audience’s psychological preference for
film and television art is realized through
aesthetic perception, and cognitive psychology is
the basic condition for the formation of
psychological preference. The co gnitive
psychological activity of the audience is positive
and initiative (Skavronskaya, Scott, Moyle et al.,
2017). From the perspective of cognitive
psychology, aesthetic cognition is fast. If
aesthetic cognition is used as the psychological
basis for elevating film and television art, it will
be found that the aesthetic system is complex
(Carrier, 2011). The audience’s psychological
mechanism of film and television art is reflected
in the role played in the cognitive process. The
audience’s perception, imagination, association,
expression and evaluation of film and television
art performance are the most realistic and
tedious psychological reactions.

Figure 1. Statistics of theme type of films of
college students

100

120

140

160

180

OthersAction
Literature

and art
Cartoon

Science

fiction
WarLoveComedy

N
u

e
r

o
f

p
e
o

p
le

Male

Female

STUDY ON THE STATUS QUO OF FILM AND

TELEVISION ART UNDER THE INTERNET

Investigation and study on the status quo of
film and television artistic accomplishment

Film and television artistic accomplishment is
actually the artistic accomplishment in the
course of designing, shooting and editing of film
and television. According to cognitive
psychology analysis, the thought exhibited by

APPLICATION OF PSYCHOLOGICAL PREFERENCE AND EMOTIONAL GUIDANCE IN THE PERFORMANCE OF FILM AND TELEVISION ART: AN ANALYSIS BASED ON BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°2, 854-859
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

856

the audience in enjoying and evaluating the
artistic performance of film and television is also
a form of artistic accomplishment. With the
rapid development of the Internet, the impact of
the Internet on the film and television artistic
accomplishment is facing not only new
opportunities, but also challenges of new
technologies. In order to probe into the status
quo of film and television artistic
accomplishment, this study uses questionnaire
survey. 1,000 questionnaires are distributed and
802 valid questionnaires are collected. Statistical
analysis method is adopted to analyze. Figure 2
shows the audience’s understanding of the
background of film and television culture. When
people have finished watching a film and
television work, most of the audience do not
know the background of film and television
culture. The survey data show that the
audience’s understanding of the background of
film and television culture accounts for less than
20%. Figure 3 shows the audience’s
understanding of film and television art plots.
The audience’s understanding of film and
television art plot setting shows the same law as
the cultural background. Most people don’t
know why such an art plot is set and lack the
ability to analyze the theme and artistic creation
of film and television art. Figure 4 shows t he
focus of the audience on film and television art.
The focus of the audience on film and television
art is influenced by gender factor. Female
audience prefers to pursue film and television
plots and actors, while male audience prefers to
focus on film and television actors and directors,
which is mainly influenced by psychological
preference.

Figure 2. Audience’s understanding of film
and television cultural background

3.33%

10%

13.33%

26.67%

46.67%

Ignorant

Basic understanding

General

Understanding

Very understanding

Figure 3. Audience’s understanding of film
and television art plot

1.74%

6.94%

17.36%

25.35%

48.61%

Ignorant
Basic understanding
General
Understanding
Very understanding

Figure 4. Research on the audience’s focus
on film and television art

0
20
40
60
80
100
120
140

Others
Lens

language
MontageClipPictureVoiceDirectorActorPlot

N
u

m
b

e
r
o
f
p
e
o
p
le
Male
Female

Factors influencing the film and television

artistic accomplishment

At present, the performance process of film

and television art is not limited to the
professional staff engaged in film and television
work. Anyone with mobile phone, camera and
DV can become the producer and publisher of
film and television works. The immediacy,
sharing and diversity of network resources make
the choice space of films and television works
larger and larger, which puts forward higher
request and challenge to people’s cognition
ability and connoisseurship. Figure 5 shows the
factors influencing the film and television artistic
accomplishment, including environmental
factors, educational factors and personal factor.
The online film and television art resources
increase the audience’s ability of self -selection
and connoisseurship. Film and television art are
not a pure commodity. It has numerous
connections with art, literature and culture. The
essence of film and television art performance
lies in that it can’t be separated from the
interests of the audience and the interest

DUNRU XIE

2020, Vol. XXIX, N°2, 854-859
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

857

combination becomes the type. The preference
of the audience determines the type of film and
television, which is inseparable from the interest
of the audience.

Figure 5. The influence factors of film and
television artistic accomplishment
performance

The influencing factors

of film and television
artistic accomplishment

performance

Environmental

factors

Educational

factors

Individual factors

The increase of network film and television

resources affects the ability of the audience to

choose independently

The audience’s ability to appreciate the complex

and influential content of film and television

Insufficient attention

The content and form of film and television art

literacy education lack novelty

Lack of new media technologies and dedicated

staff

Low participation in film reviews

APPLICATION OF PSYCHOLOGICAL PREFERENCE
AND EMOTIONAL GUIDANCE IN THE
PERFORMANCE PROCESS OF FILM AND TELEVISION
ART

An audience psychological analysis of film
and television art performance

The use of new media and the audience’s
evaluation of film and television art directly
affect the audience’s emotion towards film and
television art. Taking film and television as an
example, once the score of a film is low or the
word of mouth is poor, it directly affects the
audience’s emotion or psychological preference.
Figure 7 shows the use of new media. Most of
the current film and television art is promoted in
the form of posters, WeChat public number
push, social network site or online forum. Figure
8 is the result of film review after watching film
and television works. It can be clearly seen that
the audience participating in the film review
after watching film and television works is less
than 30%, which is in a relatively low proport ion.
Based on the mass psychological preference and
cognition, it is very important to explore the
audience’s psychological preference,
psychological demand and psychological
emotion in the history of film and television art.
The creation and performance o f film and
television art is to reflect the background of the
times, objective facts or explain the truth, and
the collision with the audience is the core of the
expression of film and television art. Different
from other art categories, film and televisio n art

is created for the audience, and the audience’s
psychological preference and emotional
guidance determine the score and value of film
and television art. Therefore, the promotion and
correction of the audience’s preference
determine the significance of the existence of
film and television art. Figure 6 is the division of
the performance dimensions of film and
television art. It mainly includes cognition,
ability and awareness of film and television art,
including cognition, selection, appreciation,
speculation, creation and evaluation of film and
television art.

Figure 6. The division of film and television
art performance dimensions

Film and television

art literacy

Film and television art

media cognition

Film and television art

media ability

Film and television art

media awareness

Cognition

Choice

Appreciation

Speculation

Creation

Evaluation

Figure 7. Use of new media

9.8%

20.26%34.64%

20.92% 14.38%

WeChat public number push

Network forum

Poster

Social networking sites

Others

Figure 8. Review results after watching film
and television art works

22.75%

16.08%

30.98%

24.31%

5.88%

Often

Occasionally

General

Few

Never

APPLICATION OF PSYCHOLOGICAL PREFERENCE AND EMOTIONAL GUIDANCE IN THE PERFORMANCE OF FILM AND TELEVISION ART: AN ANALYSIS BASED ON BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°2, 854-859
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

858

Case study of film and television art based

on cognitive psychology

Figure 9. Different examples of film and
television art

(a) Stills of Monster Company

(b) The picture of Nezha

The film and television art plot, visual

rhythm, film and television special effects, film
and television color and film and television
animation impact the psychology and emotion of
the audience. In terms of cognitive psychology,
it takes only seven seconds for things to form a
first impression in people’s thinking, and the first

feeling of the audience directly affects their
psychological preference and emotion. With the
improvement of people’s visual appreciation
ability and the higher and higher require ment for
film and television color, film and television
animation and film and television special effects,
the factors that determine the quality of film and
television art are no longer the theme, but the
expressive force and infection of film and
television art. Figure 9 shows different example
of film and television art. Figure 9 (a) shows stills
of Monster Company. It can be seen that the
visual elements of film and television works are
very different. Grotesque humor and interesting
animation impress the audience with
exaggerated artistic modeling. Figure 9 (b) is
stills of Nezha, which gains the audience’s
psychological preference through film and
television animation and color. Through the
audience’s film review after watching the film,
the film and television plot is highly praised.
Figure 9 (c) shows the bullet scene of the Matrix,
creating a scene with the same atmosphere as
the whole film through special effects, and
bringing a fresh feeling to the audience through
plot setting and special effects. Film and
television rhythm, film and television special
effects, film and television color and film and
television animation all create a clear artistic
expression way, which is reflected in
psychological cognition, psychological
preference and psychological emotion in
psychology through the change of visual rhythm.

CONCLUSIONS

Based on the principle of cognitive
psychology, this study investigates the status
quo of film and television art under the Internet,
and explores the application of psychological
preference and emotional guidance in the
performance process of film and television art.
Conclusions have been drawn as follows:

(1) The audience’s psychological mechanism
of film and television art performance is
reflected in the role played in the cognitive
process. The audience’s perception,
imagination, association, expression and
evaluation of film and television art performance
are the most realistic and tedious psychological
reactions.

(2) Online film and television art resources
increase the audience’s ability of self -selection
and connoisseurship. The essence of film and

https://image.baidu.com/search/detail?ct=503316480&z=&tn=baiduimagedetail&ipn=d&word=%E6%80%AA%E7%89%A9%E5%85%AC%E5%8F%B8%E5%89%A7%E7%85%A7&step_word=&ie=utf-8&in=&cl=2&lm=-1&st=-1&hd=&latest=&copyright=&cs=787308282,3723058028&os=570596244,3943459456&simid=3385622059,281530451&pn=4&rn=1&di=155760&ln=1137&fr=&fmq=1573634556573_R&ic=&s=undefined&se=&sme=&tab=0&width=&height=&face=undefined&is=0,0&istype=2&ist=&jit=&bdtype=0&spn=0&pi=0&gsm=0&objurl=http://pic.jj20.com/up/allimg/512/0H912125536/120H9125536-0 &rpstart=0&rpnum=0&adpicid=0&force=undefined

https://image.baidu.com/search/detail?ct=503316480&z=&tn=baiduimagedetail&ipn=d&word=%E9%BB%91%E5%AE%A2%E5%B8%9D%E5%9B%BD%E5%AD%90%E5%BC%B9%E5%9C%BA%E6%99%AF&step_word=&ie=utf-8&in=&cl=2&lm=-1&st=-1&hd=&latest=&copyright=&cs=1012039915,3033059366&os=2845335729,2090946969&simid=4133137623,454627295&pn=7&rn=1&di=12320&ln=198&fr=&fmq=1573637326572_R&ic=&s=undefined&se=&sme=&tab=0&width=&height=&face=undefined&is=0,0&istype=2&ist=&jit=&bdtype=0&spn=0&pi=0&gsm=0&objurl=http://img.mp.itc.cn/q_70,c_zoom,w_640/upload/20160926/26646ae29fe44aa89bcd8af6221fc36e_th &rpstart=0&rpnum=0&adpicid=0&force=undefined

DUNRU XIE

2020, Vol. XXIX, N°2, 854-859
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

859

television art performance lies in that it can’t be
separated from the interests of the audience and
the interest combination becomes the type. The
preference of the audience determines the type
of film and television, which is inseparable from
the interest of the audience.

(3) Film and television rhythm, film and
television special effects, film and television
color and film and television animation all create
a clear artistic expression way, which is reflected
in psychological cognition, psychological
preference and psychological emotion in
psychology through the change of visual rhythm.

REFERENCES

Balietti, S., Goldstone, R. L., & Helbing, D. (2016).
Peer review and competition in the art exhibition
game. Proceedings of the National Academy of
Sciences, 113(30), 8414-8419.

Buchheim, L., & Kolaska, T. (2016). Weather and the
psychology of purchasing outdoor-movie tickets.
Discussion Papers in Economics, 63(11), 3718-
3738.

Carrier, D. (2011). The future of art history in the
context of psychology and the cognitive sciences.
Leonardo, 43(5), 386.

Dubey, I., Ropar, D., & Hamilton, A. F. D. C. (2016).
Brief report: a comparison of the preference for
viewing social and non-social movies in typical
and autistic adolescents. Journal of Autism and
Developmental Disorders, 47(2), 514-519.

Halpern, A. R., & O”Connor, M. G. (2013). Stability of
art preference in frontotemporal dementia.
Psychology of Aesthetics, Creativity, and the Arts,
7(1), 95-99.

Haslam, S., Parsons, A., Omylinska-Thurston, J., Nair,

K., Harlow, J., Lewis, J., Thurston, S., Griffin, J.,
Dubrow-Marshall, L., Karkou, V. (2019). Arts for
the blues – a new creative psychological therapy
for depression: a pilot workshop report.
Perspectives in Public Health, 139(3), 137-146.

Karasik, R. J. (2014). Transcending dementia through
the ttap method: a new psychology of art, brain,
and cognition by linda levine madori.
Educational Gerontology, 40(2), 152-153.

Krentz, U. C., & Earl, R. K. (2013). The baby as
beholder: adults and infants have common
preferences for original art. Psychology of
Aesthetics, Creativity, and the Arts, 7(2), 181-190.

Levine-Madori, L., & Bendel, T. (2013). Research to
practice: the ttap method a new psychology of
art, brain and cognition. Alzheimer’s & Dementia,
9(4), 293.

Quigley, C., Westall, C., Wade, N. J., Longstaffe, K.,
Cavanagh, P., & Conway, B. R. (2014). Review:
visual attention and consciousness, nystagmus in
infancy and childhood, edgar rubin and
psychology in denmark: figure and ground,
cognitive search: evolution, algorithms, and the
brain, the psychology of visual art: eye, brain and
art. Perception, 43(6), 595-604.

Skavronskaya, L., Scott, N., Moyle, B., Le, D.,
Hadinejad, A., & Zhang, R., et al. (2017).
Cognitive psychology and tourism research: state
of the art. Tourism Review, 72(2), 221-237.

Smith, & Reffin, B. (2006). Constraint is freedom. an
application of zombie to certain aspects of art
and cognitive psychology. Technoetic Arts, 4(1),
55-64.

Stambulova, N. B., & Wylleman, P. (2018).
Psychology of athletes’ dual careers: a state-of-
the-art critical review of the european discourse.
Psychology of Sport and Exercise, 42, 74-88.

Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.

Revista Argentina de Clínica Psicológica
2020, Vol. XXIX, N°1, 1016-1021
DOI: 10.24205/03276716.2020.142

2020, Vol. XXIX, N°1,1016-1021

REVISTA ARGENTINA

DE CLÍNICA PSICOLÓGICA

1016

ANALYSIS ON UNCERTAINTIES IN JUDICIAL DECISION BASED ON
COGNITIVE

PSYCHOLOGY

Yongchao Li*

Abstract

A judicial decision is arrived at through fact reasoning, legal reasoning and decision reasoning. Due to the
difference between judges in cognitive psychology, the above reasoning process is affected by uncertainty
thought, resulting in uncertainties of the judicial decision. This paper explores deep into the uncertainties in
judicial decision from the perspective of cognitive psychology, and puts forward some countermeasures.
Specifically, the author analysed the cognition psychology of judges in the process of judicial decision,
examined the manifestation of uncertainties in judicial decision, and conducted a psychological analysis of
uncertainty thought. The results show that the cognitive psychology of law and justice is formed through long-
term integration of learning and work; in the process of judicial decisions, judges are limited by their own
cognition in the process of fact reasoning, legal reasoning and decision reasoning, resulting in an unfair
judgment against one party; the uncertainties of judicial decision should be reduced from aspects of legislation,
legal application procedures, the personal emotions and qualities of judges, as well as publicity and education.
The research findings lay the basis for the application of cognitive psychology in judicial decisions.

Key words: Judicial Decision, Reasoning, Uncertainty Thought, Cognitive Psychology.
Received: 19-02-19 | Accepted: 09-07-19

INTRODUCTION

The certainty of judicial decision is an
important manifestation of formal rationality,
mainly through the consistency of legal rules,
but in any judicial decision system, “certainty” is
the goal pursued (Foxall, 2014). At present, no
jurist or legal institution’s thought can occupy an
absolute dominant position; it is only based on a
certain angle, and absolute judicial uncertainty
or certainty is one-sided (Rand, 2015).
Moreover, the judicial decision process is a
passive right, initiated on the litigation of the
litigant, generally including th e identification of
legal facts, the search for relevant legal norms,
the subsumption according to the legal order,
and the final declaration of judicial decisions
(Vlek, 2010). In the entire judicial decision

Zhengzhou University School of Law, Zhengzhou 450001,
China.
E-Mail: yun815207@163.com

process, the uncertainties of the judicial pro cess
include the identification of legal facts and
evidence collection and adoption, the timeliness
of evidence, the criteria for the identification of
evidence or facts, etc.; besides, considering the
extreme complicatedness of the cases, the
conflict of legal rules between judicial staffs is
also a factor of uncertainty (Brinkman, 2017).

The judicial process is related to people’s
cognitive psychological state. As a special social
psychological activity, judicial decisions formally
express people’s attitudes, concepts and
theories about legal uncertainty, as well as their
thoughts, perspectives, knowledge and
psychology, etc. on judicial phen omena (Stone,
2010; Emma & Mcnaught, 2016). People’s
internal psychological behaviours are only ones
that are no different from external behaviours
while cognitive psychology is a by -product of
behaviourism, in which people are active
information explorers and do not change their
thoughts and behaviours under the

YONGCHAO LI

2020, Vol. XXIX, N°1,1016-1021
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

1017

environmental stimuli (Cunliffe, 2014). The
psychological cognitive process of law is a
cognition collection of legal characteristics
formed in the individual consciousness after
long-term accumulation and integration,
including the subtle learning in daily life, work
and study (Bishop, 2017). It is the uncertainty in
the process of judicial decision that makes it
impossible to anticipate the legitimacy of one’s
actions and thus fail to correctly un derstand the
outcome of the decision (Davies, 2012). Based on
the theory of cognitive psychology, this paper
aims to explore the uncertainty of ideas in the
process of judicial decision and gives
corresponding countermeasures.

THE PROCESS OF JUDICIAL COGNITION

PSYCHOLOGY

Judicial decisions are based on legal ground,
which is generated in social consciousness. The
formation of legal consciousness is an important
part of people’s cognitive psychology. People’s
legal cognition process starts with sporadic le gal
phenomena (Shapiro, Mixon, Jackson et al., 2015).
One’s cognitive psychology begins from birth.
Table 1 lists the four stages of cognitive
development: at the sensorimotor stage, there
develops a sense of good and bad, and the
emotion will be more devoted to oneself; at the
pre-operational stage, social behavioural
activities begin to manifest, and there is no
intentional concept in the process of cognitive
psychology formation; at the concrete
operational stage, human will and autonomy are
mainly formed; at the formal operational,
people’s idealistic emotions appear and
personality begins to form. Figure 1 shows the
influencing factors of judicial cognition
psychology, mainly including social learning,
communication and mass communication,
popularization of law, legal education, and legal
research.

People’s cognitive psychology of the judiciary
needs to be taught and induced, e.g., in the
course of people’s growth or learning (Flanagan
& Ahern, 2011). The process of communication
is also an important channel for the formation of
cognitive psychology, allowing people to share
news, ideas and attitudes, and establish a
cognitive identity and ideological resonance
between people (Soboleva, 2013). The
popularization and publicity of judicial
knowledge has enabled people to directly

acquire judicial knowledge, gradually cultivated
their awareness of judicial cognition, and
continued to lay a solid legal foundation for the
formation of a society ruled by law.

Judicial

lectures, seminars, reading clubs, and legal
columns have all become common popularized
forms of law so that people are always in contact
with the law and obey the law, which can also
greatly promote the cultivation of people’s
judicial cognition.

Table 1. Overview of each stage of cognitive
development

Phase Principal variation

Perception-
motion phase

The initial likes and dislikes
emerge, and emotions pour out on

the self

Preoperational
stage

For real social behavior to begin
with, there is no concept of
intention in moral reasoning

Concrete
operational stage

The formation of will and the
emergence of autonomy

Formal operation
stage

The emergence of idealistic
feelings, the formation of

personality began, began to adapt
to the adult world

Figure 1. Formation of judicial cognitive
psychology

Formation of judicial

cognitive psychology

Social learning

Communication and mass communication

Franco-prussian propaganda

Legal education

Legal research

PSYCHOLOGICAL ANALYSIS OF UNCERTAINTY IN

JUDICIAL DECISIONS

The manifestation of uncertainty in judicial
decisions

The uncertainty of judicial decisions is
generated by the coordination and balance
between the legality and rationality of the ruling.
When the legal norms of the adjudicated cases
are ambiguous or the facts are in conflict, the
judges will have an idea of uncertainty; even if
the legal norms of the judicial decisions are clear
and the facts are clearly defined, the judicial
officials may be subject to internal bias, the
trade-off between morality and law, and the
influence of public opinion, and also reveals the
idea of uncertainty. Figure 2 shows the main

ANALYSIS ON UNCERTAINTIES IN JUDICIAL DECISION BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°1,1016-1021
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

1018

manifestation of the uncertainty in terms of fact
reasoning, legal reasoning, and

decision

reasoning in judicial decision. Among them, the
uncertainty of fact reasoning includes that of
causal connection and factual reasoni ng; the
uncertainty of legal reasoning includes that of
legal norms and legal interpretation; the
uncertainty of decision reasoning includes that
of statutory law and judge-made law, etc. Figure
3 shows the causes of uncertainty in judicial
decisions, including the uncertainty of natural
language, the limitations of legislation, the non –
self-sufficiency of the legal system, and the
subjective factors of judges.

Psychological analysis of uncertainty
thought

The psychological analysis of uncertainty
thought mainly includes the acquisition of
psychological knowledge, the establishment of

psychological ideals, the cultivation of
psychological emotions, the occurrence of
psychological will and the establishment o f
psychological beliefs. Due to the interaction of
life experience and sociality, people will
gradually integrate effective information into
their existing psychological cognition and
assimilate this information. The formation of
uncertainty thoughts is related to human
maturity, social activities and judicial activities.
Among them, social activities have a more
significant influence, because people’s judicial
psychological cognition depends on social
interaction. From the perspective of cognitive
psychology, the idea of uncertainty originates
from the process of people pursuing justice,
maintaining order and realizing the rule of law.
Through long-term judicial research, the
certainty of pursuing judicial decisions is mainly
to satisfy the rules of conduct of all parties.

Figure 2. The main manifestation of the thought of uncertainty in judicial decision

The main performance of

the uncertainty thought

judicial decision

Uncertainty of fact

reasoning

Uncertainty of

legal reasoning

Uncertainty in

decision reasoning

Philosophical questioning of

uncertainty

in fact reasoning

Judicial analysis of uncertainty

in fact reasoning

Uncertainty of legal norms

Uncertainty of legal

interpretation

Uncertainty of decision

reasoning in statutory law

Uncertainty of decision

reasoning in judgment method

Diversity of the meaning of facts

Causal probability

Non-reproducibility of objective facts

The tailorability of normative facts

Legal doubts

Legal loopholes

Creativity of legal interpretation

Openness of legal interpretation

Figure 3. The causes of uncertainty thought in judicial decision

The causes of

uncertainty thought in

judicial decision

Uncertainty of natural language

Limitations of legislation

Insufficiency of the legal

system

Subjective factors of judicial officers

YONGCHAO LI

2020, Vol. XXIX, N°1,1016-1021
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

1019

SIGNIFICANCE AND COUNTERMEASURES OF

IDEOLOGICAL UNCERTAINTY IN JUDICIAL DECISION

Theoretical analysis of judicial decisions and
significance of ideological uncertainty

A complete judicial decision includes not only
statutory judicial interpretations, but also
interpretations given in the application of law
and of the judge’s authority. Current legislation
and judicature are independent systems, and
uncertainties arisen in the judicial process is
placed above the will of the legislator. Ta ble 2
lists the current choice and reasons for rural
dispute resolution; if people want to spend less
time, they will choose cadre mediation to solve
privately; if they want to spend the least, most
will choose cadre mediation; if they prefer a
more satisfactory solution, judicial decisions will
be more effective through lawsuits. The judicial
decision process must have clear objectives,
strong feasibility and legal

interpretation

applicability. Figure 4 shows the logical diagram
of the judicial decision behaviour

management

theory. If judicial decision -making behaviour
management needs to follow the deductive logic
from value to fact, the intrinsic logical
relationship between legal value research and
judicial form research cannot be ignored. Figure
5 shows the role of legal facts in judicial
decisions. The judicial decision process needs to
support facts by evidence; legal facts are formed
through legal evaluation and legal tailoring, and
then filed as the factual basis and premise of
judicial decision. A complete judicial decision
process also includes repetitive confirmation of
the case facts and the outcome of the ruling
based on legal normative logic, where repetitive
confirmation is the last link in the decision
reasoning.

Table 2. Choice and reasons of dispute
resolution ways in rural areas

Lawsuit Cadre
mediation

Compounding
in private

Minimum
time-

consuming

9.39% 45.23% 45.23%

Spend least 9.12% 51.92% 37.06%
Successful

solution
48.29% 34.84% 17.97%

Figure 4. Schematic diagram of the
theoretical management of judicial
adjudication

Judicial

justice

Judicial
decision

behavior

Social needs

Judicial adjudication

Judging mechanism

Judge Motivation

Judge competence

Decision-making process

Influencing factor

Rule control

Conflict of responsibility

Judge experience

Social expectations

Judicial
management

measures

Compliance with applicable rules of law

Human resource management of judges

Judicial macro-management system

Court internal management system

Figure 5. The role of legal facts in judicial
adjudication

Influence and countermeasure of

uncertainty in judicial decisions
The judicial decision process is a logical

inference process, but this process is
incomplete, thereby resulting in uncertainty.
China’s legislative institutions and judicial
organs have different powers. The law can only
be interpreted by the legislature, which leads to
the fact that the judicial organs do not have
universal inevitability in legal norms. Moreover,
the facts of the case are probabilistic, so, for
many times there exists no reasoning of an
implication relationship between the
preconditions and conclusions, seriously
affecting the fairness and justice of the judiciary.
The uncertainty of judicial decisions is relative
and objective. Just because of the existence of

ANALYSIS ON UNCERTAINTIES IN JUDICIAL DECISION BASED ON COGNITIVE PSYCHOLOGY

2020, Vol. XXIX, N°1,1016-1021
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

1020

relative situations or the amplification of
objective conditions, its negative impact on the
judicial system is very large, which will definitely
bring unreasonable and unacceptable judgment
to the parties, and make them suffer the loss of
material, spirit, and even life. Figure 6 shows the
legal normative process of judicial process.

Clear

rules are selected through the legal norm system
and legal rules, and used to explain the rule of
legal interpretation in the broad sense and select
appropriate legal principles.

The uncertainty of judicial decisions should
be reduced from the aspects of legislation, legal
application procedures, judges’ personal
emotions and qualities, and publicity and
education. Legislation should be forward –
looking; laws and regulations should be
formulated and updated in a timely manner to
adapt to the changing realities of society.
Furthermore, legislation should be as specific
and operational as possible. The legal
procedures should be initiated to limit the will of
the judges and reduce their subjective
psychology in the judicial decision process. From
the perspective of subjective cognitive
psychology, the judicial decision is the discretion
of the judge; although the whole process is
based on law and facts, ultimately it depends on
the psychological role of the judges. Therefore,
more stringent reforms must be carried out in
the election and appointment system and
procedures of the judges, and external
supervision and institutional constraints on the
judges should be strengthened so that the
discretion of the judges is within the scope
permitted by law and procedures. Strengthening
the publicity of the legal and judicial process is
an important measure to improve the judicial
and legal cognition of the whole people. Only by
enabling the whole people to understand the law
and obey the law can the number of judicial
decisions be greatly reduced, thereby reducing
the uncertainty of judicial decisions.

Figure 6. The legal process of judicial
adjudication

Legal

norm

system
Legal

rule

Clear

rules

A broad

interpretation

of the law

after the rule

Legal

principles

Legal norm proposition

CONCLUSIONS

Based on the theory of cognitive psychology,
this paper explores the uncertainty in the
process of judicial decision and gives
corresponding countermeasures. The specific
conclusions are as follows:

(1) Judicial lectures, seminars, reading clubs,
and legal columns have all become widely used
forms of law so that people are always in contact
with the law and obey the law, which greatly
promotes the cultivation of people’s judicial
cognition;

(2) The uncertainty in judicial decision is
mainly reflected in the uncertainty of fact
reasoning, the uncertainty of legal reasoning and
the uncertainty of decision reasoning. Its causes
include the uncertainty of natural language, the
limitations of legislation, the non -self-sufficiency
of the legal system, and the subjective factors of
the judge;

(3) The idea of uncertainty arising in the
judicial process is placed above the will of the
legislator. The management of judicial decision –
making must follow the deductive logic from
value to fact, and the judicial decision process
needs to support the facts by evidence; through
legal evaluation and legal tailoring, legal facts
are formed and then filed a as the factual basis
and premise of judicial decisions;

(4) The uncertainty of judicial judgment is
relative and objective. Measures should be
formulated from the aspects of legislation, legal
application procedures, judges’ personal
emotions and qualities, publicity and education,
etc., to reduce the emergence of the uncertainty
in judicial decisions.

Acknowledgement

This paper is supported by the National Social
Science Fund of China: The theory of interior
administrative act’s exteriorization , No.
14CFX011.

REFERENCES

Bishop, P. (2017). Salmon fishing in the Severn:
judicial deference to regulatory judgments based
on scientific assessments. Environmental Law
Review, 19(3), 201-209.

Brinkman, J. T. (2017). “Thinking like a lawyer” in an
uncertain world: The politics of climate, law and

YONGCHAO LI

2020, Vol. XXIX, N°1,1016-1021
REVISTA ARGENTINA
DE CLÍNICA PSICOLÓGICA

1021

risk governance in the united states. Energy
Research & Social Science, 34, 104-121.

Cunliffe, E. (2014). Judging, fast and slow: using
decision-making theory to explore judicial fact
determination. The International Journal of
Evidence & Proof, 18(2), 139-180.

Davies, A. (2012). State liability for judicial decisions
in European union and international law.
International and Comparative Law Quarterly,
61(3), 585-611.

Emma, M. N. J., & Mcnaught, A. (2016). Exploring
decision making around therapist self-disclosure
in cognitive behavioural therapy. Australian
Psychologist, 53(1), 33-39.

Flanagan, B., & Ahern, S. (2011). Judicial decision-
making and transnational law: A survey of
common law supreme court judges.
International and Comparative Law Quarterly,
60(1), 1-28.

Foxall, G. R. (2004). What judges maximize: toward
an economic psychology of the judicial utility
function. Liverpool Law Review, 25(3), 177-194.

Rand, E. J. (2015). Fear the frill: Ruth Bader Ginsburg
and the uncertain futurity of feminist judicial
dissent. Quarterly Journal of Speech, 101(1), 72-
84.

Shapiro, D. L., Mixon, L. K., Jackson, M., & Shook, J.
(2015). Psychological expert witness testimony
and judicial decision making trends.
International Journal of Law and Psychiatry, 42-
43, 149-153.

Soboleva, A. (2013). Use and misuse of language in
judicial decision-making: Russian experience.
International Journal for the Semiotics of Law-
Revue internationale de Sémiotique juridique,
26(3), 673-692.

Stone, A. (2010). Democratic objections to structural
judicial review and the judicial role in
constitutional law. University of Toronto Law
Journal, 60(1), 109-135.

Vlek, C. (2010). Judicious management of uncertain
risks: ii. simple rules and more intricate models
for precautionary decision-making. Journal of
Risk Research,13(4), 545-569.

Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.

Building a Testing-Based Training Paradigm From Cognitive
Psychology Principles

Daniel Corral, Alice F. Healy, Erica V. Rozbruch, and Matt Jones
University of Colorado Boulder

Cognitive psychology often produces findings that are relevant to educational instruc-
tion. However, many of these studies rely on artificial conditions, which often fail to
transfer to realistic settings, resulting in a disconnection between cognitive psychology
and education. This article begins to address this issue by taking established principles
from cognitive psychology and applying them to teach participants real academic
concepts. We report a training paradigm that applies established principles from
cognitive psychology: retrieval practice, feedback, self-paced studying, cognitive an-
tidote, and levels of processing. This paradigm was used to teach undergraduates basic
concepts of research design that are typically taught in university science courses.
Participants studied PowerPoint-style slides that were divided into three sections. At the
end of each section, participants were presented quiz questions. After each quiz
response, the participant was shown the correct answer. This study also tested different
forms of responding to quiz questions (between subjects): (a) fill-in-the-blank, (b)
multiple-choice, and (c) fill-in-the-blank followed by a multiple-choice version of the
same question. Participants completed two posttests, one immediately after training and
another 1 week later. Both posttests consisted of items that tested retention and
conceptual understanding. A control condition (wherein participants received no train-
ing) was used to assess the effectiveness of the training paradigm. Participants who
used this paradigm outperformed control participants on both posttests. However, no
differences in performance were found among participants who used different forms of
responding.

Keywords: retrieval practice, complex concept acquisition, technology-based learning
and instruction, translational research

Supplemental materials: http://dx.doi.org/10.1037/stl0000146.supp

One of the primary challenges in education is
finding effective methods that increase stu-
dents’ retention and comprehension of course
material. Factors that facilitate learning are thus
of great interest to instructors. Over the past 70
years, many findings from cognitive psychology

have shed light on this goal. This work has led
to the discovery of various learning principles
(e.g., correct-answer feedback: Benassi, Over-
son, & Hakala, 2014; self-paced study: Ariel,
2013; cognitive antidote: Healy, Jones, Lal-
chandani, & Tack, 2017; levels of processing:
Craik & Lockhart, 1972). One of the most ro-
bust findings from cognitive psychology is that
retrieving information from memory improves
the retention of the information that was re-
trieved (formally known as retrieval practice;
Carrier & Pashler, 1992; Kang, Gollan, & Pa-
shler, 2013; Kang & Pashler, 2014; Karpicke &
Roediger, 2008; Pan & Rickard, 2018; Pyc &
Rawson, 2010; Roediger & Butler, 2011; Roe-
diger & Karpicke, 2006a, 2006b). Specifically,
work on the testing effect has demonstrated that

This article was published Online First June 6, 2019.
Daniel Corral, Alice F. Healy, Erica V. Rozbruch, and

Matt Jones, Department of Psychology and Neuroscience,
University of Colorado Boulder.

This research was supported by National Science Foun-
dation Grant DRL1246588.

Correspondence concerning this article should be ad-
dressed to Daniel Corral, who is now at Department of
Psychology, Iowa State University, W112 Lagomarcino
Hall, Ames, IA 50011. E-mail: dcorral@iastate.edu

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

189

–208
2332-2101/19/$12.00 http://dx.doi.org/10.1037/stl0000146

189

mailto:dcorral@iastate.edu

http://dx.doi.org/10.1037/stl0000146

testing learners on previously studied material
(i.e., retrieval practice) often leads to better
learning and retention than having them restudy
that material (Butler, Black-Maier, Raley, &
Marsh, 2017; Carpenter & Yeung, 2017; Eg-
lington & Kang, 2018; Lehman & Karpicke,
2016; Pan & Rickard, 2018; Rickard & Pan,
2018). Retrieval practice has also been shown to
aid learning in the classroom, as students who
engage in retrieval practice, either through in-
class clicker questions (Anderson, Healy, Kole,
& Bourne, 2011, 2013; Mayer et al., 2009) or
online practice quizzes (Carpenter et al., 2017;
Corral, Carpenter, Perkins, & Gentile, 2019),
often demonstrate better learning and retention
than students who do not engage in these tasks.

On the other hand, many findings from cog-
nitive psychology that appear to be relevant to
education are often not translated to the class-
room (Horvath, Lodge, & Hattie, 2017; Roedi-
ger, 2013). One reason for this lack of cross-
fertilization may be that cognitive psychology
studies often use artificial learning tasks and
materials (e.g., participants are asked to learn to
distinguish among simple geometric figures;
e.g., Corral, 2017; Corral & Jones, 2014; Corral,
Kurtz, & Jones, 2018) that are not representa-
tive of the concepts that are taught in the class-
room (e.g., a physics professor teaching the
concept of buoyancy). The use of artificial con-
ditions and simplified stimuli and concepts is
fairly common in cognitive psychology and
may lead instructors to view findings from such
studies with skepticism, as it may seem unlikely
that a given effect will hold under more ecolog-
ically valid conditions (Horvath et al., 2017;
Oliver & Conole, 2003; Smeyers & Depaepe,
2013).

Adapting laboratory studies to real-world set-
tings is a common issue in translational science—
the application of laboratory findings to real-
world settings—as researchers often struggle to
apply findings from basic and theoretical re-
search to real-world scenarios (Horvath et al.,
2017; Oliver & Conole, 2003; Smeyers & De-
paepe, 2013; Roediger, 2013; Woolf, 2008).
One potential issue is that cognitive psychology
studies typically use rigorous methodology to
isolate the variable(s) of interest. Although this
approach is appropriate for controlled scientific
studies, it might not be conducive to translation
in the classroom, which often involves many
additional facets beyond what is required in a

laboratory experiment (Horvath et al., 2017;
Oliver & Conole, 2003; Smeyers & Depaepe,
2013).

For example, although retrieval practice
might aid learning and retention (Carrier & Pa-
shler, 1992; Kang et al., 2013; Kang & Pashler,
2014; Pan & Rickard, 2018), an instructor
might not know how to implement this principle
in the classroom, as translation requires the in-
structor to make various decisions about how to
implement numerous facets of retrieval practice.
In particular, an instructor must decide what type
of retrieval practice to provide students (e.g.,
recall vs. recognition), when to present retrieval
practice during a lecture (e.g., beginning of lec-
ture vs. interspersed throughout lecture vs. end
of lecture), as well as the type of feedback
students should be presented after retrieval
practice (e.g., no feedback vs. correct-answer
feedback). As this example illustrates, each of
these components offers the instructor an op-
portunity to translate different learning princi-
ples to the classroom, but this flexibility can
produce uncertainty about when and how to
apply these principles, and might thus make
translation rather difficult.

Given these challenges, one way forward might
be to develop a training paradigm that fully
specifies each of its facets. The efficacy of this
paradigm could then be tested in the laboratory
with ecologically valid learning materials. With
this aim in mind, the current article takes well-
established learning principles from cognitive
psychology and integrates them with current
instructional practices that are used in the class-
room to develop a training paradigm that can be
easily implemented by educators to supplement
instruction. We therefore build a training para-
digm around retrieval practice, one of the most
reliable principles in cognitive psychology
(Roediger, 2013), and specify and include ad-
ditional learning principles for each of its facets.
Specifically, this training paradigm incorporates
the following four learning principles: (a) re-
trieval practice, (b) correct-answer feedback, (c)
self-paced study, and (d) cognitive antidote.
Correct-answer feedback involves showing par-
ticipants the correct answer after they respond,
and self-paced study allows them to control the
time they spend studying. Cognitive antidote
includes the idea that boredom or disengage-
ment can be offset by alternating the tasks that
learners complete, wherein two or more tasks

190 CORRAL, HEALY, ROZBRUCH, AND JONES

are interspersed (as opposed to completing one
task in its entirety and then the other).

These principles were selected for two rea-
sons. The first reason is that each of these prin-
ciples has been shown to aid learning and re-
tention across numerous studies (e.g., retrieval
practice: Brame & Biel, 2015; Carpenter, Pash-
ler, & Cepeda, 2009; Carpenter & Yeung, 2017;
Dunlosky, Rawson, Marsh, Nathan, & Willing-
ham, 2013; Roediger & Butler, 2011; Rowland,
2014; correct-answer feedback: Benassi et al.,
2014; Butler, Karpicke, & Roediger, 2007; Mc-
Daniel, Anderson, Derbish, & Morrisette, 2007;
Pashler, Cepeda, Wixted, & Rohrer, 2005; Vo-
jdanoska, Cranney, & Newell, 2010; self-paced
study: Ariel, 2013; de Jonge, Tabbers, Pecher,
Jang, & Zeelenberg, 2015; Tullis & Benjamin,
2011; and cognitive antidote: Chapman, Healy,
& Kole, 2016; Healy et al., 2017; Kole, Healy,
& Bourne, 2008). The second reason is that by
applying one of these principles for each facet
that is involved in translating retrieval practice
to a real-world paradigm we are able to fully
specify this process (as discussed in detail in the
Experiment and Training Paradigm and

Method

sections), which might greatly aid instructors in
using or adapting this training paradigm.

It is important to note that none of these
principles were manipulated between experi-
mental groups, as our primary goal was to ex-
amine the efficacy of the paradigm as a whole as
compared to a control group that benefited from
none of the training principles. This approach
highlights an important distinction between lab-
oratory studies and translational research. Lab-
oratory studies take a reductionist approach, as
their typical goal is to isolate underlying mech-
anisms for a given phenomenon. In contrast, the
goal of translational research is to produce a
working, integrated system. As the example on
translating retrieval practice demonstrates,
translation is a complex process that involves
multiple facets (Horvath et al., 2017; Oliver &
Conole, 2003; Smeyers & Depaepe, 2013; Roe-
diger, 2013; Woolf, 2008). Translation of a
given learning principle is therefore likely to
involve a multifaceted, complex training para-
digm. Moreover, it is not clear that when a
learning principle is translated and embedded
within a larger system that any given compo-
nent will aid learning, as it is possible that the
different parts of the paradigm do not work well
together and might offset or counteract the ben-

efits of any single component. For these rea-
sons, it is essential for translational research to
take a more holistic approach and examine
whether a training paradigm as a whole im-
proves learning.

However, as a secondary question, we exam-
ine whether manipulating mode of responding
(i.e., type of retrieval practice), which is integral
to implementing retrieval practice, produces
differential learning and retention across exper-
imental groups. One possibility is that how par-
ticipants respond to a given question affects the
extent to which they encode its information.
Thus, a fifth principle we incorporate into our
training paradigm (by manipulating form of re-
sponding) is levels of processing—the extent to
which connections are formed between the in-
formation that is encoded and long-term mem-
ory (LTM; Craik & Lockhart, 1972; Craik &
Tulving, 1975).

Recognition Versus Recall

When an instructor translates retrieval prac-
tice to a real-world setting, he or she must
decide what type of retrieval practice to use.
Many studies on retrieval practice (e.g., Butler
et al., 2017; Carpenter, 2009; Carpenter &
Yeung, 2017) involve recall, wherein partici-
pants must generate a response from memory.
However, other forms of responding are possi-
ble, such as selecting an answer from a list of
multiple-choice options, a process that often
relies on recognition memory (Jacoby, 1991).

Recognition and recall are two distinct mem-
ory processes by which people access informa-
tion from LTM (Kintsch, 1970). In a recogni-
tion task, participants are presented with a given
item and are asked to determine whether or not
it matches information that was previously en-
countered, as is often the case for multiple-
choice questions. When students are presented
with a multiple-choice question, they must se-
lect the correct response from a list of options.
The option that is selected is often determined
by whether the student recognizes the given
option as correct or finds it more familiar than
other options (Marsh, Roediger, Bjork, &
Bjork, 2007; see also Bjork, Little, & Storm,
2014; Little & Bjork, 2015). Likewise, in stud-
ies on recognition memory, participants are typ-
ically presented with a list of items and are
asked to memorize them within an allotted pe-

191TESTING-BASED TRAINING PARADIGM

riod of time. At testing, participants are pre-
sented with a series of items and are asked to
select which of those items were on the studied
list. Similar to multiple-choice responding, the
items that are selected in a recognition memory
task are those that participants recognize or find
most familiar (Kahana, 2012).

In contrast to this process, recall involves the
retrieval of information from LTM, as is often
the case for fill-in-the-blank questions. Fill-in-
the-blank questions require students to provide
a short response by recalling information from
LTM. The process of generating a response is
often more challenging than recognizing a pre-
viously encountered item (Anderson & Bower,
1972; Kintsch, 1970). As a result of this gener-
ation, recall has been posited to lead to deeper
encoding—a greater number of connections are
formed between the information that was
probed in LTM and the information that was
recalled—than does recognition, which can con-
sequently improve retention (Hogan & Kintsch,
1971).

The differential effects of recall and recogni-
tion responding have been well documented in
many memory experiments, but the manner in
which they affect learning and transfer of
knowledge is less clear. One prediction that
follows directly from the experimental psychol-
ogy literature is that, because questions that
require recall processes (i.e., fill in the blank)
lead to deeper encoding (Hogan & Kintsch,
1971; Kintsch, 1970), recall will produce supe-
rior learning. Another prediction is that engag-
ing in both of the retrieval processes (recall
followed by recognition) will produce cumula-
tive effects, thus leading to better learning and
retention than either recall or recognition alone.

We tested these predictions by examining
whether there are learning differences (mea-
sured through performance scores at testing)
between participants who are trained by engag-
ing in retrieval practice that invokes recognition
versus recall. We also examined whether engag-
ing in both recall and recognition, wherein par-
ticipants first attempt recall followed by recog-
nition, can aid learning and retention above and
beyond engaging in only one of these retrieval
processes.

The comparison among these three experi-
mental groups (recall, recognition, and recall-
then-recognition) complements the main ques-
tion of this study, which is a comparison of all

three of these groups to the control group. For
this latter comparison, participants in the exper-
imental groups were predicted to demonstrate
better learning and retention than participants in
the control group.

Experiment and Training Paradigm

We conducted a study to examine whether
our training paradigm can be used to aid stu-
dents in learning core scientific concepts that
are typically taught in university-level statistics
and research methods courses. These materials
were chosen due to their direct relevance to all
scientific fields (because these fields rely on
sound research methodology), and thus wide
applicability to education and instruction.

Three groups of undergraduate students, re-
ferred to as the experimental groups, were
trained under our paradigm. These groups var-
ied only in the type of retrieval practice partic-
ipants were given (recall vs. recognition vs.
recall-then-recognition). Participants were first
asked to study PowerPoint-style slides (in-
cluded in the online supplemental materials)
that were divided into three sections. Although
the range of times participants were required or
allowed to spend on each section was deter-
mined before the study (explained further in the
Procedure), within these time restrictions par-
ticipants could choose how long they studied
each slide within a given section (a form of
self-paced studying). At the end of each section,
participants were quizzed on the material for
that section (thus, we implement the cognitive
antidote principle by alternating between study
and retrieval practice) and after each response
were shown the correct answer (correct-answer
feedback). These participants completed two
posttests, one immediately after training and
another 1 week later. It is important to note that
the first posttest affords participants in the ex-
perimental conditions additional retrieval prac-
tice, which might further benefit learning (But-
ler et al., 2017; Carpenter & Yeung, 2017;
Eglington & Kang, 2018; Lehman & Karpicke,
2016; Pan & Rickard, 2018). For this reason,
the first posttest can be viewed as another facet
of the training paradigm.

Participants in a separate, control condition
did not receive any training (i.e., were not
shown any study materials or presented with
any quiz questions) and were only asked to

192 CORRAL, HEALY, ROZBRUCH, AND JONES

http://dx.doi.org/10.1037/stl0000146.supp

complete a single test, which was identical to
the second posttest that participants who re-
ceived the training paradigm completed. Post-
test performance was compared between the
control group and the trained groups.

The control condition was used to assess
whether participants in the experimental condi-
tions were indeed able to learn the concepts that
were trained, as this condition provides a base-
line measure of participants’ knowledge of the
material. Although extensive work has shown
that retrieval practice can indeed aid retention
(e.g., Carpenter et al., 2009; Carrier & Pashler,
1992; Dunlosky et al., 2013; Kang et al., 2013;
Karpicke & Roediger, 2008; Pyc & Rawson,
2010; Roediger & Butler, 2011; Roediger &
Karpicke, 2006a, 2006b), most of this literature
is limited to direct memorization and does not
typically involve true concept learning. More-
over, the limited work that has been conducted
on this topic has yielded inconclusive results, as
some of this work has shown a modest benefit
of retrieval practice and testing on concept
learning and transfer (Butler, 2010; Butler et al.,
2017; Eglington & Kang, 2018; Pan & Rickard,
2018), but other studies have failed to replicate
this finding (Peterson & Wissman, 2018; Tran,
Rohrer, & Pashler, 2015; van Gog & Kester,
2012; Wissman, Zamary, & Rawson, 2018). It
is therefore an empirical question as to whether
these principles can be used to help people learn
ecologically valid, complex concepts.

The three trained groups were defined ac-
cording to the format in which they were
quizzed during training: recall, recognition, and
recall-then-recognition. Participants in the rec-
ognition condition were provided with multiple-
choice quiz questions, and participants in the
recall condition were provided with fill-in-the-
blank quiz questions. Participants in the recall-
then-recognition condition responded to each
quiz question twice, first with a fill-in-the-blank
response and then with a multiple-choice re-
sponse (multiple-choice options were shown
only after the first response was given). This
ordering was necessary to keep the multiple-
choice options from contaminating the recall
process for a given question, as the multiple-
choice options might serve as memory cues for
the correct response, and thereby trivialize the
recall process.

Method

Participants

One hundred eighty-three undergraduate stu-
dents participated for course credit in an intro-
ductory psychology course at the University of
Colorado Boulder. This population consists pri-
marily of freshmen and contains approximately
45% women and 71% White students, with an
average age of 20 (5% of the students are 25
years of age or older); 17% of this population is
classified as low-income students. One hundred
fifty-four of these participants were randomly
assigned to three experimental conditions (be-
tween subjects): recall only (n � 51), recogni-
tion only (n � 51), and recall-then-recognition
(n � 52). The other participants (n � 29) were
sampled concurrently from the same population
and were assigned to the control condition. True
random assignment was not possible because
the online system participants use to sign up for
studies requires that one-part and two-part stud-
ies (such as our control and experimental con-
ditions, respectively) be posted as separate
sign-up options. However, this system random-
izes the order of listed studies and provides
prospective participants with no information
other than time and location, which allows for a
degree of random assignment. Thus there is a
mild self-selection issue, because participants
who chose to sign up for one- and two-part
studies might differ from one another (although
all students were subject to the same class re-
quirement of 6 hr of total research participation
that semester). However, we stress that the
number of sessions participants signed up for
was the only difference in sampling procedure
between the experimental and control condi-
tions.

Design and Materials

All materials (instructions, study slides, and
quiz and posttest questions) were presented on a
computer monitor and were shown on a black
background. All responses were entered using a
computer keyboard. The training session con-
sisted of PowerPoint-style slides that were mod-
ified from an undergraduate statistics lecture,
which covered basic principles of research
methods. These slides were adapted to exclude
extraneous information, and each slide was care-

193TESTING-BASED TRAINING PARADIGM

fully checked by the authors to ensure that each
concept was fully explained. These slides cov-
ered 16 concepts, which were divided into three
sections, and each section followed a concep-
tual progression (see Table 1). Section 1 (Slides
1–2) introduced the basic components related to
scientific experiments. Section 2 (Slides 3– 6)
introduced issues related to causal inference and
nonexperimental studies. Section 3 (Slides
7–10) introduced methods that true experiments
use to control for confounding variables and
other related topics. Figure 1 shows an example
study slide, Slide 9 from Section 3 in the train-
ing session.

Question types. Five question types were
created for this study: (a) repeated, (b) defini-
tional, (c) transfer, (d) analysis, and (e) appli-
cation; all test items for each question type are
provided in the online supplemental materials.
These question types were divided into two
subsets. We refer to Question Types 1–3 as core
questions and Question Types 4 –5 as concep-
tual questions. The immediate posttest com-
prised eight questions from each of Types 1–3.
The retention test comprised eight different
questions from each of Types 1–3, and 14 ques-
tions from each of Types 4 –5.

Core questions. To increase the chance that
condition differences would be detected, the
core questions were pilot tested with two dif-
ferent groups to ensure that no ceiling or floor
effects were present; these participants were not
trained on these materials. The first round of
pilot testing was conducted with paid subjects
from the university’s paid subject pool and the
second with undergraduate students (within the
first few weeks of the semester) in an upper
division psychology course on research meth-
ods. Given the course content, the participants
in this latter group should have some back-

ground with these materials, and thus likely
represent a more knowledgeable sample than
the introductory psychology students who par-
ticipated in the main experiment. The initial
version of the core questions consisted of four
multiple-choice options per question, but these
materials proved too easy for students and were
thus modified to be more challenging; one of
these modifications was to switch from four
multiple-choice options to five. This iterative
process of pilot testing and revising these ma-
terials was concluded once an intermediate level
of performance was found (between 50 and
60%).

The core question types tested the basic con-
cepts that participants encountered during train-
ing. Repeated questions were identical in con-
tent to the recognition version of the quiz
questions that were used during the training
session (see Figure 4). Definitional questions
were the inverse of repeated questions: Partici-
pants were shown a term and were asked to
select the correct definition from the multiple-
choice options, as shown in Figure 5. Transfer
questions were similar to repeated questions,

Table 1
A Complete List of the Concepts and the Order in Which They Were Covered in Training

Section 1 (Slides 1–2) Section 2 (Slides 3–6) Section 3 (Slides 7–10)

1. Variables 4. Nonexperimental study 11. Independent and dependent variables
2. Hypothesis 5. Causal inference 12. Experimental control
3. Experimental study 6. Correlation 13. Confounds

7. Reverse causation 14. Random assignment
8. Third variable problem 15. Quasi-independent variables
9. Self-selection 16. Addressing confounds

10. Manipulation

Figure 1. Study Slide 9 from Section 3 of the training
session.

194 CORRAL, HEALY, ROZBRUCH, AND JONES

http://dx.doi.org/10.1037/stl0000146.supp

but the description of the tested term was
grounded in a hypothetical scenario (as shown
in Figure 6).

Each of the core questions thus provides a
different measure of retention. Repeated ques-
tions provide a direct measure of retention, as
these questions can be answered correctly through
rote memorization of the content that was
trained and quizzed. Definitional questions
measure whether participants can transfer their
memory of the training material to the inverse
of the concepts that were quizzed (i.e., matching
a given term to the correct definition instead of
matching a given definition to the correct term).
In contrast, transfer questions provide a more
robust measure of concept learning and transfer
than definitional questions, in that they require
participants to recognize the instantiation of a
given concept in a superficially different sce-
nario than what was encountered in training.
Moreover, the transfer questions did not explic-
itly define the corresponding concept (as the
repeated and definitional questions did), and
thus recognizing these concepts required that
the participant actually comprehends their
meaning. Transfer questions therefore provide a
measure of both retention and concept learning,
as these questions require participants to re-
member a concept’s definition and comprehend
its meaning.

Sixteen items were constructed for each core
question type (i.e., repeated, definitional, and
transfer), one covering each of the 16 concepts
that were introduced during training (as dis-
cussed in the first paragraph of the Design and
Materials). Thus, there was a one-to-one corre-
spondence among the three core question types
in terms of the concepts they tested. For pur-
poses of explaining the experimental design, we
refer to the questions of each core question type
as numbered 1–16, following the numbering of
training concepts (see Table 1). For example,
Question 8 tested the concept of the third-
variable problem for all three question types.
Core questions for each posttest were sampled
using this numbering, as discussed in the fol-
lowing paragraph.

The core questions were divided into two
equal subsets. Each experimental participant
completed one of these subsets during the im-
mediate posttest and the other subset during the
delayed posttest, with this assignment counter-
balanced across participants within each exper-

imental condition. One subset covered even-
numbered repeated questions and odd-
numbered definitional and transfer questions;
the other subset covered odd-numbered re-
peated questions and even-numbered defini-
tional and transfer questions. Thus for each
posttest, repeated questions covered different
concepts than did transfer and definitional ques-
tions, whereas transfer and definitional ques-
tions covered the same concepts. Because def-
initional and repeated questions were the
inverses of each other, this design avoided pre-
senting participants highly similar questions on
a given posttest.

Conceptual questions. Conceptual ques-
tions consisted of 14 analysis and 14 application
questions.1 The two conceptual question types
(i.e., analysis and application) tested abstract
principles that were not directly covered or
quizzed during training, but which could be
inferred with a sufficient conceptual grasp of the
training material. Each of these questions con-
tained a detailed description of a hypothetical
experiment. Analysis questions required partic-
ipants to determine which confounding vari-
able(s), if any, were present (Figure 7 shows an
example of an analysis question). Application
questions required participants to determine
how to eliminate confounding variables, if any
were present (Figure 8 shows an example of an
application question). Half of the analysis and
application questions contained confounding
variables, and half did not.

Conceptual questions thus tested the extent to
which participants grasped the principles of
sound research methodology, internal validity,
and true experiments. These topics were chosen
because they are of primary importance in re-
search methods courses, and these questions
examine the extent to which participants can
transfer and apply the knowledge they acquired
during training to complex study scenarios. For
example, the question in Figure 7 examines
whether participants can recognize the specific
confounding in the hypothetical study scenario.
Such recognition requires a strong grasp of the
concepts of confounding, experimental manip-

1 Conceptual questions were not included on the first
posttest so that we could administer them on the retention
test (1 week later) to assess participants’ comprehension of
the materials.

195TESTING-BASED TRAINING PARADIGM

ulation and control, true experiments, and inter-
nal validity. Correctly answering these ques-
tions therefore involves more than simply
memorizing a given definition. For these rea-
sons, conceptual questions provide a strong
measure of concept learning and far transfer.

Procedure

Participants in the experimental training
groups participated in two sessions, each lasting
a maximum of 55 min. At the start of the study,
these participants were told they would be
shown slides that contained information about
basic scientific principles.

Training session. The training session was
partitioned into three sections. Participants were
instructed to study each slide carefully, as they
would be tested on the material later in the
experiment. The study slides were shown one at
a time at the center of the screen. A participant
could view the next slide by pressing the right
arrow key and the previous slide by pressing the
left arrow key. Below each slide, a counter
indicated which slide number the participant
was viewing out of the total number of slides
contained in the section, as well as which sec-
tion the participant was working on (e.g., Sec-
tion 2, Slide 2 out of 4). Participants could view
slides only from the section they were studying
and could not move ahead prematurely to the
next section or return to a previous section once
it was complete.

Navigating each section. Minimum and
maximum time limits were implemented for
each section, based on the number of slides
contained in the section. These time constraints
were meant to partially simulate real-world
study conditions in which students are required
to learn multiple concepts within a limited time
frame. In such cases, students must devote a
sufficient amount of study toward each concept
to learn all the concepts, but must also balance
the amount of time they allocate toward any
single concept. Under such circumstances learn-
ers can control the amount of time they spend
studying any given concept (as in the present
study).

At the start of each section, a prompt showed
the participant the number of slides that were
contained in the section and the maximum study
time that would be allowed. Time limits were
set to allow an average study time of 2.5–3.5

min per slide. This range was intended to ac-
commodate a wide spectrum of preferred pacing
across different students. Section 1 contained
two study slides and Sections 2–3 each con-
tained four. Section 1 ran for 5–7 min, and
Sections 2 and 3 ran for 10 –14 min each.2

If participants attempted to move past the last
slide in a section before the minimum study
time had been reached, the screen was cleared
and a prompt instructed them to return to the
last slide by pressing the spacebar and to con-
tinue studying for at least the minimum duration
of time that remained in the section. Once a
section’s minimum study time was reached, the
screen was cleared and a prompt was presented
that gave the participant the option of exiting
the study phase by pressing the Enter key or
continuing to study and returning to the slide
they were previously viewing by pressing the
spacebar. If participants elected to continue
studying, they could continue navigating be-
tween slides by pressing the left- and right-
arrow keys. Once a section’s maximum time
was exceeded, the screen was cleared and a
prompt instructed the participant to press the
spacebar to exit the section and continue to the
quiz.

Quiz instructions. After studying each sec-
tion, participants were given a self-paced rest
break and were notified that they would be
quizzed on the material that was covered in the
section they had just completed. After complet-
ing their study of the slides in the first section,
all participants were provided specific details on
the format of quizzes they were going to be
administered. Participants in the recall condi-
tion were instructed that they would need to
type in a response for each quiz item. Partici-
pants in the recognition condition were in-
structed that they would be given a multiple-
choice quiz and would be required to select a
response for each quiz item. Participants in the
recall-then-recognition condition were in-
structed that they would be shown two versions
of the same question for each quiz item—a
fill-in-the-blank version followed by a multiple-
choice version—and would need to respond to
each accordingly. Additionally, after respond-
ing to the first fill-in-the-blank question, these

2 These time limits were used to accommodate the con-
straints of running a laboratory experiment.

196 CORRAL, HEALY, ROZBRUCH, AND JONES

participants were shown a prompt reminding
them they would be presented with two versions
of each question throughout the quiz. All par-
ticipants were asked to press the spacebar when
they were ready to begin the quiz.

Quiz questions. Quiz questions were pre-
sented at the end of each section, which queried
the material for that section. Sections 1–3 con-
tained three, seven, and six quiz questions, re-
spectively (one per concept covered). The dis-
play for all quiz and posttest questions included
a text box, located directly beneath the question,
where participants were asked to enter their
responses. Each quiz question consisted of a
description of a given term, and participants
were required to either type the correct term
(recall-only, as shown in Figure 2), select the
correct term from a list of five multiple-choice
options (recognition-only, as shown in Figure
3), or complete both of these tasks in succession
(recall-then-recognition). For each quiz ques-
tion in the recall-then-recognition condition, the
participant was first provided with a fill-in-the-
blank form of the question (as in Figure 2),
followed by the same question in multiple-
choice format (as in Figure 3).

Correct-answer feedback. After typing in
a response, participants were required to press
the enter key (this was also required for both
posttests). Participants were then shown the cor-
rect answer at the bottom of the display. In all
experimental conditions, only the correct an-
swer was shown; the corresponding letter op-
tion was not displayed for multiple-choice
items. Thus the feedback was identical in all
conditions, matching verbatim the correct alter-
native from the multiple-choice version of the
question. For the recall-then-recognition condi-

tion, participants were not shown the correct
answer until after they entered their second re-
sponse, on the multiple-choice version of the
question. After being shown the correct answer,
participants were asked to press the spacebar
when they were ready to move on to the next
question. There was a 300-ms interval follow-
ing the feedback for each question on the quiz
(as well as each question on both posttests).

Immediate posttest. All questions in both
posttests were presented in multiple-choice for-
mat to explicitly test recognition learning,
which is a common form of assessment in the
classroom. The immediate posttest comprised
24 core questions, which were presented in a
random order (different for each participant).
After completing the immediate posttest, partic-
ipants in the experimental conditions were
thanked for their participation and reminded
that they would be required to return in 7 days.

Delayed posttest. The delayed posttest
consisted of 52 questions and followed the same
procedure as the immediate posttest. Upon re-
turning, participants in the experimental condi-
tions were notified that they would be tested on

Figure 2. An example fill-in-the-blank quiz question. The
correct response is self-selection.

Figure 3. An example multiple-choice quiz question. The
correct response is option a.

Figure 4. An example from the repeated question type
(identical to the recognition version of questions given
during training). The correct response is option a.

197TESTING-BASED TRAINING PARADIGM

the material that was covered in the first session
of the experiment (i.e., the previous week). The
delayed posttest was partitioned into two sec-
tions. The first section consisted of 24 core
questions (the subset not used in that partici-
pant’s immediate posttest, as explained in the
Design and Materials section), and the second
section consisted of all 28 conceptual questions.
The order in which questions were presented
within each section was randomized, separately
for each participant.

Control group. Participants in the control
condition were notified that they would be given a
test on basic scientific principles. These partici-
pants were only asked to complete a single test,
which was identical to the delayed posttest that
participants in the experimental conditions com-
pleted. The rest of the procedure was identical to
the second session that participants in the experi-
mental conditions completed. Because there were
two versions of this test, the version that was
completed by each control participant was ran-
domly selected, subject to the constraint that half
of these participants completed one version and
the other half completed the other version.

Results

Nine experimental participants were excluded
from the analyses because they did not return for
the second posttest (two from the recall-only con-
dition, three from the recall-then-recognition con-
dition, and four from the recognition-only condi-
tion), leaving 174 total participants.

It is important to note that core and concep-
tual questions assessed different aspects of par-
ticipants’ knowledge of the training material. It
was possible for participants to correctly answer

core questions by directly memorizing the training
material. These questions hence provide a direct
measure of retention. In contrast, conceptual ques-
tions tested participants’ conceptual understand-
ing, as they required participants to apply their
knowledge of the training material to scenarios
that tested these concepts’ underlying principles.
As a result, it was not possible for participants to
correctly answer conceptual questions just by
memorizing the training material. Participants’
performance on core and conceptual questions
was therefore analyzed separately.

Experimental Conditions Versus Control
Condition

First, we examined whether participants in
the experimental conditions were able to learn
and retain the material they studied during the
training session.3 Thus, performance on the core
questions (i.e., repeated, definitional, and trans-
fer) was compared between participants in the
experimental and control conditions.

Performance on core questions. Figure 9
shows the mean performance on each type of
core question for participants in the experimen-
tal and control conditions. Performance by par-
ticipants in the experimental conditions on the
immediate posttest exceeded control partici-
pants’ performance, Mexperimental-immediate � .76;
Mcontrol � .49, t(172) � 8.47, p � .001, SE �
.031, d � 1.74. Experimental participants’ de-
layed posttest performance also exceeded control
participants’ performance, Mexperimental-delayed �
.69, t(172) � 6.19, p � .001, SE � .031, d � 1.28.

Performance on conceptual questions.
Furthermore, participants in the experimental
conditions (M � .30) outperformed participants
in the control condition (M � .23) on concep-
tual questions (analysis and application ques-
tions), t(172) � 2.36, p � .020, SE � .029, d �
.456. It is also important to note that participants
in the control condition did not perform reliably
above chance (20%) on conceptual questions,
t(28) � .977, p � .337, SE � .029, d � .37,
whereas participants in the experimental condi-
tions performed significantly above chance,
t(144) � 8.43, p � .001, SE � .012, d � 1.41.

3 All reported analyses comparing the experimental and
control groups meet the assumption of equal variance, as
indicated by Levene’s test.

Figure 5. An example item from the definitional question
type. The correct response is option e.

198 CORRAL, HEALY, ROZBRUCH, AND JONES

Recall Versus Recognition Versus
Recall-Then-Recognition

A separate analysis examined whether there
were performance differences on core questions
among the experimental conditions, and if so,
whether such differences depended on the test and
question types. This analysis was a mixed-model
analysis of variance with a between-subjects fac-
tor of training condition (recall only vs. recogni-
tion only vs. recall-then-recognition) and within-
subject factors of question type (repeated
questions vs. transfer questions vs. definitional
questions) and test (immediate vs. delayed).

The analysis revealed a main effect of test,
F(1, 142) � 34.68, p � .001, MSE � .032, �p2 �

.196, such that participants performed better on
the first posttest than on the second. There was
also a main effect of question type, F(2, 284) �
114.90, p � .001, MSE � .019, �p2 � .447, as
participants performed best on repeated ques-
tions. Additionally, there was an interaction be-
tween test and question type, F(2, 284) � 3.29,
p � .039, MSE � .02, �p2 � .023, as there was
a greater decrease in performance between the
first and second posttest for repeated and defi-
nitional questions than for transfer questions (as
shown in Figure 9). No differences in perfor-
mance among the experimental conditions were
found, and there were no interactions between
condition and question or test type (all ps �
.216, including all least-significant-difference
post hoc comparisons among the experimental
conditions). Likewise, no performance differ-
ences were found among the experimental con-
ditions on the conceptual questions (p � .979).
Table 2 shows the mean performance of each
experimental group on each of the core question
types on the immediate and delayed posttests.

Exploratory Analysis

One concern with the analyses contrasting
the three experimental conditions is that they
may not adequately capture true differences that
might exist in conceptual understanding among
these groups. Conceptual questions were meant

Figure 6. An example item from the transfer question
type. The correct response is option a.

Figure 7. An example item from the analysis question type. The correct response is
option b.

199TESTING-BASED TRAINING PARADIGM

to capture such differences, but the challenging
nature of these questions might have obscured
the effects of the experimental manipulation. As
noted in the second paragraph of the Results
section, repeated and definitional questions
could be correctly answered by memorizing the
material presented during training, and thus
they allowed for an adequate measure of reten-
tion but not of conceptual understanding. Al-
though memorization could be used for transfer
questions, doing so was more challenging be-
cause these questions were presented in novel
contexts from what was encountered during
training, and therefore required a deeper level of
understanding. More specifically, it was neces-
sary for participants to understand these con-
cepts well enough to recognize them in unique
scenarios. Transfer questions hence provide the
best measure of conceptual understanding among
the three core question types.

An exploratory analysis was thus conducted
on transfer questions, to further examine whether
participants who engaged in recall developed a
better understanding and formed more durable
memories of the concepts in the study material
than did participants who did not engage in
recall. Because participants in the recall-only
and the recall-then-recognition conditions were
asked to engage in recall during training, both
groups were combined for this analysis. A
mixed-model analysis of variance was used to
test for an interaction between type of training
(between-subjects factor: recall conditions vs.
recognition-only) and test type (within-subjects
factor: immediate vs. delayed posttests). Com-
paring the immediate and delayed tests allows
for an assessment of participants’ retention and
conceptual understanding of the study material.

Figure 9B shows the mean performance on
transfer questions by type of training and type
of test. The analysis revealed a significant in-
teraction between condition and test type, F(1,
143) � 3.97, p � .048, MSE � .026, �p2 � .027,
as there was less of a decrease in performance
between the first and second posttests for par-
ticipants who engaged in recall (Mimmediate �
.657; Mdelayed � .647) than for participants who
engaged only in recognition (Mimmediate � .710;
Mdelayed � .620). Thus, this exploratory analysis
suggests that recall quizzing produced more du-
rable knowledge that was less susceptible to
forgetting, at least for the transfer questions,
which required more conceptual understanding
than the repeated or definitional questions.

Discussion

This article presents a training paradigm that
is built on the principle of retrieval practice.
Translating this principle into a real-world par-
adigm requires addressing multiple facets, such
as how much time to allow learners to study a
given set of concepts, when to include retrieval
practice, what type of retrieval practice to in-
clude, and whether to provide participants feed-
back on their responses. At each of these deci-
sion points, we fully specified the translation
process by implementing findings from basic
experimental psychology, regarding interspersed
retrieval practice, different forms of responding, a
restricted form of self-paced studying, and cor-
rect-answer feedback. To briefly summarize these
facets: Participants were allowed to navigate the
study slides within each section, permitting them
to control which slides they spent more time
studying (within the allotted time for each sec-
tion). Concepts were divided into three sections,
and interspersed retrieval practice was used,
wherein participants were quizzed at the end of
each section. After participants responded to a
quiz question they were provided correct-answer
feedback.

It is important to note that only form of
responding was manipulated among the exper-
imental groups (recall vs. recognition vs. recall-
then-recognition), as the implementations of the
other learning principles were held constant.
Manipulating all of these principles as a unit
enables a holistic test of their combined effect,
which is more relevant to translation than is the
reductionist approach of assessing each princi-

Figure 8. An example item from the application question
type. The correct response is option d.

200 CORRAL, HEALY, ROZBRUCH, AND JONES

ple individually. Moreover, if left unspecified,
each of the facets can lead to ambiguity in
regards to the translation of retrieval practice to
a real-world paradigm. To avoid this ambiguity
impeding translation (Horvath et al., 2017; Ol-
iver & Conole, 2003; Smeyers & Depaepe,
2013; Roediger, 2013), we explicitly specify
each facet of our training paradigm and base our

decisions for each on the vast literatures on the
learning sciences.

This training paradigm was developed with
the goal that it might serve as a teaching tool
that can be used to enhance student learning.
Thus, we were not specifically interested in
whether any one of these principles could en-
hance learning on its own, as each has been

Figure 9. (A) The experimental and control groups’ mean performance on each type of core
question (repeated, definitional, and transfer questions) for each posttest. (B) Mean perfor-
mance for transfer questions on each posttest for the recall conditions (recall condition and
recall-then-recognition condition) and the recognition condition. Error bars indicate standard
errors of the mean.

201TESTING-BASED TRAINING PARADIGM

shown to do so in the context of the laboratory.
Instead, our goal was to examine whether these
principles could be translated into a realistic,
complex learning system to aid learners in ac-
quiring ecologically valid concepts, which
could then be used by instructors in the class-
room. Thus, we were interested in whether com-
bining all of these principles into a single inter-
vention would substantively impact performance
in a realistic educational learning task. This
holistic approach is often appropriate for trans-
lational research, because the translation of a
given principle involves numerous facets be-
yond the variables that are manipulated in the
laboratory (Horvath et al., 2017; Oliver &
Conole, 2003; Smeyers & Depaepe, 2013; Roe-
diger, 2013). With these issues in mind, the
training paradigm was constructed in a manner
that would allow for instructors to directly apply
(in cases where the same concepts as those
presented in this study are covered) or easily
modify and adapt the paradigm accordingly
(changing out the study slides and quiz ques-
tions), based on the course curriculum (dis-
cussed further below).

The training paradigm was effective in help-
ing participants in the experimental conditions
learn the concepts they were taught during train-
ing, and moreover these concepts were retained
1 week later. Importantly, the training paradigm
also aided participants in correctly answering
conceptual (application and analysis) questions,
which required participants to have a thorough
understanding of the study material. These
question types tested complex scientific princi-
ples, which, as many university professors who
have taught a research methods course can af-
firm, can be extremely difficult for students to
learn and retain (as indicated by the control
group’s chance performance on conceptual
questions). Moreover, participants in the exper-
imental conditions were not quizzed on these

question types during training and were not
tested on them until 1 week after they com-
pleted the training session. Thus, this finding
seems to reflect the experimental participants’
genuine conceptual understanding of the study
material.

Perhaps more important is the extent to which
such concepts were learned by participants who
received training. On each of the posttests that
participants in the experimental conditions com-
pleted, they outperformed control participants
on core questions by approximately 20%, which
amounts to a difference of two full letter grades.
Notably, these learning gains were achieved
with only a single training session, which con-
sisted of less than an hour of actual training.
Furthermore, participants who received training
scored approximately 76% and 70% on the core
questions in the first and second posttests, re-
spectively, translating to passing letter grades of
C and C–. This level of performance is notewor-
thy given that the amount of training participants
in the experimental conditions were given is many
orders of magnitude less than the instruction and
study time that students in actual statistics and
research methods courses receive. Taken together,
these findings serve as a powerful demonstration
of how the current training paradigm can aid stu-
dents in acquiring and subsequently retaining
complex concepts.

Type of Quizzing Format

Despite the evidence for the strong benefit of
the training paradigm overall, performance ap-
peared to be equivalent among the experimental
conditions, suggesting that all three quizzing
formats are equally effective. It is therefore
unclear which format is ideal for presenting
quiz questions for this training paradigm. One
possibility is that the benefits of recall-based
quizzing were masked by the fact that the ques-

Table 2
Mean (SD) Performance for Each Experimental Group on Each of the Core Question Types for
Each Posttest

Immediate posttest Delayed posttest

Group Repeated Definitional Transfer Repeated Definitional Transfer

Recall .842 (.18) .732 (.22) .658 (.21) .753 (.18) .635 (.22) .660 (.20)
Recognition .899 (.16) .747 (.19) .710 (.21) .790 (.17) .650 (.19) .620 (.23)
Recall-then-recognition .872 (.13) .747 (.19) .656 (.22) .789 (.17) .694 (.18) .635 (.23)

202 CORRAL, HEALY, ROZBRUCH, AND JONES

tion format of the posttests matched that of the
quiz questions that were presented to the recog-
nition group. Research on transfer-appropriate
processing has shown that test performance is
superior when the training and testing condi-
tions are similar (Balota & Neely, 1980). Thus,
performance for participants in the recognition-
only condition may have been inflated, reducing
the performance advantage for the recall condi-
tions. Future work will be required to more
directly test this possibility.

An exploratory analysis, which examined
whether the decline in performance between the
two posttests on transfer questions differed be-
tween the recall conditions and the recognition-
only condition, suggests that retention and
transfer of concepts may have been stronger for
participants who engaged in recall. Participants
in the recall conditions performed equally well
on the transfer questions on both posttests, sug-
gesting that their memory for the concepts that
were learned during training was not weakened
by the 1-week delay between the first and sec-
ond posttest. In contrast, performance on the
transfer questions for participants in the recog-
nition-only condition decreased considerably
between the first and second posttests (by ap-
proximately 9%), suggesting that their memory
of the study material was somewhat tenuous in
comparison to that of participants who engaged
in recall during training. Thus, instructors who
employ this training paradigm may wish to use
a version that includes recall responding during
quizzing. In the classroom, recall questions can
be used during quizzing by asking students to
write out their response to a given quiz question
and then showing students the correct response
(similarly to the type of feedback used in our
paradigm4).

Guide and Implications for Instructors

Instructors who wish to use this paradigm to
train students on different content (e.g., physics,
chemistry, mathematics) can do so by simply
following our training procedure (discussed
above in the Method section), and replacing our
slides and quiz questions with those that corre-
spond to the topic of interest. In this process, we
recommend creating training slides that are con-
cise and devoid of superfluous information, so
that the slides fully and clearly explain all of the
concepts that are introduced. Additionally, in

cases where the training content builds on con-
cepts that were introduced in earlier slides, we
suggest presenting slides in a manner that fol-
lows a conceptual progression.

One area that instructors might wish to devi-
ate from our training procedure is in the amount
of time that students are permitted to study a
given slide. Here, participants’ study time was
limited (although participants were given some
autonomy in the amount of time they could
spend studying) due to the time restrictions of
the laboratory experiment. However, based on
the principles of self-pacing (Ariel, 2013; de
Jonge et al., 2015; Tullis & Benjamin, 2011), it
might be more useful to allow participants full
control over how much time they spend study-
ing a given slide. On the other hand, one issue
that this approach introduces is that some stu-
dents might not spend a sufficient amount of
time studying a given slide. Thus, it might be
wise to keep a minimum study time in place for
any given set of slides, but provide participants
the ability to advance to the next set of slides
once the minimum time has been reached.

Furthermore, as in our paradigm, we recom-
mend that instructors quiz students on any con-
cepts that are presented in the training slides. It
is important to note that our quiz questions were
presented in an abstract format so we could
directly test participants’ ability to transfer their
knowledge to novel scenarios during testing.
This aspect of the training paradigm was thus
implemented for reasons of experiment design,
and instructors may or may not wish to adopt a
similar approach.

We also recommend that instructors imple-
ment an immediate posttest after training to
assess how well participants are able to learn
and retain the training material. This type of
assessment can be particularly useful in helping
both the student and instructor identify the as-
pects of the material that the student does not
yet fully grasp. One noteworthy finding is that
participants performed best on repeated ques-
tions, which were identical to the questions that
were quizzed, and worst on transfer questions.

4 Instructors might also consider using more complex
forms of feedback that encourage students to think carefully
about the material, such as explanation feedback, wherein
the correct answer is coupled with a detailed explanation
(Butler, Godbole, & Marsh, 2013; Corral & Carpenter,
2019).

203TESTING-BASED TRAINING PARADIGM

However, performance decreased substantially
on repeated (and definitional) questions be-
tween the first and second posttest, whereas
performance was relatively stable for transfer
questions. One reason for this finding might be
that rote memorization could be used to answer
repeated (and definitional) questions, but trans-
fer questions required conceptual understand-
ing. Thus, when participants were given the
second test 1 week later, they may have forgot-
ten the information that was memorized during
training. In contrast, because performance on
transfer questions might have been driven by
conceptual understanding, as opposed to rote
memorization, performance on these questions
might have been more stable. These findings
and explanation are in line with work on levels
of processing (Craik & Lockhart, 1972; Craik &
Tulving, 1975), wherein information that is pro-
cessed in a deeper and more meaningful manner
(e.g., information that is comprehended by the
learner) is more robust to decay than informa-
tion that is learned through rote memorization
(Symons & Johnson, 1997).

This explanation suggests that transfer items
can better assess students’ knowledge than
items that can be answered through rote mem-
orization. Moreover, training performance on
the latter type of items might lead both students
and instructors to form an inaccurate perception
of the student’s actual understanding of the
tested content. This misperception can be prob-
lematic in cases where pretests are used to help
prepare students for an upcoming exam, as stu-
dents might develop a false sense of security
due to their high performance on the items that
were memorized during study or training. Con-
sequently, students might reduce their study time,
leaving them ill-prepared for an exam. The find-
ings presented here therefore have direct impli-
cations for instructors who use clicker questions
or pretests to assess their students’ knowledge
of course material. Our findings suggest that
any such assessments should incorporate trans-
fer-like questions, which are fairly similar to the
type of test questions that instructors often use
on exams.

Lastly, although instructors can use this train-
ing paradigm during lecture, it can also be ap-
plied outside of the classroom. For instance, our
training paradigm can be implemented as an
automated tutoring system that is made avail-
able to students. This option would allow stu-

dents autonomy over when they study, and also
provide them a structured and controlled train-
ing environment outside of the classroom. Our
training paradigm might also be particularly
well-suited for classroom laboratory courses
(e.g., research methods, statistics), in which stu-
dents are often required to complete assign-
ments independently within a given time period
(typically 1–3 hr). This context is highly similar
to what participants in the experimental condi-
tions encountered, and thus students in labora-
tory courses might greatly benefit from a train-
ing paradigm like the one used in the present
study.

Limitations and Future Directions

From a translational and applied perspective,
the implementation of multiple learning princi-
ples within a single training paradigm is a par-
ticular strength of this article. However, a lim-
itation of this approach from a theoretical
perspective is that we did not isolate and test
each of these principles. Thus, we do not know
the extent to which each of these principles
affected learning, as we examined only their
combined impact. Nevertheless, a researcher or
instructor might be interested in this question.
Thus, a potential direction for future work is to
methodically vary which facets are included in
the paradigm and compare those conditions to
the full paradigm (e.g., full paradigm vs. para-
digm without correct-answer feedback or full
paradigm vs. paradigm without retrieval prac-
tice).

One potential critique of the present study is
that the control condition did not receive any
instruction, and thus these results might be
taken to demonstrate that the training paradigm
merely leads to better learning than not receiv-
ing training at all. However, as we state above,
the materials used in this study were highly
complex (particularly the conceptual question
types) and it is by no means a given that they
can be readily acquired, even with extensive
training. Indeed, as many research method in-
structors will likely attest, there are numerous
students who fail to learn these exact concepts
over an entire semester of rigorous instruction.
Moreover, many training procedures fail to pro-
duce learning whatsoever, as is exemplified in
studies where participants in some conditions
perform at chance (e.g., Johnstone & Shanks,

204 CORRAL, HEALY, ROZBRUCH, AND JONES

2001; Quinn, Palmer, & Slater, 1999; Shanks,
Johnstone, & Staggs, 1997). Thus, demonstrat-
ing that this training paradigm benefits complex
learning is a critical first step of the present
work.

Nevertheless, an instructor might certainly be
interested in the extent to which this training
paradigm benefits learning above and beyond
simply studying the materials. One way to an-
swer this question in future work would be to
provide one group of participants the full train-
ing paradigm and another group the Power-
Point-style slides for study. Another potential
future direction is to examine how this para-
digm might fare in comparison to how students
typically study. Recent work suggests that stu-
dents use suboptimal study strategies (Corral et
al., 2019), and given that the training paradigm
used here is premised on well-established learn-
ing principles, we would predict learning to be
better for students who use the training para-
digm than for those who receive the same study
materials and are left to their own devices. To
build on this idea, a particularly strong test of
this paradigm’s efficacy might be to select stu-
dents in a course who are struggling (e.g., stu-
dents with a letter grade of C- or lower) and
randomly assign them to complete the training
paradigm or to continue to study using their
preferred method. These students’ progress
could also be monitored throughout the semes-
ter to examine whether the benefits of the train-
ing paradigm are observed over an extended
period.

Conclusion

Translating basic and theoretical research to-
ward real-world applications can be challenging
(Woolf, 2008) and often fails to occur in the
fields of cognitive psychology and education.
One reason for this failure is that many cogni-
tive psychology studies require participants to
learn artificial concepts, which can make in-
structors skeptical of how well a given effect
will transfer to the classroom. The current study
lays out a blueprint for how principles from
cognitive psychology, specifically the testing
effect, form of responding, self-paced studying,
and feedback, can be integrated to construct a
valuable training paradigm. Furthermore, we
have demonstrated the efficacy and applicabil-
ity of this training paradigm with ecologically

valid learning materials. These materials cov-
ered various core concepts of the scientific
method, and quiz and posttest items were sim-
ilar in structure and difficulty to exam questions
that are typically presented to students in a
university-level research methods course. The
findings for the current study are thus applicable
to educators from a wide range of scientific
domains. However, the current project takes
only a small step toward utilizing cognitive
psychology to aid students with the learning of
real academic concepts. If the translation of
cognitive psychology principles is to improve in
the domain of education, future work must care-
fully demonstrate the efficacy of such principles
with real academic concepts.

References

Anderson, J. R., & Bower, G. H. (1972). Recognition
and retrieval processes in free recall. Psychologi-
cal Review, 79, 97–123. http://dx.doi.org/10.1037/
h0033773

Anderson, L. S., Healy, A. F., Kole, J. A., & Bourne,
L. E., Jr. (2011). Conserving time in the classroom:
The clicker technique. Quarterly Journal of Experi-
mental Psychology, 64, 1457–1462. http://dx.doi.org/
10.1080/17470218.2011.593264

Anderson, L. S., Healy, A. F., Kole, J. A., & Bourne,
L. E., Jr. (2013). The clicker technique: Cultivating
efficient teaching and successful learning. Applied
Cognitive Psychology, 27, 222–234. http://dx.doi
.org/10.1002/acp.2899

Ariel, R. (2013). Learning what to learn: The effects
of task experience on strategy shifts in the alloca-
tion of study time. Journal of Experimental Psy-
chology: Learning, Memory, and Cognition, 39,
1697–1711. http://dx.doi.org/10.1037/a0033091

Balota, D. A., & Neely, J. H. (1980). Test-expectancy
and word-frequency effects in recall and recogni-
tion. Journal of Experimental Psychology: Human
Learning and Memory, 6, 576–587. http://dx.doi
.org/10.1037/0278-7393.6.5.576

Benassi, V., Overson, C., & Hakala, C. (Eds.).
(2014). Applying science of learning in education:
Infusing psychological science into the curricu-
lum. Washington, DC: American Psychological
Association.

Bjork, E. L., Little, J. L., & Storm, B. C. (2014).
Multiple-choice testing as a desirable difficulty in
the classroom. Journal of Applied Research in
Memory & Cognition, 3, 165–170. http://dx.doi
.org/10.1016/j.jarmac.2014.03.002

Brame, C. J., & Biel, R. (2015). Test-enhanced learn-
ing: The potential for testing to promote greater
learning in undergraduate science courses. CBE—

205TESTING-BASED TRAINING PARADIGM

http://dx.doi.org/10.1037/h0033773

http://dx.doi.org/10.1080/17470218.2011.593264

http://dx.doi.org/10.1002/acp.2899

http://dx.doi.org/10.1037/a0033091

http://dx.doi.org/10.1037/0278-7393.6.5.576

http://dx.doi.org/10.1016/j.jarmac.2014.03.002

Life Sciences Education, 14, 1–12. http://dx.doi
.org/10.1187/cbe.14-11-0208

Butler, A. C. (2010). Repeated testing produces su-
perior transfer of learning relative to repeated
studying. Journal of Experimental Psychology, 5,
1118–1133. http://dx.doi.org/10.1037/a0019902

Butler, A. C., Black-Maier, A. C., Raley, N. D., &
Marsh, E. J. (2017). Retrieving and applying
knowledge to different examples promotes transfer
of learning. Journal of Experimental Psychology:
Applied, 23, 433– 446. http://dx.doi.org/10.1037/
xap0000142

Butler, A., Godbole, N., & Marsh, E. (2013). Expla-
nation feedback is better than corrective feedback
for promoting transfer of learning. Journal of Ed-
ucational Psychology, 105, 290–298. http://dx.doi
.org/10.1037/a0031026

Butler, A. C., Karpicke, J. D., & Roediger, H. L., III.
(2007). The effect of type and timing of feedback
on learning from multiple-choice tests. Journal of
Experimental Psychology: Applied, 13, 273–281.
http://dx.doi.org/10.1037/1076-898X.13.4.273

Carpenter, S. K. (2009). Cue strength as a moderator
of the testing effect: The benefits of elaborative
retrieval. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 35, 1563–
1569. http://dx.doi.org/10.1037/a0017021

Carpenter, S. K., Pashler, H., & Cepeda, N. J. (2009).
Using tests to enhance 8th grade students’ retention
of U.S. history facts. Applied Cognitive Psychology,
23, 760–771. http://dx.doi.org/10.1002/acp.1507

Carpenter, S. K., Rahman, S., Lund, T. J., Armstrong,
P. I., Lamm, M. H., Reason, R. D., & Coffman, C. R.
(2017). Students’ use of optional online Reviews and
its relationship to summative assessment outcomes in
introductory biology. CBE Life Sciences Education,
16, 1–9. http://dx.doi.org/10.1187/cbe.16-06-0205

Carpenter, S. K., & Yeung, K. L. (2017). The role of
mediator strength in learning from retrieval. Jour-
nal of Memory and Language, 92, 128–141. http://
dx.doi.org/10.1016/j.jml.2016.06.008

Carrier, M., & Pashler, H. (1992). The influence of
retrieval on retention. Memory & Cognition, 20,
633– 642. http://dx.doi.org/10.3758/BF03202713

Chapman, M. J., Healy, A. F., & Kole, J. A. (2016).
Memory load as a cognitive antidote to performance
decrements in data entry. Memory, 24, 1182–1196.
http://dx.doi.org/10.1080/09658211.2015.1086380

Corral, D. (2017). A dual model of relational concept
representation (Unpublished doctoral dissertation).
University of Colorado Boulder, Boulder, CO.

Corral, D., & Carpenter, S. K. (2019). Facilitating
transfer through incorrect examples and explana-
tory feedback. Manuscript under review.

Corral, D., Carpenter, S. K., Perkins, K. M., & Gen-
tile, D. A. (2019). Assessing students’ use of op-
tional online reviews. Manuscript under review.

Corral, D., & Jones, M. (2014). The effects of relational
structure on analogical learning. Cognition, 132,
280–300. http://dx.doi.org/10.1016/j.cognition.2014
.04.007

Corral, D., Kurtz, K. J., & Jones, M. (2018). Learning
relational concepts from within- versus between-
category comparisons. Journal of Experimental
Psychology: General, 147, 1571–1596. http://dx
.doi.org/10.1037/xge0000517

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of
processing: A framework for memory research.
Journal of Verbal Learning and Verbal Behavior,
11, 671– 684. http://dx.doi.org/10.1016/S0022-
5371(72)80001-X

Craik, F. I. M., & Tulving, E. (1975). Depth of
processing and the retention of words in episodic
memory. Journal of Experimental Psychology:
General, 104, 268–294. http://dx.doi.org/10.1037/
0096-3445.104.3.268

de Jonge, M., Tabbers, H. K., Pecher, D., Jang, Y., &
Zeelenberg, R. (2015). The efficacy of self-paced
study in multitrial learning. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 41,
851– 858. http://dx.doi.org/10.1037/xlm0000046

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan,
M. J., & Willingham, D. T. (2013). Improving
students’ learning with effective learning tech-
niques: Promising directions from cognitive and
educational psychology. Psychological Science in
the Public Interest, 14, 4–58. http://dx.doi.org/10
.1177/1529100612453266

Eglington, L. G., & Kang, S. H. K. (2018). Retrieval
practice benefits deductive inference. Educational
Psychology Review, 30, 215–228. http://dx.doi
.org/10.1007/s10648-016-9386-y

Healy, A. F., Jones, M., Lalchandani, L. A., & Tack,
L. A. (2017). Timing of quizzes during learning:
Effects on motivation and retention. Journal of
Experimental Psychology: Applied, 23, 128–137.
http://dx.doi.org/10.1037/xap0000123

Hogan, R. M., & Kintsch, W. (1971). Differential
effects of study and test trials on long-term recog-
nition and recall. Journal of Verbal Learning and
Verbal Behavior, 10, 562–567. http://dx.doi.org/10
.1016/S0022-5371(71)80029-4

Horvath, J. C., Lodge, J. M., & Hattie, J. (2017).
From the laboratory to the classroom: Translating
science of learning for teachers. New York, NY:
Routledge/Taylor & Francis Group.

Jacoby, L. L. (1991). A process dissociation frame-
work: Separating automatic from intentional
uses of memory. Journal of Memory and Lan-
guage, 30, 513–541. http://dx.doi.org/10.1016/
0749-596X(91)90025-F

Johnstone, T., & Shanks, D. R. (2001). Abstractionist
and processing accounts of implicit learning. Cog-
nitive Psychology, 42, 61–112. http://dx.doi.org/10
.1006/cogp.2000.0743

206 CORRAL, HEALY, ROZBRUCH, AND JONES

http://dx.doi.org/10.1187/cbe.14-11-0208

http://dx.doi.org/10.1037/a0019902

http://dx.doi.org/10.1037/xap0000142

http://dx.doi.org/10.1037/a0031026

http://dx.doi.org/10.1037/1076-898X.13.4.273

http://dx.doi.org/10.1037/a0017021

http://dx.doi.org/10.1002/acp.1507

http://dx.doi.org/10.1187/cbe.16-06-0205

http://dx.doi.org/10.1016/j.jml.2016.06.008

http://dx.doi.org/10.3758/BF03202713

http://dx.doi.org/10.1080/09658211.2015.1086380

http://dx.doi.org/10.1016/j.cognition.2014.04.007

http://dx.doi.org/10.1037/xge0000517

http://dx.doi.org/10.1016/S0022-5371%2872%2980001-X

http://dx.doi.org/10.1037/0096-3445.104.3.268

http://dx.doi.org/10.1037/xlm0000046

http://dx.doi.org/10.1177/1529100612453266

http://dx.doi.org/10.1007/s10648-016-9386-y

http://dx.doi.org/10.1037/xap0000123

http://dx.doi.org/10.1016/S0022-5371%2871%2980029-4

http://dx.doi.org/10.1016/0749-596X%2891%2990025-F

http://dx.doi.org/10.1006/cogp.2000.0743

Kahana, M. J. (2012). Foundations of human mem-
ory. New York, NY: Oxford University Press.

Kang, S. H. K., Gollan, T. H., & Pashler, H. (2013).
Don’t just repeat after me: Retrieval practice is
better than imitation for foreign vocabulary learn-
ing. Psychonomic Bulletin & Review, 20, 1259–
1265. http://dx.doi.org/10.3758/s13423-013-
0450-z

Kang, S. H. K., & Pashler, H. (2014). Is the benefit of
retrieval practice modulated by motivation? Jour-
nal of Applied Research in Memory & Cognition,
3, 183–188. http://dx.doi.org/10.1016/j.jarmac
.2014.05.006

Karpicke, J. D., & Roediger, H. L., III. (2008). The
critical importance of retrieval for learning. Science,
319, 966–968. http://dx.doi.org/10.1126/science
.1152408

Kintsch, W. (1970). Models for free recall and rec-
ognition. In D. A. Normal (Ed.), Models of human
memory (pp. 331–373). New York, NY: Academic
Press. http://dx.doi.org/10.1016/B978-0-12-
521350-9.50016-4

Kole, J. A., Healy, A. F., & Bourne, L. E., Jr. (2008).
Cognitive complications moderate the speed-
accuracy tradeoff in data entry: A cognitive anti-
dote to inhibition. Applied Cognitive Psychology,
22, 917–937. http://dx.doi.org/10.1002/acp.1401

Lehman, M., & Karpicke, J. D. (2016). Elaborative
retrieval: Do semantic mediators improve mem-
ory? Journal of Experimental Psychology: Learn-
ing, Memory, and Cognition, 42, 1573–1591.
http://dx.doi.org/10.1037/xlm0000267

Little, J. L., & Bjork, E. L. (2015). Optimizing mul-
tiple-choice tests as tools for learning. Memory &
Cognition, 43, 14–26. http://dx.doi.org/10.3758/
s13421-014-0452-8

Marsh, E. J., Roediger, H. L., III, Bjork, R. A., &
Bjork, E. L. (2007). The memorial consequences
of multiple-choice testing. Psychonomic Bulletin
& Review, 14, 194–199. http://dx.doi.org/10.3758/
BF03194051

Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K.,
Bimber, B., Chun, D., . . . Zhang, H. (2009).
Clickers in college classrooms: Fostering learning
with questioning methods in large lecture classes.
Contemporary Educational Psychology, 34, 51–
57. http://dx.doi.org/10.1016/j.cedpsych.2008.04
.002

McDaniel, M. A., Anderson, J. L., Derbish, M. H., &
Morrisette, N. (2007). Testing the testing effect in the
classroom. The European Journal of Cognitive Psy-
chology, 19, 494–513. http://dx.doi.org/10.1080/
09541440701326154

Oliver, M., & Conole, G. (2003). Evidence-based
practice in e-learning and higher education: Can
we and should we? Research Papers in Education,
18, 385–397. http://dx.doi.org/10.1080/0267152
032000176873

Pan, S. C., & Rickard, T. C. (2018). Transfer of
test-enhanced learning: Meta-analytic review and
synthesis. Psychological Bulletin, 144, 710–756.
http://dx.doi.org/10.1037/bul0000151

Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer,
D. (2005). When does feedback facilitate learning
of words? Journal of Experimental Psychology:
Learning, Memory, and Cognition, 31, 3– 8. http://
dx.doi.org/10.1037/0278-7393.31.1.3

Peterson, D. J., & Wissman, K. T. (2018). The testing
effect and analogical problem-solving. Memory,
26, 1460–1466. http://dx.doi.org/10.1080/0965
8211.2018.1491603

Pyc, M. A., & Rawson, K. A. (2010). Why testing
improves memory: Mediator effectiveness hypoth-
esis. Science, 330, 335. http://dx.doi.org/10.1126/
science.1191465

Quinn, P. C., Palmer, V., & Slater, A. M. (1999).
Identification of gender in domestic-cat faces with
and without training: Perceptual learning of a nat-
ural categorization task. Perception, 28, 749–763.
http://dx.doi.org/10.1068/p2884

Rickard, T. C., & Pan, S. C. (2018). A dual memory
theory of the testing effect. Psychonomic Bulletin
& Review, 25, 847– 869. http://dx.doi.org/10.3758/
s13423-017-1298-4

Roediger, H. L., III. (2013). Applying cognitive psy-
chology to education: Translational educational
science. Psychological Science in the Public Inter-
est, 14, 1–3. http://dx.doi.org/10.1177/15291006
12454415

Roediger, H. L., III, & Butler, A. C. (2011). The
critical role of retrieval practice in long-term re-
tention. Trends in Cognitive Sciences, 15, 20–27.
http://dx.doi.org/10.1016/j.tics.2010.09.003

Roediger, H. L., III, & Karpicke, J. D. (2006a).
Test-enhanced learning: Taking memory tests im-
proves long-term retention. Psychological Science,
17, 249–255. http://dx.doi.org/10.1111/j.1467-
9280.2006.01693.x

Roediger, H. L., III, & Karpicke, J. D. (2006b). The
power of testing memory: Basic research and im-
plications for educational practice. Perspectives on
Psychological Science, 1, 181–210. http://dx.doi
.org/10.1111/j.1745-6916.2006.00012.x

Rowland, C. A. (2014). The effect of testing versus
restudy on retention: A meta-analytic review of the
testing effect. Psychological Bulletin, 140, 1432–
1463. http://dx.doi.org/10.1037/a0037559

Shanks, D. R., Johnstone, T., & Staggs, L. (1997).
Abstraction processes in artificial grammar learn-
ing. Quarterly Journal of Experimental Psychol-
ogy A: Human Experimental Psychology, 50, 216–
252. http://dx.doi.org/10.1080/713755680

Smeyers, P., & Depaepe, M. (2013). Making sense of
the attraction of psychology: On the strengths and
weaknesses for education and educational re-

207TESTING-BASED TRAINING PARADIGM

http://dx.doi.org/10.3758/s13423-013-0450-z

http://dx.doi.org/10.1016/j.jarmac.2014.05.006

http://dx.doi.org/10.1126/science.1152408

http://dx.doi.org/10.1016/B978-0-12-521350-9.50016-4

http://dx.doi.org/10.1002/acp.1401

http://dx.doi.org/10.1037/xlm0000267

http://dx.doi.org/10.3758/s13421-014-0452-8

http://dx.doi.org/10.3758/BF03194051

http://dx.doi.org/10.1016/j.cedpsych.2008.04.002

http://dx.doi.org/10.1080/09541440701326154

http://dx.doi.org/10.1080/0267152032000176873

http://dx.doi.org/10.1037/bul0000151

http://dx.doi.org/10.1037/0278-7393.31.1.3

http://dx.doi.org/10.1080/09658211.2018.1491603

http://dx.doi.org/10.1126/science.1191465

http://dx.doi.org/10.1068/p2884

http://dx.doi.org/10.3758/s13423-017-1298-4

http://dx.doi.org/10.1177/1529100612454415

http://dx.doi.org/10.1016/j.tics.2010.09.003

http://dx.doi.org/10.1111/j.1467-9280.2006.01693.x

http://dx.doi.org/10.1111/j.1745-6916.2006.00012.x

http://dx.doi.org/10.1037/a0037559

http://dx.doi.org/10.1080/713755680

search. In P. Smeyers & M. Depaepe (Eds.), Edu-
cational research: The attraction of psychology
(pp. 1–10). Dordrecht, The Netherlands: Springer.
http://dx.doi.org/10.1007/978-94-007-5038-8_1

Symons, C. S., & Johnson, B. T. (1997). The self-
reference effect in memory: A meta-analysis. Psy-
chological Bulletin, 121, 371–394. http://dx.doi
.org/10.1037/0033-2909.121.3.371

Tran, R., Rohrer, D., & Pashler, H. (2015). Retrieval
practice: The lack of transfer to deductive infer-
ences. Psychonomic Bulletin & Review, 22, 135–
140. http://dx.doi.org/10.3758/s13423-014-0646-x

Tullis, J. G., & Benjamin, A. S. (2011). On the
effectiveness of self-paced learning. Journal of
Memory and Language, 64, 109–118. http://dx.doi
.org/10.1016/j.jml.2010.11.002

van Gog, T., & Kester, L. (2012). A test of the testing
effect: Acquiring problem-solving skills from
worked examples. Cognitive Science, 36, 1532–
1541. http://dx.doi.org/10.1111/cogs.12002

Vojdanoska, M., Cranney, J., & Newell, B. R.
(2010). The testing effect: The role of feedback
and collaboration in a tertiary classroom setting.
Applied Cognitive Psychology, 24, 1183–1195.
http://dx.doi.org/10.1002/acp.1630

Wissman, K. T., Zamary, A., & Rawson, K. A. (2018).
When does practice testing promote transfer on de-
ductive reasoning tasks? Journal of Applied Re-
search in Memory & Cognition, 7, 398– 411. http://
dx.doi.org/10.1016/j.jarmac.2018.03.002

Woolf, S. H. (2008). The meaning of translational
research and why it matters. Journal of the Amer-
ican Medical Association, 299, 211–213. http://dx
.doi.org/10.1001/jama.2007.26

Received April 19, 2018
Revision received March 26, 2019

Accepted April 10, 2019 �

E-Mail Notification of Your Latest Issue Online!

Would you like to know when the next issue of your favorite APA journal will be
available online? This service is now available to you. Sign up at https://my.apa.org/
portal/alerts/ and you will be notified by e-mail when issues of interest to you become
available!

208 CORRAL, HEALY, ROZBRUCH, AND JONES

http://dx.doi.org/10.1007/978-94-007-5038-8_1

http://dx.doi.org/10.1037/0033-2909.121.3.371

http://dx.doi.org/10.3758/s13423-014-0646-x

http://dx.doi.org/10.1016/j.jml.2010.11.002

http://dx.doi.org/10.1111/cogs.12002

http://dx.doi.org/10.1002/acp.1630

http://dx.doi.org/10.1016/j.jarmac.2018.03.002

http://dx.doi.org/10.1001/jama.2007.26

Building a Testing-Based Training Paradigm From Cognitive Psychology Principles

Recognition Versus Recall
Experiment and Training Paradigm
Method
Participants
Design and Materials
Question types
Core questions
Conceptual questions

Procedure
Training session
Navigating each section
Quiz instructions
Quiz questions
Correct-answer feedback
Immediate posttest
Delayed posttest
Control group

Results
Experimental Conditions Versus Control Condition
Performance on core questions
Performance on conceptual questions
Recall Versus Recognition Versus Recall-Then-Recognition
Exploratory Analysis
Discussion
Type of Quizzing Format
Guide and Implications for Instructors
Limitations and Future Directions
Conclusion
References

INTRODUCTION

Intersection of Minority Health, Health Disparities, and Social
Determinants of Health With Psychopharmacology and Substance Use

Hector I. Lopez-Vergara1, Tamika C. B. Zapolski2, and Adam M. Leventhal3, 4
1 Department of Psychology, University of Rhode Island

2 Department of Psychology, Indiana University–Purdue University Indianapolis
3 Department of Population and Public Health Sciences, Keck School of Medicine, University of Southern California

4 Institute for Addiction Science, University of Southern California

aaa
Although the United States (U.S.) is an increasingly multicultural

society (Vespa et al., 2018), the incorporation of psychopharma-
cology and basic behavioral addiction science into research on
minority health and health disparities is underleveraged. Similarly,
psychopharmacology and substance use research has often over-
looked the societal context in which drugs are consumed. The
relative schism of psychopharmacology and basic behavioral addic-
tion science with research on minority health and health disparities is
an area of opportunity for improving public health in an ever more
diverse society (Muennig et al., 2018). In efforts to stimulate
research on these topics, this special issue focuses on research
addressing the intersection of minority health, health disparities,
and social determinants of health (MHDS) with psychopharmacol-
ogy and substance use.
The articles in the special issue are transdisciplinary in nature,

encompassing topics that range in focus from how sociocultural
variables at the individual and societal level influence risk for
substance use, to statistical and methodological issues in execution
on minority health and health disparities research, to etiology
focused studies and clinical applications. The special issue begins
with Ozga et al. (2021) who contextualize tobacco use health
disparities in rural communities via a cumulative disadvantage
framework. By reviewing the development of tobacco use

disparities among rural communities with the consideration of
systemic and pharmacological factors that contribute to the dispa-
rities, the authors identify potential points of macrolevel interven-
tions, as well as articulate how health outcomes can be shaped by
societal forces. This is followed by Phillips et al. (2021) who
delineate the various factors that impact the development of tobacco
use among Asian and Pacific Islander (API) youth using the
National Institute on Minority Health and Health Disparities
(NIMHD) multidimensional research framework, which serves as
a tool to “unpack” distinct levels of analysis in the development of
health disparities from cell to society. Reviewing psychometric
critiques of cross-cultural research, Lopez-Vergara et al. (2021)
emphasize the need to statistically test for the cultural equivalence
of measurement (within a falsifiable psychometric framework)
when making inferences across cultural groups in addiction and
clinical science.

These reviews are followed by five novel studies examining
sociocultural, MHDS, and psychopharmacological factors that influ-
ence substance use risk across racial, ethnic, and sexual minority
populations. First, in an empirical search for mechanisms of health
disparities, Bacio (2021) uses structural equation modeling to inves-
tigate drinking motives as pathways to problematic drinking among
Latinx college students. Bacio (2021) provides evidence that socio-
cultural variables may influence drinking among Latinx students via
motivational pathways (drinking to cope), demonstrating how socio-
cultural variables can influence individual-level processes. This is
followed by Clifton et al. (2021), who compared differences in racial
identity among Black young adults based on both explicit and implicit
measurement strategies. By leveraging basic behavioral science
principles, the authors provide an indirect way of assessing aspects
of racially based self-concept that may be difficult to measure directly
due to social desirability, as well as how such assessments relate to
substance use outcomes among Black young adults. Next, demon-
strating how intersectionality of identities can overlap with distinct
risk and protective factors, Albuja et al. (2021) disaggregate corre-
lates of increased alcohol involvement among Monoracial and Mul-
tiracial Native American/American Indian college students.

Finally, in this section, we have two studies that utilize novel study
designs to examine sociocultural factors, psychopharmacology, and
substance use among minority populations. In an empirical study at
the within-person level of analysis, Lewis et al. (2021) use a daily
diary study to investigate the proximal effects of sexual minority

Editor’s Note. This is an introduction to the special issue “Intersection of
Minority Health, Health Disparities, and Social Determinants of Health with
Psychopharmacology and Substance Use.” Please see the Table of Contents
here: http://psycnet.apa.org/journals/pha/29/5—WWS

Tamika C. B. Zapolski https://orcid.org/0000-0003-0675-560X
None of the authors have a conflict of interest to declare.
Hector I. Lopez-Vergara played a equal role in conceptualization, writing

of original draft, and writing of review and editing. Tamika C. B. Zapolski
played an equal role in conceptualization, writing of original draft, and
writing of review and editing. Adam M. Leventhal played an equal role in
conceptualization, writing of original draft, and writing of review and
editing.
Correspondence concerning this article should be addressed to Hector I.

Lopez-Vergara, Department of Psychology, University of Rhode Island,
142 Flagg Road, Kingston, RI 02881, United States. Email: hlopez-verga
ra@uri.edu

Experimental and Clinical Psychopharmacology

© 2021 American Psychological Association 2021, Vol. 29, No. 5, 427–428
ISSN: 1064-1297 https://doi.org/10.1037/pha0000522

427

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed

b
y
th
e
A
m
er
ic
an

P
sy
ch
o
lo
g
ic
al

A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed

p
u
b
li
sh
er
s.

C
o
n
te
n
t
m
ay

b
e
sh
ar
ed

at
n
o
co
st
,
b
ut

an
y
re
q
ue
st
s
to

re
u
se

th
is
co
n
te
n
t
in

p
ar
t
o
r
w
h
ol
e
m
u
st
g
o
th
ro
ug
h
th
e
A
m
er
ic
an

P
sy
ch
o
lo
g
ic
al

A
ss
o
ci
at
io
n
.

http://psycnet.apa.org/journals/pha/29/5

https://orcid.org/0000-0003-0675-560X

mailto:hlopez-vergara@uri.edu

https://doi.org/10.1037/pha0000522

stress on alcohol involvement among sexual minority women, pro-
viding insights into how experiences of marginalization can unfold in
day-to-day drinking experiences and may intersect with negative
reinforcement mechanisms underlying substance use. Leveraging
experimentally induced nicotine withdrawal, Liautaud et al. (2021)
show that symptoms of anhedonia may be a phenotypic feature of
acute withdrawal among African–American smokers (which provides
hypotheses for phenotypically personalized treatments) in a popula-
tion subject to tobacco-related health disparities.
The special issue concludes with articles addressing interven-

tions. Nalven et al. (2021) provide a systematic review of diversity
inclusion in opioid pharmacological treatment trials, finding evi-
dence that minoritized populations are frequently underrepresented
in treatment trials; whereas Fogg et al. (2021) document that
minoritized individuals are typically omitted from samples in the
reemerging field of Psychedelic-Assisted Psychotherapies. Finally,
in a remote adaptation to a contingency management intervention
for problematic drinking, Koffarnus et al. (2021) show that a
contingency management intervention for alcohol use disorder
(with a participant-funded incentive) is effective but is less accessi-
ble to participants with lower income and greater alcohol use.
Overall, these articles provide a window into the breadth of issues

at the intersection of MHDS with psychopharmacology and substance
use. Integrating the fields of psychopharmacology and basic behav-
ioral addictions science with research on MHDS is not only of public
health importance, but can help further elucidate our understanding of
human behavior in all of its complexity. As demonstrated here, a
better understanding of the synergy between societal context(s) and
individual-level processes can lead to interventions tailored to specific
risk and resilience factors; interventions that are personalized and
contextualized have the potential to improve the health of our society.
We are very grateful to the authors for their contributions to this
special issue. We hope that professionals from various disciplines
who read this special issue become inspired to bridge psychophar-
macological and social determinants perspectives in their own work,
and, in turn, accelerate scientific progress within each field.

References

Albuja, A., Straka, B., Desjardins, M., Swartzwelder, H. S., & Gaither, S.
(2021). Alcohol use and related consequences for monoracial and multi-
racial Native American/American Indian college students. Experimental
and Clinical Psychopharmacology, 29(5), 487–500. https://doi.org/10
.1037/pha0000475

Bacio, G. A. (2021). Motivational pathways to problematic drinking among
Latinx college drinkers. Experimental and Clinical Psychopharmacology,
29(5), 466–478. https://doi.org/10.1037/pha0000516

Clifton, R. L., Rowe, A. T., Banks, D. E., Ashburn-Nardo, L., & Zapolski,
T. C. B. (2021). Examining the effects of implicit and explicit racial

identity on psychological distress and substance use among Black young
adults. Experimental and Clinical Psychopharmacology, 29(5), 479–486.
https://doi.org/10.1037/pha0000489

Fogg, C., Michaels, T. I., de la Salle, S., Jahn, Z. W., & Williams, M. T.
(2021). Ethnoracial health disparities and the ethnopsychopharmacology
of psychedelic-assisted psychotherapies. Experimental and Clinical
Psychopharmacology, 29(5), 537–552. https://doi.org/10.1037/pha
0000490

Koffarnus, M. N., Kablinger, A. S., Kaplan, B. A., & Crill, E. M. (2021).
Remotely administered incentive-based treatment for alcohol use disorder
with participant-funded incentives is effective but less accessible to low-
income participants. Experimental and Clinical Psychopharmacology,
29(5), 526–536. https://doi.org/10.1037/pha0000503

Lewis, R. J., Romano, K. A., Ehlke, S. J., Lau-Barraco, C., Sandoval, C. M.,
Glenn, D. J., & Heron, K. E. (2021). Minority stress and alcohol use in
sexual minority women’s daily lives. Experimental and Clinical Psycho-
pharmacology, 29(5), 501–510. https://doi.org/10.1037/pha0000484

Liautaud, M. M., Kechter, A., Bello, M. S., Guillot, C. R., Oliver, J. A.,

Banks, D. E., D’Orazio, L. M., & Leventhal, A. M. (2021). Anhedonia in

tobacco withdrawal among African-American smokers. Experimental and

Clinical Psychopharmacology, 29(5), 553–565. https://doi.org/10.1037/

pha0000474
Lopez-Vergara, H. I., Yang, M., Weiss, N. H., Stamates, A. L., Spillane,
N. S., & Feldstein Ewing, S. W. (2021). The cultural equivalence of
measurement in substance use research. Experimental and Clinical
Psychopharmacology, 29(5), 456–465. https://doi.org/10.1037/pha
0000512

Muennig, P. A., Reynolds, M., Fink, D. S., Zafari, Z., & Geronimus, A. T.
(2018). America’s declining well-being, health, and life expectancy: Not
just a white problem. American Journal of Public Health, 108, 1626–
1631. https://doi.org/10.2105/AJPH.2018.304585

Nalven, T., Spillane, N. S., Schick, M. R., & Weyandt, L. L. (2021).
Diversity inclusion in United States opioid pharmacological treatment
trials: A systematic review. Experimental and Clinical Psychopharma-
cology, 29(5), 511–525. https://doi.org/10.1037/pha0000510

Ozga, J. E., Romm, K. F., Turiano, N. A., Douglas, A., Dino, G., Alexander,
L., & Blank, M. D. (2021). Cumulative disadvantage as a framework for
understanding rural tobacco use disparities. Experimental and Clinical
Psychopharmacology, 29(5), 429–439. https://doi.org/10.1037/pha
0000476

Phillips, K. T., Okamoto, S. K., Johnson, D. L., Rosario, M. H., Manglallan,
K. S., & Pokhrel, P. (2021). Correlates of tobacco use among Asian and
Pacific Islander youth and young adults in the U.S.: A systematic review of
the literature. Experimental and Clinical Psychopharmacology, 29(5),
440–455. https://doi.org/10.1037/pha0000511

Vespa, J., Armstrong, D., & Medina, L. (2018). Demographic turning points
for the United States: Population projections for 2020 to 2060. In Current
population reports (pp. 25–1144). U.S. Census Bureau.

Received August 2, 2021 ▪

428 LOPEZ-VERGARA, ZAPOLSKI, AND LEVENTHAL

T
h
is
d
o
cu
m
en
t
is
co
p
y
ri
g
ht
ed
b
y
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
o
r
o
n
e
o
f
it
s
al
li
ed
p
u
b
li
sh
er
s.
C
o
n
te
n
t
m
ay
b
e
sh
ar
ed
at
n
o
co
st
,
b
ut
an
y
re
q
ue
st
s
to
re
u
se
th
is
co
n
te
n
t
in
p
ar
t
o
r
w
h
ol
e
m
u
st
g
o
th
ro
ug
h
th
e
A
m
er
ic
an
P
sy
ch
o
lo
g
ic
al
A
ss
o
ci
at
io
n
.

https://doi.org/10.1037/pha0000475

https://doi.org/10.1037/pha0000516

https://doi.org/10.1037/pha0000489

https://doi.org/10.1037/pha0000490

https://doi.org/10.1037/pha0000503

https://doi.org/10.1037/pha0000484

https://doi.org/10.1037/pha0000474

https://doi.org/10.1037/pha0000512

https://doi.org/10.2105/AJPH.2018.304585

https://doi.org/10.1037/pha0000510

https://doi.org/10.1037/pha0000476

https://doi.org/10.1037/pha0000511

Intersection of Minority Health, Health Disparities, and Social Determinants of Health With Psychopharmacology and Substance Use

References

Psychoso

Review Article

Psychoso

Psychopharmacology of COVID-19

Melanie Bilbul, M.D., C.M., F.R.C.P.(C), Patricia Paparone, M.D., Anna M. Kim, M.D.,
Shruti Mutalik, M.D., Carrie L. Ernst, M.D.

Background: With the rapid, global spread of severe
acute respiratory syndrome coronavirus 2, hospitals have
become inundated with patients suffering from corona-
virus disease 2019. Consultation-liaison psychiatrists are
actively involved in managing these patients and should
familiarize themselves with how the virus and its
proposed treatments can affect psychotropic
management. The only Food and Drug Administration–
approved drug to treat COVID-19 is remdesivir, and
other off-label medications used include chloroquine and
hydroxychloroquine, tocilizumab, lopinavir/ritonavir,
favipiravir, convalescent plasma therapy, azithromycin,
vitamin C, corticosteroids, interferon, and colchicine.
Objective: To provide an overview of the major safety
considerations relevant to clinicians who prescribe psy-
chotropics to patients with COVID-19, both related to
the illness and its proposed treatments. Methods: In this
targeted review, we performed structured literature
searches in PubMed to identify articles describing the
impacts of COVID-19 on different organ systems, the

matics 61:5, September/October 2020

neuropsychiatric adverse effects of treatments, and any
potential drug interactions with psychotropics. The
articles most relevant to this one were included. Results:
COVID-19 impacts multiple organ systems, including
gastrointestinal, renal, cardiovascular, pulmonary,
immunological, and hematological systems. This may
lead to pharmacokinetic changes that impact psycho-
tropic medications and increase sensitivity to
psychotropic-related adverse effects. In addition, several
proposed treatments for COVID-19 have
neuropsychiatric effects and potential interactions with
commonly used psychotropics. Conclusions: Clinicians
should be aware of the need to adjust existing psycho-
tropics or avoid using certain medications in some pa-
tients with COVID-19. They should also be familiar
with neuropsychiatric effects of medications being used
to treat this disease. Further research is needed to
identify strategies to manage psychiatric issues in this
population.

(Psychosomatics 2020; 61:411–427)

Key words: COVID-19, psychotropic, psychopharmacology, side effects.

Received April 24, 2020; revised May 11, 2020; accepted May 12, 2020.
From the Department of Psychiatry(M.B., P.P., A.M.K., S.M., C.L.E.),
Icahn School of Medicine at Mount Sinai, New York, NY; Department
of Medical Education (C.L.E.), Icahn School of Medicine at Mount
Sinai, New York, NY. Send correspondence and reprint requests to
Carrie L. Ernst, MD, One Gustave L. Levy Place, Box 1230, New York,
NY 10029; e-mail: carrie.ernst@mssm.edu

INTRODUCTION

With the rapid, global spread of severe acute respira-
tory syndrome coronavirus 2 (SARS-CoV-2), hospitals
have become inundated with patients suffering from
COVID-19 infection. Remdesivir was recently
approved by the US Food and Drug Administration
(FDA) to treat severe COVID-19,1 and many other
medications are either being studied in clinical trials or
being used off-label and/or for compassionate use.2

As the pandemic spreads, consultation-liaison
psychiatrists are being called upon to help manage the

psychiatric conditions of individuals with COVID-19
and are encountering challenging clinical scenarios
of multiple medical comorbidities and unfamiliar
drugs. Psychiatrists should familiarize themselves

www.psychosomaticsjournal.org 411

http://crossmark.crossref.org/dialog/?doi=10.1016/j.psym.2020.05.006&domain=pdf

mailto:carrie.ernst@mssm.edu

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

with the mechanism of action of these treatments,
neuropsychiatric side effects, and possible interactions
with psychotropics. In addition, as COVID-19 affects
multiple organ systems, psychiatrists will need to be
aware of safety concerns inherent in prescribing psy-
chotropics to these patients.

This article is divided into 2 main sections. The first
provides an update on the organ systems that may be
negatively impacted by COVID-19 and recommenda-
tions for safer use of psychotropics in these patients.
The second section reviews potential neuropsychiatric
side effects of the early approved and investigational
treatments for COVID-19 as well as pharmacokinetic
and pharmacodynamic drug interactions when used
concurrently with psychotropics. COVID-19 therapies
reviewed include remdesivir, chloroquine, hydroxy-
chloroquine, azithromycin, tocilizumab, lopinavir/
ritonavir, favipiravir, convalescent plasma therapy, cor-
ticosteroids, interferon (IFN), vitamin C, and colchicine.

Given the limited literature in this area, we un-
dertook a nonsystematic narrative review that was
focused on practical clinical concerns. We used a
structured PubMed search using the following search
terms in combination with the names of the medica-
tions mentioned previously: “COVID-19”, “coronavi-
rus”, “Psychotropic medications”, “QT prolongation”,
“Psychiatric side effects”, “Neuropsychiatric side ef-
fects”, “drug interactions”, and pertinent organ sys-
tems, for example, “hepatic”, “renal”, “hematological”,
“pulmonary”, and “cardiac”. This was followed by a
search of manufacturer’s package inserts for pertinent
facts about specific medications, including drug
interactions.

We selected the aforementioned medications as
they were the ones most commonly being used in health
care settings and clinical trials at the time of prepara-
tion of this article, although we are aware that this is a
rapidly evolving field and thus this list is not meant to
be comprehensive.

IMPACT OF COVID-19 ON PSYCHOTROPIC
DRUG SAFETY

COVID-19 is believed to impact multiple organs,
including the liver, kidneys, lungs, and heart, as well as
the immune and hematological systems.3 Damage to
these organs or systems may lead to pharmacokinetic
changes that impact absorption, distribution,

412 www.psychosomaticsjournal.org

metabolism, and/or excretion of psychotropic medica-
tions as well as increased sensitivity to certain psycho-
tropic adverse effects. As such, clinicians should be
aware of the potential need to make adjustments to
existing psychotropic regimens or avoid using certain
psychotropic agents if such safety concerns arise
(Tables 1 and 2).

Hematological Effects

An early report noted the presence of lymphopenia
(lymphocyte count less than 1.0 3 109/L) in 63% and
leukopenia (white blood cell count less than 4 3 109/L)
in 25% of patients with COVID-19.4 It has been pro-
posed that lymphopenia is a feature of severe COVID-
19 cases and may serve as a poor prognostic factor.
Contributing factors likely include direct infection of
lymphocytes and cytokine storm.5 It therefore seems
prudent to use caution and consider avoiding medica-
tions that have the potential to further impact white
blood cell production, particularly lymphocytes. By
contrast, clinicians might determine that it is acceptable
from a safety standpoint to continue psychotropics
which have only been associated with agranulocytosis
and neutropenia, assuming the patient does not have a
secondary bacterial infection. Several psychotropics
have been implicated in hematological adverse effects,
including leukopenia, neutropenia, and agranulocy-
tosis. The most commonly implicated psychotropics
include carbamazepine and clozapine, but there is a
class effect FDA warning on all first and secondary
generation antipsychotics for the potential association
with leukopenia, neutropenia, and agranulocytosis, as
well as a number of published case reports. Carba-
mazepine is more likely to be associated with an early
transient leukopenia but has also been associated with
agranulocytosis and aplastic anemia.6

While the leukopenia and lymphopenia observed in
patients with COVID-19 may be less of a concern for
clozapine prescribers in the setting of a normal
neutrophil count, clozapine deserves unique mention
given several potential challenges associated with its use
during the COVID-19 pandemic. These challenges have
been recently reviewed along with recommendations for
management in a consensus statement by Siskind and
colleagues.7 Patients on clozapine may have difficulty
accessing routine absolute neutrophil count moni-
toring, and the FDA has released guidance allowing
health care providers to use medical judgment to delay

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

TABLE 1. Potential Psychotropic Safety Concerns in COVID-19 Organized by Drug Class

Drug class Specific drugs Problem Solution

Antipsychotics Clozapine Patients with difficulty accessing ANC monitoring
May be associated with increased risk of
pneumonia and its complications

Levels can increase with acute infection leading to
clozapine toxicity

COVID-19 associated with leukopenia and
lymphopenia; unclear impact on neutrophils;
clozapine associated with neutropenia and
agranulocytosis and more rarely lymphopenia
or aplastic anemia

COVID-19 associated with seizures; clozapine can
lower seizure threshold

Reduce frequency of ANC monitoring at
discretion of provider

Education of patients and urgent clinical
assessment including ANC for those with
symptoms of infection

Consider halving clozapine dose in patients with
fever, pneumonia, and/or flu-like symptoms;
temporarily discontinue clozapine if toxicity
emerges

Monitor complete blood count (CBC); if
persistent white blood cell abnormalities, weigh
risks versus benefits of continuing clozapine;
when total white blood cell count is decreased
but neutrophil count is normal, consider
continuing clozapine

Recognize potential for lowered seizure threshold;
assure nontoxic clozapine level; consider
holding clozapine, decreasing dose, or adding
antiepileptic

Other
antipsychotics

COVID-19 associated with decreased white blood
cell and lymphocyte counts; rare reports of
antipsychotic-associated aplastic anemia or
lymphopenia, especially with phenothiazines
(chlorpromazine, fluphenazine, thioridazine)

Coagulation abnormalities (PT and aPTT
prolongation, thrombocytopenia) are observed
in patients with COVID-19; rare reports of
thrombocytopenia associated with multiple
antipsychotics

Concern for COVID-19 associated
tachyarrhythmias and cardiac injury and
potential for several medications being used to
treat COVID-19 to cause QT prolongation; all
antipsychotics with potential for QT
prolongation

Acute liver injury in patients with COVID-19;
antipsychotics (especially chlorpromazine) with
potential for drug-induced liver injury

COVID-19 associated with seizures; all
antipsychotics can lower seizure threshold

Monitor CBC; if persistent hematologic
abnormalities (e.g., lymphopenia, neutropenia,
thrombocytopenia) weigh risks versus benefits
of continuing antipsychotic agent

Baseline EKG for QTc; caution in patients with
baseline prolonged QTc and/or other risk
factors for drug-induced QT prolongation and
TdP; daily EKG and electrolyte monitoring,
reduce other risk factors, and cardiology consult
in high-risk cases if opt to use antipsychotic;
case-by-case risk-benefit discussion

Monitor liver function tests and avoid
chlorpromazine in patients with liver injury; risk
versus benefit assessment for other antipsychotic
use

Consider avoiding antipsychotics (especially
clozapine, quetiapine, olanzapine, and first-
generation drugs) or adding antiepileptic drug
(AED) in patients who have seizures

Antiepileptics Carbamazepine COVID-19 associated with leukopenia and
lymphopenia; leukopenia and rare reports of
aplastic anemia associated with carbamazepine
use;

Acute liver injury in patients with COVID-19;
carbamazepine with potential for drug-induced
liver injury

Monitor CBC; if persistent white blood cell
abnormalities or aplastic anemia, use
alternative AED

Monitor liver function tests and avoid
carbamazepine in patients with liver injury

Valproic acid Coagulation abnormalities (PT and aPTT
prolongation, thrombocytopenia) observed in
patients with COVID-19; valproic acid
associated with thrombocytopenia

Acute liver injury in patients with COVID-19;
valproic acid with potential for drug-induced
liver injury

Monitor platelet count; avoid valproic acid if
thrombocytopenia

Monitor liver function tests and avoid valproic
acid in patients with liver injury

Gabapentin COVID-19 with potential for acute kidney injury;
gabapentin clearance dependent on intact renal
function

Adjust gabapentin dose based on renal function

Bilbul et al.

Psychosomatics 61:5, September/October 2020 www.psychosomaticsjournal.org 413

http://www.psychosomaticsjournal.org

TABLE 1. (Continued)

Drug class Specific drugs Problem Solution

Selective
serotonin
reuptake
inhibitors
(SSRIs) and
serotonin
norepinephrine
reuptake
inhibitors
(SNRIs)

All Coagulation abnormalities observed in patients
with COVID-19 and many patients with
COVID-19 receiving anticoagulation; SSRIs
and SNRIs associated with impaired platelet
aggregation and abnormal bleeding

Concern for COVID-19–associated
tachyarrhythmias and cardiac injury and
potential for several medications being used to
treat COVID-19 to cause QT prolongation;
citalopram with potential for QT prolongation

Acute liver injury in patients with COVID-19;
duloxetine with a potential for drug-induced
liver injury

Monitor coagulation factors and platelet count;
weigh risks and benefits for individual patient
but consider avoiding SSRIs and SNRIs in
patients with recent bleeding or high risk for
bleeding (e.g., thrombocytopenia, concurrent
anticoagulation therapy, history of
hemorrhage); can instead use nonserotonin
reuptake inhibitor antidepressant such as
bupropion

Baseline EKG for QTc; caution in patients with
baseline prolonged QTc and/or other risk
factors for drug-induced QT prolongation and
TdP; consider using SSRI other than citalopram
in high-risk cases

Monitor liver function tests, avoid duloxetine in
patients with liver injury

Bupropion COVID-19 associated with seizures; bupropion
can lower seizure threshold

Avoid bupropion in patients with seizures or
lowered seizure threshold

Lithium COVID-19 with potential for acute kidney injury;
lithium clearance dependent on intact renal
function; lithium with nephrotoxic potential

Adjust lithium dose based on renal function;
consider temporarily holding lithium until acute
kidney injury resolves

Benzodiazepines All COVID-19 associated with delirium;
benzodiazepines can exacerbate delirium

COVID-19 associated with prominent respiratory
symptoms; benzodiazepines can suppress
respiratory drive

Lopinavir/Ritonavir contraindicated with
midazolam and triazolam (and can raise levels
of some other benzodiazepines) due to CYP450
inhibition

Avoid or taper existing benzodiazepines in
patients with delirium if possible

Weigh risks versus benefits in using
benzodiazepines in patients with prominent
respiratory symptoms; a low dose may be able
to be used safely in nondelirious patients

Avoid midazolam and triazolam and consider
using lorazepam, temazepam, or oxazepam in
patients taking lopinavir/ritonavir

ANC = absolute neutrophil count; aPTT = activated partial thromboplastin time; COVID-19 = coronavirus disease 2019; EKG =
electrocardiogram; PT = prothrombin time; TdP = torsades de pointes.

Psychopharmacology of COVID-19

laboratory testing for drugs subject to Risk Evaluation
and Mitigation Strategy.8 While there are no data yet
available on COVID-19 in patients on clozapine, it has
been suggested that clozapine is associated with a
higher risk of pneumonia and its complications. Ex-
planations include aspiration, sialorrhea, sedation, and
poorly understood effects on the immune system.7,9

Patients should be educated on symptoms of pneu-
monia and urgently evaluated by a clinician if symp-
toms of infection emerge. Complicating the picture
further, elevation of clozapine levels has been observed
with multiple acute viral and bacterial infections. This
may in part be related to effects of systemic infection
and inflammation on CYP450 enzymes.10 Clinicians
should closely monitor clozapine levels and consider
reducing the dose by up to a half in patients with fever
and other signs of infection.

Coagulation abnormalities such as prothrombin
time and activated partial thromboplastin time

414 www.psychosomaticsjournal.org

prolongation, thrombocytopenia, and disseminated
intravascular coagulation are also frequently observed
in patients with COVID-19. At the same time, many
patients with COVID-19 experience increased throm-
botic risk and may be prescribed prophylactic antico-
agulants.5 These factors may impact the decision to
prescribe psychotropics that have been associated with
platelet dysfunction and increased bleeding risk (e.g.,
selective serotonin reuptake inhibitors [SSRIs] and
valproic acid). Clinicians should be especially mindful
of using these medications in patients who have other
risk factors for bleeding, such as concomitant anti-
coagulation therapy and a history of significant
bleeding event.

Cardiac Effects

There is limited available information regarding car-
diovascular involvement in COVID-19 infection,

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

TABLE 2. Potential Psychotropic Safety Concerns in COVID-19 Organized by Organ System

Organ system
affected by
COVID-19

Systemic effects and symptoms Potential psychotropic safety concerns

Hematologic Lymphopenia
Coagulopathy (increased PT, aPTT; decreased
platelets)

Consider avoiding medications that can negatively impact white
blood cell production

Highest risk: carbamazepine, clozapine, olanzapine
Moderate risk: all first and second generation antipsychotics
(especially low-potency conventionals)

Rare reports: TCAs, benzodiazepines (chlordiazepoxide),
gabapentin, and valproate

Consider avoiding medications that can increase bleeding risk (via
thrombocytopenia or impaired platelet aggregation): valproic
acid, SSRIs, SNRIs

Cardiac Concern for tachyarrhythmias, heart failure,
myopericarditis, acute cardiac injury

Several medications being used for COVID-19
(azithromycin, hydroxychloroquine, chloroquine,
lopinavir/ritonavir) reported to prolong QT interval

Caution with psychotropics known to prolong QTc and in patients
with other underlying risk factors for QT prolongation

Highest risk: antipsychotics, citalopram, tricyclic antidepressants

Hepatic Risk of acute liver injury, especially in severe cases In patients with hepatic injury or failure:
Consider avoiding psychotropics that can also cause serious drug-
induced liver injury: chlorpromazine, carbamazepine, valproate,
duloxetine, and nefazodone.

Refer to prescribing information to determine if dose adjustments
are needed

Renal Acute kidney injury has been observed, particularly
in patients with COVID-19–associated acute
respiratory distress syndrome (ARDS) and
preexisting chronic kidney disease

Consider dose adjustment with some psychotropics (e.g., lithium,
gabapentin, topiramate, pregabalin, paliperidone, and duloxetine)

Consider avoiding potentially nephrotoxic drugs

Nervous system Central nervous system: headache, dizziness,
impaired consciousness, ataxia, stroke, delirium,
seizures

Peripheral nervous system: impaired taste/smell/
vision, neuropathic pain

In patients with delirium, caution with deliriogenic medications:
benzodiazepines, opioids, sedative-hypnotics, and those drugs
with strong anticholinergic effects (tertiary amine tricyclic
antidepressants, low-potency first-generation antipsychotics, some
second-generation antipsychotics, benztropine, and
diphenhydramine)

Caution with medications that can lower seizure threshold:
antipsychotics and certain antidepressants (bupropion, tricyclics)

Pulmonary Cough, shortness of breath, pneumonia and ARDS In COVID-19 patients with anxiety or panic symptoms, weigh risks
versus benefits in using benzodiazepines in patients with
prominent respiratory symptoms, given potential to suppress
respiratory drive

aPTT = activated partial thromboplastin time; COVID-19 = coronavirus disease 2019; PT = prothrombin time; QTc = corrected QT
interval; SNRI = serotonin norepinephrine reuptake inhibitors; SSRI = selective serotonin reuptake inhibitor; TCA = tricyclics antidepressant.

Bilbul et al.

although tachyarrhythmias and heart failure have been
described with other SARS beta-coronavirus in-
fections.11 A recent report described acute myoper-
icarditis in a patient with COVID-19,12 and a meta-
analysis found acute cardiac injury in at least 8% of
patients with COVID-19.13 It has been suggested that
COVID-19 most likely has an arrhythmogenic effect.14

Proposed mechanisms of myocardial injury include
derangement of angiotensin-converting enzyme 2 signal
pathways, cytokine storm, and myocarditis. In addi-
tion, several medications being used off-label in the

Psychosomatics 61:5, September/October 2020

management of COVID-19 (azithromycin, hydroxy-
chloroquine, chloroquine, and lopinavir/ritonavir) have
been reported to prolong the QT interval. QT prolon-
gation, particularly in those with underlying medical
risk factors, has been linked to lethal ventricular ar-
rhythmias, such as torsades de pointes.

A complete discussion of the cardiac side effects of
psychotropics is beyond the scope of this article, except
to note that it has been well described in the literature
that a number of psychotropic medications can prolong
the QT interval. Although the data are often difficult to

www.psychosomaticsjournal.org 415

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

interpret because of confounding factors, antipsy-
chotics, tricyclic antidepressants, and the SSRI cit-
alopram appear to be the agents of most concern. It is
difficult to stratify antipsychotic medications by QT
prolongation risk. Of the typical antipsychotics, thio-
ridazine causes the greatest QT prolongation, although
intravenous haloperidol has also been implicated. The
greatest risk among the atypicals appears to be related
to ziprasidone and possibly iloperidone. Aripiprazole
and possibly lurasidone have been associated with the
lowest risk based on available data.15

Health care providers should be aware of the
baseline corrected QT interval (QTc) and all concomi-
tant medications, laboratory test results, medical
comorbidities, and family history before prescribing
psychotropics in patients with COVID-19. Caution
should be used in patients with a baseline prolonged
QTc and/or other risk factors for drug-induced QT
prolongation and torsades de pointes: the use of QT-
prolonging medications, cardiac comorbidities, age
.65, female sex, family history of sudden cardiac
death, hypokalemia/hypomagnesemia, and illicit sub-
stance use. If QT-prolonging medications are used in a
patient with a QTc .500 ms or other significant risk
factors, electrocardiograms should be monitored
frequently (daily in high-risk cases), potassium and
magnesium should be repleted, cardiology involvement
should be considered, and every attempt made to
reduce risk factors.15 In patients who test positive for
COVID-19 but are already taking a psychotropic drug
that has inherent potential for QTc prolongation, risk-
benefit decisions must be made on a case-by-case basis
regarding continuation versus switching to an alterna-
tive medication.

Hepatic Effects

Several studies have reported acute liver injury,
particularly in severe COVID-19 cases.4,16,17 The
etiology of the liver injury is not known, and hy-
potheses include viral infection, drug-induced liver
injury, and systemic inflammation due to cytokine
storm or hypoxia.16 Laboratory abnormalities
observed include elevated aspartate aminotransferase,
alanine aminotransferase, and bilirubin.17 Liver
function tests should be monitored, and if abnormal,
consideration should be given to avoiding psycho-
tropics that can also cause hepatic injury or making
dose adjustments if heavily dependent on hepatic

416 www.psychosomaticsjournal.org

metabolism. As most psychotropics are lipid soluble
and require hepatic metabolism before clearance,
clinicians should review the package insert to deter-
mine if a dose adjustment is needed. In addition,
many psychotropics (valproate, carbamazepine, tri-
cyclic antidepressants, serotonin norepinephrine re-
uptake inhibitors, and second-generation
antipsychotics) have been associated with mild hep-
atoxicity that manifests with modest, transient in-
creases in liver enzymes. Only a few are thought to
have a high risk of causing serious drug-induced liver
injury, including chlorpromazine, carbamazepine,
valproate, duloxetine, and nefazodone.18,19 Such
high-risk psychotropics should be preferentially
avoided in patients with COVID-19–associated liver
disease.

Renal Effects

Acute kidney injury has been observed, particularly in
patients with COVID-19–associated acute respiratory
distress syndrome and preexisting chronic kidney dis-
ease. Several causes have been proposed, including
impaired gas exchange, hemodynamic alterations,
sepsis, and an inflammatory/immune reaction involving
release of circulating mediators that cause injury to
kidney cells.20 In such patients, avoiding potentially
nephrotoxic drugs, such as lithium, may be required. In
addition, psychiatrists should be aware of any renal
impairment and make necessary dose adjustments as
per the manufacturer’s prescribing information. Psy-
chotropics highly dependent on renal excretion include
lithium, gabapentin, topiramate, pregabalin, and pal-
iperidone. Many other psychotropics have caused renal
excretion of active metabolites. Levels of these medi-
cations or their metabolites can increase in the setting
of impaired renal clearance such that reduced dosing or
avoiding the medication may be required. For example,
administration of duloxetine is not recommended for
patients with severe renal impairment (CrCL of ,30
mL/min).18

Neurological Effects

Based on similarities between SARS-CoV2 and other
coronaviruses, it is thought likely that SARS-CoV2 has
a neuroinvasive potential,21 but there remain many
unanswered questions about neurological manifesta-
tions of COVID-19. Initial observations note a variety
of neurological syndromes in patients with COVID-19,

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

Bilbul et al.

particularly the more severely affected ones. These
include stroke, delirium, seizures, and an encephalitis-
type presentation. A recent article from Wuhan22 re-
ports neurologic symptoms in 36.4% of patients with
COVID-19, falling into 3 categories: (1) central nervous
system symptoms or diseases (headache, dizziness,
impaired consciousness, ataxia, acute cerebrovascular
disease, and seizure); (2) peripheral nervous system
symptoms (impairment in taste, vision, and smell,
neuropathic pain); and (3) skeletal muscular injury. It is
not known whether these neurologic syndromes are a
direct effect of the virus entering the central nervous
system or an indirect response to the cytokine storm
that patients are experiencing. A specific prevalence
rate of delirium was not reported but is presumed to be
very high and to contribute to poor adherence with care
and other safety concerns. Certainly, for patients with
severe COVID-19 infections, there are many other po-
tential etiologies of delirium, including organ failure,
hypoxia, sepsis, medication effects, and electrolyte/
metabolic abnormalities. Observational studies have in
fact reported high rates of benzodiazepine use for
sedation in ventilator-dependent patients with COVID-
19.23 Environmental factors such as isolation from
family members and difficulty mobilizing patients also
contribute.24

In patients with COVID-19 and delirium, clinicians
should be mindful about prescribing benzodiazepines,
opioids, and drugs with strong anticholinergic proper-
ties (tertiary amine tricyclic antidepressants, low-
potency antipsychotics, benztropine, and diphenhy-
dramine) as these medications have the potential to
cause or exacerbate confusion, sedation, and/or falls.
Clinicians should also be cautious about prescribing
psychotropics that can lower the seizure threshold in
patients with seizures or structural brain lesions. Such
medications include most antipsychotics (especially
clozapine, quetiapine, olanzapine, and first-generation
antipsychotics)25 and certain antidepressants (bupro-
pion, tricyclics).26

Pulmonary Effects

As the lung is considered the primary organ that is
affected by COVID-19, most patients present with
respiratory symptoms, such as cough and shortness of
breath. Affected individuals may develop pneumonia
and acute respiratory distress syndrome leading to high
supplemental oxygen requirements and, in the most

Psychosomatics 61:5, September/October 2020

severe cases, invasive ventilation.4 Psychiatric consul-
tants may be asked to evaluate and manage patients
with COVID-19 and anxiety or panic symptoms in
addition to respiratory distress. While there may be
circumstances in which the use of small doses of a
benzodiazepine is appropriate, it is important to be
aware of the potential of benzodiazepines to suppress
respiratory drive, particularly at higher doses. Clini-
cians therefore need to consider risks versus benefits in
using benzodiazepines in patients with prominent res-
piratory symptoms.

PSYCHIATRIC CONSIDERATIONS OF
PROPOSED COVID-19 TREATMENTS

Many of the proposed COVID-19 treatments have the
potential for neuropsychiatric side effects as well as
drug-drug interactions. These are reviewed in the
following section and summarized in Table 3.

Remdesivir

Remdesivir is an antiviral medication that interacts
with RNA polymerase and evades proofreading by
viral exonuclease leading to a decrease in viral RNA.27

On May 1, 2020, the US FDA issued an Emergency
Use Authorization to use remdesivir for treatment of
suspected or confirmed severe COVID-19 infection,1

with severe defined as “patients with an oxygen
saturation #94% on room air or requiring supple-
mental oxygen, mechanical ventilation, or extracorpo-
real membrane oxygenation.” The Emergency Use
Authorization was based on early promising data from
a randomized double-blinded, placebo-controlled28 and
an open-label trial.29 Remdesivir is administered by
infusion, with a treatment course of 5 or 10 days,
depending on severity of disease.

Neuropsychiatric Effects

No information is available regarding neuropsychiatric
side effects, but administration has been associated with
infusion-related reactions that can present with hypo-
tension, diaphoresis, and shivering.1 Such symptoms
might be misconstrued as a panic attack.

Psychotropic Considerations

Remdesivir carries a risk of transaminase eleva-
tions,30 specifically but not limited to alanine

www.psychosomaticsjournal.org 417

http://www.psychosomaticsjournal.org

TABLE 3. Psychiatric Side Effects and Drug Interactions with Proposed COVID-19 Treatments

Proposed COVID-19
treatment

Mechanism of action Psychiatric side effects Drug-drug interactions

Azithromycin Used with
hydroxychloroquine.

Antibacterial (primarily)
Antiviral and anti-
inflammatory (potential)

Psychotic depression, catatonia,
delirium, aggressive reaction,
anxiety, dizziness, headache,
vertigo, and somnolence

� Risk of QTc prolongation—caution with psy-
chotropics known to prolong QTc

� Risk of hepatotoxicity—caution with hepato-
toxic drugs

Chloroquine and
hydroxychloroquine

Anti-inflammatory
Antiviral: interference with
virus-receptor binding

Immune-modulating effects

Psychosis, delirium, suicidality,
personality changes,
depression, nervousness,
irritability, compulsive
impulses, preoccupations, and
aggressiveness

� Risk of QTc prolongation—caution with QT
prolonging drugs. Do not use outside of the
hospital setting or a clinical trial due to risk of
heart rhythm problems (FDA)

� Metabolized by CYP3A4—potential drug in-
teractions with CYP3A4 inhibitors (e.g., flu-
voxamine) and inducers (e.g., carbamazepine,
oxcarbazepine, modafinil)

� Risk of hepatotoxicity—caution with hepato-
toxic drugs

� Risk of seizures—caution with psychotropics
that can lower the seizure threshold

� Higher risk of neuropsychiatric side effects
when combined with CYP3A4 inhibitors, low-
dose glucocorticoids, alcohol intake, family
history of psychiatric disease, female gender,
low body weight, and supratherapeutic dosing

� Long half-life (40 h)—adverse effects and drug-
interactions may continue for days after the
drug has been discontinued

Colchicine Anti-inflammatory
Immune modulator: targets
IL-6 pathway, inhibition
of NLRP3 inflammasome.
May attenuate cytokine
storm.

At toxic doses: delirium,
seizures, muscle weakness,
depressed reflexes

� Narrow therapeutic index—potential for
toxicity

� Caution in renal and hepatic failure
� Caution with P-gp and CYP3A4 inhibitors
(e.g., fluvoxamine)

� CYP3A4 inducers may decrease levels
Convalescent plasma
therapy

Antibody containing
convalescent plasma from
patients who have
recovered from viral
infections

No specific psychiatric effects
(Note: allergic reactions can
produce shortness of breath
and palpitations that mimic
panic attacks)

Potential psychological effects
for donors

� There are no specific interactions (Note:
patients who develop transfusion reactions
might receive steroids or diphenhydramine
which can have negative synergistic effects with
existing psychotropics.)

Corticosteroids Immune modulators and
anti-inflammatory: may
lessen cytokine storm and
hyperinflammation
syndrome

Depression, mania, agitation,
mood lability, anxiety,
insomnia, catatonia,
depersonalization, delirium,
dementia, and psychosis

� Inconsistently reported to be weak CYP3A4
and CYP2C19 inducers

� Phenytoin—increases hepatic metabolism of
systemic corticosteroids

� Caution with bupropion—lowers seizure
threshold

� Majority of neuropsychiatric side effects occur
early in treatment course, usually within days,
and dosing is the most significant risk factor
(i.e., at prednisone equivalents of .40 mg/d)

Favipiravir Antiviral: RNA-dependent
RNA polymerase
inhibitor

No information � Possible QT prolongation

Psychopharmacology of COVID-19

aminotransferase elevations up to 20 times the
upper limit of normal.1 This may impact the decision
to use hepatically metabolized psychotropics, such
as valproic acid.

418 www.psychosomaticsjournal.org

Chloroquine and Hydroxychloroquine

Chloroquine, a synthetic form of quinine used for the
treatment and prophylaxis of malaria, and

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

TABLE 3. (Continued)

Proposed COVID-19
treatment
Mechanism of action Psychiatric side effects Drug-drug interactions

Interferon Immune modulator,
antiproliferative, and
hormone-like activities

Antiviral

IFN alpha: boxed warning for
“life-threatening or fatal
neuropsychiatric disorders.”
Specific effects include
fatigue, mood disorders,
suicidality, anxiety disorders,
irritability, lability, apathy,
sleep disturbance, and
cognitive deficits

IFN beta: fatigue, weight loss,
myalgia, arthralgia

� No known pharmacokinetic interactions with
psychotropics

� Potential for bone marrow suppression—safety
concerns with some psychotropics (e.g., carba-
mazepine, valproate, and clozapine)

� May lower seizure threshold: caution with
psychotropics that also lower seizure threshold

Lopinavir/Ritonavir Antiviral
Lopinavir: protease
inhibitor

Ritonavir: boosts plasma
levels of lopinavir

Possible abnormal dreams,
agitation, anxiety, confusion,
and emotional lability

All protease inhibitors
associated with paresthesias,
taste alterations, and
neurotoxicity

� Extensively metabolized by cytochrome P450–
risk of multiple possible interactions

� May get increased concentrations of coad-
ministered CYP3A4 or CYP2D6 substrates

� May get decreased concentrations of CYP1A2
or CYP2B6 substrates

� Contraindicated with pimozide, midazolam,
and triazolam due to increased drug levels and
potentiation of adverse effects

� Lowers concentrations of some psychotropics
(e.g., bupropion, methadone, lamotrigine, and
olanzapine)

� Other potential side effects that may impact
psychotropic use: Stevens Johnson syndrome,
diabetes mellitus, QTc prolongation, pancrea-
titis, neutropenia, hepatotoxicity, and chronic
kidney disease

Remdesivir Only FDA-approved
medication for severe
COVID-19

Interacts with RNA
polymerase, leads to
decrease in viral RNA

No information � No information is available about pharmaco-
kinetic drug-drug interactions

� Risk of elevated aminotransferase levels (e.g.
ALT up to 203 upper limit of normal)—
caution with potentially hepatotoxic
psychotropics

Tocilizumab Immune modulator:
recombinant humanized
monoclonal antibody that
acts as an IL-6 inhibitor;
may lessen cytokine storm

Possible positive effects on
depressive symptoms

� No major interactions reported

Vitamin C Enhances immune response,
antioxidant and reducing
agent

No evidence for
neuropsychiatric adverse
effects;

Of note, lower levels associated
with depression, confusion,
anger, delirium

� Coadministration with barbiturates may
decrease the effects of vitamin C

ALT = alanine aminotransferase; COVID-19 = coronavirus disease 2019; FDA = Food and Drug Administration; IFN = interferon; IL =
interleukin; P-gp = P-glycoprotein.

Bilbul et al.

hydroxychloroquine, a derivative compound used in
the treatment of inflammatory disorders such as
rheumatic arthritis and systemic lupus erythematosus,
are being considered as a possible treatment for
COVID-19 infection. Interest in these medications is
in part because of their potential for interference with

Psychosomatics 61:5, September/October 2020

virus-receptor binding and immune-modulating ef-
fects.31 The most promising study is a small open-
label trial from France,32 although a recent large
observational study showed that the risk of intuba-
tion or death was not significantly higher or lower
among patients who received the drug than among

www.psychosomaticsjournal.org 419

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

those who did not.33 The authors suggest that their
findings do not support continued use of the drug in
patients with COVID-19 outside of clinical trials.

Neuropsychiatric Effects

Neuropsychiatric side effects of chloroquine and
hydroxychloroquine include psychosis, delirium, agita-
tion, suicidality, personality changes, depression, and
sleep disturbances.34,35 Risk factors for
hydroxychloroquine-induced neuropsychiatric effects
may be concurrent use of CYP3A4 inhibitors or low-
dose glucocorticoids, alcohol intake, family history of
psychiatric disease, female gender, low body weight,
and supratherapeutic dosing.36

A number of mechanisms have been postulated
for the pathogenesis of hydroxychloroquine-induced
neuropsychiatric effects, such as cholinergic imbal-
ance due to acetylcholinesterase inhibition, inhibition
of the serotonin transporter protein, and N-methyl-D-
aspartate and gamma aminobutyric acid
antagonism.34

Psychotropic Considerations

Hydroxychloroquine and chloroquine can cause heart
conduction disorders, including QT interval prolon-
gation, bundle branch block, atrioventricular block,
and torsades de pointes.37 On April 24, 2020, the FDA
issued a safety announcement against the use of
hydroxychloroquine or chloroquine for COVID-19
outside of the hospital setting or a clinical trial
because of risk of heart rhythm problems.38 Caution
should be used when combining them with QT-
prolonging psychotropics. These agents can also be
hepatotoxic39 and epileptogenic,40 so caution should
be exercised in patients with hepatic disease, or in
conjunction with psychotropics that may be hepato-
toxic or may lower the seizure threshold.

Both chloroquine and hydroxychloroquine are
metabolized by CYP3A4,41 so CYP3A4 inhibitors (e.g.,
fluvoxamine) could raise plasma levels and increase the
potential for adverse effects. By contrast, CYP3A4 in-
ducers, such as carbamazepine, oxcarbazepine, and
modafinil, could decrease levels of chloroquine or
hydroxychloroquine, potentially rendering them less
effective. Given hydroxychloroquine’s long half-life (40
h), the potential for continued adverse effects and drug
interactions may continue for days after the drug has
been discontinued.35

420 www.psychosomaticsjournal.org

Tocilizumab

Tocilizumab is a recombinant humanized monoclonal
antibody that acts as an interleukin-6 receptor inhibi-
tor42 and is FDA approved to treat several types of
arthritis.43 Tocilizumab is being trialed in patients with
severe COVID-19 and elevated interleukin-6 because
interleukin-6 appears to be involved in cytokine storms
that have been observed in critically ill patients with
COVID-19.44

Neuropsychiatric Effects

Data from rheumatic arthritis patients suggest that
tocilizumab may have some positive effects on depres-
sive symptoms in rheumatoid arthritis45,46; however,
unpublished data from a small study surprisingly sug-
gest that patients who received tocilizumab after allo-
geneic hematopoietic cell transplantation experienced
worse symptoms of depression, anxiety, pain, and
sleep.47

Psychotropic Considerations

No major interactions have been reported.

Favipiravir

Favipiravir is an antiviral thought to act as an RNA-
dependent RNA polymerase inhibitor.48 It was
approved in China in February 2020 for treatment of
influenza,48 and there are current trials evaluating its
efficacy on SARS-Cov-2. It is not currently approved
for use in the United States.

Neuropsychiatric Effects

No published information is available.

Psychotropic Considerations

There is no published information available. One
published case report suggested a mild QT prolonga-
tion in a patient with Ebola virus who received
favipiravir.49

Lopinavir/Ritonavir (Kaletra)

Lopinavir/Ritonavir is an antiviral medication used to
treat HIV-1 infection.50 The 2 medications work syn-
ergistically: Lopinavir is a protease inhibitor, and ri-
tonavir helps to boost plasma levels of lopinavir by

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

Bilbul et al.

inhibiting its metabolism.50 Unfortunately, a recently
published randomized, controlled, open-label trial
found no additional benefit with lopinavir-ritonavir
treatment in hospitalized patients with SARS-CoV-2
as compared with standard care.51

Neuropsychiatric Effects

The manufacturer’s prescribing information lists
possible psychiatric side effects, including abnormal
dreams, agitation, anxiety, confusion, and emotional
lability although there is limited information in pub-
lished case reports or trials regarding the incidence of
such effects.50 Protease inhibitors as a class have been
associated with neurological adverse events, such as
paresthesias, taste alterations, and neurotoxicity.52

Psychotropic Considerations

Protease inhibitors are extensively metabolized by the
cytochrome P450 system and have been shown to
interact with many drugs, including psychotropics.53

The use of ritonavir may lead to increased concentra-
tions of coadministered drugs that are CYP3A4 or
CYP2D6 substrates or decreased concentrations of
CYP1A2 or CYP2B6 substrates, many of which are
psychotropics.

The use of lopinavir/ritonavir is contraindicated
with medications that include pimozide, midazolam,
and triazolam because of increased drug levels and
potentiation of adverse effects. The use of benzodiaze-
pines not dependent on CYP metabolism (lorazepam,
temazepam, or oxazepam) is recommended. Owing to
CYP450 enzyme or glucuronidation-inducing effects,
ritonavir-boosted protease inhibitors also have been
shown to lower concentrations of some psychotropics
(e.g., bupropion, methadone, lamotrigine, and olanza-
pine), thus leading to increased dose requirements for
these medications.53

As most psychotropics are substrates for CYP
isoenzymes, there are many additional theoretical in-
teractions, but the clinical significance varies by agent.
Clinicians should assess each potential interaction
individually by reviewing available literature and
manufacturer’s prescribing information.

Other potential nonpsychiatric side effects that
may have implications for psychiatrists include Stevens
Johnson syndrome, diabetes mellitus, QTc prolonga-
tion, pancreatitis, neutropenia, hepatotoxicity, and
chronic kidney disease.50

Psychosomatics 61:5, September/October 2020

Convalescent Plasma Therapy

Antibody containing convalescent plasma from recov-
ered patients has been used with some success as a last
resort to treat severe viral respiratory infections such as
SARS-CoV, Middle Eastern respiratory syndrome-
CoV, and Ebola, although large clinical trials are ab-
sent.54 Trials are currently underway to study the
effectiveness of convalescent plasma therapy in the
treatment of individuals with severe respiratory failure
associated with COVID-19.

Neuropsychiatric Effects

When used for the treatment of other severe acute
viral respiratory infections, convalescent plasma ther-
apy was not associated with serious adverse events,55

although in general, plasma transfusions can cause a
range of adverse events from mild fever and allergic
reactions to life-threatening bronchospasm/anaphylaxis,
transfusion-related acute lung injury, and transfusion-
associated circulatory overload.56

Specific neuropsychiatric effects have not been re-
ported, although allergic reactions, cardiovascular
complications, and bronchospasm can produce symp-
toms such as shortness of breath and palpitations that
mimic panic attacks.

A potential psychological adverse effect of conva-
lescent plasma therapy relates to ethical concerns about
coercion, confidentiality, and privacy of the prospective
donors that were initially raised during the Ebola
outbreak57 and led to a World Health Organization
document providing guidance on the ethical use of
convalescent plasma.58

Psychotropic Considerations

There are no specific interactions between psychotro-
pics and plasma transfusions, but patients who develop
transfusion reactions might receive steroids or diphen-
hydramine which can have negative synergistic effects
with existing psychotropics.

Azithromycin

Azithromycin is an antibacterial agent which may have
antiviral and anti-inflammatory activities.32 It is under
investigational use for treatment of COVID-19 when
given in conjunction with chloroquine or hydroxy-
chloroquine. In one small French study (n = 20), azi-
thromycin added to hydroxychloroquine was

www.psychosomaticsjournal.org 421

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

significantly more efficient for virus elimination than
hydroxychloroquine alone.32

Neuropsychiatric Effects

Side effects that have been reported include psychotic
depression, catatonia, delirium, aggressive reaction,
anxiety, dizziness, headache, vertigo, and
somnolence.59,60

Psychotropic Considerations

Azithromycin has not been implicated in pharmacoki-
netic interactions with psychotropics but has been
associated with QTc prolongation and life-threatening
torsades de pointes arrhythmias. It has also been
associated with hepatotoxicity.61

Vitamin C

High-dose intravenous vitamin C (ascorbic acid), an
antioxidant and reducing agent, has been investigated
in the treatment of sepsis because of its enhancement of
the immune response.62 In the intensive care setting,
vitamin C administration has been correlated with
preventing progressive organ dysfunction and reducing
mortality in sepsis and acute respiratory distress syn-
drome63 and is being investigated in critically ill pa-
tients with COVID-19.

Neuropsychiatric Effects

There are no known adverse neuropsychiatric conse-
quences of high-dose intravenous vitamin C adminis-
tration, but some studies have associated lower levels of
vitamin C with depression, confusion, and anger.64

Vitamin C deficiency has also been identified as a
possible risk factor for delirium.65

Psychotropic Considerations

Coadministration with barbiturates may decrease the
effects of vitamin C.62

Corticosteroids

Corticosteroids are involved in immune function,
inflammation, and carbohydrate metabolism and are
used in the treatment of endocrinopathies, autoimmune
disorders, and asthma/allergies.66 In previous pan-
demics, such as SARS and Middle Eastern respiratory
syndrome, corticosteroids were not recommended

422 www.psychosomaticsjournal.org

because of the concern that they may exacerbate lung
injury.67 Given evidence suggesting that severe
COVID-19 may be associated with a cytokine storm
and hyperinflammation syndrome,67 corticosteroids
may have a role in treatment.

Neuropsychiatric Effects

The neuropsychiatric side effects of corticosteroids have
been well described in the literature and include
depression, mania, agitation, mood lability, anxiety,
insomnia, catatonia, depersonalization, delirium, and
psychosis.66 Most neuropsychiatric side effects occur
early in the treatment course, usually within days, and
dosing is the most significant risk factor (i.e., at pred-
nisone equivalents of .40 mg/d).66

Psychotropic Considerations

Corticosteroids have been inconsistently reported to be
weak CYP3A4 and CYP2C19 inducers,68 which could
lead to decreased effects of CYP3A4 or CYP2C19
substrate psychotropics.69 In addition, phenytoin has
been shown to increase hepatic metabolism of systemic
corticosteroids.70

Interferon

IFNs are glycoproteins that have immunomodulatory,
antiproliferative, and hormone-like activities.71 IFN
alpha and beta have anti-SARS-CoV-1 activity in vitro,
and IFN beta reduces the replication of Middle Eastern
respiratory syndrome-coronavirus in vitro.72,73 Based
on this information, IFN has been considered as a
potential treatment for COVID-19, including in com-
bination with ribavirin, a guanosine analogue with a
broad-spectrum antiviral potency.74

Neuropsychiatric Effects

IFN alpha has a boxed warning for “life-threatening or
fatal neuropsychiatric disorders.”75 Specific effects
include fatigue, mood disorders, suicidality, anxiety
disorders, irritability, lability, apathy, sleep distur-
bance, and cognitive deficits.76 Side effects of IFN beta
can include fatigue, weight loss, myalgia, and
arthralgia,77 but not generally depression. Given the
potential for significant psychiatric side effects of IFN
alpha, it is important for clinicians to screen for base-
line psychiatric history and monitor closely for emer-
gence of any symptoms.

Psychosomatics 61:5, September/October 2020

http://www.psychosomaticsjournal.org

Bilbul et al.
Psychotropic Considerations

There are no known pharmacokinetic interactions with
psychotropics, but clinicians should be mindful of the
potential for bone marrow suppression which may raise
safety concerns with concurrent use of psychotropics,
such as carbamazepine, valproate, and clozapine. In
addition, seizures in conjunction with bupropion use
have been reported.78

Colchicine

Colchicine is a plant-derived alkaloid with anti-
inflammatory properties that is used for a variety of
rheumatological and cardiac conditions.79 It is hy-
pothesized that colchicine could treat COVID-19
through targeting the overactive interleukin-6
pathway.80

Neuropsychiatric Effects

Colchicine does not typically produce any neuropsy-
chiatric effects, but at toxic doses, it can cause delirium,
seizures, and muscle weakness.81

Psychotropic Considerations

Colchicine has a narrow therapeutic index, and atten-
tion must be paid to potential drug interactions that
might increase toxicity. Colchicine is metabolized by
CYP3A4 and excreted via the P-glycoprotein transport
system as well as cleared by the kidneys through
glomerular filtration. Dose adjustment is recommended
with concurrent use of CYP3A4 or P-glycoprotein in-
hibitors as well as in patients with hepatic or renal
impairment.82 CYP3A4 inducers can lead to increased
metabolism and theoretically decreased effectiveness of
colchicine.

DISCUSSION

COVID-19 and its treatments can impact many organ
systems and contribute to a host of drug interactions
and neuropsychiatric effects. This can have safety im-
plications for use of psychotropics, which are highly
metabolized by the hepatic cytochrome p450 system
and carry their own potential for drug-interactions and
end-organ adverse effects.

While there are no absolute contraindications to
the use of psychotropics in patients with COVID-19,
psychiatrists must be mindful of potential adverse

Psychosomatics 61:5, September/October 2020

effects and conduct a thoughtful risk-benefit analysis as
part of their clinical decision-making process. For
example, chloroquine, hydroxychloroquine, and azi-
thromycin have the potential for QT prolongation,
which can be problematic in patients with tenuous
cardiac status. Generally, psychiatrists might avoid
antipsychotic medications in the setting of a prolonged
QT interval. However, in our experience, hyperactive
delirium in patients with COVID-19 is highly prevalent,
manifests with severe agitation that can be difficult to
treat, and leads to dangerous behaviors such as
removing oxygen or assaulting staff. While there is
limited evidence to support the use of any interventions
in the management of agitation in COVID-19–
associated delirium, most consultation-liaison psychia-
trists consider antipsychotics such as haloperidol the
gold standard for managing agitation in delirious pa-
tients. In these situations, the consultation-liaison psy-
chiatrist should assist the medical team in reasoning
through the cardiac risks of using an antipsychotic
balanced against effective management of the agitation.
The use of an antipsychotic with cardiology involve-
ment and frequent electrocardiogram monitoring or
telemetry may be deemed acceptable. Alternatives such
as alpha-2 agonists (dexmedetomidine and clonidine) or
antiepileptics (valproic acid) should be considered if the
individual patient’s cardiac risk is determined to be
high and/or if the antipsychotic is clinically ineffective.
Melatonin has been proposed for addressing con-
sciousness and sleep-wake cycle disturbances in delir-
ious patients with COVID-19, especially given its
potential for antioxidative, anti-inflammatory, and
immune-enhancing effects.83 With the exception of
patients who chronically use alcohol or benzodiaze-
pines and may be at risk for withdrawal, benzodiaze-
pines should be avoided if possible and considered only
as a last resort for highly agitated delirious patients for
whom other treatments are unavailable or ineffective.
Early delirium screening and nonpharmacological
strategies to prevent or treat delirium such as frequent
orientation and early mobilization should be used if
practically feasible.24

As another example, we have observed many
nondelirious patients with COVID-19 and significant
anxiety in the setting of respiratory distress. In some
cases, the anxiety leads to requests to leave against
medical advice or refusal to remain isolated. For these
patients, psychiatrists should consider whether the
benefit of a low-dose benzodiazepine outweighs the

www.psychosomaticsjournal.org 423

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

potential risk of respiratory depression. The use of
benzodiazepines may be reasonable in patients with
adequate oxygen saturation and in the absence of
confusion or a depressed sensorium. Depending on the
individual patient’s circumstances and symptoms,
alternative medications such as gabapentin, buspirone,
hydroxyzine, a low-dose atypical antipsychotic, or a
SSRI may be appropriate. Nonpharmacological/psy-
chosocial interventions (e.g., behaviorally oriented
therapies) should also be used.

Other important tasks for the psychiatrist treating a
patient with COVID-19 include review of all medica-
tions, monitoring for neuropsychiatric side effects of
medications such as hydroxychloroquine or corticoste-
roids, and differentiating between primary psychiatric
symptoms versus those that are secondary to COVID-
19 or other medications.

Interestingly, several psychotropics, including
haloperidol and valproic acid, were recently named on
a list of FDA-approved medications with potential for
in vitro action against SARS-CoV-2.84 Fluvoxamine is
also under investigation for its potential to reduce the

424 www.psychosomaticsjournal.org

inflammatory response during sepsis by inhibiting
cytokine production,85 and melatonin for its anti-
oxidative and anti-inflammatory properties.86 If more
data become available, psychiatrists might consider
preferentially using these agents if clinically
appropriate.

In summary, psychiatrists must be aware of the
likelihood of encountering patients with COVID-19
infection and must remain cognizant of the neuropsy-
chiatric effects and drug-drug interactions of COVID-
19 treatments as well as the end-organ effects of
COVID-19.

Funding: This research did not receive any specific
grant from funding agencies in the public, commercial, or
not-for-profit sectors.

Disclosure: C.L.E. receives royalty payments from
American Psychiatric Publishing, Inc. The other au-
thors report no proprietary or commercial interest in
any product mentioned or concept discussed in this
article.

References

1. Fact Sheet for Healthcare Providers – Emergency Use
Authorization (EUA) of Remdesivir (GS-5734TM) [Internet].
U.S. Food and Drug Administration (FDA). 2020.
Available from: https://www.fda.gov/media/137566/download.
[Accessed 24 May 2020]

2. Kalil AC: Treating COVID-19—off-label drug use,
compassionate use, and randomized clinical trials during
pandemics. JAMA 2020. https://doi.org/10.1001/jama.2020.
4742

3. Wang T, Du Z, Zhu F, et al: Comorbidities and multi-
organ injuries in the treatment of COVID-19. Lancet 2020;
395:e52

4. Huang C, Wang Y, Li X, et al: Clinical features of patients
infected with 2019 novel coronavirus in Wuhan, China.
Lancet 2020; 395:497–506

5. Terpos E, Ntanasis-Stathopoulos I, Elalamy I, et al: Hema-
tological findings and complications of COVID-19. Am J
Hematol 2020; 95:834–847

6. Oyesanmi O, Kunkel EJS, Monti DA, Field HL: Hemato-
logic side effects of psychotropics. Psychosomatics 1999;
40:414–421

7. Siskind D, Honer WG, Clark S, et al: Consensus statement
on the use of clozapine during the COVID-19 pandemic.
J Psychiatry Neurosci 2020; 45:200061

8. Policy for Certain REMS Requirements During the COVID19
Public Health Emergency [Internet]. FDA. Available from:
https://www.fda.gov/media/136317/download. [Accessed 24
May 2020]

9. de Leon J, Sanz EJ, Norén GN, De las Cuevas C: Pneumonia
may be more frequent and have more fatal outcomes with
clozapine than with other second-generation antipsychotics.
World Psychiatry 2020; 19:120

10. Clark SR, Warren NS, Kim G, et al: Elevated clozapine
levels associated with infection: a systematic review. Schiz-
ophr Res 2018; 192:50–56

11. Yu C-M, Wong RS-M, Wu EB, et al: Cardiovascular com-
plications of severe acute respiratory syndrome. Postgrad
Med J 2006; 82:140–144

12. Inciardi RM, Lupi L, Zaccone G, et al: Cardiac involvement
in a patient with coronavirus disease 2019 (COVID-19).
JAMA Cardiol 2020. https://doi.org/10.1001/jamacardio.
2020.1096

13. Li B, Yang J, Zhao F, et al: Prevalence and impact of car-
diovascular metabolic diseases on COVID-19 in China. Clin
Res Cardiol 2020; 109:531–538

14. Wu C-I, Postema PG, Arbelo E, et al: SARS-CoV-2,
COVID-19 and inherited arrhythmia syndromes.
Hear Rhythm 2020. https://doi.org/10.1016/j.hrthm.2020.03.
024

15. Beach SR, Celano CM, Sugrue AM, et al: QT prolongation,
torsades de Pointes, and psychotropic medications: a 5-year
update. Psychosomatics 2018; 59:105–122

16. Lee I-C, Huo T-I, Huang Y-H: Gastrointestinal and
liver manifestations in patients with COVID-19.
J Chin Med Assoc 2020; 83:521–523

17. Xie H, Zhao J, Lian N, Lin S, Xie Q, Zhuo H: Clinical
characteristics of non-ICU hospitalized patients with

Psychosomatics 61:5, September/October 2020

https://www.fda.gov/media/137566/download

https://doi.org/10.1001/jama.2020.4742

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref3

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref4

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref5

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref6

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref7

https://www.fda.gov/media/136317/download

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref9

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref10

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref11

https://doi.org/10.1001/jamacardio.2020.1096

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref13

https://doi.org/10.1016/j.hrthm.2020.03.024

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref15

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref16

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref17

http://www.psychosomaticsjournal.org

Bilbul et al.

coronavirus disease 2019 and liver injury: a retrospective
study. Liver Int 2020; 40:1321–1326

18. Goldberg J, Ernst C: Managing the side effects of psycho-
tropic medications, 2nd ed. Washington DC: American
Psychiatric Association Publishing; 2019

19. Telles-Correia D, Barbosa A, Cortez-Pinto H, Campos C,
Rocha NBF, Machado S: Psychotropic drugs and liver dis-
ease: a critical review of pharmacokinetics and liver toxicity.
World J Gastrointest Pharmacol Ther 2017; 8:26–38

20. Fanelli V, Fiorentino M, Cantaluppi V, et al: Acute kidney
injury in SARS-CoV-2 infected patients. Crit Care 2020;
24:155

21. Li Y-C, Bai W-Z, Hashikawa T: The neuroinvasive potential
of SARS-CoV2 may play a role in the respiratory failure of
COVID-19 patients. J Med Virol 2020; 92:552–555

22. Mao L, Jin H, Wang M, et al: Neurologic manifestations of
hospitalized patients with coronavirus disease 2019 in
Wuhan, China. JAMA Neurol 2020:e201127

23. Helms J, Kremer S, Merdji H, et al: Neurologic features in
severe SARS-CoV-2 infection. N Engl J Med 2020;
382:2268–2270

24. Kotfis K, Roberson SW, Wilson JE, Dabrowski W, Pun BT,
Ely EW: COVID-19: ICU delirium management during
SARS-CoV-2 pandemic. Crit Care 2020; 24:1–9

25. Wu C-S, Wang S-C, Yeh I-J, Liu S-K: Comparative risk of
seizure with use of first- and second-generation antipsychotics
in patients with schizophrenia and mood disorders. J Clin
Psychiatry 2016; 77:e573–e579

26. Johannessen Landmark C, Henning O, Johannessen SI:
Proconvulsant effects of antidepressants — what is the cur-
rent evidence? Epilepsy Behav 2016; 61:287–291

27. Al-Tawfiq JA, Al-Homoud AH, Memish ZA: Remdesivir as
a possible therapeutic option for the COVID-19. Trav Med
Infect Dis 2020; 34:101615

28. A Multicenter, Adaptive, Randomized Blinded Controlled Trial
of the Safety and Efficacy of Investigational Therapeutics for the
Treatment of COVID-19 in Hospitalized Adults Adaptive
COVID-19 Treatment Trial (ACTT) [Internet]. 2020. Available
from: https://ichgcp.net/de/clinical-trials-registry/NCT04280705.
[Accessed 20 May 2020]

29. Study to Evaluate the Safety and Antiviral Activity of
Remdesivir (GS-5734TM) in Participants With Severe
Coronavirus Disease (COVID-19) [Internet]. 2020.
Available from: https://clinicaltrials.gov/ct2/show/NCT042
92899. [Accessed 24 May 2020].

30. COVID-19 Investigation Team. Clinical and virologic charac-
teristics of the first 12 patients with coronavirus disease 2019
(COVID-19) in the United States. Nat Med 2020. https://doi.
org/10.1038/s41591-020-0877-5

31. Sahraei Z, Shabani M, Shokouhi S, Saffaei A: Aminoquinolines
against coronavirus disease 2019 (COVID-19): chloroquine or
hydroxychloroquine. Int J Antimicrob Agents 2020; 55

32. Gautret P, Lagier J-C, Parola P, et al: Hydroxychloroquine
and azithromycin as a treatment of COVID-19: results of an
open-label non-randomized clinical trial. Int J Antimicrob
Agents 2020:105949

Psychosomatics 61:5, September/October 2020

33. Geleris J, Sun Y, Platt J, et al: Observational study of hydroxy-
chloroquine in hospitalized patients with Covid-19. N Engl J
Med 2020. https://doi.org/10.1056/NEJMoa2012410

34. Good MI, Shader RI: Behavioral toxicity and equivocal
suicide associated with chloroquine and its derivatives. Am J
Psychiatry 1977; 134:798–801

35. Manufacter’s Package Insert, Hydroxychloroquine [Internet].
Available from: https://www.accessdata.fda.gov/drugsatfda_
docs/label/2017/009768s037s045s047lbl . [Accessed 20
May 2020]

36. Mascolo A, Berrino PM, Gareri P, et al: Neuropsychiatric
clinical manifestations in elderly patients treated with
hydroxychloroquine: a review article. Inflammopharmacol-
ogy 2018; 26:1141–1149

37. McGhie TK, Harvey P, Su J, Anderson N, Tomlinson G,
Touma Z: Electrocardiogram abnormalities related to anti-
malarials in systemic lupus erythematosus. Clin Exp Rheu-
matol 2018; 36:545–551

38. FDA cautions against use of hydroxychloroquine or chlo-
roquine for COVID-19 outside of the hospital setting or a
clinical trial due to risk of heart rhythm problems [Internet].
FDA. 2020. Available from: https://www.fda.gov/media/
137250/download. [Accessed 24 May 2020]

39. Makin AJ, Wendon J, Fitt S, Portmann BC, Williams R:
Fulminant hepatic failure secondary to hydroxychloroquine.
Gut 1994; 35:569–570

40. Krzeminski P, Lesiak A, Narbutt J: Seizures as a rare adverse
effect of chloroquine therapy in systemic lupus erythematosus
patients: a case report and literature survey. Postepy Der-
matol Alergol 2018; 35:429–430

41. Browning DJ. In: Browning DJ, (ed) Pharmacology of
chloroquine and hydroxychloroquine BT – hydroxy-
chloroquine and chloroquine retinopathy. New York,
NY: Springer New York; 2014. p. 35–63. Available from:
https://doi.org/10.1007/978-1-4939-0597-3_2. [Accessed 24
May 2020]

42. Sheppard M, Laskou F, Stapleton PP, Hadavi S,
Dasgupta B: Tocilizumab (actemra). Hum Vaccin Immun-
other 2017; 13:1972–1988

43. Manufacterer’s Packge Insert, Actemra (tocilizumab)
[Internet]. Available from: https://www.actemrahcp.com/.
[Accessed 20 May 2020]

44. Luo P, Liu Y, Qiu L, Liu X, Liu D, Li J: Tocilizumab
treatment in COVID-19: a single center experience. J Med
Virol 2020; 92:814–818

45. Singh JA, Beg S, Lopez-Olivo MA: Tocilizumab for rheu-
matoid arthritis: a Cochrane systematic review. J Rheumatol
2011; 38:10–20

46. Harrold LR, John A, Reed GW, et al: Impact of tocilizumab
monotherapy on clinical and patient-reported quality-of-life
outcomes in patients with rheumatoid arthritis. Rheumatol
Ther 2017; 4:405–417

47. Knight JM, Costanzo ES, Singh S, et al: Inflammation and
the brain: tocilizumab may redefine our understanding of
depression and related symptoms in the medically ill.
Orlando, FL: ACLP; 2018

www.psychosomaticsjournal.org 425

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref17

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref18

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref19

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref20

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref21

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref22

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref23

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref24

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref25

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref26

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref27

https://ichgcp.net/de/clinical-trials-registry/NCT04280705

https://clinicaltrials.gov/ct2/show/NCT04292899

https://doi.org/10.1038/s41591-020-0877-5

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref31

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref32

https://doi.org/10.1056/NEJMoa2012410

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref34

https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/009768s037s045s047lbl

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref36

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref37

https://www.fda.gov/media/137250/download

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref39

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref40

https://doi.org/10.1007/978-1-4939-0597-3_2

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref42

https://www.actemrahcp.com/

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref44

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref45

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref46

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref47

http://www.psychosomaticsjournal.org

Psychopharmacology of COVID-19

48. Dong L, Hu S, Gao J: Discovering drugs to treat
coronavirus disease 2019 (COVID-19). Drug Discov Ther
2020; 14:58–60

49. Chinello P, Petrosillo N, Pittalis S, et al: QTc interval
prolongation during favipiravir therapy in an
Ebolavirus-infected patient. PLoS Negl Trop Dis 2017;
11:e0006034

50. Kaletra (lopinavir and ritonavir): Manufacturer’s Prescribing
Information [Internet]. Available from: https://www.
accessdata.fda.gov/drugsatfda_docs/label/2016/021251s052_
021906s046lbl . [Accessed 23 April 2020]

51. Cao B, Wang Y, Wen D, et al: A trial of lopinavir–ritonavir
in adults hospitalized with severe Covid-19. N Engl J Med
2020; 382:1787–1799

52. Abers MS, Shandera WX, Kass JS: Neurological and psy-
chiatric adverse effects of antiretroviral drugs. CNS Drugs
2014; 28:131–145

53. Goodlet KJ, Zmarlicka MT, Peckham AM: Drug–drug in-
teractions and clinical considerations with co-administration
of antiretrovirals and psychotropic drugs. CNS Spectr 2019;
24:287–312

54. Chen L, Xiong J, Bao L, Shi Y: Convalescent plasma as a
potential therapy for COVID-19. Lancet Infect Dis 2020;
20:398–400

55. Cunningham AC, Goh HP, Koh D: Treatment of
COVID-19: old tricks for new challenges. Crit Care 2020;
24:91

56. Pandey S, Vyas GN: Adverse effects of plasma transfusion.
Transfus 2012; 52:65S–79S

57. Van Griensven J, De Weiggheleire A, Delamou A, et al: The
use of Ebola convalescent plasma to treat Ebola virus disease
in resource-constrained settings: a perspective from the field.
Clin Infect Dis 2016; 62:69–74

58. World Health Organization (2015). Ethics of using convalescent
wholebloodandconvalescentplasmaduringthe Ebolaepidemic:
interim guidance for ethics review committees, researchers, na-
tional health authorities and blood transfusion services. World
Health Organization 2015. Available from: https://apps.who.int/
iris/handle/10665/161912

59. ZITHROMAX® (azithromycin tablets) and (azithromycin
for oral suspension) [Internet]. FDA. Available from: https://
www.accessdata.fda.gov/drugsatfda_docs/label/2013/05071
0s039,050711s036,050784s023lbl . [Accessed 22 April
2020]

60. Ginsberg DL: Azithromycin-induced psychotic depression
and catatonia. Prim Psychiatry 2006; 13:22–26

61. Leitner JM, Graninger W, Thalhammer F: Hepatotoxicity of
antibacterials: pathomechanisms and clinical data. Infection
2010; 38:3–11

62. Linster CL, Van Schaftingen E: Vitamin C. FEBS J 2007;
274:1–22

63. Truwit JD, Hite RD, Morris PE, et al: Effect of vitamin C
infusion on organ failure and biomarkers of inflammation
and vascular injury in patients with sepsis and severe acute
respiratory failure: the CITRIS-ALI randomized clinical
trial. JAMA 2019; 322:1261–1270

426 www.psychosomaticsjournal.org

64. Pullar JM, Carr AC, Bozonet SM, Vissers M: High vitamin
C status is associated with elevated mood in male tertiary
students. Antioxidants 2018; 7:91

65. Torbergsen AC, Watne LO, Frihagen F, Wyller TB,
Brugaard A, Mowe M: Vitamin deficiency as a risk factor for
delirium. Eur Geriatr Med 2015; 6:314–318

66. Dubovsky AN, Arvikar S, Stern TA, Axelrod L: The
neuropsychiatric complications of glucocorticoid use: steroid
psychosis revisited. Psychosomatics 2012; 53:103–115

67. Mehta P, McAuley DF, Brown M, Sanchez E, Tattersall RS,
Manson JJ: COVID-19: consider cytokine storm syndromes
and immunosuppression. Lancet 2020; 395:1033–1034

68. Flockhart DA, Oesterheld JR: Cytochrome P450-mediated
drug interactions. Child Adolesc Psychiatr Clin N Am
2000; 9:43–76

69. Villikka K, Varis T, Backman J, Neuvonen P, Kivistö K:
Effect of methylprednisolone on CYP3A4-mediated drug
metabolism in vivo. Eur J Clin Pharmacol 2001; 57:457–460

70. McLelland J, Jack W: Phenytoin/dexamethasone interaction:
a clinical problem. Lancet 1978; 311:1096–1097

71. Jacobs L, Johnson KP: A brief history of the use of in-
terferons as treatment of multiple sclerosis. Arch Neurol
1994; 51:1245–1252

72. Hensley LE, Fritz EA, Jahrling PB, Karp C, Huggins JW,
Geisbert TW: Interferon-b 1a and SARS coronavirus repli-
cation. Emerg Infect Dis 2004; 10:317

73. Ströher U, DiCaro A, Li Y, et al: Severe acute respiratory
syndrome-related coronavirus is inhibited by interferon-a.
J Infect Dis 2004; 189:1164–1167

74. Lu C-C, Chen M-Y, Chang Y-L: Potential therapeutic agents
against COVID-19: what we know so far. J Chin Med Assoc
2020; 83:534–536

75. PEGASYS (peginterferon alfa-2a) injection – Highlights of
Prescribing Information [Internet]. FDA. 2020. Available
from: https://www.accessdata.fda.gov/drugsatfda_docs/
label/2017/103964s5270lbl . [Accessed 20 April 2020].

76. Davoodi L, Masoum B, Moosazadeh M, Jafarpour H,
HaghshenasMR,MousaviT:Psychiatricsideeffectsofpegylated
interferon-aandribavirintherapyinIranianpatientswithchronic
hepatitis C: a meta-analysis. Exp Ther Med 2018; 16:971–978

77. Reder AT, Feng X: How type I interferons work in multiple
sclerosis and other diseases: some unexpected mechanisms.
J Interferon Cytokine Res 2014; 34:589–599

78. Ahmed F, Jacobson IM, Herrera JL, et al: Seizures during
pegylated interferon and ribavirin therapy for chronic Hep-
atitis C: observations from the WIN-R trial. J Clin Gastro-
enterol 2011; 45:286–292

79. Slobodnick A, Shah B, Krasnokutsky S, Pillinger MH: Up-
date on colchicine, 2017. Rheumatology (Oxford) 2017;
57(suppl_l):i4–i11

80. Tardif J-C, Bassevitch Z: Colchicine Coronavirus SARS-
CoV2 Trial (COLCORONA) (COVID-19) [Internet]. Avail-
able from: https://clinicaltrials.gov/ct2/show/NCT04322682.
[Accessed 24 May 2020]

81. Finkelstein Y, Aks SE, Hutson JR, et al: Colchicine poisoning:
the dark side of an ancient drug. Clin Toxicol 2010; 48:407–414

Psychosomatics 61:5, September/October 2020

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref48

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref49

https://www.accessdata.fda.gov/drugsatfda_docs/label/2016/021251s052_021906s046lbl

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref51

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref52

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref53

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref54

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref55

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref56

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref57

https://apps.who.int/iris/handle/10665/161912

https://www.accessdata.fda.gov/drugsatfda_docs/label/2013/050710s039,050711s036,050784s023lbl

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref60

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref61

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref62

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref63

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref64

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref65

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref66

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref67

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref68

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref69

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref70

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref71

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref72

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref73

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref74

https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/103964s5270lbl

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref76

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref77

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref78

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref79

https://clinicaltrials.gov/ct2/show/NCT04322682

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref81

http://www.psychosomaticsjournal.org

Bilbul et al.

82. Nuki G: Colchicine: its mechanism of action and efficacy in
crystal-induced inflammation. Curr Rheumatol Rep 2008;
10:218

83. Zambrelli E, Canevini M, Gambini O, D’Agostino A:
Delirium and sleep disturbances in COVID–19: a possible
role for melatonin in hospitalized patients? Sleep Med 2020;
70:111

84. Gordon DE, Jang GM, Bouhaddou M, et al: A SARS-
CoV-2-Human Protein-Protein Interaction Map Reveals
Drug Targets and Potential Drug-Repurposing. bioRxiv

Psychosomatics 61:5, September/October 2020

[Internet]. 2020. Available from: http://biorxiv.org/content/
early/2020/03/27/2020.03.22.002386.abstract. [Accessed 24
May 2020].

85. Rosen DA, Seki SM, Fernández-Castañeda A, et al: Modu-
lation of the sigma-1 receptor–IRE1 pathway is beneficial in
preclinical models of inflammation and sepsis. Sci Transl
Med 2019; 11:eaau5266

86. Zhang R, Wang X, Ni L, et al: COVID-19: melatonin
as a potential adjuvant treatment. Life Sci 2020; 250:
117583

www.psychosomaticsjournal.org 427

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref82

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref83

http://biorxiv.org/content/early/2020/03/27/2020.03.22.002386.abstract

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref85

http://refhub.elsevier.com/S0033-3182(20)30144-4/sref86

http://www.psychosomaticsjournal.org

Discussion
References

The Journal of Psychohistory 48 (2) Fall 2020

Psychopharmacology for
the Psycho-Historian:
The Evils of “Big Pharma,”
Lobbying, Corruption
and Serious Side Effects
of Medications

Doctors pour drugs of which they know little, to cure diseases of which they know
less, into patients of which they know nothing.

—Moliere (1622-1673, French dramatist)

ABStrACt: Psychohistorians have an interest in understanding terrorism, gun vio-
lence, suicide, homicide, war crimes and other forms of criminal activities. In some
of these situations side effects of medications, often psychotropics were considered
a contributing factor to the violence and aggression involved in these acts. there
are cases of war criminals who claimed their medications caused them to murder
innocent civilians. In several mass gun shooting, the gun industry claimed that
the offender was on psychiatric medication and claimed that mental sickness and
the side effect of medication caused their homicidality. In several civil cases, drug
manufacturers have been sued and a number of them settled their case without a
court finding and without claiming responsibility for any wrong doing. However,
in other cases they were held liable resulting in million-dollar fines. We review
scientific articles on the side effects of psychiatric medications and the presence
of any connection to violence, suicide and criminal activities. We also explore cor-
ruption in a number of pharmaceutical companies and their financial relationship
with government officials and prescribers.

A PSyCHO-HISTORICAL PERSPECTIvE: TALK vS DRUGS

Telling stories and tales to share learning and provide healing has ex-isted since ancient times. The collection of Middle Eastern folk tales
commonly known as the “1001 Nights” illustrated the healing effect these

JamsHid a. marvasti & claire c. olivier

101Evils of Psychopharmacology; Implications for Psycho-Historians

stories could have.1 They provided “sessions” for the listeners as they took
in the lessons and integrated them as a talking remedy. The difference
from the Freudian school is that through folk tale type of storytelling the
doctor does the talking, sharing tales that meet the patients’ concerns. In
the 1970’s and 1980’s metaphors, and fairytales were used as a therapeutic
intervention by Milton Erikson, Peseschkian and Bettelheim. While hair-
dressers and bartenders filled the role of amateur modern therapists listen-
ing to their clients and offering life stories.2

Clinical literature indicates a replacement of talk therapy by pharmaco-
therapy during the last 30-40 years. In the 1970’s, the treatments of choice
were psychotherapy, behavioral therapy, and/or environmental therapy.
However, it gradually changed to the point that treatment now relies heav-
ily on medication. There are multiple reasons for this change. One is the
pressure from the insurance companies and the managed care industry to
use drugs, which are cheaper and may possibly work faster (on the sur-
face).3 Because certain psychiatric symptoms are connected to neurotrans-
mitters, insurance companies may encourage doctors to use medications
to modify these neuromessengers to minimize symptoms.4 Yet inhibiting
these neurotransmitters may not cure the patient. Researchers are trying to
connect mental disorders to biochemical causes. If such connections are
found, it is believed that the most effective approach would be a biochem-
ical treatment. However, in our opinion, various types of therapy (psycho-
therapy, somatic (body based) therapy, etc.), may also positively impact a
person’s brain chemistry, without negative side effects.

PREvALENCE OF UTILIzING PSyCHIATRIC MEDICATIONS
Many Americans have come to expect substances to provide instant relief.
Legal substances such as coffee, tea, alcohol and marijuana (in states where
it is legal) are commonplace and can help provide energy, comfort, cre-
ativity, relaxation and distraction. When these methods are not enough to
combat one’s mood the media is more than willing to step in and provide
other suggestions. Marketing in the media and news provides a way to de-
crease the unknown about medications, regardless of their veracity. Adults
then become more comfortable asking doctors about medications they see
on the television for them or their children.5

In regards to prevalence of medication utilization, IQVia, the compa-
ny that is the best source for physician prescribing data, reported that in
2017, over 7,200,000 children under 17 were on psychiatric medication.6
The highest number of children were on ADHD medication, followed by
antidepressants, then antipsychotics, and finally anti-anxiety agents.7 The

102 Jamshid A. Marvasti & Claire C. Olivier

total of people (all ages in the US) taking psychiatric medications in 2017,
was over 80,200,000.8

NATURE OF SIDE EFFECTS AND FDA IMPACT ON WARNING LABELS
Side effects are generally dose oriented (which means that they may in-
crease if the dose increases) and time related. Time related refers to how
the body can adjust to side effects over time; therefore, they may decrease
gradually and some of them eventually disappear.

The range of side effects of psychiatric medications may include insig-
nificant and temporary ones (e.g., dry mouth, frequent yawning, stuffy
nose), those which are significant and permanent, (e.g., tardive dyskine-
sia-uncontrollable jerky movements of the body), and finally, life threaten-
ing disorders (e.g., serotonin syndrome, neuroleptic malignant syndrome,
suicidality or aggression).9 Almost all antidepressant and antipsychotic
medication may cause akathisia which is a neurological disorder charac-
terized by a feeling of inner restlessness, rocking while standing or sitting,
lifting the feet as if marching on the spot, and constantly crossing and un-
crossing the legs while sitting. The other name for akathisia is Restless Leg
Syndrome, however the restlessness may not be limited to the legs. It may
transfer to the upper extremities and generally can include a feeling of “in-
ner tension.” Clinical literature indicates that this kind of internal tension
and severe anxiety/restlessness may result in aggression or suicidality.

Phenobarbital, which is used for the treatment of epilepsy, is associated
with memory disorders, hyperactivity, irritability, aggression, inattentive-
ness, learning difficulties and depressed mood.10 Benzodiazepines such as Va-
lium and Ativan may cause disinhibition in children and also in adults with
TBI (traumatic brain injury). They may become aggressive and hyperactive.
Long term use of Lithium may cause kidney and thyroid damage which may
lead to cognitive dulling, weight gain, tremors and slowed down movement.

CHANGES TO THE LABEL OF PSyCHIATRIC MEDICATION WARNING
There were approximately 14,775 cases reported to the US FDA’s MedWatch
system between 2004-2012, which noted that a number of psychiatric
drugs had violent side effects. In 1,500 cases there was homicidal ideation/
homicide. There were over 8,000 cases of aggression and approximately
3,280 cases of developing mania.11 Drug companies have been forced to
change their warning labels over time, given the severity of side effects or
the discovery of new negative impacts.

In 2004, the FDA requested drug companies to change the warning la-
bel of 10 popular antidepressants. Patients who were on these medications
were to be closely monitored for increased depression or suicidality. Addi-

103Evils of Psychopharmacology; Implications for Psycho-Historians

tionally, the FDA shared that agitation, hostility, impulsivity and mania
(among other side effects), had been reported by child and adult patients
who were taking these antidepressants.12 Yet the Black Box label change
to Selective Serotonin Reuptake Inhibitor (SSRI) antidepressants regarding
possible suicidality in 2004, only warned patients up to age 18. It was not
until 2007 that the FDA extended this warning to people under age 24.13

In 2007, the FDA changed the label of the ADHD medication Desoxyn
(methamphetamine), to include multiple warnings regarding potential
side effects of: aggression, bipolar disorder and psychotic symptoms.

In 2009, warning labels of drugs for smoking cessation and depression
(Zyban and Wellbutrin) were changed to include warnings of hostility, agi-
tation, and suicidal ideation/behavior.

In 2011 in Australia, a safety update on the psycho-stimulant medica-
tion Provigil stated that this drug was associated with aggressive behavior,
suicidal ideation/behavior, psychosis and mania, among other warnings.
In 2010 the FDA in the U.S. added “aggression,” to the warning label.14

The most frequent prescribed SSRI antidepressants are Prozac, Paxil,
Zoloft, Effexor, Cymbalta, Lexapro and Celexa. Their side effects may in-
clude the development of mania and psychosis which may result in poor
judgment and violence. Additionally, these medications may also cause
somnambulism, or sleepwalking. Dr. Ohayon conducted a sleep study poll
of nearly 16,000 adults in the U.S.15 He found that people who were taking
an antidepressant were twice as likely to experience sleepwalking. Despite
this clinical research, sleepwalking is not listed as a side effect for most anti-
depressants. Legally if someone commits a crime while sleepwalking, they
are not held responsible, as the defendant, “does not have the capacity to
form intent.”16

Additionally, in 2017, the FDA changed the label of ADHD medication
Dyanavel XR (amphetamine) to include the warning that it may exacer-
bate behavior disturbance symptoms or thought disorders for those with
a history of psychotic diagnosis. More significantly, this medication could
cause psychotic symptoms including hallucinations or mania for patients
with no previous psychotic symptoms.17

CRIMINAL ASPECTS OF PSyCHIATRIC MEDICATIONS
AND DRUG MANUFACTURERS

Case of Wesbecker Trial
In 1989 Joseph Wesbecker killed 8 people and injured 12 at his workplace,
before committing suicide and a case was brought against the drug man-
ufacturer. Wesbecker had been taking an SSRI antidepressant (Prozac) for

104 Jamshid A. Marvasti & Claire C. Olivier

one month before committing these crimes.18 Dr. Breggin was the scientific
and medical expert for the case. He was asked to evaluate for the plain-
tiffs the scientific basis for the claim that the antidepressant was causing
violence and suicide. Additionally, he was to evaluate the potential negli-
gence of the manufacturer, Eli Lilly, in how it developed and marketed this
medication, which included any possible attempts to minimize the risk of
drug-induced suicide and violence.19

Initially, it appeared that Eli Lilly had won the case, as this medication
was found not to be at fault. However, the judge then found out the trial
had been rigged, as the manufacturing company had paid the plaintiffs to
withhold damaging evidence.20 As part of the settlement in 1994, the drug
manufacturer released internal documents from 1985 which showed evi-
dence that the risks of this drug outweighed the benefit.

Even without knowing how much the drug was at fault in the Wesbecker
case, what is still significant is the possible withholding of information by
drug manufacturer. Eli Lilly conducted research on animals which demon-
strated that previously friendly cats began showing aggressive behavior
(such as growling and hissing) when given this medication. This response
was consistent with their unpublished work on a SSRI (Sertraline).21 While
Eli Lilly denied that suicidality could be caused by medications, unpub-
lished reports showed healthy volunteers on this antidepressant dropping
out of the clinical trials because of developing akathisia and suicidality. Yet
published reports by Eli Lilly did not include material on behavioral toxic-
ity. Additionally, Kaufman stated that in other clinical trial reports, it was
not mentioned that this drug company had been co-prescribing Benzodi-
azepines (tranquilizers) along with this SSRI antidepressant to minimize
the agitation caused by this drug alone.22

Case of Donald Schell
In Wyoming in 2001, Donald Schell murdered his wife, daughter and
granddaughter before shooting himself under the influence of another
SSRI antidepressant (Paxil). The plaintiff was awarded a multimillion-dol-
lar settlement.23 As put forth by Anne Thompson, “plaintiffs’ lawyers have
argued for years that the so-called miracle antidepressants have a darker
side that pharmaceutical makers have hidden from the general public, oc-
casionally with lethal consequences.”24 In her article Thompson reviews
the legal case of Donald Schell who had been taking this drug for only 48
hours before his crimes. This was a ground-breaking case because it was the
first time a jury found a pharmaceutical company liable for the homicides
committed by a person taking an antidepressant. A key element of the case
were internal documents by the pharmaceutical company, which showed
that they were aware that a small number of people could become violent

105Evils of Psychopharmacology; Implications for Psycho-Historians

as a side effect. Despite having this information, the company did not in-
clude such warnings about aggression or suicide on drug packaging.

It was not contested that Schell had been depressed. He had previous-
ly been on a different antidepressant medication. Yet he had no significant
marital problems and loved his daughter and granddaughter, so there was
no apparent motivation for the murder. The plaintiffs’ lawyer Vickery, fo-
cused the argument on whether it is, “possible for the drug to produce a
violent reaction in some people.” A critical element was Vickery’s win in
pre-trial when a motion by this drug company was overruled. They had
wanted to exclude testimonies by two expert witnesses, Dr. Healy a psychia-
trist and Dr. Maltsburger an associate clinical professor of psychiatry at Har-
vard. Healy was known for his lectures and writings regarding SSRIs and how
they should have warning labels. His own research illustrated that one in
four healthy volunteers on this medication could develop agitation and sui-
cidality. He shared that these results were supported by studies that Beecham
labs had conducted at the request of the manufacturer. Within their inter-
nal confidential documents, were the results of a study with 2000 healthy
volunteers taking either a placebo or anti-depressant. The plaintiffs’ team
shared that in these results were hundreds of volunteers who had negative
side effects, like attempted suicide which Beecham physicians stated were
either, “possible, probably, or definitely caused by this SSRI anti-depressant.”

The defense put forth that their medication was very effective and could
have been for Schell if he had been able to take it for a longer duration. They
also claimed that two pills couldn’t have caused his actions. However, the
plaintiffs showed the results from their study included volunteers experienc-
ing anxiety, nightmares, hallucinations and other symptoms within two days
of taking this drug. Two other volunteers attempted suicide within 2-3 weeks.

Although the plaintiffs asked for $25 million, the 8-person jury award-
ed $8 million. The manufacturer was found to be 80% at fault, Schell 20%.
Therefore, his relatives received $6.4 million. This was a significant win as it
encouraged other plaintiff lawyers to begin taking up antidepressant cases.25

Case of Stewart Dolin
In 2010 in Chicago, Stewart Dolin, a 57-year-old attorney, committed sui-
cide by jumping in front of a train.26 His wife sued GlaxoSmithKline (GSK),
a drug manufacturer, because just days before his suicide he started taking
a generic version of an SSRI anti-depressant (Paxil). The suit claimed that
GSK did not sufficiently warn Dolin’s doctors, about the suicidal risk for
adults using this drug. The suit held that Dolin would not have taken his
life if he had not taken this medication. The Black Box warning on the
medication stated that there was a suicide risk but only for children and
young adults up to age 24. During the trial it was argued that the manu-

106 Jamshid A. Marvasti & Claire C. Olivier

facturer had known for 20 years that this medication could trigger suicidal
behavior in a person of any age but hid the risk and ignored numerous sui-
cides committed during clinical trials.

GSK had claimed no fault because they had not manufactured the ge-
neric version, however the judge ruled that they were responsible for the
label. However, they stated they were not responsible because the FDA did
not require them to add a suicidal risk on the warning label. The 9-panel
jury found this company liable in 2017 for the charges and Dolin’s widow
was awarded $2 million for damages and $1million for the suffering of her
husband. However, the original suit asked for $39 million in damages.27
The defense later appealed and in 2018, the verdict was overturned. GSK
claimed that they had repeatedly asked the FDA to update this drug’s warn-
ings to include risk of suicidality to adults. The 7th Circuit panel found that
the FDA would have rejected an adult suicide warning and that this drug
company was following the FDA’s mandated drug label warning.28

One wonders if the FDA could be sued for negligence if they had been
provided with clinical studies which supported a known risk of older (over
24 years old) adult suicide, yet did not require this information on medica-
tions’ labels. While the FDA is entrusted with ensuring the safety of all food
and drug products in the United States, it is not clear how some medications
are approved to be on the market despite a mix of success and failure during
clinical trials. During the pre-market trials of one of the SSRI antidepressants,
16 people taking this medication attempted suicide, two successfully.29 Yet it
was still FDA approved in 1987, without a warning label of possible suicidali-
ty. The German version of the FDA would not allow this medication to come
to their market until it included a suicide warning on the label in 2004.30

PHARMACEUTICAL COMPANIES PAyOUT
Pharmaceutical companies have been in the news for years regarding claims
of false advertising or promoting their medication for unapproved usage.
Additionally, they are accused of being more focused on making a profit
than in disclosing all significant information about their drugs and the side
effects they cause. Major drug companies have had to settle hundreds of
millions of dollars in government cases or are under investigation.31

In 2005, the major drug company Eli Lilly, had to pay out $690 mil-
lion in a lawsuit to settle claims by 8000 plaintiffs that an antipsychotic
(Zyprexa), had caused diabetic and hyperglycemic side effects. They also
had to pay an additionally $500 million in 2007 for another 18,000 suits
against the same medication.32

In 2009, the same company, Eli Lilly, paid a $515 million dollar fine, in
addition to pleading guilty to criminal charges, that it unlawfully promot-

107Evils of Psychopharmacology; Implications for Psycho-Historians

ed their antipsychotic medication (Zyprexa) for uses that had not been
FDA approved which included treating dementia. They settled an addi-
tional federal civil investigation paying out $800 million to the federal
government and various states.

In 2009, Pfizer, a major drug company, was ordered to pay $2.3 billion
by the U.S. Department of Justice in criminal and civil fines. They were
charged with illegal promotions to encourage doctors to use several of their
products including Geodon, an antipsychotic. Additionally, to curb fur-
ther similar offenses they were monitored for five years by the Department
of Health and Human Services Office of Inspector General.33

In 2009, another maker of an SSRI anti-depressant GlaxoSmithKline
(GSK) was found guilty of failing to adequately warn doctors of the risks of
using their medication (Paxil) during pregnancy. Lyam Kilker, age 3, was
born with heart defects which his mother attributed to this medication,
taken during her pregnancy. He was one of the first out of approximately
600 similar cases, and his family was paid $2.5 million in damages.34

THE ISSUES OF KICKBACKS
Psychiatrist, Michael Reinstein, had his license suspended for overprescrib-
ing clozapine, an antipsychotic medication. In the case brought against
him he was accused of submitting 140,000 false claims that involved kick-
backs from drug manufacturers. Additionally, the disciplinary board stated
that he did not consider alternative treatments that would have been less
harmful to his patients. This antipsychotic is known for having seizures
as one of its side effects. The company which makes the generic version of
this anti-psychotic, had to pay over $27 million for persuading Dr. Rein-
stein to overprescribe their drug.35

Case of Andrew yount
Andrew Yount was a 4-year-old boy when he was prescribed an antipsy-
chotic, Risperdal, for his behavioral problems and ADHD.36 Yet this drug
had only been approved by the FDA for adult schizophrenia. However
Johnson & Johnson, the pharmaceutical company, began to market this
drug to doctors, encouraging them to prescribe it to children for a range of
behavioral concerns. The FDA has specifically rejected it for the pediatric
population because there was not enough data to show its safety. This an-
ti-psychotic increased the hormone prolactin which promoted the growth
of female breasts, and affected Yount and thousands of other boys. These
boys became men with full sized breasts, a condition that is irreversible un-
less they are surgically removed. Some of these men have won settlements
against the manufacturer, which included Yount, who received $76 mil-

108 Jamshid A. Marvasti & Claire C. Olivier

lion. In their 2016 annual report, Johnson & Johnson stated that they have
18,500 lawsuits against them related to this medication.37

LOST COURT CASES WHEN MEDICATIONS
WERE USED AS A DEFENSE
Although there are multiple drug lawsuits and settlements against large drug
manufacturers,38 there are also cases that claiming medications as a cause of
abnormal behavior were ruled out. Following are a few of these cases:

Texting suicide case
In the Commonwealth v Michelle Carter case, better known as the “texting
suicide case,” Carter had been sending text messages to her boyfriend Roy,
encouraging him to kill himself, which he ultimately did.39 A psychiatrist
for the defense claimed that Carter’s use of antidepressants made her invol-
untarily intoxicated and delusional in thinking she was helping Roy. She
was later convicted of involuntary manslaughter. While it is reasonable that
defense attorneys will use whatever material, they can to help their clients,
psychiatrists testifying in court have an obligation to share relevant psychi-
atric clinical science that illustrates the current reality of understanding.

“Ambien Excuse”
Roseanne Barr, a television actress, had to face consequences for posting
a racist comment on twitter. She blamed her bad behavior on her pre-
scription medication Ambien in response to public outrage, however the
medical community has found this excuse hard to believe. Ambien is a
sedative hypnotic prescribed as a sleeping aid in patients with insomnia.
Although the side effects of the psychotropic medication include con-
fusion, drowsiness and muscle weakness, the drug maker tweeted that,
“…racism is not a side effect of any Sanofi medication.” Ms. Barr was
certainly not thinking clearly in her “Ambien blackout” when she “sleep
tweeted,” but that does not excuse or explain the racist nature of her
post. Ms. Barr was tried in the Court of Public Opinion and her sitcom
was cancelled. Allen Frances M.D, a psychiatrist at Duke University com-
mented in a STAT News article, “Allowing fake medical excuses to go un-
challenged has 3 harmful consequences: encouraging more bad behavior,
discouraging those who really need medications from using them, and
unfairly stigmatizing the mentally ill.”40

Sleeping Pill Crime
Brian Browning killed his wife while she was asleep, but blamed his ac-
tions on Unisom, a sleeping pill he had been taking. Browning’s doctor had
shared that this medication could trigger behavior outside of one’s normal

109Evils of Psychopharmacology; Implications for Psycho-Historians

character. However, the courts in Australia, where this took place, did not
accept this argument and Browning was convicted. The court acknowl-
edged that Browning’s judgement and emotional abilities could have been
impacted, however there were other factors to consider that carried more
weight. Browning had claimed that Unisom caused psychosis, yet evidence
showed that psychosis is not one of the side effects, therefore this assertion
was abandoned at the trial.41

U.S. SOLDIER WHO MURDERED AFGHANI CIvILIANS
Some of the U.S. veterans who have been involved with war crime activ-
ities have claimed that their medications contributed to their crimes.42
One such case involved Army Staff Sgt. Robert Bales, who was charged
with murdering 16 Afghani civilians during his deployment in 2012, but
believes that his homicidal behavior was tied to taking Lariam, a prophy-
lactic antimalarial drug given to troops in endemic regions.43 This drug is
controversial since it has a number of neurological and psychiatric side
effects (hallucinations, anxiety, paranoia) and can cause vestibular prob-
lems (inner ear functioning affecting balance, eye movement and causing
vertigo). Due to its side effects profile, in 2009, the Defense Department
made it a last-choice option for the U.S. military, only to be used in areas
with drug-resistant strains of malaria. In 2013 the FDA placed a Black Box
warning on Lariam. Bales cited a number of possible contributing factors
for his war crimes, such as traumatic brain injury (TBI) which he sustained
in 2009. This is significant because U.S. Military regulations stipulate that
Lariam is not allowed for individuals with TBI because of its likelihood to
cross the blood-brain barriers and cause psychotic, homicidal or suicid-
al behavior. However, there was no documented evidence that Bales was
taking Lariam, yet it is not uncommon for deployed service members to
be issued this drug without it being recorded in their medical records. It is
possible that Bales was experiencing hallucinations and psychosis during
the massacre if he was administered Lariam in direct contradiction to the
U.S. military rules.44 In order to avoid the death penalty, Bales pleaded
guilty to 16 counts of murder and six counts of assault and attempted
murder. He was sentenced to life in prison without the possibility of pa-
role. His legal team had asked the U.S. Army Court of Criminal Appeals to
review his life sentence, given that the implications of being prescribed
this drug, in addition to witness information, were not part of the orig-
inal investigation. His appeal was denied however, and his life sentence
stands.45 If he runs out of options in the courts, Bales may seek clemency
from President Trump.46

110 Jamshid A. Marvasti & Claire C. Olivier

DRUG COMPANIES AS REPEAT OFFENDERS?
There are several explanations as to why major pharmaceutical companies
continue to be repeat offenders. As Morgan Statt mentioned in her article,
“Taking Big Pharma to Court,” one is that drug companies are permitted
to provide the funding for clinical trials.47 The National Institute of Health
puts a portion of its budget toward the cost of research trial however, the
amount has been drastically decreased in recent years. Obviously the source
of money should not dictate the result of trial. But, there are those who
believe one who finances the project, possibly can shape the research to
its own benefit. For example drug companies may exclude certain popula-
tions (like the elderly) from the trials, (even if that population is most likely
to need the medication), in order to have more positive outcomes. Addi-
tionally, government investigations revealed that certain research studies
regarding antipsychotics were ghostwritten by the drug companies’ market-
ing department and then signed by well-known physicians. In this way the
study falsely appears to be conducted independently by physicians.48 Dr.
Stefan Kruszewski, a psychiatrist who previously worked as a paid speaker
for multiple drug companies shared that in the beginning, he was able to
speak about the drugs related to the science. Later he was told what to say,
which included false data. One presentation included information about
an antipsychotic drug which was promoted as having no neurological side
effects. He stated, “‘They made it all up. It was never true.”’49

Lobbying
The health and pharmaceutical companies are also a very powerful lob-
bying group and just in the first quarter of 2017 they spent $78 million in
their lobbying efforts.50 There is a concern that there is a conflict of interest
between these companies and the FDA. In 1992 the Prescription Drug User
Fee Act was passed (and is renewed every 5 years) which allowed the FDA
to collect fees for processing medication approvals. While originally meant
to offer the FDA a nominal resource, as of 2017 it accounted for over 40%
of the FDA’s budget. Raising the question of how neutral the FDA can be
towards these companies’ demands regarding getting certain drugs to the
market. While this regulation was altered in May 2018, prescription drug
user fees still have a major impact on the FDA’s overall budget. These fees
accounted for 45% of the agency’s FY2018 total fundung.51

Members from the 7 largest pharmaceutical companies appeared before
the Senate Finance Committee in February of 2019 to discuss high drug
costs.52 The committee criticized the companies of finger pointing and
greedy policies, including only cutting costs of drugs which were not draw-
ing in much money. Yet it is important to note that the members of this
committee also took a combined $7.9 million in campaign contributions

111Evils of Psychopharmacology; Implications for Psycho-Historians

from the pharmaceutical industry during the last 6-year Senate election
cycle. Pharmaceutical Research and Manufacturers of American (PhRMA)
alone spent $28 million on lobbying efforts in 2018 to impact federal poli-
cy regarding drug pricing bills.53

New Legislation
Two senators, Grassley and Klobuchar have put forth two bills to lower costs
of prescription drugs. One is aimed at limiting “pay-for-delay deals,” where
pharmaceutical companies are paying generic companies to delay releasing
generic versions of their drugs; if there is no generic option, the public is
forced to buy the more expensive name brand. The second bill would allow
U.S. citizens to buy medications from Canada for lower prices.54

A HISTORICAL AND ETHICAL PERSPECTIvE
There were various periods of psychopharmacology, from its conceptual be-
ginnings in the 18th century, to the start of modern psychopharmacology
in the 1950’s and now its present-day usage. However ethical concerns
arose during each time period naming the potential dangers of these med-
ications.55 One historical perspective on psychopharmacology suggests
that its rise was not necessarily caused by more effectiveness in medica-
tion treatment of nuanced psychiatric diseases.56 Rather, a combination of
pharmaceutical marketing, advances in science, political/economic situa-
tions, changes in mental health care and a desire for a quick fix, supported
the success of psychopharmacology. There is an ongoing mindset/hope in
Western culture that technology and science can create a magic pill which
will cut through the messiness of being human and provide simple solu-
tions for madness and misery. Doctors Braslow and Marder put it this way.
“It is not that psychopharmacology has failed. Instead, we have failed our
patients by adopting a myopia that sees only symptoms and their allevi-
ation by psychoactive drugs.”57 Yet with a fragmented health care system,
psychiatrists may feel these drugs are all they can offer.

In the world of medications, researchers are also exploring drugs which
could target the memory processes for those suffering with PTSD like vet-
erans.58 While the desire for a remedy that could disrupt dysfunctional
aversive memories is warranted, it does raise the question of how is our
humanity impacted? One of the significant challenges in this drug research
is the ability to be selective when attempting to alter aversive memories.59
One doctor who lectures in medical ethics, Dr. Daniel Sokol, made this
point, ‘‘Removing bad memories is not like removing a wart or a mole. It
will change our personal identity since who we are is linked to our mem-
ories.’”60 Additionally, would a drug that removed memories, also prevent
those who take it from feeling remorse, or learning from their mistakes?61

112 Jamshid A. Marvasti & Claire C. Olivier

CONCLUSION
We have illustrated the various types of court cases that have arisen re-
garding the potential connection between psychiatric medication and vio-
lence. Yet as Kaufman stated, “While manufacturers have a vested interest
in exonerating their drugs, plaintiffs have an interest in blaming it, and
defendants in exonerating themselves. We need careful, independent anal-
ysis of existing study data.”62 However, clinical trials have demonstrated
that there can be side effects from psychiatric medications which may im-
pact suicidality and aggression/homicidality. Additionally, we have report-
ed how various pharmaceutical companies have a history of withholding
unfavorable clinical research and at times, falsely marketing medications.

There are some physicians, such as Allen Frances,63 who believe that the
pattern of illegal activity committed by various pharmaceutical companies
will not stop unless a few of these companies’ executives are handcuffed
and sentenced to jail.

Literature has put forth various recommendations to improve the health
of our nation. One idea concerns the relationship between drug company
sales representatives and physicians. Doctors should not receive any kind
of reward or compensation for giving patients certain medications, nor
should drug companies be allowed to advertise directly to consumers. Drug
companies spend billions of dollars to advertise to the public, who are then
encouraged to ask their physician for a specifically advertised medication.
Yet the public does not know the actual clinical trial results of this med-
ication. Furthermore, when a drug company pays for a clinical trial, an
independent party should review the details and results before they are
submitted to the FDA. Drug companies must disclose all of their research,
instead of cherry-picking favorable results.

Jamshid A. Marvasti, M.D. is a child and adult psychiatrist practicing at Prospect
Manchester Hospital, Manchester, Connecticut. He is a clinical assistant professor
of psychiatry at the University of New England College of Osteopathic Medicine.
Dr. Marvasti has published and edited a number of articles and books including
War Trauma in Veterans and Their Families (2012), and Psycho-Political
Aspects of Suicide Warriors, Terrorism, and Martyrdom (2008). He can be
reached at jmarvasti@aol.com.

Claire C. Olivier, MSW Received her master’s degree from the University of Cali-
fornia at Berkeley. Ms. Olivier previously wrote a chapter entitled, “The Battle Af-
ter the War: Cultural Challenges for those Coming Home,” in Dr. Marvasti’s book
War Trauma in Veterans and Their Families (2012), and contributed to Dr.
Marvasti’s earlier book, Psycho-Political Aspects of Suicide Warriors, Terror-
ism and Martyrdom (2008). She can be reached at claireolivier4@yahoo.com

113Evils of Psychopharmacology; Implications for Psycho-Historians

REFERENCES
1. Marvasti JA, Using Metaphors, Fairy Tales, and Storytelling in Psychotherapy with Chil-

dren, in 101 Favorite Play Therapy Techniques. Edited by Kaduson HG, Schaefer CE.
New Jersey, 1997, pp 35-39.

2. Ibid
3. Marvasti JA, Wu P, Merritt R: “Psychopharmacology for Play Therapists.” Interna-

tional Journal of Play Therapy: 27:1:35-45, 2018.
4. Ibid
5. Ibid
6. Citizens Commission on Human Rights International: The Mental Health Watch-

dog. Number of Children & Adolescents Taking Psychiatric Drugs in the U.S. Avail-
able at https://www.cchrint.org/psychiatric-drugs/children-on-psychiatric-drugs/.
Accessed March 1, 2019

7. Ibid
8. Citizens Commission on Human Rights International: The Mental Health Watch-

dog. Total Number of People Taking Psychiatric Drugs in the United States. Avail-
able at https://www.cchrint.org/psychiatric-drugs/people-taking-psychiatric-drugs/.
Accessed March 1, 2019

9. Marvasti JA, Wu P, Merritt R: “Psychopharmacology for Play Therapists.” Interna-
tional Journal of Play Therapy: 27:1:35-45, 2018

10. Dulcan M, Lake M: Child and Adolescent Psychiatry (4th ed.,). Washington, DC:
American Psychiatric Publishing, 2012. pp 303

11. Citizens Commission on Human Rights International: The Mental Health Watch-
dog. Psychiatric Drugs and Violence-The Facts. Available at https://www.cchrint.org/
psychiatric-drugs/drug_warnings_on_violence/ Accessed February 13, 2019

12. Ibid
13. Kauffman J: “Selective Serotonin Reuptake Inhibitor (SSRI) Drugs: More Risks than

Benefits?” Journal of American Physicians and Surgeons. Vol 14, Number 1, Spring
2009

14. Citizens Commission on Human Rights International: The Mental Health Watch-
dog. Total Number of People Taking Psychiatric Drugs in the United States.

15. Doheny K: Sleepwalking may be more common than you think. WebMD. May 14,
2012. Available at https://www.webmd.com/sleep-disorders/news/20120514/sleep-
walking-may-be-more-common-than-you-think#1 Accessed March 7, 2019

16 Healy D, Herxheimer A, Menkes D: “Antidepressants and Violence: Problems at the
Interface of Medicine and Law.” Plos Medicine. September 2006; 3(9). E372.

17. U.S. Food and Drug Administration (2017) Available at https://www.accessdata.fda.
gov/drugsatfda_docs/label/2017/208147s003lbl . Accessed March 1, 2019

18. Healy D, Herxheimer A, Menkes D: Antidepressants and Violence: Problems at the In-
terface of Medicine and Law.

19. Breggin P: Wesbecker-Prozac Product Liability Suits. Available at https://breggin
.com/wesbecker-prozac-product-liability-suits/ Accessed March 15, 2019

20. Ibid
21. Kauffman J: Selective Serotonin Reuptake Inhibitor (SSRI) Drugs: More Risks than Benefits?
22. Ibid
23. Hilts PJ: “Jury Award $6.4 Million in Killings Tied to Drug.” NY Times, June 08, 2001
24. Thompson A: Paxil Maker Held Liable in Murder/Suicide. Will $6.4 Million Verdict

Open a New Mass Tort? Baum Hedlund Aristei Goldman. July 9, 2001. Available at
https://www.baumhedlundlaw.com/drug-injury-press-releases/paxil-maker-held-li-
able-in-murder-suicide.php Accessed March 7, 2019

25. Ibid

114 Jamshid A. Marvasti & Claire C. Olivier

26. CBS (Chicago): Jury Awards $3M to Widow Who Sued Drug Firm Over Husband’s
Suicide. April 21,2017. Available at https://chicago.cbslocal.com/2017/04/21/jury-
awards-3m-to-widow-who-sued-drug-firm-over-husbands-suicide/ Accessed March
15, 2019

27. Ibid
28. Bellon T: U.S. appeals court says GSK cannot be sued over generic drug suicide. Re-

uters. August 22, 2018. Available at https://www.reuters.com/article/us-gsk-lawsuit/
u-s-appeals-court-says-gsk-cannot-be-sued-over-generic-drug-suicide-idUSKCN-
1L72D1 Accessed March 15, 2019

29. Walker A: “Take Two of These and Sue Me in the Morning; The Emergence of Litiga-
tion Regarding Psychotropic Medication in the United States and Europe.” Arizona
Journal of International and Comparative Law. 2002. Vol.19. No.2. Available at http://
arizonajournal.org/wp-content/uploads/2015/11/Walker Accessed March 1, 2019

30. Ibid
31. Wilson D: “Side Effects May Include Lawsuits.” The New York Times. October 2,

2010. Available at https://www.nytimes.com/2010/10/03/business/03psych.html
Accessed March 9, 2019

32. Citizens Commission on Human Rights: Chronology of Sample Lawsuits about Psy-
chotropic Drugs. Available at https://files.ondemandhosting.info/data/www.cchr
.org/files/education/appendix/10-chronology-of-sample-lawsuits Accessed Feb-
ruary 15, 2019

33. Ibid
34. The Telegraph: GSK antidepressant Paxil to blame for baby’s heart defects, U.S. jury

rules. October 15, 2009. Available at https://www.telegraph.co.uk/news/health/
news/6335822/GSK-antidepressant-Paxil-to-blame-for-babys-heart-defects-US-jury-
rules.html Accessed March 12, 2019

35. Harrison P: Psychiatrist Suspended for Antipsychotic Overprescribing. Medscape.
August 12, 2014. Available at https://www.medscape.com/viewarticle/829790 Accessed
March 9, 2019

36. Liston B: Makers of Risperdal Sued for Breast Development in Boys. Mad in Ameri-
ca. July 21, 2017. Available at https://www.madinamerica.com/2017/07/risperdal-
sued-breast-development-boys/ Accessed March 17, 2019

37. Ibid
38. Saunders J: Top Eight Largest Drug Lawsuit Settlements of All Time. Saunders Walk-

er. October 23, 2017. Available at https://www.saunderslawyers.com/top-eight-
largest-drug-lawsuit-settlements-time/ Accessed March 2, 2019

39. Knoll J, Annas G: “Warning: Antidepressants may cause messaging manslaughter.”
Psychiatric Times. September 19, 2017. Vol 34:9. Available at http://www.psychiatric
times.com/psychopharmacology/warning-antidepressants-may-cause-messaging-
manslaughter Accessed March 19, 2019

40. Biscaldi L: Roseanne Barr’s “Ambien Excuse” Tried in the Court of Public Opin-
ion. Medical Bag. June 6, 2018. Available at https://www.medicalbag.com/medicine/
ambien-excuse-roseanne-barr-tweets/article/771396/ Accessed March 21, 2019

41. Loughnan A: The drugs made me do it: can prescription side-effects be an excuse for
crime? The Conversation. July 8, 2016. Available at http://theconversation.com/the-
drugs-made-me-do-it-can-prescription-side-effects-be-an-excuse-for-crime-45821
Accessed March 29, 2019

42. Marvasti JA, Podolski J E: Forensic Aspects of Combat Trauma and PTSD: Special Vet-
erans’ Court, Malingering, And Criminal Conduct, In War Trauma In Veterans And
Their Families: Diagnosis And Management of PTSD, TBI and Comorbidities Of Combat
Trauma. Edited by Marvasti JA. Springfield, IL: Charles Thomas Publishers, 2012,
pp154-168

115Evils of Psychopharmacology; Implications for Psycho-Historians

43. Kime P: “U.S. Soldier Who Murdered Afghan Civilians Blames Malaria Drug Used By
Army.” Miami Herald. August 8, 2017. Available at https://www.miamiherald.com/
news/nation-world/national/article165990147.html Accessed March 13, 2019

44. Ibid
45. Associated Press: Court upholds ex-soldier’s life Sentence In Slaying Of Afghan

Civilians. Army Times. September 28, 2017. Available at https://www.armytimes
.com/news/your-army/2017/09/28/court-upholds-us-soldiers-life-sentence-in-
slayings-of-afghan-civilians/ Accessed March 28, 2019

46. Kime P: Ex-soldier Who Shot Up Afghan Village May Seek Clemency From Pres-
ident, Lawyer Says. McClatchy Washington Bureau. June 20, 2018. Available at
https://www.mcclatchydc.com/news/nation-world/national/article213092124.html
Accessed March 19, 2019

47. Statt M: Taking Big Pharma to Court: Why Lawsuits Have Little Effect on
Drug Company. Mad in America. February 22, 2018. Available at https://www
.madinamerica.com/2018/02/big-pharma-lawsuits-little-effect-drug-companies/
Accessed March 26, 2019

48. Wilson D: Side Effects May Include Lawsuits. October 2, 2010.
49. Ibid
50. Statt M: Taking Big Pharma to Court: Why Lawsuits Have Little Effect on Drug Company.

February 22, 2018.
51. Dabrowska A, Green V: The Food and Drug Administration (FDA) Budget: Fact

Sheet. Congressional Research Service. Updated September 12, 2018. Available at
https://fas.org/sgp/crs/misc/R44576 Accessed March 25, 2019

52. Evers-Hillstrom K: Senators Publicly Grill ‘Big Pharma’ Executives After Accepting
Millions From Industry. Open Secrets. February 26, 2019. Available at https://www
.opensecrets.org/news/2019/02/senators-grill-big-pharma-executives/ Accessed March
16, 2019

53. Ibid
54. Smith J: Senators Grill Pharma Execs Over Prescription Drug Prices. Yahoo Finance. Feb-

ruary 26, 2019. Available at https://finance.yahoo.com/news/senators-grill-pharma-
execs-over-prescription-drug-prices-154953281.html Accessed March 22, 2019

55. Barbara JG: History of Psychopharmacology: From Functional Restitution to Function-
al Enhancement. 2015. 489-504. 10.1007/978-94-007-4707-4_26. Available at https://
www.researchgate.net/publication/283653231_History_of_Psychopharmacology_
From_Functional_Restitution_to_Functional_Enhancement Accessed April 8, 2020

56. Braslow JT, Marder SR: History of Psychopharmacology. Annu.Rev.Clin.Psychol.
2019. 15:25–50. Available at https://www.annualreviews.org/doi/pdf/10.1146/
annurev-clinpsy-050718-095514 Accessed April 5, 2020

57. Ibid
58. Giustino TF, Fitzgerald PJ, Maren S: Revisiting propranolol and PTSD: Memory erasure

or extinction enhancement? Neurobiol Learn Mem. 2016 Apr; 130: 26–33. Available
at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4818733/ Accessed April 8, 2020

59. Ibid
60. Derbyshire D: Pill to erase bad memories: Ethical furor over drugs ‘that threaten

human identity.’ Daily Mail. February 16, 2009. Available at https://www.dailymail
.co.uk/news/article-1145777/Pill-erase-bad-memories-Ethical-furore-drugs-
threaten-human-identity.html Accessed April 6, 2020

61. Ibid
62. Kauffman J: Selective Serotonin Reuptake Inhibitor (SSRI) Drugs: More Risks than Bene-

fits? 2009
63. Frances A: Saving Normal. New York, NY: HarperCollins Publishers, 2013.

Copyright of Journal of Psychohistory is the property of Association for Psychohistory Inc.
and its content may not be copied or emailed to multiple sites or posted to a listserv without
the copyright holder’s express written permission. However, users may print, download, or
email articles for individual use.

Dement Neuropsychol 2019 December;13(4):422-426

422422 Can psychopharmacology improve cognitive rehabilitation? Caixeta and Caixeta

http://dx.doi.org/10.1590/1980-57642018dn13-040009

Original Article

This study was conducted at the Universidade Federal de Goiás Faculdade de Medicina – Neurology, Goiânia, GO, Brazil.

1Universidade Federal de Goiás Faculdade de Medicina – Neurology, Goiânia, GO, Brazil. 2Federal University of Goiás Faculty of Medicine Ringgold standard institu-
tion – Post-Graduation, Goiânia, GO, Brazil.

Leonardo Caixeta. Universidade Federal de Goiás Faculdade de Medicina – Neurology – Avenida Cristo Rei, 626, Setor Jaó – 74605-020 Goiânia GO – Brazil.
E-mail: leonardocaixeta1@gmail.com

Disclosure: The authors report no conflicts of interest.

Received August 01, 2019. Accepted in final form September 25, 2019.

Therapeutic synergism
How can psychopharmacology improve cognitive rehabilitation?

Leonardo Caixeta1,2 , Victor Melo Caixeta1,2

ABSTRACT. Despite recent advances in cognitive rehabilitation of patients with cognitive disorders, there are many

major obstacles to the optimized global use of this therapeutic resource. Objective: The authors outline the concept of
‘therapeutic synergism’, i.e. the concurrent use of pharmacological and cognitive rehabilitation therapies to maximize

functional benefits, addressing the optimization of therapeutic approaches for cognitive disorders. Methods: Three
psychopharmacological and rehabilitation interrelationship paradigms are presented in three different clinical settings.

Results: Paradigm 1: Behavioral and cognitive symptoms that hinder a cognitive rehabilitation program, but can be
improved with psychopharmacology. Paradigm 2: Cognitive symptoms that hinder cognitive rehabilitation, but can be

improved with anticholinesterases. Paradigm 3: Behavioral symptoms that hamper the use of cognitive rehabilitation,

but can be improved by psychotropic drugs. Conclusion: Judicious use of psychotropic drugs in cognitive disorders can
benefit, directly or indirectly, cognitive functions, thereby favoring other treatment modalities for cognitive impairment,

such as neuropsychological rehabilitation.

Key words: cognitive rehabilitation, treatment engagement, psychopharmacology, synergism.

SINERGISMO TERAPÊUTICO: COMO A PSICOFARMACOLOGIA PODE MELHORAR A REABILITAÇÃO COGNITIVA?

RESUMO. Apesar dos recentes avanços na reabilitação cognitiva de pacientes com distúrbios cognitivos, existem muitos e

graves obstáculos ao uso otimizado globalmente desse recurso terapêutico. Objetivo: Os autores destacam o conceito de
‘sinergismo terapêutico’, ou seja, o uso simultâneo de terapias de reabilitação farmacológica e cognitiva, maximizando os

benefícios funcionais, a fim de abordar a otimização da abordagem terapêutica dos distúrbios cognitivos. Métodos: Três
paradigmas de inter-relacionamento psicofarmacológico e de reabilitação são apresentados em três contextos clínicos

diferentes. Resultados: Paradigma 1: sintomas comportamentais e cognitivos que dificultam um programa de reabilitação
cognitiva, mas podem ser melhorados com a psicofarmacologia. Paradigma 2: sintomas cognitivos que dificultam a

reabilitação cognitiva, mas podem ser melhorados com anticolinesterásicos. Paradigma 3: sintomas comportamentais que

dificultam o uso da reabilitação cognitiva melhorada por drogas psicotrópicas. Conclusão: O uso criterioso das drogas
psicotrópicas nos distúrbios cognitivos pode beneficiar, direta ou indiretamente, as funções cognitivas, favorecendo,

portanto, outras modalidades de tratamento para o comprometimento cognitivo, como a reabilitação neuropsicológica.

Palavras-chave: reabilitação cognitiva, tratamento, psicofarmacologia, sinergismo.

Despite recent advances in cognitive rehabilitation of patients with cognitive
disorders, there are many major obstacles to
the optimized global use of this therapeutic
resource. Some patients may find it difficult
to adhere to cognitive rehabilitation due to

the lack of insight regarding cognitive deficits,
or because of compromised brain systems, or
even general difficulty in performing daily
tasks, sense of hopelessness, lack of energy
and apathy, symptoms that may be due to
the disease itself or associated to depression

Dement Neuropsychol 2019 December;13(4):422-426

423Caixeta and Caixeta Can psychopharmacology improve cognitive rehabilitation?

or other comorbid psychiatric disorders.1 For cognitive
rehabilitation methods to be effective, patients must be
adequately engaged and motivated not only to begin a
rehabilitation process, but also to remain involved in the
intervention until a therapeutic dosage can be achieved.1
Many patients do not benefit from rehabilitation or can-
not be indicated for this procedure because of partially
treated behavioral symptoms, either for lack of a diagno-
sis, or for inadequate or underdosing of medications. On
the other hand, misuse of drugs by patients, especially
medications with cognitive effects, can compromise the
efficacy of cognitive rehabilitation.2

Notwithstanding recent evidence suggesting that
concurrent pharmacological and behavioral methods
may maximize functional benefits for patients suffer-
ing from, for example, dementia,3-5 there is an inex-
plicable scarcity of studies concerning the therapeutic
synergism between psychopharmacology and cognitive
rehabilitation, reflecting the unfortunate absence of
contact between these two domains: pharmacological
(biomedical approach) and non-pharmacological (essen-
tially the psychological approach).

In this article, we present three paradigmatic cases
on how these two domains can be interconnected and
through which strategies the psychopharmacological
approach can optimize the implementation of cognitive
rehabilitation techniques to enhance improvement in
real-world functioning.

METHODS
Using a qualitative approach, the main principles or
strategies of association between psychoactive drugs
and cognitive rehabilitation used in the Memory Clinic
at the Federal University of Goiás (UFG), in Central
Brazil, were reviewed. We focused attention on pharma-
cological management that addresses the optimization
of cognitive rehabilitation techniques.

The Institute of Memory at the UFG is a referral cen-
ter for cognitive disorders in Central Brazil with 20 years
of experience in the evaluation, diagnosis and multidis-
ciplinary treatment of cognitive disorders. Centers such
as this have academic credentials to seek, through their
accumulated experience, some subjective principles
that govern conduct and therapeutic strategies in areas
where there is little literature.

RESULTS
We identified three models of interaction between
psychoactive drugs and cognitive rehabilitation that
seek to optimize rehabilitation methods. Each of these
models will be exemplified by a clinical case illustrating

the way psychopharmacology and cognitive rehabilita-
tion interact.

CASE 1
Paradigm 1: Behavioral and cognitive symptoms that
hinder a cognitive rehabilitation program, but can be
improved with psychopharmacology

Cloney (fictitious name), 25 years old, is a patient
with invasive developmental disorder (autistic syn-
drome), presenting with severe behavioral and cognitive
changes. The behavior alterations that prevented a cog-
nitive rehabilitation approach included impulsivity and
aggressiveness (he took the papers from the teacher’s
desk and ripped them up compulsively, despite being
asked not to do so several times; whenever a child passed
him by, he would grab them by the arm and attack them;
he presented several episodes of direct violence towards
people who assisted him; disobeyed commands and
did not respect the social limits and rules previously
imposed), dysphoria, compulsive overeating, motor rest-
lessness and hyperactivity. Cognitive disorders included
severe mental retardation, with impairments in several
cognitive domains. The cognitive disorders that most
affected his functional adaptation and social life were
cognitive inflexibility, insight absence, Theory of Mind
deficit, impaired decision-making, expressive language
difficulties, and severe attention deficit. Cloney also pre-
sented extreme intolerance to any modification of his
environment, reacting aggressively when this occurred
(he broke everything around him).

Due to prejudice held by both his mother and the
multi-professional care team, Cloney was not taking any
medication for his disorder. After explaining to them
how modern psychopharmacy could help him control
some of his worse behavioral issues, and maybe even
improve some of his basic cognitive functions, thereby
allowing a rehabilitation approach, a pharmacological
regimen was started consisting of aripiprazole 15mg/
day (prescribed to improve social behavior, reduce
aggressiveness and control restlessness), fluoxetine 20
mg/day (indicated to control compulsive overeating and
dysphoria) and methylphenidate 40mg/day (prescribed
to improve attention span, and reduce both hyperactiv-
ity level and appetite).

One month after starting on this medication, Cloney
showed a marked improvement in many aspects of his
behavior, which also presented as benefits in a variety
of cognitive functions: 1) motor restlessness ceased
and, consequently, he could sit still in a chair, focusing
his attention better, allowing a better verbal approach
and eye-to-eye interaction; 2) impulsivity also ceased,

Dement Neuropsychol 2019 December;13(4):422-426

424 Can psychopharmacology improve cognitive rehabilitation? Caixeta and Caixeta

and his actions became more predictable, improving
the safety of the care team since they could prevent cer-
tain responses or undesired actions; 3) improved many
aspects of his relationships, as he obeyed social rules
(started to accept his mother and teachers’ authority)
and became more sociable in general, so he could engage
in group interactions, including with other children; 4)
he became more tolerant to the limits established and
to the external rules imposed, eliminating the explo-
sive reactions when his immediate desires were denied;
5) it became easier to negotiate with him on decisions
that involved immediate desires – after medication
he started to accept that his desires may be satisfied
in exchange for some effort (for example, helping his
mother or following teachers’ requests). These improve-
ments promoted a better-structured cognitive-behavior
base, more amenable to the application of adequate
rehabilitation techniques. Before the psychopharma-
cological intervention, even simple cognitive-behavior
approaches were impossible. Currently, Cloney is rea-
sonably engaged in cognitive rehabilitation, and his
team of health professionals and teachers, as well as the
other students and patients, no longer fear him.

CASE 2
Paradigm 2: Cognitive symptoms that hinder cogni-
tive rehabilitation, but can be improved with anti-
cholinesterases

Homero (fictitious name), 74 years old, is a patient
with Alzheimer’s disease, naïve to treatment with anti-
cholinesterases (most effective medication group for
this dementia).4 As the treatment with galantamine
did not work because of side effects (severe nausea,
tachycardia and dizziness), the family were reluctant to
try this pharmacological group again. A glutamatergic
antagonist (memantine) was prescribed, without any
significant benefit in cognition, particularly in memory.

Homero presented a severe memory deficit and
inability to learn new information, which made neu-
ropsychological rehabilitation approaches even harder.
Despite the three sessions he had every week, there was
no effective improvement of the patient, comparing to
previous sessions, so it was not possible to advance to
the next phases of the process. The family noted no ben-
efits in the social-functional sphere. Since there was no
benefit of the neuropsychological rehabilitation, the
therapeutic approach was discontinued.

Donepezil (another anticholinesterase, although
with more favorable side-effect profile) was then pre-
scribed at the dosage of 10mg/day in order to improve
cognitive outcome, especially recent memory and, con-

sequently, enhance his learning mechanisms. Three
months after starting use of the medication, Homero
presented a clear memory improvement, and his
attention capacity was better, favoring the learning
process. In fact, the improvement was very evident
when he resumed rehabilitation: his attention span
had developed, he could maintain recently learned
information available for longer in working memory
(for example, during the execution of a task, he could
gather information that was necessary later for use in a
new task).

CASE 3
Paradigm 3: Behavioral symptoms that hamper the use of
cognitive rehabilitation, but can be improved by psycho-
tropic drugs

Thelma (fictitious name), 67 years old, suffering
from depression (not previously diagnosed) associated
with dementia in Parkinson’s disease. Despite being in
use of an anticholinesterasic drug and memory defi-
cit improvements achieved, her greatest difficulty was
adhering to the cognitive rehabilitation. Thelma pre-
sented intense fatigability, being incapable of remaining
in continuous consultation for more than five seconds.
Her low attentional span impaired all the rehabilitation
approaches that relied on attention for task execution.
She also exhibited economy of effort with many answers
like “I don’t know”. She had a pessimistic attitude to cog-
nitive rehabilitation, believing that she couldn’t obtain
any benefit from it, and was unable to develop any
involvement or affective bonding with the rehabilitation
professional, proving averse to the activity. In many situ-
ations, she was anxious and irritated with the activity,
creating ploys to leave and stop the process.

Thelma was referred to a psychiatrist who diagnosed
masked depression. She was started on a noradrenergic
antidepressant (mirtazapine 30 mg /day). A month later,
clear improvement in the patient’s mood was observed:
she was more active and had more physical/men-
tal energy, was more interested in the ongoing tasks,
regained the pleasure associated with social contact and
other activities, could maintain her attentional focus
for much longer, could see the point in investing, more
actively, in the rehabilitation. Indeed, after the depres-
sion treatment, her involvement and performance in the
cognitive rehabilitation activities increased markedly.

DISCUSSION
From the reported cases, we can infer three models of
interaction between psychopharmacology and cognitive
rehabilitation:

Dement Neuropsychol 2019 December;13(4):422-426

425Caixeta and Caixeta Can psychopharmacology improve cognitive rehabilitation?

1. In the patient receiving psychotropic drugs for
an underlying disease state where cognitive rehabilita-
tion is indicated in order to improve the residual cogni-
tive deficits associated with the disease (for instance, in
schizophrenia, in which the medication is necessary to
control the disease, but does not always act on the com-
mon cognitive symptoms of this medical condition);6,7

2. In patients with executive dysfunction (one of
the areas in which cognitive rehabilitation is known for
having more limited results), the use of some drugs can
optimize treatment (for example, the use of memantine
in patients with cognitive inflexibility, such as in the
case of pathological gamblers);8

3. Many patients may not be suitable for cognitive
rehabilitation due to psychiatric symptoms that hinder
the full conducting of the process (e.g. aggressive, agi-
tated patients that are incapable of therapeutic bonding,
or apathetic and asthenic patients that do not engage
sufficiently in the therapeutic process).1,9 In others,
despite attempts to rehabilitate, they have only dis-
crete or diminished improvement because of psychiatric
symptoms. In both cases, medication may be used as an
agent for reducing dysfunctional behavior, allowing the
application of the rehabilitation in a safe and effective
manner, or improving results when the rehabilitation is
already underway, but in a limited way.

Our study highlights the concept of ‘therapeutic
synergism’, i.e. the concurrent use of pharmacological
and cognitive rehabilitation therapies maximizing func-
tional benefits in order to address the optimization of
the therapeutic approach for cognitive disorders.

Many other authors have been working on this
approach in different scenarios and within different
rationales.5,8,10-13 The three paradigms presented describe
different scenarios in which a precise drug intervention
(precision that must be almost ‘surgical’) helps in the
process of cognitive rehabilitation in many ways.

In the first paradigm, the psychopharmacological
intervention must provide the basic conditions to ensure
the patient can be indicated for cognitive rehabilita-
tion, otherwise they would not be an eligible candidate.

In the second paradigm, the pharmacological inter-
vention in cognition enables and facilitates the rehabili-
tation, which may then have a real chance of success. In
other words, the prescription of a cognitive enhancer to
augment cognitive rehabilitation outcomes, based on a
rationale in which a cognitive enhancer proceeds by tar-
geting more basic discrete cognitive skills, so that cogni-
tive rehabilitation can progress to more complex skills.
This assumes that the basic skills must be refined before

more complex skills can work effectively. Some authors
also claim that, currently, cognitive training exercises
are used to improve basic cognitive skills, but pharma-
cotherapy holds promise as a more effective treatment.5

In the third paradigm, the pharmacological interven-
tion in behavior optimizes the response to the ongoing
rehabilitation process, since it overcomes an obstacle
to fully exploit the therapeutic process. For cognitive
rehabilitation therapies to be successful, patients must
be adequately involved and motivated not only to begin
cognitive intervention but also to keep engaged in the
rehabilitation program until a therapeutic dosage can
be reached.1

In a literature review about cognitive rehabilitation,
Manzine & Pavarine14 found that, in most of the stud-
ies reviewed, cognitive rehabilitation can provide more
benefit for the patient’s rehabilitation when combined
with other interventions, such as pharmacological
treatment. Provided that both treatment modalities
are aligned and optimized, the synergistic therapeutic
effects become evident. To this end, it is fundamental
that the attending physician is aware of the whole thera-
peutic program in which the patient is engaged, hav-
ing consistent notions of how one treatment modality
may impact (positively or negatively) the other, and of
how delicate the relationship dynamic is between them.
Without such tools, there is a risk of wrongly assessing
the risk-benefit ratio involved in each pharmacological
choice. In practice, unaware physicians run the risk of
prescribing a medication option that may, in some cases,
have negative effects on the patient’s cognitive function
and also on their cognitive rehabilitation.

In conclusion, judicious use of psychotropic drugs
can benefit, directly or indirectly, cognitive functions,
thereby favoring other treatment modalities for cogni-
tive impairment, such as neuropsychological rehabilita-
tion. This finding reflects those of other authors.10-12 For-
tunately, with better knowledge of the available drugs in
general (and psychotropics in particular), greater invest-
ment in medical training, as well as a better technical
and affective rapport between medical and non-medical
professionals, a more optimistic scenario will be possible
in the coming years.

Author contributions. Leonardo Ferreira Caixeta and
Victor Melo Caixeta: conceptualization, data curation,
formal analysis, funding acquisition, investigation,
methodology, project administration, resources, soft-
ware, supervision, validation, visualization, writing –
original draft, writing – review & editing.

Dement Neuropsychol 2019 December;13(4):422-426

426 Can psychopharmacology improve cognitive rehabilitation? Caixeta and Caixeta

REFERENCES
1. Choi J, Twamley EW. Cognitive rehabilitation therapies for Alzheimer’s

disease: a review of methods to improve treatment engagement and
self-efficacy. Neuropsychol Rev. 2013;23(1):48-62.

2. HaeRi Na, SangYun Kim, Yu Kyeong Kim, Moon Ho Park, Sung Tae Cho,
and Woo Jung Kim. Cognitive Therapy Combined with Drug Treatment
in Patients with Alzheimer’s Disease: A Neuropsychological and Positron
Emission Tomography Investigation, a Pilot Study. Dement Neurocogn
Disord. 2015;14(2):76-82.

3. Buschert V, Bokde AL, Hampel H. Cognitive intervention in Alzheimer
disease. Nat Rev Neurol. 2010;6(9):508-17.

4. Campos C, Rocha NB, Vieira RT, Rocha SA, Telles-Correia D, Paes F, et
al. Treatment of Cognitive Deficits in Alzheimer’s disease: A psychophar-
macological review. Psychiatr Danub. 2016;28(1):2-12.

5. Medalia A, Opler LA, Saperstein AM. Integrating psychopharmacology
and cognitive remediation to treat cognitive dysfunction in the psychotic
disorders. CNS Spectr. 2014;19(2):115-20.

6. Harvey PD. Pharmacological approaches to cognitive enhancement.
In: Harvey PD, ed. Cognitive Impairment in Schizophrenia. New York:
Cambridge University Press; 2013.

7. Silverstein SM, Hatashita-Wong M, Wilkniss S, Lapasset J, Bloch A,
McCarthy R. Clinical management and rehabilitation of the treatment
refractory patient: conceptual foundations and outcome data.Sante Ment
Que. 2004;29(2):15-44.

8. Grant JE, Chamberlain SR, Odlaug BL, Potenza MN, Kim SW. Meman-
tine shows promise in reducing gambling severity and cognitive inflex-
ibility in pathological gambling: a pilot study. Psychopharmacology (Berl).
2010;212(4):603-12.

9. McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT. A meta-
analysis of cognitive remediation in schizophrenia. Am J Psychiatry.
2007;164(12):1791-802.

10. Meguro M, Kasai M, Akanuma K, Ishii H, Yamaguchi S, Meguro K.
Comprehensive approach of donepezil and psychosocial interventions
on cognitive function and quality of life for Alzheimer’s disease: the Osaki-
Tajiri Project. Age Ageing 2008;37:469-73.

11. Bottino C, Carvalho I, Alvarez AM, et al. Cognitive rehabilitation combined
with drug treatment in Alzheimer’s disease patients: a pilot study. Clin
Rehabil 2005;19:861-9.

12. Onder G, Zanetti O, Giacobini E, et al. Reality orientation therapy
combined with cholinesterase inhibitors in Alzheimer’s disease: random-
ized controlled trial. Br J Psychiatry 2005;187:450-5.

13. De Vreese, LP, Verlato C, Emiliani S et al. Effect size of a three-month
drug treatment in Alzheimer’s disease when combined with individual
cognitive retraining: preliminary results. Neurobiol Aging 1998;19(suppl
4):S213.

14. Manzine PR, Pavarini SCI. Cognitive rehabilitation: literature review based
on levels of evidence. Dement Neuropsychol. 2009;3(3):248-55.

RESEARCH ARTICLE Open Access

Attitudes towards psychopharmacology
and psychotherapy in psychiatric patients
with and without migration background
Eva J. Brandl1,2*†, Nora Dietrich1,2†, Nicoleta Mell1,2, Johanna G. Winkler1,2, Stefan Gutwinski1,2,
H. Joachim Bretz1,2 and Meryam Schouler-Ocak1,2

Abstract

Background

: Sociodemographic factors, attitude towards treatment and acculturation may be important factors
influencing the decision of immigrants to seek and maintain psychiatric treatment. A better understanding of these
factors may significantly improve treatment adherence and outcome in these patients. Therefore, we investigated
factors associated the attitude towards psychotherapy and medication in a sample of psychiatric outpatients with
and without migration background.

Methods

: N = 381 patients in a psychiatric outpatient unit offering specialized treatment for migrants were
included in this study. Attitude towards psychotherapy was assessed using the Questionnaire on Attitudes Toward
Psychotherapeutic Treatment, attitude towards medication with the Drug Attitude Inventory-10. Acculturation,
symptom load and sociodemographic variables were assessed in a general questionnaire. Statistical analyses
included analyses of covariance and hierarchical regression.

Results

: Patients of Turkish and Eastern European origin reported a significantly more positive attitude towards
medication than patients without migration background. When controlling for sociodemographic and clinical
variables, we did not observe any significant differences in attitude towards psychotherapy. Acculturation neither
influenced the attitude towards psychotherapy nor towards medication.

Conclusion: Our study indicates that sociodemographic and clinical factors may be more relevant for patients´
attitudes towards treatment than acculturation. Considering these factors in psychiatric treatment of patients with
migration background may improve treatment outcome and adherence.

Keywords: Migrants, Attitude, Medication, Psychotherapy

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article’s Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: eva.brandl@charite.de
†Eva J. Brandl and Nora Dietrich contributed equally to this work.
1Charité Universitätsmedizin Berlin, corporate member of Freie Universität
Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health,
Department of Psychiatry and Psychotherapy, Campus Mitte, Berlin, Germany
Charité Universitätsmedizin Berlin, Berlin, Germany
2Psychiatrische Universitätsklinik der Charité im St. Hedwig Krankenhaus,
Große Hamburger Str. 5-11, 10115 Berlin, Germany

Brandl et al. BMC Psychiatry (2020) 20:176
https://doi.org/10.1186/s12888-020-02585-1

http://crossmark.crossref.org/dialog/?doi=10.1186/s12888-020-02585-1&domain=pdf

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/publicdomain/zero/1.0/

mailto:eva.brandl@charite.de

Background
With rising numbers of migrants and refugees over the
past years, there has been increasing interest in mental
health issues of these groups. A variety of psychosocial
risk factors, including lower socioeconomic status,
higher risk for unemployment [1], discrimination [2] and
experience of violence as well as migration stress [3]
contribute to higher rates of psychiatric disorders in mi-
grant populations. Although the risk for specific psychi-
atric disorders varies depending on the region of origin
[3] as well as on the circumstances of being a migrant or
a refugee [4], generally a higher prevalence of most psy-
chiatric disorders has been reported [5–12]. Despite this
increased risk and a higher symptom load compared to
individuals without migration background [13–15], mi-
grants tend to use mental health services, including psy-
chotherapy, less often [16–19]. In addition, treatment
adherence to psychopharmacological treatment has been
reported to be lower in migrants and ethnic minorities
[20–24]. Insufficient consideration of sociodemographic
differences between migrants and non-migrants seeking
treatment [15, 25] in clinical practice as well as relevant
language and cultural barriers provide partial explana-
tions for these issues. Another important, yet insuffi-
ciently investigated factor influencing treatment seeking
and adherence is the attitude towards psychiatric and
psychotherapeutic treatment in migrants. A negative at-
titude towards psychotherapy may be one of the main
reasons not to seek treatment [26]. Only few studies on
attitude towards psychotherapy in migrants have been
performed to date, and most of these have been con-
ducted in the United States, indicating a generally less
positive attitude towards psychotherapy in migrants. A
high impact of sociodemographic and symptom-related
factors on the attitude has been reported [27]. Accultur-
ation of migrants has also been identified as a factor in-
fluencing attitude towards psychotherapy [28–30].
However, a recent meta-analysis found ethnic differences
in the impact of acculturation on attitudes towards psy-
chological treatment with little impact in most ethnic
groups except for individuals of Asian heritage [31]. Lit-
erature on the attitude towards psychotherapy in mi-
grant populations in Germany and Europe is sparse but
also indicates a less positive attitude in these groups
[32–35]. However, the influence of acculturation on atti-
tude towards psychotherapy of migrants in Germany has
not been investigated extensively yet. Education, age,
sex, (e.g., [27, 36, 37]) as well as psychiatric symptom
load (e.g., [36, 38, 39]) have been investigated regarding
an influence on attitude towards psychotherapy with
heterogeneous results, indicating a need for further stud-
ies in this field.
Attitude towards medication has been shown to be an

important predictor of medication adherence, e.g. [40–

42]. The attitude towards pharmacological treatment in
migrants and ethnic minorities has only been examined
in a few studies. Similar to the attitude towards psycho-
therapy, a less positive attitude towards medication has
been found in ethnic minority patients [43–48].
The influence of acculturation on medication adher-

ence in patients with mental disorders has not been in-
vestigated extensively, but better adherence in
individuals with stronger orientation towards the host
culture has been reported [49, 50]. However, accultur-
ation was not associated with attitude towards medica-
tion in all studies [51]. In other medical areas,
acculturation has been associated with better drug ad-
herence, e.g. [49, 52–54]. To the best of our knowledge,
there are no data on the influence of acculturation of at-
titude towards medication in psychiatric patients with
migration background in Germany.
In summary, attitude towards psychotherapy and

medication may influence treatment adherence and out-
come. However, the specific relevance of factors poten-
tially influencing these attitudes towards treatment,
including clinical and sociodemographic factors, migra-
tion background and acculturation in migrants is not
well understood yet. Therefore, we set out to a) examine
the attitude of psychiatric patients with and without mi-
gration background towards psychotherapy as well as to-
wards medication and b) to identify the association of
relevant sociodemographic and clinical factors and ac-
culturation with the attitudes towards psychotherapy
and medication.

Methods
Participants
All patients treated in the outpatient unit of the Psychi-
atric University Hospital of Charité at St.Hedwig-Hos-
pital in Berlin, Germany, between April and June 2015
and who did not fulfill our exclusion criteria (acute
psychosis, severe cognitive impairment, acute emergency
treatment) were invited to fill out a questionnaire pro-
vided in seven languages (German, English, French,
Arabic, Farsi, Turkish and Russian). The outpatient unit
offers general psychiatric outpatient treatment to two
large downtown districts of Berlin and additionally is
specialized in treatment of patients with migration
background.
Questionnaires were handed out to the patients who

came to their appointments in the outpatient unit and
filled out in the waiting area after informed consent was
obtained. Information on current medication and diag-
noses according to ICD-10 criteria was obtained from
electronic medical records. The study was approved by
the ethics board of Charité – Universitätsmedizin Berlin
and conducted in accordance with the Declaration of

Brandl et al. BMC Psychiatry (2020) 20:176 Page 2 of 10

Helsinki. All participants gave written informed consent
before participation in the study.

Measures
The questionnaire contained a general part with demo-
graphic and clinical data (such as marital status, duration
of illness, employment status etc.). Current symptom
load was assessed with the Symptom Checklist 14 (SCL-
14), a short version of the Symptom Checklist 90 [55].
These general characteristics of the sample have been
described previously [15]. The SCL-14 subscales reached
Cronbach’s alpha of α = 0.89 for somatization, α = 0.83
for anxiety and α = 0.87 for depression in our dataset.
For the purpose of this study, only patients without mi-
gration background and the largest migrant groups
(Turkish, Eastern European, middle Eastern/north Afri-
can (MENA [56];) plus Afghanistan/Pakistan (MENAP))
migration background) were included since the other
groups were too small for meaningful analyses.

Attitude towards psychotherapy
Attitude towards psychotherapy was assessed using the
Questionnaire on Attitudes Toward Psychotherapeutic
Treatment (QAPT [36]) which consists of 20 statements
rated on a Likert-type scale ranging from 1 (“I do not
agree”) to 4 (“I agree”). Four subscales are created to as-
sess the attitude towards psychotherapy: psychothera-
pist’s competence, anticipated judgment by others,
general attitude towards psychotherapy and personal ac-
ceptance. Higher scores indicate a more positive attitude
toward psychotherapy. The validity of the instrument
was confirmed in the original publication of the ques-
tionnaire. The internal consistency of the subscales has
been confirmed in the original publication [36]. In our
own data set, the QAPT subscales reached the following
α-values: competence: α = 0.52, judgment: α = 0.64, gen-
eral attitude: α = 0.58, acceptance: α = 0.61. The QAPT
has been used in other cross-cultural studies on attitude
towards psychotherapy before with higher α-values for
the QAPT subscales in some studies [35, 57] and com-
parable α-values to our sample in others [34].

Attitude towards medication
To examine attitudes towards and subjective experience
with medication, we applied the 10-item version of the
Drug Attitude Inventory (DAI [58]). The scale consists
of ten statements (for example: “For me, the good things
about medication outweigh the bad”; “I feel more normal
on medication”; “It is ununatural for my mind and body
to be controlled by medication”) with a dichotomous re-
sponse option (true/ false) and assesses general attitude
towards medication. Several studies have underlined the
validity and reliability of the DAI [59]. Cronbach’s α of
the DAI in our dataset was 0.68.

Acculturation
In patients with migration background (defined as not
holding German citizenship per birth, having immi-
grated to Germany and/or having at least one parent not
holding German citizenship following the definition of
the Federal Statistical Office [1]), acculturation was
assessed using the Acculturation Index by Ward &
Rana-Deuba [60]. Based on a two-dimensional approach
to acculturation it contains two subscales: “host national
identification” and “co-national identification”. Both
scales range between 1 and 7 with higher values indicat-
ing a stronger identification with that culture. A high re-
liability (co-national identification scale α = .93 and host
identification scale α = .96) and good validity of the Ac-
culturation Index has previously been reported [61] with
the same α-values being obtained in our own dataset .

Statistical analyses
Data were analyzed using RStudio 0.99.489 for Windows.
Differences between the included migrant groups and
patients without migration background in sociodemo-
graphic and clinical parameters were explored with ana-
lysis of variance (ANOVA), Chi-Quadrat-tests and
Fisher-Yates-tests, respectively.
Analyses of covariance (ANCOVA) were conducted in

order to assess if the subsamples with migration back-
ground differed on the five dimensions (four QAPT
scales and DAI) from the subsample without migration
background. Potentially relevant covariates (SCL-14 sub-
scale values for anxiety, somatization and depression;
age; education; gender; religious affiliation; medication
intake; psychiatric inpatient stays) were theoretically de-
rived, e.g. [27, 30]. Only those covariates that showed a
significant correlation with the respective dependent
variable (QAPT subscales and DAI) were included in the
final analyses and are provided for each analysis in
Table 2. Two ANCOVA were conducted per dimension.
Due to the gender distribution differences in our sub-
samples, the first analysis included only gender as covar-
iate in case it correlated with the dependent variable.
The second analysis also included further sociodemo-
graphic (e.g. education, religious affiliation) and clinical
factors (e.g. symptom severity, inpatient stays, medica-
tion intake). The adjusted means were compared with
the Dunnett-test using the sample without migration
background as control.
Hierarchical regressions were conducted to test if ac-

culturation predicts a significant additional amount of
variance in the samples with migration background after
accounting for sociodemographic and clinical variables.
The covariates from the prior analysis were adopted for
each dependent variable. In the second step both scales
of the Acculturation Index were added. Due to the ex-
ploratory character of the analyses, we did not correct p-

Brandl et al. BMC Psychiatry (2020) 20:176 Page 3 of 10

values for multiple testing. Patients who had returned
questionnaires with more than 20% of missing values
were excluded from the analyses. In the total sample,
6.6% of values were missing. We applied listwise deletion
to missing values for the ANCOVA and the hierarchical
regression to avoid a high loss of information.

Results
Sociodemographic data
The original sample comprised N = 423 participants who
had returned completed questionnaires out of N = 700
patients who were invited to participate in the study re-
sponse rate of 60.5% [15]. Due to the limited sample
sizes, patients from Asia (N = 5), Africa (N = 10) Western
Europe and America (N = 19) were not included in the
analyses. N = 8 patients had to be excluded due to in-
complete questionnaires, resulting in a total sample of
N = 381 individuals. The sample included patients with-
out migration background (N = 194), and patients of
Turkish (N = 111), Eastern European (N = 39) or
MENAP (N = 37) background. We found significant dif-
ferences in terms of gender, education, religiousness,
medication intake and diagnoses among the subsamples
(see Table 1) as previously described for the overall sam-
ple [15]. There were also significant differences in re-
ported symptom severity regarding somatic and anxiety
symptoms. Due to the observed differences, sociodemo-
graphic and clinical variables were incorporated in the
following statistical analyses as covariates.

Attitudes toward psychotherapy and medication
First, we analyzed whether patients with Turkish, East-
ern European and MENAP background differed signifi-
cantly in their attitude towards psychotherapy as
measured by the four scales of the QAPT and in their
attitude towards medication measured by the DAI as
compared to patients without migration background.
Two ANCOVA were conducted per QAPT scale and
DAI. In the first ANCOVA, we only controlled for gen-
der if necessary. In the second ANCOVA, we also added
further relevant sociodemographic and clinical control
variables. Sociodemographic and clinical variables with
significant association with at least one of the QAPT
subscales were education, number of inpatient stays in
the history, current symptom load on the SCL subscales
somatization and depression.
The mean value of the QAPT-judgment scale was sig-

nificantly lower among the samples with East European
and MENAP background compared to the sample with-
out migration background, indicating a less positive atti-
tude on this subscale of the QAPT (see Supplementary
Table S1). However, after controlling for sociodemo-
graphic variables, no significant differences remained.
On the QAPT scales competence, acceptance and

general attitudes, the samples with Eastern European,
Turkish and MENAP background did not differ signifi-
cantly from the sample without migration background in
both analyses (see Supplementary Tables S2-S4).
Regarding the attitude towards medication, patients

with Turkish and Eastern European background had a
significantly more positive attitude compared to the
sample without migration background. This remained
significant after controlling for potentially relevant socio-
demographic and clinical variables (see Supplementary
Table S5). There was no statistically significant differ-
ence in attitude towards medication between the
MENAP-subgroup and patients without migration
background.

Acculturation and attitudes
In the next step, we examined if acculturation explained
an additional amount of variance beyond the identified
relevant sociodemographic and clinical variables. We
conducted a hierarchical regression with the two scales
of the acculturation index (host national identification
and co-national identification) added in the second step.
The main results are presented in Table 2 (for further
details, see Supplementary Table S6). The first p-value
indicates if the model explains a significant amount of
variance as compared to a null model. The second p-
value indicates whether the second model including the
acculturation index (step 2) explains significantly more
variance than the model without the acculturation index
(step 1). For reasons of simplicity only the test statistics
of the additional variables are presented in the table.
The F-tests for ΔR2 did not reach significance (with one
exception in the East European sample on the QAPT-
judgment scale). Hence, the models including the accul-
turation indexes (apart from one exception) did not ex-
plain significantly more variance than the models
without the acculturation indexes, indicating no major
association of acculturation with the attitude towards
psychotherapy as well as towards medication in our
sample.

Discussion

To the best of our knowledge, this is the first study to
investigate attitude towards psychotherapy and medica-
tion in a sample of patients with and without migration
background in a psychiatric outpatient unit. We did not
find major differences in the attitude towards psycho-
therapy after controlling for relevant sociodemographic
and clinical factors. The attitude towards medication
was more positive in patients with Turkish and Eastern
European background. Acculturation did not have a sig-
nificant association with patients´ attitudes towards
treatment in our sample except for the QAPT-judgment
scale in the Eastern European subsample. In this

Brandl et al. BMC Psychiatry (2020) 20:176 Page 4 of 10

T
a
b
le

1
So
ci
o

d
em

o
g
ra
p
h
ic
an
d
sy
m
p
to
m

re
la
te
d
ch
ar
ac
te
ris
ti
cs

o
f

th
e
to
ta
l
sa
m
p
le
an
d
th
e
fo
u
r
su
b
sa
m
p
le
s

Va
ria
b
le

To
ta
l
sa
m
p
le
n
=

38
1

n
(%
)

±
SD

Sa
m
p
le
w
it
h
o
u
t
m
ig
ra
ti
o
n

b
ac
kg
ro
u
n
d
n
=
19
4

n
(%
)
/M

±
SD

Sa
m
p
le
w
it
h
Tu
rk
is
h
m
ig
ra
ti
o
n

b
ac
kg
ro
u
n
d
n
=
11
1
n
(%
)
/M

±
SD

Sa
m
p
le
w
it
h

Ea
st

Eu
ro
p
ea
n

m
ig
ra
ti
o
n
b
ac
kg
ro
u
n
d
n
=
39

n
(%
)

/M
±
SD

Sa
m
p
le
w
it
h
M
EN

A
P
m
ig
ra
ti
o
n

b
ac
kg
ro
u
n
d
n
=
37

n
(%
)
/M
±
SD

p- va
lu
e

G
en

d
er

m
al
e

22
4
(5
8.
8)

10
0
(5
1.
6)

85
(7
6.
6)

21
(5
3.
9)

18
(4
8.
6)

< .0 01 *

m
al
e

15
7
(4
1.
2)

94
(4
8.
5)

26
(2
3.
4)

18
(4
6.
2)

19
(5
1.
4)

A
g
e

43
.9
(1
2.
5)

44
.3
(1
3.
4)

45
.3
(1
0.
9)

43
.2
(1
3.
2)

38
.2
(1
0.
5)

.0
61

Sc
h
o
o
l
ed

u
ca
ti
o
n

.0
09
*

h
ig
h

21
7
(5
7.
0)

12
6
(6
6.
0)

48
(4
3.
2)

25
(6
4.
1)

18
(4
8.
6)

lo
w

13
8
(3
6.
2)

60
(3
0.
9)

52
(4
6.
8)

12
(3
0.
8)

14
(3
7.
8)

n
o
t
in
d
ic
at
ed

26
(6
.8
)

8
(4
.1
)

11
(9
.9
)

2
(5
.1
)

5
(1
3.
5)

C
o
u
n
tr
y
o
f
b
irt
h

G
er
m
an
y

22
3
(5
8.
5)

19
3
(9
9.
5)

19
(1
7.
1)

7
(1
7.
9)

4
(1
0.
8)

O
th
er

co
u
n
tr
y

15
8
(4
1.
5)

1
(U
SA

)
92

(T
u
rk
ey
)

30
(8
2.
1)

(3
Bu

lg
ar
ia
,

1
C
ze
ch

Re
p
u
b
lic
,

1
H
u
n
g
ar
y,
1
Ko

so
vo
,

1
La
tv
ia
,1
0
Po

la
n
d
,

1
Ru

m
an
ia
,8

Ru
ss
ia
,

4
Se
rb
ia
,1

U
kr
ai
n
e,

1
fo
rm

er
Yu

g
o
sl
av
ia
)

33
(8
9.
2)

(4
Ira
q
,2

Ira
n
,

1
Is
ra
el
,6

Le
b
an
o
n
,

2
M
o
ro
cc
o
,1

Pa
ki
st
an
,

16
Sy
ria
,1

Tu
n
is
ia
)

Re
lig
io
u
s
af
fil
ia
ti
o
n

< .0 01 *

Ye
s

23
0
(6
0.
4)

76
(3
9.
2)

97
(8
7.
4)

25
(6
4.
1)

N
o

12
2
(3
2.
0)

94
(4
8.
5)
11
(9
.9
)

13
(3
3.
3)

N
o
t
in
d
ic
at
ed

29
(7
.6
)

24
(1
2.
4)

3
(2
.7
)

1
(2
.6
)

M
ed

ic
at
io
n

in
ta
ke

.5
97

Ye
s

32
8
(8
6.
1)

16
0
(8
2.
5)

98
(8
8.
3)

35
(8
9.
7)

35
(3
4.
6)

N
o

53
(1
3.
9)

34
(1
7.
5)

13
(1
1.
7)

4
(1
0.
3)

2
(5
.4
)

N
u
m
b
er

o
f

m
ed

ic
at
io
n
s

1.
5
(0
.7
)

1.
3
(1
.0
)

1.
5
(0
.9
)

1.
7
(1
.1
)

1.
5(
0.
7)

.0
14
*

Ty
p
e
o
f
m
ed

ic
at
io
n

.0
03
*

A
n
ti
d
ep

re
ss
an
t

26
9(
70
.6
)

10
9
(5
6.
2)

98
(8
8.
3)

31
(7
9.
5)

31
(8
3.
8)

N
eu
ro
le
p
ti
c

20
7
(5
4.
3)

11
1
(5
7.
2)

50
(4
5.
0)

22
(5
6.
4)

24
(6
4.
9)

Tr
an
q
u
ill
iz
er

23
(6
.9
)

13
(6
.7
)

5
(4
.5
)

4
(1
0.
3)

1
(2
.7
)

M
o
o
d
st
ab
ili
ze
r

37
(7
.1
)

20
(1
0.
3)

8
(7
.2
)

9
(2
3.
1)

0
(0
.0
)

A
n
ti
d
em

en
ti
va

0(
0.
0)

0
(0
.0
)
0
(0
.0
)
0
(0
.0
)
0
(0
.0
)

Brandl et al. BMC Psychiatry (2020) 20:176 Page 5 of 10

T
a
b
le
1
So
ci
o
d
em
o
g
ra
p
h
ic
an
d
sy
m
p
to
m
re
la
te
d
ch
ar
ac
te
ris
ti
cs

o
f
th
e
to
ta
l
sa
m
p
le
an
d
th
e
fo
u
r
su
b
sa
m
p
le
s
(C
o
n
tin
u
ed
)

Va
ria
b
le
To
ta
l
sa
m
p
le
n
=

38
1
n
(%
)
/M

±
SD
Sa
m
p
le
w
it
h
o
u
t
m
ig
ra
ti
o
n

b
ac
kg
ro
u
n
d
n
=
19
4
n
(%
)
/M

±
SD
Sa
m
p
le
w
it
h
Tu
rk
is
h
m
ig
ra
ti
o
n
b
ac
kg
ro
u
n
d
n
=
11
1
n
(%
)
/M
±
SD
Sa
m
p
le
w
it
h
Ea
st
Eu
ro
p
ea
n
m
ig
ra
ti
o
n
b
ac
kg
ro
u
n
d
n
=
39
n
(%
)
/M
±
SD
Sa
m
p
le
w
it
h
M
EN
A
P
m
ig
ra
ti
o
n
b
ac
kg
ro
u
n
d
n
=
37
n
(%
)
/M
±
SD
p- va
lu
e

Pa
in

M
ed
ic
at
io
n

12
(4
.3
)

2
(1
.0
)

8
(7
.2
)
2
(5
.1
)
0
(0
.0
)

In
te
rn
al
m
ed

ic
at
io
n

56
(1
4.
7)

35
(1
8.
0)

15
(1
3.
5)

3
(7
.7
)

3
(8
.1
)

Su
b
st
it
u
ti
o
n
/

ad
d
ic
ti
o
n
tr
ea
tm

en
t

m
ed
ic
at
io
n

1
(0
.3
)

1
(0
.5
)

0
(0
.0
)
0
(0
.0
)
0
(0
.0
)
O
th
er

12
(3
.1
)

5
(2
.6
)

5
(4
.5
)
0
(0
.0
)
2
(5
.4
)

In
p
at
ie
n
t
st
ay
s

(p
sy
ch
ia
tr
y)

.0
05
*

n
ev
er

13
2
(3
4.
6)

49
(2
5.
3)

55
(4
9.
5)

7
(1
7.
9)

21
(5
6.
7)

se
ld
o
m

14
4
(3
7.
8)

76
(3
9.
2)

38
(3
4.
2)

20
(5
1.
3)

10
(2
7.
0)

o
ft
en

92
(2
4.
1)

63
(3
1.
5)

14
(1
2.
6)

10
(2
5.
6)

5
(1
3.
5)
n
o
t
in
d
ic
at
ed

13
(3
.4
)

6
(3
.1
)

4
(3
.6
)

2
(5
.1
)
1
(2
.7
)

Sy
m
p
to
m

se
ve
rit
y

SC
L
So
m
at
iz
at
io
n

2.
4
(1
.2
)

2.
0
(1
.0
)

3.
1
(1
.3
)

2.
4
(1
.2
)

2.
6
(1
.0
)

< .0 01 *

SC
L
D
ep

re
ss
io
n

2.
8(
1.
1)

2.
6
(1
.0
)

3.
0
(1
.1
)

3.
2
(1
.2
)

3.
0
(1
.0
)

.1
84

SC
L
A
n
xi
et
y

2.
0
(1
.0
)

1.
7
(0
.9
)

2.
3
(1
.1
)

1.
9
(1
.1
)

2.
5
(1
.2
)

< .0 01 *

D
ia
g
n
o
si
s
(IC

D
-1
0)

F0
8
(2
.1
)

7
(3
.6
)

1
(0
.9
)

0
(0
.0
)
0
(0
.0
)
< .0 01 *

F1
85

(3
0.
2)

65
(3
3.
5)

9
(8
.1
)

5
(1
2.
8)

6
(1
6.
2)

F2
86

(3
0.
6)

51
(2
6.
3)

18
(1
6.
2)

9
(2
3.
1)

8
(2
1.
6)

F3
16
4
(4
3.
0)

79
(4
0.
7)

49
(4
4.
1)

15
(3
8.
5)

21
(5
6.
8)

F4
14
1
(3
7.
0)

50
(2
5.
8)

61
(5
5.
0)

15
(3
8.
5)

15
(4
0.
5)

F5
7
(1
.8
)

3
(1
.5
)

2
(1
.8
)

2
(5
.1
)
0
(0
.0
)

F6
61

(1
6.
0)

46
(2
3.
7)

7
(6
.3
)

8
(2
0.
5)

0
(0
.0
)

F7
7
(1
.8
)

7
(3
.6
)
0
(0
.0
)
0
(0
.0
)
0
(0
.0
)

F8
0(
0.
0)

0
(0
.0
)
0
(0
.0
)
0
(0
.0
)
0
(0
.0
)

F9
2
(0
.5

1
(0
.5
)
1
(0
.9
)
0
(0
.0
)
0
(0
.0
)

N
o
te
.
Lo

w
le
ve
l
o
f
sc
h
o
o
l
ed

u
ca
ti
o
n
w
as

d
ef
in
ed

as
0
–
9
ye
ar
s
o
f
sc
h
o
o
l,
a
h
ig
h
le
ve
l
o
f
sc
h
o
o
l
ed

u
ca
ti
o
n
as

1
0
–
1
3
ye
ar
s
o
f
sc
h
o
o
l
ed

u
ca
ti
o
n
(f
o
llo
w
in
g
P
et
ro
w
sk
i
u
n
d
K
o
lle
g
en

,
2
0
1
4
).
R
eg

ar
d
in
g
th
e
cl
as
si
fi
ca
ti
o
n
o
f

in
p
at
ie
n
t
st
ay
s,
1
–
2
in
p
at
ie
n
t
st
ay
s
w
er
e
ca
ti
g
o
ri
ze
d
as

se
ld
o
m

an
d
m
o
re

th
an

2
as

o
ft
en

.M
EN

A
=
M
id
d
le

Ea
st

an
d
N
o
rt
h
A
fr
ic
a
re
g
io
n
,S
C
L
=
Sy
m
p
to
m
-C
h
ec
k-
Li
st
,n

=
sa
m
p
le

si
ze

=
ar
it
h
m
et
ic
m
ea
n
,S
D
=
st
an

d
ar
d

d
ev
ia
ti
o
n
.N

o
te
.
Th

e
n
u
m
b
er

o
f
d
ia
g
n
o
si
s
d
o
es

n
o
t
co
rr
es
p
o
n
d
w
it
h
th
e
sa
m
p
le

si
ze

as
so
m
e
p
at
ie
n
ts

h
av
e
m
u
lt
ip
le

d
ia
g
n
o
si
s.
Th

e
d
ia
g
n
o
si
s
h
av
e
b
ee
n
cl
as
si
fi
ed

ac
co
rd
in
g
to

th
e
In
te
rn
a
ti
o
n
a
l
C
la
ss
ifi
ca
ti
o
n
o
f
D
is
ea
se
s

10
(G
ra
u
b
n
er
,
2
0
1
4
).
F0

=
O
rg
an

ic
,
in
cl
u
d
in
g
sy
m
p
to
m
at
ic
,
m
en

ta
l
d
is
o
rd
er
s,
F1

=
M
en

ta
l
an

d
b
eh

av
io
u
ra
l
d
is
o
rd
er
s
d
u
e
to

u
se

o
f
p
sy
ch
o
ac
ti
ve

su
b
st
an

ce
s,
F2

=
Sc
h
iz
o
p
h
re
n
ia
,
sc
h
iz
o
ty
p
al

an
d
d
el
u
si
o
n
al

d
is
o
rd
er
s,
F3

=
M
o
o
d
/
af
fe
ct
iv
e
d
is
o
rd
er
s,
F4

=
N
eu

ro
ti
c,
st
re
ss
-r
el
at
ed

an
d
so
m
at
o
fo
rm

d
is
o
rd
er
s,
F5

=
B
eh

av
io
u
ra
l
sy
n
d
ro
m
es

as
so
ci
at
ed

w
it
h
p
h
ys
io
lo
g
ic
al

d
is
tu
rb
an

ce
s
an

d
p
h
ys
ic
al

fa
ct
o
rs
,
F6

=
D
is
o
rd
er
s
o
f
p
er
so
n
al
it
y
an

d
b
eh

av
io
u
r
in

ad
u
lt
p
er
so
n
s,
F7

=
M
en

ta
l
re
ta
rd
at
io
n
,
F8

=
D
is
o
rd
er
s
o
f
p
sy
ch
o
lo
g
ic
al

d
ev
el
o
p
m
en

t,
F9

=
B
eh

av
io
u
ra
l
an

d
em

o
ti
o
n
al

d
is
o
rd
er
s
w
it
h
o
n
se
t
u
su
al
ly

o
cc
u
rr
in
g
in

ch
ild

h
o
o
d
an

d
ad

o
le
sc
en

ce
*
p
< .0 5

Brandl et al. BMC Psychiatry (2020) 20:176 Page 6 of 10

subsample, a higher level of acculturation was associated
with a more positive attitude towards psychotherapy re-
garding anticipated judgment by others. However, due
to the very limited sample size of this subsample and the
low Cronbach’s α of the QAPT-judgment scale, this
finding needs to be considered with caution. Since this
association was not observed in the two other subsam-
ples with MENAP- and Turkish background, we do not
assume a major impact of acculturation on anticipated
judgement for utilizing psychotherapy by others;

however, a replication in a larger sample would be re-
quired before final conclusions can be drawn.
These findings are partially in line with results of pre-

vious studies. Calliess et al. [32] also did not report an
impact of acculturation on the attitude towards psycho-
therapy in young adult individuals with Turkish back-
ground in Germany. However, they found a significant
influence of migration background on the attitude to-
wards psychotherapy after controlling for sociodemo-
graphic variables, whereas these differences did not

Table 2 Association of acculturation with attitude towards psychotherapy and medication

Turkish background (N = 111) Eastern European background (N = 39) MENAP background (N = 37)

Dependent Variable R2 ΔR2 F for ΔR2 p for ΔR2 n R2 ΔR2 F for ΔR2 p for ΔR2 n R2 ΔR2 F for ΔR2 p for ΔR2 n

QUAPT judgment

Step 1: .17* .17 3.83 .003* .27 .27 1.66 .186 .16 .16 0.86 .523

Step 2: .20* .03 1.91 .154 .50 .23 4.61 .022* .19 .03 0.42 .662

Host national identification

Co-national identification

100 26 29

QUAPT competence

Step 1: .03 .03 1.36 .263 .12 .12 1.50 .244 .07 .07 1.07 .359

Step 2: .06 .03 1.17 .315 .23 .11 1.54 .237 .07 .00 0.01 .994

Host national identification
Co-national identification

90 26 30

QUAPT-acceptance

Step 1: .07 .07 3.09 .051 .35* .35 6.10 .008* .01 .01 0.11 .897

Step 2: .12* .05 2.56 .083 .39* .04 .76 .480 .04 .03 0.42 .663

Host national identification

Co- national Identification

90 26 31

QUAPT general attitude

Step 1: .19* .19 3.12 .008* .47* .47 2.71 .047* .02 .02 0.07 .998

Step 2: .19* .00 0.34 .715 .51 .04 0.51 .607 .10 .08 0.81 .459

Host national identification
Co-national identification

89 25 27

Drug Attitude Inventory

Step 1: .09 .09 2.37 .058 .28 .28 2.24 .096 .19 .19 1.57 .212

Step 2: .09 .00 0.04 .966 .34 .06 0.87 .432 .20 .02 0.14 .871

Host national identification

2003Co-national identification

98 28 31

Main results of the hierarchical regression predicting the QUAPT scales judgment, competence, acceptance and general attitude as well as the DAI scale in the
samples with Turkish, East European and MENAP background. The association of control variables with attitude towards psychotherapy and medication are
included in Step 1. Acculturation scales are added to the other variates in the second step. For simplicity reasons, the control variables as well as the B- and β-
values are not shown in this Table, details can be found in Supplementary Table S6. MENA = Middle East and North Africa Region, QUAPT = Questionnaire on
Attitudes Toward Psychotherapeutic Treatment, DAI = Drug Attitude Inventory
* p < .05

Brandl et al. BMC Psychiatry (2020) 20:176 Page 7 of 10

remain significant after controlling for confounders in
our sample. In most ethnic groups, a recent meta-
analysis did not report a major impact of acculturation
as well [31]. Knipscheer & Kleber [33] reported signifi-
cant differences between migrants and non-migrants in
their attitude towards psychotherapy in a Dutch sample;
however, while statistically significant, the observed dif-
ferences were rather small. Ditte et al. reported a less fa-
vorable attitude towards psychotherapy in Russian
migrants as compared to German participants [35]. Our
group found a less positive attitude towards psychother-
apy in individuals of Turkish background in a previous
study [34], where migration background was the most
important predictor beyond sociodemographic factors.
Nonetheless, the participants in the previous study were
recruited in waiting rooms of general practitioners
whereas the participants for the current study were
already in psychiatric treatment, which may in parts ex-
plain the observed differences in the results. It can be
hypothesized that patients already actively seeking psy-
chiatric treatment in general may have a more positive
attitude towards psychiatry and psychotherapy than indi-
viduals not seeking psychiatric treatment and that there-
fore migration background may play a smaller role in
our sample than in samples from the general population.
In addition, the outpatient unit from which patients
were recruited for the study is specialized in treatment
of migrants. The use of professional interpreters and the
presence of staff with migration background may reduce
feelings of stigmatization and could also contribute to a
less negative view on psychotherapy in patients with mi-
gration background.
The finding that sociodemographic and clinical vari-

ables influence attitude towards psychotherapy is in line
with previous studies. For example, Constantine and
Gainor [39] found that individuals with higher depres-
sion symptom load were more likely to seek treatment.
When correcting for education level, differences in atti-
tude towards medication were smaller. Attitude towards
treatment is generally considered to be more positive in
patients with higher education levels (e.g., [27, 36]). Gen-
der only partially predicted attitude towards psychother-
apy in our analyses, which is in line with mixed findings
of previous studies [33, 37, 62].
The attitude towards medication was more positive in

patients of Turkish and Eastern European background.
While gender, depression symptom load and current
medication intake were associated with attitude towards
medication in our sample, acculturation was, similar to
the attitude towards psychotherapy, not a significant
predictor. The more positive attitude in these two sub-
groups contradicts other studies which reported a less
favorable attitude towards medication in ethnic minor-
ities [43–48]. However, most of the previous studies

have been conducted in the US examining individuals of
Hispanic or African-American origin. One study con-
ducted in Switzerland included mainly immigrants from
Western European countries who were excluded from
our analyses due to the small sample size in our sample
[48]. Therefore, our result indicates cultural differences
in attitude towards medication and underlines the im-
portance in considering specific cultural factors when
initiating medication in psychiatric patients with migra-
tion background. The finding that acculturation did not
influence attitude towards medication beyond sociode-
mographic factors is in line with an earlier study in His-
panic patients [51]. However, since other studies found
an impact of acculturation on medication adherence [49,
50, 52–54], which may in parts represent attitude to-
wards medication, final conclusions cannot be drawn
and more research in this field is required.
Several limitations need to be considered in interpret-

ation of our findings. The sample was a convenience
sample and not a representative data set, so the results
cannot be applied to the general population. In particu-
lar, since the participants were all patients in a psychi-
atric outpatient unit, conclusions about reasons for
migrants to not utilize psychiatric treatment cannot be
drawn. In addition, the sample size of the subgroups was
rather small, limiting statistical power to identify signifi-
cant effects. Due to the small sample size, duration of
stay in Germany and comparisons between 1st vs. 2nd
migrant generation could not be incorporated in our
analyses. Subgroup analyses by type of medication or
psychiatric diagnose could also not be performed due to
the limited sample size. Although we controlled for con-
founding variables in our analyses, the results may be
biased due to other differences among the groups. The
questions in the DAI were related to general attitude
towards medication and not to psychopharmacology
specifically; therefore, the attitude towards specific anti-
depressant or antipsychotic treatment cannot be
assessed with our data. Finally, the Cronbach’s alpha of
the QAPT subscales and the DAI in our sample was not
very high, indicating low reliability and limiting the abil-
ity to detect significant differences.

Conclusions

In summary, our study contributes to a better un-
derstanding of views on psychotherapy and medica-
tion in migrants. Since sociodemographic differences
among different migrant groups and patients without
migration background seem to be stronger associated
with patients´ views as compared to acculturation,
our study underlines the need to consider these
sociodemographic factors in psychiatric treatment of
migrants.

Brandl et al. BMC Psychiatry (2020) 20:176 Page 8 of 10

Supplementary information

Supplementary information accompanies this paper at https://doi.org/10.
1186/s12888-020-02585-1.

Additional file 1: Supplementary Tables. Tables S1–5 Results and
descriptive statistics of the two analysis of covariance with the factor
migration background and the Drug Attitude Inventory (DAI) as
dependent variable. R2 = .19*, corrected R2 = .17 (both for analysis 2). The
corrections are based on the mean value of SCL Somatization M = 2.41,
SCL Depression M = 2.83, SCL Anxiety M = 1.99. The DAI value represents
an arithmetic mean of a 2 point Likert scale (1 = True, 2 = False) with
higher values indicating a more positive attitude. MENA = Middle East
and North Africa Region, MG = migration background, DAI = Drug
Attitude Inventory, SCL = Symptom Check List, SE = standard error, Sum
Sq = Sum of Squares, df = degrees of freedom, MSS = Mean sum of
squares. Table S6. Complete results of the hierarchical Regression
predicting the QUAPT scales judgment, competence, acceptance and
general attitude as well as the DAI scale within the samples with Turkish,
East European and MENAP background. The acculturation scales are
added in the second step. School education: 0 = low, 1 = high, Gender:
0 = female, 1 = male, religious affiliation: 0 = yes, 1 = no. Higher scores on
the scales of the QUAPT and DAI indicate a more positive attitude on
that scale. For simplicity reasons the control variables are only presented
in step 1. MENA = Middle East and North Africa Region, QUAPT =
Questionnaire on Attitudes Toward Psychotherapeutic Treatment, DAI =
Drug Attitude Inventory, SCL = Symptom Check List.* p < .05

Abbrevations

ANCOVA: Analysis of covariance; DAI: Drug attitude inventory; MENA: Middle
East, North Africa; MENAP: Middle East, North Africa, Afghanistan/Pakistan;
QAPT: Questionnaire on attitudes toward psychotherapeutic treatment; SCL-
14: Symptom Checklist-14

Acknowledgments

EJB participated in the Clinician Scientist Program of Charité and the Berlin
Institute of Health. We acknowledge support from the German Research
Foundation (DFG) and the Open Access Publication Fund of Charité –
Universitätsmedizin Berlin.

Authors´ contributions

EJB: Study design, recruitment, data analysis, writing of manuscript. ND:
recruitment, data analysis, writing of manuscript. NM: recruitment, data
management. JGW: recruitment, data management. SG: recruitment, data
management. HJB: study design, data analysis. MSO: study design, writing of
manuscript. All authors read and approved the final manuscript.

Funding

No funding was obtained for the presented study.

Availability of data and materials

The datasets used for the current study are available from the corresponding
author on reasonable request.

Ethics approval and consent to participate

The study was approved by the ethics board of Charité – Universitätsmedizin
Berlin (reference number: EA4/007/15) and conducted in accordance with
the Declaration of Helsinki. All participants gave written informed consent
before participation in the study.

Consent for publication

Not applicable.

Competing interests

EJB: speaker fees from Servier and Medice. MSO: Speaker from Servier and
Forum für medizinische Fortbildung – FomF, expert opinion for the court.
JGW, ND, NM, SG and HJB declare no conflicts of interest.

Received: 12 September 2019 Accepted: 5 April 2020

References

1. Bundesamt S. Statistisches Jahrbuch 2016. Wiesbaden: Statistisches

Bundesamt; 2016. 2016.
2. Aichberger MC, Bromand Z, Rapp MA, Yesil R, Montesinos AH, Temur-Erman

S, et al. Perceived ethnic discrimination, acculturation, and psychological
distress in women of Turkish origin in Germany. Soc Psychiatry Psychiatr
Epidemiol. 2015;50(11):1691–700.

3. Jurado D, Alarcon RD, Martinez-Ortega JM, Mendieta-Marichal Y, Gutierrez-
Rojas L, Gurpegui M. Factors associated with psychological distress or
common mental disorders in migrant populations across the world. Rev
Psiquiatr Salud Ment. 2017;10(1):45–58.

4. Dapunt J, Kluge U, Heinz A. Risk of psychosis in refugees: a literature review.
Transl Psychiatry. 2017;7(6):e1149.

5. Dingoyan D, Schulz H, Kluge U, Penka S, Vardar A, von Wolff A, et al.
Lifetime prevalence of mental disorders among first and second generation
individuals with Turkish migration backgrounds in Germany. BMC Psychiatry.
2017;17(1):177.

6. Tortelli A, Errazuriz A, Croudace T, Morgan C, Murray RM, Jones PB, et al.
Schizophrenia and other psychotic disorders in Caribbean-born migrants
and their descendants in England: systematic review and meta-analysis of
incidence rates, 1950-2013. Soc Psychiatry Psychiatr Epidemiol. 2015;50(7):
1039–55.

7. Cantor-Graae E, Selten JP. Schizophrenia and migration: a meta-analysis and
review. Am J Psychiatry. 2005;162(1):12–24.

8. Levecque K, Lodewyckx I, Bracke P. Psychological distress, depression and
generalised anxiety in Turkish and Moroccan immigrants in Belgium: a
general population study. Soc Psychiatry Psychiatr Epidemiol. 2009;44(3):
188–97.

9. Aichberger MC, Neuner B, Hapke U, Rapp MA, Schouler-Ocak M, Busch MA.
Association between migrant status and depressive symptoms in the older
population in Germany. Psychiatr Prax. 2012;39(3):116–21.

10. Bermejo I, Mayninger E, Kriston L, Harter M. Mental disorders in people with
migration backround compared with German general population. Psychiatr
Prax. 2010;37(5):225–32.

11. Crafa D, Warfa N. Maternal migration and autism risk: systematic analysis. Int
Rev Psychiatry. 2015;27(1):64–71.

12. Cantor-Graae E, Pedersen CB. Full spectrum of psychiatric disorders related
to foreign migration: a Danish population-based cohort study. JAMA
Psychiatry. 2013;70(4):427–35.

13. Sariaslan S, Morawa E, Erim Y. Mental distress in primary care patients:
German patients compared with patients of Turkish origin. Nervenarzt. 2014;
85(5):589–95.

14. Bermejo I, Nicolaus L, Kriston L, Holzel L, Harter M. Culture sensitive analysis
of psychosomatic complaints in migrants in Germany. Psychiatr Prax. 2012;
39(4):157–63.

15. Brandl EJ, Dietrich N, Mell N, Winkler J, Gutwinski S, Bretz J, et al. Clinical
and sociodemographic Differences Between Patients with and without
Migration Background in a Psychiatric Outpatient Service. Psychiatr Prax.
2018;45(7):367-74.

16. Durbin A, Moineddin R, Lin E, Steele LS, Glazier RH. Mental health service
use by recent immigrants from different world regions and by non-
immigrants in Ontario, Canada: a cross-sectional study. BMC Health Serv
Res. 2015;15:336.

17. Schaffer A, Cairney J, Cheung AH, Veldhuizen S, Levitt AJ. Use of treatment
services and pharmacotherapy for bipolar disorder in a general population-
based mental health survey. J Clin Psychiatry. 2006;67(3):386–93.

18. Antoniades J, Mazza D, Brijnath B. Efficacy of depression treatments for
immigrant patients: results from a systematic review. BMC Psychiatry. 2014;
14:176.

19. Koch E, Hartkamp N, Siefen RG, Schouler-Ocak M. German pilot study of
psychiatric inpatients with histories of migration. Nervenarzt. 2008;79(3):328–
39.

20. Hung CI. Factors predicting adherence to antidepressant treatment. Curr
Opin Psychiatry. 2014;27(5):344–9.

21. Wallach-Kildemoes H, Thomsen LT, Kriegbaum M, Petersen JH, Norredam M.
Antidepressant utilization after hospitalization with depression: a
comparison between non-Western immigrants and Danish-born residents.
BMC Psychiatry. 2014;14:77.

Brandl et al. BMC Psychiatry (2020) 20:176 Page 9 of 10

https://doi.org/10.1186/s12888-020-02585-1

22. Forcada I, Pera V, Cruz I, Pifarre J, Serna C, Rue M, et al. Comparison of
immigrant and native-born population adherence to antipsychotic
treatment in a Spanish health region. Community Ment Health J. 2013;49(2):
199–205.

23. Rossom RC, Shortreed S, Coleman KJ, Beck A, Waitzfelder BE, Stewart C,
et al. Antidepressant adherence across diverse populations and healthcare
settings. Depress Anxiety. 2016;33(8):765–74.

24. Diaz E, Woods SW, Rosenheck RA. Effects of ethnicity on psychotropic
medications adherence. Community Ment Health J. 2005;41(5):521–37.

25. Schouler-Ocak M, Bretz HJ, Hauth I, Montesinos AH, Koch E, Driessen M,
et al. Patients of immigrant origin in outpatient psychiatric facilities: a
comparion between Turkish, eastern European and German patients.
Psychiatr Prax. 2010;37(8):384–90.

26. Thompson A, Hunt C, Issakidis C. Why wait? Reasons for delay and prompts
to seek help for mental health problems in an Australian clinical sample.
Soc Psychiatry Psychiatr Epidemiol. 2004;39(10):810–7.

27. Selkirk M, Quayle E, Rothwell N. A systematic review of factors affecting
migrant attitudes towards seeking psychological help. J Health Care Poor
Underserved. 2014;25(1):94–127.

28. Miller MJ, Yang M, Hui K, N. Y. M, Lim RH. A bilinear multidimensional
measurement model of Asian American acculturation and enculturation:
implications for counseling interventions. J Couns Psychol. 2011;58(3):346–
57.

29. Kim BSK. Adherence to Asian and European American cultural values and
attitudes toward seeking professional psychological help among Asian
American college students. J Couns Psychol. 2007;54(4):474–80.

30. Jang Y, Chiriboga DA, Okazaki S. Attitudes toward mental health services:
age-group differences in Korean American adults. Aging Ment Health. 2009;
13(1):127–34.

31. Sun S, Hoyt WT, Brockberg D, Lam J, Tiwari D. Acculturation and
enculturation as predictors of psychological help-seeking attitudes (HSAs)
among racial and ethnic minorities: A meta-analytic investigation. J Couns
Psychol. 2016;63(6):617–32.

32. Calliess IT, Schmid-Ott G, Akguel G, Jaeger B, Ziegenbein M. Attitudes
towards psychotherapy of young second-generation Turkish immigrants
living in Germany. Psychiatr Prax. 2007;34(7):343–8.

33. Knipscheer JW, Kleber RJ. Help-seeking behaviour regarding mental health
problems of Mediterranean migrants in the Netherlands: familiarity with
care, consultation attitude and use of services. Int J Soc Psychiatry. 2005;
51(4):372–82.

34. Bretz J, Sahin D, Brandl EJ, Schouler-Ocak M. Cultural Influence on Attitude
towards Psychotherapy – A Comparison of Individuals of Turkish Origin with
Individuals without Migration Background. Psychother Psychosom Med
Psychol. 2019;69(5):176-81.

35. Ditte D, Schulz W, Schmid-Ott G. Attitude towards psychotherapy in the
Russian population and in the population with a Russian/soviet cultural
background in Germany. A pilot study. Nervenarzt. 2006;77(1):64–72.

36. Schmid-Ott G, Reibold S, Ernst GH, Niederauer HH, Künsebeck HW, Schulz
W, Lamprecht F, et al. Development of a Questionnaire to Assess Attitudes
towards Psychotherapeutic Treatment. Dermatol Psychosom. 2003;(4):187–
93.

37. Petrowski K, Hessel AK, A. Weidner, K., Brähler E, Hinz A. Attitudes towards
psychotherapy in the general population. Psychother Psychosom Med
Psychol 2014;64(2):82–85.

38. Obasi EM, Leong FTL. Psychological distress, acculturation, and mental
health-seeking attitudes among people of African descent in the United
States: A preliminary investigation. J Couns Psychol. 2009;56(2):227–38.

39. Constantine MG, Gainor KA. Depressive symptoms and attitudes toward
counseling as predictors of biracial college Women’s psychological help-
seeking behavior. Women Therapy. 2004;27(1–2):147–58.

40. De Las CC, Penate W. Explaining pharmacophobia and pharmacophilia in
psychiatric patients: relationship with treatment adherence. Hum
Psychopharmacol. 2015;30(5):377–83.

41. De Las CC, Penate W, Cabrera C. Perceived health control: A promising step
forward in our understanding of treatment adherence in psychiatric care. J
Clin Psychiatry. 2016;77(10):e1233–e9.

42. Kondratova L, Konig D, Mlada K, Winkler P. Correlates of negative attitudes
towards medication in people with schizophrenia. Psychiatr Q. 2019;90(1):
159–69.

43. Wagner AW, Bystritsky A, Russo JE, Craske MG, Sherbourne CD, Stein MB,
et al. Beliefs about psychotropic medication and psychotherapy among

primary care patients with anxiety disorders. Depress Anxiety. 2005;21(3):99–
105.

44. Cooper LA, Gonzales JJ, Gallo JJ, Rost KM, Meredith LS, Rubenstein LV, et al.
The acceptability of treatment for depression among African-American,
Hispanic, and white primary care patients. Med Care. 2003;41(4):479–89.

45. Dwight-Johnson M, Sherbourne CD, Liao D, Wells KB. Treatment preferences
among depressed primary care patients. J Gen Intern Med. 2000;15(8):527–
34.

46. Hazlett-Stevens H, Craske MG, Roy-Byrne PP, Sherbourne CD, Stein MB,
Bystritsky A. Predictors of willingness to consider medication and
psychosocial treatment for panic disorder in primary care patients. Gen
Hosp Psychiatry. 2002;24(5):316–21.

47. Givens JL, Houston TK, Van Voorhees BW, Ford DE, Cooper LA. Ethnicity and
preferences for depression treatment. Gen Hosp Psychiatry. 2007;29(3):182–
91.

48. Thorens G, Gex-Fabry M, Zullino DF, Eytan A. Attitudes toward
psychopharmacology among hospitalized patients from diverse ethno-
cultural backgrounds. BMC Psychiatry. 2008;8:55.

49. Denktas S, Koopmans G, Birnie E, Foets M, Bonsel G. Underutilization of
prescribed drugs use among first generation elderly immigrants in the
Netherlands. BMC Health Serv Res. 2010;10:176.

50. Telles C, Karno M, Mintz J, Paz G, Arias M, Tucker D, et al. Immigrant families
coping with schizophrenia. Behavioral family intervention v. case
management with a low-income Spanish-speaking population. Br J
Psychiatry. 1995;167(4):473–9.

51. Cabassa LJ, Lester R, Zayas LH. “It’s like being in a labyrinth:” Hispanic
immigrants’ perceptions of depression and attitudes toward treatments. J
Immigr Minor Health. 2007;9(1):1–16.

52. Tailakh AK, Evangelista LS, Morisky DE, Mentes JC, Pike NA, Phillips LR.
Acculturation, medication adherence, lifestyle behaviors, and blood pressure
control among Arab Americans. J Transcult Nurs. 2016;27(1):57–64.

53. Foster BA, Read D, Bethell C. An analysis of the association between
parental acculturation and children’s medication use. Pediatrics. 2009;124(4):
1152–61.

54. Padilla R, Steiner JF, Havranek EP, Beaty B, Davidson AJ, Bull S. A comparison
of different measures of acculturation with cardiovascular risk factors in
Latinos with hypertension. J Immigr Minor Health. 2011;13(2):284–92.

55. Harfst T, Koch U, Kurtz von Aschoff C, Nutzinger DO, Rüddel H, Schulz H.
Development and validation of a short version of the symptom Checklist-
90-R. DRV-Schriften. 2002;33:71–3.

56. Hamidi S, Akinci F. Measuring efficiency of health Systems of the Middle
East and North Africa (MENA) region using Stchastic frontier analysis. Appl
Health Econ Health Policy. 2016;14(3):337–47.

57. Schulz W, Shin MA, Schmid-Ott G. Attitudes towards psychotherapy in
South Korea and Germany : A cross-cultural comparative study. Nervenarzt.
2018;89(1):51–7.

58. Hogan TP, Awad AG, Eastwood R. A self-report scale predictive of drug
compliance in schizophrenics: reliability and discriminative validity. Psychol
Med. 1983;13(1):177–83.

59. Townsend L, Floersch J, Findling RL. Adolescent attitudes toward psychiatric
medication: the utility of the drug attitude inventory. J Child Psychol
Psychiatry. 2009;50(12):1523–31.

60. Ward C, Rana-Deuba A. Acculturation and adaptation revisited. J Cross-Cult
Psychol. 1999;30(4):422–42.

61. Ward C, Kennedy A. Acculturation strategies, psychological adjustment, and
sociocultural competence during cross-cultural transitions. Int J Intercult
Relat. 1994;18(3):329–43.

62. Mackenzie CS, Scott T, Mather A, Sareen J. Older adults’help-seeking
attitudes and treatment beliefs concerning mental health problems. Am J
Geriatr Psychiatry. 2008;16(12):1010–9.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

Brandl et al. BMC Psychiatry (2020) 20:176 Page 10 of 10

BioMed Central publishes under the Creative Commons Attribution License (CCAL). Under
the CCAL, authors retain copyright to the article but users are allowed to download, reprint,
distribute and /or copy articles in BioMed Central journals, as long as the original work is
properly cited.

InternationalJournal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 764

www.internationaljournalofcaringsciences.org

Original Article

A Case Study Approach: Psychopharmacology for Atypical Antidepressants
Snap Shot

Danita R. Potter, PhD, RN
Program Director, College of Nursing, Northwestern State University, Shreveport, LA. USA

Steven Stockdale, PharmD, BS
Pharmacist, Highlands Behavioral Health System, Denver, CO, USA

Marilyn O’Mallon, PhD, APRN, CNS
Online Program Director, Boise State University, Boise, Idaho, USA

Corresponding Author: Danita R. Potter, PhD, Professor, Program Director, College of Nursing,
Northwestern State University, Shreveport, LA. E-mail: potterd@nsula.edu drpotter41@yahoo.com

Abstract

Major depression is one of most common mental illnesses affecting 6.7% of American adults each year. Depression
leads to disruption in daily lives and life’s pleasures accompanied by serious medical problems which may lead to
suicide (MHA, 2018). The advance practice psychiatric nurse practitioner must conduct an assessment and workup
to rule out disorders such as hypothyroidism, anemia, kindness or renal impairment, cancers, or cardiac illness
(Weber & Estes, 2016). Children with a traumatic childhood, particularly those that constitute major setbacks in life
are at greater risk for depression later in life. The aim of this paper is to discuss the review of diagnostic criteria and
considerations, pseudonym case study, over and review of general indications of atypical antidepressants, and
conclusions and implications for the case approach.

Key words: antidepressants, psych mental health nurse practitioners, major depression, atypical

Introduction

Major depression is one of most common
mental illnesses affecting 6.7% of American
adults each year. Depression leads to
disruption in daily lives and life’s pleasures
accompanied by serious medical problems
which may lead to suicide (MHA, 2018).
According to the latest DSM-V, depression
does not discriminate it affects persons from
every walk of life including children and the
elderly (APA, 2013). Major Depressive
Disorder (MDD), also known Clinical
Depression), is characterized by an
inescapable and ongoing low mood often

accompanied by low self-esteem, loss of
interest or pleasure in activities than a person
used to fine enjoyable (MHA, 2018). The
advance practice psychiatric nurse practitioner
must conduct an assessment and workup to
rule out disorders such as hypothyroidism,
anemia, kindness or renal impairment, cancers,
or cardiac illness (Weber & Estes, 2016). Risk
and prognostic factors include temperamental,
environmental, genetic and physiological, and
course modifiers. Neuroticism, a negative
affectivity) is well-established risk factor for
the onset of major depressive disorder, and
high levels appear to render individual more
likely to develop depressive episodes in

International Journal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 765

www.internationaljournalofcaringsciences.org

response to stressful life events. Children with
a traumatic childhood, particularly those that
constitute major setbacks in life are at greater
risk for depression later in life. First-degree
family members of individual with major
depressive disorder have a two to fourfold
higher risk than the general population to
develop depression. Any major non-mood
disorders can increase the risk of developing
depression later in life (APA, 2013). The Food
and Drug Administration (FDA) approved five
atypical antidepressants used to treat
depression. These five drugs are Bupropion
(Wellbutrin, Forfivo XL, Aplenzin),
Mirtazapine (Remeron), Nefazodone (Serzone,
Dutonin), Trazodone (Desyrel, Oleptro), and
Vortioxetine (Trintellix). The aim of this paper
is to discuss the review of diagnostic criteria
and considerations, case study, over and
review of general indications of atypical
antidepressants, and conclusions and
implications for the case approach.

Review of Diagnostic Criteria and
Considerations: MDD on is an episodic,
frequently recurring syndrome requiring five
or more criteria present for two weeks. One of
these nine criteria must be either persistent
depressed mood or pervasive anhedonia. Other
symptoms can include sleep disturbance, loss
of appetite loss or gain and or weigh gain loss
or gain, fatigue, psychomotor retardation or
agitation including feelings of worthlessness
or thoughts of suicide (DSM-5). The DSM-5
includes a note indicated to do not include
symptoms that are clearly attributable to
another medical condition. Coding and
recording procedures according to the DSM-5
indicates that for recurrent moderate episode
296.32 (F33.1) (APA, 2013).

Neurobiology: The neurobiology of
depression has been evolving and changing
over the last decade. In the classic monoamine
theory of depression, the emphasis was on a
decadency of norepinephrine (NE, serotonin
(5HT), and dopamine (DA). Although this
theory corresponds to the use of current

antidepressant, there is little data to support it
and some research results give conflicting
evident (Stahl, 2013, Cogburn, 2018). This
theory has been supplemented with a more
complicated view that involves how the
neurotransmitter symptom regulates
information process in key areas of the
neurological system related to symptoms of
depression (Stahl, 2013).

Assessment and Screening: According to
Weber and Estes (2016), screening and
assessment for persons suspected with
depressive mood or probable diagnosis of
depression must go through ha workup to
exclude disorders other possible illness. In
addition to a work-up, the clinician can use an
important screen tool which can help the
clinical rule out depressive disorder or bipolar
disorder. This is the Mood Disorder
Questionnaire (MDQ). This tool can help the
provider form a differentiation whether the
patient has had prior hypomania or manic
episodes which may indicate bipolar disorder.
Another screening tool is the Patient Health
Questionnaire (PHQ-9) and the Center for
Epidemiological Studies Depression Scale
(CEDS) has both been used in primary care for
depression and can be used in the waiting
room to screen for mood disorders. Both used
as screening tools and should not be used for
diagnostic purposes. When the clinician
chooses tools for diagnostic purposes, the tools
should be reliable and valid such as the Beck
Depression Inventory and the Inventory
Depressive Symptomatology (IDS) and Self
Report. These have been used to assist he
clinician to diagnose and manage progress of
treatment (Weber & Estes, 2016).

Another major responsibility of the clinician is
screening for the presence of suicidality and
level of or severity o risk of suicide. Once tool
to assess for suicide is the Substance Abuse
and Mental Health Services Administration
(SAMHSA, 2018) has developed a five-step
suicide assess, evaluation, and triage method
to identify both risk and protective factors. The

International Journal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 766

www.internationaljournalofcaringsciences.org

Suicide assessment Five-Step Evaluation and
Triage, SAFE-T Assessment of Suicide risk
incudes
1) identify risk factors; 2) identify protective
factors; 3) conduct suicide inquiry; 4)
determine risk or level of intervention, and 5)
documentation (Weber & Estes, 2016, p.899).
Children and the elderly are the most
vulnerable when it comes to antidepressants
with increased risks of suicide. Cautions exist
to use SSRIS with children or teenagers. As
suicidal depressed patient begins to improve
with treatment, the act of suicide is carried out
due to an increase in physical energy (as cited
in Weber & Estes, p. 909). A faux/pseudonym
case presentation is discussed below for
learning purposes. The actual case does not
exist.

Case Study (pseudonym)

Ms. T. is a 72-year-old African American
woman who is recently divorced with 2
children and 5 grandchildren. She was
employed by Tell Tell South Metrics of
American for 15 years and now enjoys
retirement. Her hobbies include going to the
casinos to gamble four times a week with
friends. She takes her retirement check and
exhausts it all on gambling, leaving no money
to pay her bills or personal items. She lives
with her mother in a rural community. In the
last 6 weeks, her oldest daughter noticed that
Ms. T does not want to go gambling anymore
and she is often very sad and uninterested in
hanging out with friends. The daughter
decided to bring her to a therapist. The
waiting room assessment tool was used to
screen for any possible behaviors that would
warrant further evaluation. A suicide screen
tool was used to assess risk of suicidal level
and safety. Denies any recent losses or deaths
in family. Patient denied suicidal thoughts.
Daughter reports dry and irritated skin to
lower legs, vital signs 120/82, 80, 12, 98.2.
weight-265 lbs.

Upon interviewing the Ms. T. and her
daughter, the daughter indicated that for the
last month her mother has been very tired
staying in her room on most days,
disinterested in her normal routines or
hobbies, neglecting hygiene, and increased
appetite. The patient responses to yes or no
answers and her head is face down to the floor
most of the session. Patient denies pain, SOB,
her past medical history is without significant
falls, head injury, heart/respiratory conditions,
NKDA. Upon mini-mental (Mini-Cog):
Appearance: Hair unkept, clean today
(daughter stated that earlier she gave her a
bath) and dressed in pants and t-shirt. Mood
was described as “been feeling down and
out”, Affect is flat. Memory, language,
attention and executive functioning were
intact. Old records revealed she had a prior
diagnosis of MDD and upon asking the
daughter she replied, “oh yeah! Momma did
go to the doctor in her early 40s when she was
that medicine got momma messed up and she
gained a lot of weight. She ate all day plus my
dad would complain she wouldn’t let him
touch her”.

Old records indicated she had been previously
treated for depression with bupropion and
developed a rash and noncompliance with it.
Today’s visit labs reveal chemistry levels
within normal limits, Complete blood count
(CBC) within normal limits, cholesterol within
normal limits, and glucose within normal
limits. Body mass index greater than 24 with a
fasting blood sugar of 112 mg/dl. Liver
function studies within normal limits, Bilirubin
Urea Nitrates (BUN) and creatinine within
normal limits. Denies suicidal attempts or
thinking in past or currently.

Review of General Indications

Mirtazapine (Remeron) is Food and Drug
Administration (FDA) approved for Major
Depressive Disorder (MDD). Off-label uses
may include Panic Disorder, Generalized
(GAD) and Posttraumatic Stress Disorder

International Journal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 767

www.internationaljournalofcaringsciences.org

(PTSD). leads to rapid and sustained
improvement in depressive symptoms and is
effective in subgroups of depressed patients,
particular anxious patient and those with
melancholic depression treatment -resistant
depression, geriatric depression, depression
and anxiety associated with alcohol
dependence, and agitated elderly patients.
Mirtazapine has a range of clinically useful
applications including improving sleep,
antiemetic, appetite improvement,
management of pain, weight gain (Alam,
Voronovich, & Carley, 2013). A snap shot
(overview) of atypical antidepressants drugs,
developed by Potter (2018) are provided in
Table 1. 1. It provides information on drug
class, generic name, brand name, mechanism
of action, FDA approved indications and off-
label indications, dosing, side effects including
black box warnings, special populations
precautions, and drug interactions.

According to Stahl (2017), Mirtazapine boosts
neurotransmission and blocks alpha 2
adrenergic presynaptic receptor, increases
serotonin neurotransmission, and blocks
5HT2C, 5HT3, and histamine 1 receptors.
Indications for this drug includes MDD,
Seasonal affective disorder, Nicotine
addiction, Bipolar Disorder, Attentional
Deficit Hyperactivity Disorder (ADHD), and

sexual disorders (Stahl, 2017). This
medication was chosen to treat Ms. T with
because of it side effect profile. Out of all the
other atypical antidepressants, Mirtazapine
was found to have the fewest side effects,
adverse reactions and unique mechanisms of
action then some of the other atypical
antidepressants.

Conclusions and Implications for Advanced
Practiced Registered Nurses (APRNs): After
ruling out Bipolar and other psychiatric
disorders along with anemic, and suicidality, I
started Ms. T. on Mirtazapine 15 mg by mouth
every night. Because it is safe long-term and
not habit forming, Mirtazapine maybe
tolerated better than Bupropion. The patient
presented to the clinic to day with an existing
raised generalized rash to her skin, thus
Bupropion has a warning of potential for
Steven’s Johnston Syndrome (Stahl, 2018).
Mirtazapine may also cause some notable side
effects of lowering white blood cell count,
may increase cholesterol, may cause
photosensitivity, included teaching patient and
her daughter on side effects, skin protective
measures, and check weekly labs CBC, LDL
& HDL cholesterol, triglycerides, liver
function studies, glucose, monitor body mass
index (BMI), screen for suicidal ideation each
visit. Follow up visit next week.

International Journal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 768

www.internationaljournalofcaringsciences.org

Table 1.1. Psychopharmacology for Atypical Antidepressants: Snap Shot

Class Atypical
Antidepressants

Generic
Agent

Brand
Name(s)

MOA Indications
*FDA

Dosing SE/ADR/BBW Drug-
Interactions

Dopamine
reuptake inhibitor
& releaser, NDRI
(NE DA reuptake
inhibitor

1.Bupropion Wellbutrin
Forfivo XL,
Aplenzin

Boosts
neurotransmitt
ers NE & DA,
blocks NE
reuptake pump
increasing NE
neurotransmis
sion, blocks
DA reuptake
increasing DA
neurotransmis
sion,

*MDD,
*Seasonal
affective
disorder,
*Nicotine
addiction,
Bipolar,
ADHD,
Sexual
disorders

225-450 mg in
3 divided
doses
SR 200-
445mg in 2
divided doses
XL 150mg,
300mg,
450mg
hydrobromide
ER 174 mg,
378mg, 522
mg

Dry mouth,
constipation,
nausea, weight
loss, anorexia,
tremor, HA,
constipation,
sweating, Abd.
Pain, HTN, rash,
rare Seizures,
Steven-Johnston
Syndrome,
Hypomania, rare
Suicidal ideation

Tramadol,
MAOIs,
Fluoxetine, SSRIs,
Warfarin CYP450
2D6, CYP450
3A4 inhibition,
Haldol, general
anesthetics, HTN
increases with
nicotine
TCAs, Lithium,
Levodopa
Zyban
HX Seizures,
Thioridazine
Proven allergy to
Bupropion

serotonin, NE
receptor
antagonist, Alpha
2 antagonist,
NaSSA
(noradrenaline &
specific
serotonergic
agent)

2.Mirtazapine Remeron boosts
neurotransmitt
ers 5HT &
NE, blocks
alpha 2
adrenergic
presynaptic
receptor,
increases 5HT
neurotransmis
sion, blocks
5HT2C,
5HT3, & H1
receptors

*MDD
PD, GAD,
PTSD

15-45 mg at
HS

Low WBC,
photosensitivity,
Avoid Alcohol,
Risk2Benefits
4Children,
Possible
activating SEs,
Suicidal
iieeatin4Children
& Adolescents,
Avoid if known
allergy-Remeron

MAOIs,
Tramadol, may
cause SS

serotonin receptor
antagonist, SARI

3.Nefazodone Dutononin,
Serzone

blocks
serotonin 2A
receptors
potently,
blocks
serotonin
reuptake pump
and NE
reuptake pump

*Depression
,
PD, PTSD

300-600mg/d Hepatotoxicity,
HX Seizures,
Fetal SS,
Risk2Benefits
4Children,
Possible
activating SEs,
Suicide, Cardiac
Problems
Elderly
Hepatic & Renal

Tramadol,
MAOIs,
Fluoxetine, SSRIs,
Warfarin CYP450
2D6, CYP450
3A4 inhibition,
Haldol, general
anesthetics

serotonin receptor
antagonist (S-
MM), SARI

4.Trazodone Oleptro
Desyrel

blocks 2A
receptors,
blobs
serotonin
reuptake pump

*Depression
,
insomnia
(primary/sec
ondary),
anxiety

150-600mg/d
150-375 mg/d
ER

N/V/, edema,
blurred vision,
dry mouth,
constipation,
dizziness,
sedation, fatigue,
HA,
incoordination,
tremor, syncope,
rare rash, sinus
bradycardia
(long-term)

Tramadol,
MAOIs,
Fluoxetine, SSRIs,
Warfarin

serotonin
multimodal (S-
MM), Multimodal
antidepressant

5.Vortioxetine Trintellix increases
release of
several
neurotransmitt
ers: serotonin,
NE, DA,
Glutamate,
Acetylcholine,
Histamine

*MDD
GAD,
Cognitive
S/S of
Depression,
Geriatric
depression

5-20 mg/d N/V,
constipation,
sexual
dysfunction, rare
seizures, rare
mania & SI

Tramadol,
MAOIs, CYP450
2D6,, Warfarin,
NSAIDS

International Journal of Caring Sciences January – April 2020 Volume 13 | Issue 1| Page 769

www.internationaljournalofcaringsciences.org

References

Alam, A., Voronovich, Z., & Carley, J. A. (2013). A
review of therapeutic uses of Mirtazapine in
psychiatric and medical conditions. The Primary
Care Companion for CNS Disorders, 15(3),
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC161
726/pdf/20030100s00006p55

American Psychiatric Association (2013). Diagnostic
and statistical manual of mental disorders. (5th
Eds.). Arlington, VA: American Psychiatric
Publishing.

Cogburn, M. (2018). Antidepressants PowerPoint
Lecture. Northwestern State University College of
Nursing.

Elley, C. R., Dawes, D., Dawes, M., Price, M., Draper,
H., Goodyear-Smith, F. 2014). Screening for
lifestyle and mental health risk factors in the waiting
room: Feasibility study of the Case-fining Health
Assessment Tool. Canadian Family Physician
60(11), 527-534.

Mental Health America (2013). Basic facts about
Depression.
http://www.mentalhealthamerica.net/conditions/depr
ession

Potter, D. R. (2018) A case study approach:
Psychopharmacology for Atypical Antidepressants-
Snap Shot. In Mark Cogburn’s Course
Psychopharmacology, Summer 2018, Northwestern
State University, Louisiana.

Stahl, S. M. (2013). Stahl’s essential
psychopharmacology: Neuroscientific basis and
practical applications. (4th Eds.). UK, Cambridge:
University Press.

Stahl, S. M. (2017). Stahl’s essential
psychopharmacology: Prescriber’s guide. (6th Eds.).
UK Cambridge: University Press.

Weber, M. & Estes, K. (2016). Anxiety and depression,
(pp. 897-912), In T. Woo & M. V. Robinson’s
Pharmacotherapeutics: For advanced practice
nurse prescribes. (4th Eds.). Philadelphia, PA: F. A.
Davis.

Copyright of International Journal of Caring Sciences is the property of International Journal
of Caring Sciences and its content may not be copied or emailed to multiple sites or posted to
a listserv without the copyright holder’s express written permission. However, users may
print, download, or email articles for individual use.

Psychopathology

Feist, G. J., Dostal, D., & Kwan, V. (2021). Psychopathology in world-class artistic and scientific creativity. Psychology of Aesthetics, Creativity, and the Arts. https://doi.org/10.1037/aca0000440.supp (Supplemental)

Lee, S. A., & Jobe, M. C. (2022). Does fear mediate the neuroticism-psychopathology link for adults living through the COVID-19 pandemic? Clinical Medicine Insights: Psychiatry, 1–5. https://doi.org/10.1177/11795573211069912

Lantagne, A., & Furman, W. (2021). A dyadic perspective on psychopathology and young adult physical dating aggression. Psychology of Violence, 11(6), 569–579. https://doi.org/10.1037/vio0000386.supp (Supplemental)

Schreuder, M. J., Wigman, J. T. W., Groen, R. N., Wichers, M., & Hartman, C. A. (2021). On the transience or stability of subthreshold psychopathology. Scientific Reports, 11(1), 23306. https://doi.org/10.1038/s41598-021-02711-3

Hawkins-Elder, H., & Ward, T. (2021). Describing disorder: The importance and advancement of compositional explanations in psychopathology. Theory & Psychology, 31(6), 842–866. https://doi.org/10.1177/09593543211021157

Terry, C., & Lecci, L. (2021). Examining cognitive performance and psychopathology in individuals undergoing parental competency evaluations. Professional Psychology: Research and Practice. https://doi.org/10.1037/pro0000436

Developmental Psychology

Bauger, L., & Bongaardt, R. (2018). Structural developmental psychology and health promotion in the third age. Health Promotion International, 33(4), 686–694. https://doi.org/10.1093/heapro/daw104

Legare, C. H., Clegg, J. M., & Wen, N. J. (2018). Evolutionary developmental psychology: 2017 redux. Child Development, 89(6), 2282–2287. https://doi.org/10.1111/cdev.13018

Bland, A. M., & DeRobertis, E. M. (2020). Maslow’s unacknowledged contributions to developmental psychology. Journal of Humanistic Psychology, 60(6), 934–958. https://doi.org/10.1177/0022167817739732

Cochet, H., & Guidetti, M. (2018). Contribution of developmental psychology to the study of social interactions: Some factors in play, joint attention and joint action and implications for robotics. Frontiers in Psychology, 9. https://doi.org/10.3389/fpsyg.2018.01992

Koops, W., & Kessel, F. (2017). Developmental psychology without positivistic pretentions: An introduction to the special issue on historical developmental psychology. European Journal of Developmental Psychology, 14(6), 629–646. https://doi.org/10.1080/17405629.2017.1382344

Kline, M. A., Shamsudheen, R., & Broesch, T. (2018). Variation is the universal: making cultural evolution work in developmental psychology. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 373(1743). https://doi.org/10.1098/rstb.2017.0059

Cognitive Psychology

Schmidt, H. G., & Mamede, S. (2020). How cognitive psychology changed the face of medical education research. Advances in Health Sciences Education, 25(5), 1025–1043. https://doi.org/10.1007/s10459-020-10011-0

Lloyd, M. E. (2020). Sometimes a demo is not just a demo: When demonstrating cognitive psychology means confronting assumptions. Scholarship of Teaching and Learning in Psychology. https://doi.org/10.1037/stl0000192

Yao, N., & Wang, L. (2020). Application of game activities in mental health education of kindergartens based on cognitive psychology. Revista Argentina de Clínica Psicológica, 29(2), 871–877.

Xie, D. (2020). Application of psychological preference and emotional guidance in the performance of film and television art: An analysis based on based on cognitive psychology. Revista Argentina de Clínica Psicológica, 29(2), 854–859.

Li, Y. (2020). Analysis on uncertainties in judicial decision based on cognitive psychology. Revista Argentina de Clínica Psicológica, 29(1), 1016–1021.

Corral, D., Healy, A. F., Rozbruch, E. V., & Jones, M. (2019). Building a testing-based training paradigm from cognitive psychology principles. Scholarship of Teaching and Learning in Psychology, 5(3), 189–208. https://doi.org/10.1037/stl0000146.supp (Supplemental)

Psychopharmacology

Lopez-Vergara, H. I., Zapolski, T. C. B., & Leventhal, A. M. (2021). Intersection of minority health, health disparities, and social determinants of health with psychopharmacology and substance use. Experimental and Clinical Psychopharmacology, 29(5), 427–428. https://doi.org/10.1037/pha0000522

Bilbul, M., Paparone, P., Kim, A. M., Mutalik, S., & Ernst, C. L. (2020). Psychopharmacology of COVID-19. Psychosomatics: Journal of Consultation and Liaison Psychiatry, 61(5), 411–427. https://doi.org/10.1016/j.psym.2020.05.006

MARVASTI, J. A., & OLIVIER, C. C. (2020). Psychopharmacology for the Psycho-Historian: The Evils of “Big Pharma,” Lobbying, Corruption and Serious Side Effects of Medications. Journal of Psychohistory, 48(2), 100–115.

Caixeta, L., & Caixeta, V. M. (2019). Therapeutic synergism: How can psychopharmacology improve cognitive rehabilitation? Dementia & Neuropsychologia, 13(4), 422–426. https://doi.org/10.1590/1980-57642018dn13-040009

Brandl, E. J., Dietrich, N., Mell, N., Winkler, J. G., Gutwinski, S., Bretz, H. J., & Schouler-Ocak, M. (2020). Attitudes towards psychopharmacology and psychotherapy in psychiatric patients with and without migration background. BMC Psychiatry, 20(1), 176. https://doi.org/10.1186/s12888-020-02585-1

Potter, D. R., Stockdale, S., & O’Mallon, M. (2020). A Case Study Approach: Psychopharmacology for Atypical Antidepressants Snap Shot. International Journal of Caring Sciences, 13(1), 764–769.

Turn in your highest-quality paper
Get a qualified writer to help you with

“ Integrative Literature Review ”

Get high-quality paper

NEW! AI matching with writer

Order an Essay Now & Get These Features For Free:

Turnitin Report

Formatting

Title Page

Citation

Outline

Place an Order