DQ

Assignment

ETHICAL LEADERSHIP AND UNETHICAL EMPLOYEE
BEHAVIOR: A MODERATED MEDIATION MODEL

CHENJING GAN
Ningbo University

I explored the role of employee moral justification as a cognitive mediator in the relationship
between ethical leadership and unethical employee behavior, and then investigated employee
moral identity as a moderator of this indirect relationship

.

I based my moderated meditation
model on social learning theory and tested it by analyzing data collected from 271 employees
of 17 firms in China at 2 time points separated by approximately 3 weeks. The results showed
that the negative indirect relationship between ethical leadership and unethical employee
behavior through moral justification was significant when employee moral identity was
strong. Theoretical and practical implications of the findings are discussed.

Keywords: ethical leadership, unethical employee behavior, moral justification, moral
identity, social learning.

Various types of unethical behavior occur in virtually every sector of society,
and ethical issues have become topics of great interest in both mass media and
the academic community (O’Fallon & Butterfield, 2011, 2012). The concept
of ethical leadership, which is defined as “the demonstration of normatively
appropriate conduct through personal actions and interpersonal relationships, and
the promotion of such conduct to followers through two-way communication,
reinforcement, and decision making” (Brown, Treviño, & Harrison, 2005, p. 120),
has received much attention from many scholars in recent years. Researchers
have mainly drawn upon social learning theory (Bandura, 1977) to explain
the influence of ethical leadership on unethical employee behavior, whereby
employees learn what behaviors are expected, rewarded, and punished by

SOCIAL BEHAVIOR AND PERSONALITY, 2018, 46(8), 1271–128

4

© 2018 Scientific Journal Publishers Limited. All Rights Reserved.
https://doi.org/10.2224/sbp.7328

1271

Chenjing Gan, Business School, Ningbo University.
This research was supported by the Natural Science Foundation of Zhejiang Province (LQ18G020002).
Correspondence concerning this article should be addressed to Chenjing Gan, Business School,
Ningbo University, No. 18 Fenghua Road, Ningbo 3125211, People’s Republic of China. Email:
ganchenjing@nbu.edu.cn

ETHICAL LEADERS AND UNETHICAL EMPLOYEES127

2

observing and imitating ethical leaders’ behaviors (Mayer, Aquino, Greenbaum,
& Kuenzi, 2012). However, it remains unclear how employees learn to inhibit
their unethical behavior, or why some employees learn from ethical leaders
and some do not. In social learning theory it is posited that the effects of
reinforcement are cognitively mediated, which means that learning cannot take
place without awareness of what is being reinforced (Bandura, 1977). In other
words, employees learn to eliminate their unethical behaviors only when they
are aware of which behaviors are unacceptable to ethical leaders. Thus, in the
current study I explored employee moral justification as a cognitive mediating
mechanism in the relationship between ethical leadership and unethical employee
behavior, and examined employee moral identity as a moderator of this indirect
relationship.

Moral justification involves cognitive reconstruction of an ethically questionable
behavior and is an attempt by the individual to legitimize this behavior rather
than to assume responsibility for an outcome (Barsky, 2011; Green, 1991; Hunt
& Vitell, 1986; Stevens, Deuling, & Armenakis, 2012; Vitell, Keith, & Mathur,
2011). In other words, moral justification involves the individual reconstructing
harmful behavior to make it appear personally and socially acceptable (Aquino,
Reed, Thau, & Freeman, 2007; Detert, Treviño, & Sweitzer, 2008). When
employees are engaged in moral justification, they cognitively legitimize their
unethical behavior and are not aware that the behavior is forbidden; thus, the
learning effect of ethical leadership on unethical employee behavior is absent.
Therefore, I proposed that moral justification would act as a cognitive mediator
of the relationship between ethical leadership and unethical employee behavior.

According to social learning theory, people self-regulate by setting behavioral
standards for themselves and responding to their own behavior in accordance
with their self-imposed demands (Bandura, 1977). Researchers have posited that
unethical behaviors occur only when individuals’ self-regulatory mechanisms
are deactivated through several cognitive mechanisms, such that they are unable
to anticipate and judge their actions in comparison to a set of internal moral
standards (Bandura, 1986; Detert et al., 2008; Moore, Detert, Klebe Treviño,
Baker, & Mayer, 2012; Stevens et al., 2012). This being so, some employees may
be reluctant to learn from an ethical leader, as the behaviors of that leader are
inconsistent with the employee’s own behavioral standards.

Moral identity refers to the extent to which one’s self-concept incorporates
the importance of being a moral person (Aquino & Reed, 2002), and it affects
individual self-regulatory mechanisms by setting standards for individual
behavior (Aquino, Freeman, Reed, Lim, & Felps, 2009; Bergman, 2002; Mulder
& Aquino, 2013; Reynolds & Ceranic, 2007). The self-imposed demands of
employees with a weak moral identity are quite different from those of ethical
leaders, and these employees are reluctant to learn from their ethical leaders.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 127

3

Therefore, I argued that the negative effect of ethical leadership on unethical
employee behavior through moral justification would be diminished for
employees with a weak moral identity. Moral identity may operate as a moderator
that strengthens the negative relationship between ethical leadership and moral
justification, thereby further inhibiting unethical employee behavior.

Figure 1 illustrates the proposed moderated mediation model and provides an
overview of the study.

Moral identity

Ethical leadership Moral justification
Unethical employee

behavior

Figure 1. Mediated moderation research model.

Theoretical Background and Hypotheses

Ethical Leadership, Moral Justification, and Unethical Behavior
Brown and Treviño (2006) elaborated on a proposition that moral persons and

moral managers are the two components of ethical leadership. As moral people,
ethical leaders have desirable characteristics, such as being fair and trustworthy.
As moral managers, ethical leaders use transactional efforts to hold followers
to acting according to moral standards, such as by communicating ethics and
punishing followers’ unethical behavior. The influence of ethical leadership on
employee moral justification can also be explained through these two aspects
of leadership style (Bandura, 1986; Brown & Treviño, 2006; Liu, Long, & Loi,
2012). First, as moral people, ethical leaders pay attention to, and consistently
align their own behavior with, moral standards. From observing their leaders’
behaviors, employees learn that unethical behavior is morally unacceptable and,
therefore, do not justify it (Brown & Treviño, 2006; Brown et al., 2005). Because
ethical leaders are credible role models for ethical behaviors, employees cannot
dilute their own faulty actions by diffusing responsibility; this inhibits employee
moral justification (Liu et al., 2012). Second, as moral managers, ethical leaders
demand that employees have high ethical standards, are accountable for the moral
consequences of their behaviors, and care for others’ interests (Brown & Treviño,
2006; Brown et al., 2005). However, individuals use moral justification to make
harming others appear acceptable to both themselves and others. Individuals
who engage in moral justification are concerned about their self-interests and
shirk responsibility for their unethical behaviors (Moore et al., 2012; Vitell et al.,

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1274

2011). The demand and influence of ethical leadership are contradictory to the
cognitive process of moral justification and inhibit individuals from engaging in
moral justification. Therefore, I argued that ethical leadership would decrease
employee moral justification.

As already described, in social learning theory it is posited that learning
cannot take place without awareness of what is being reinforced (Bandura,
1977). Employees who engage in moral justification do not sincerely approve
of ethical leaders’ behaviors and do not learn to inhibit their own unethical
behaviors. Furthermore, researchers have described how unethical behaviors are
inhibited when individuals’ self-regulatory mechanisms are working effectively,
as engaging in unethical behaviors is inconsistent with the individual’s internal
moral standard, which can result in feelings of guilt (Bandura, 1986; Moore et
al., 2012, Vitell et al., 2011). However, moral justification deactivates one’s self-
regulatory mechanism by reducing the negative impact of unethical behaviors
on self-concept through finding excuses to legitimize one’s unethical behavior
(Barsky, 2011; Hunt & Vitell, 1986; Stevens et al., 2012; Vitell et al., 2011).
Thus, the inconsistency between an individual behaving unethically and his or
her internal moral standard is reduced through moral justification. Therefore,
I argued that, through the reframing of their cognitive perceptions, moral
justification would allow individuals to engage in unethical behavior without
experiencing distress. In some empirical studies, scholars have also found that
moral justification is positively related to unethical decision making (Stevens et
al., 2012) and unethical behavior (Barsky, 2011; Moore et al., 2012). On the basis
of these arguments and findings, I proposed the following hypothesis:
Hypothesis 1: Employee moral justification will mediate the negative relationship
between ethical leadership and unethical employee behavior, such that ethical
leadership will decrease employee moral justification and employee moral
justification will increase unethical employee behavior.

Moral Identity as a Moderator
As already described, according to social learning theory, people self-regulate

by setting themselves certain behavioral standards. Moral identity influences
an individual’s self-regulatory mechanism by setting the parameters for his or
her moral standard, which creates a need for the individual to act consistently
with his or her identity (Bergman, 2002; Reynolds & Ceranic, 2007). The high
ethical standards set by ethical leaders for their employees are inconsistent with
the self-concept and behavioral standards of employees with a weak moral
identity (Aquino et al., 2009; Mulder & Aquino, 2013; Reynolds & Ceranic,
2007). Therefore, these employees are not susceptible to the influence of ethical
leadership and continue to avoid responsibility for their own unethical behaviors
through moral justification. Additionally, although ethical leaders are moral

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 127

5

persons who behave ethically and appropriately, employees with a weak moral
identity still find ways to justify their unethical behavior, such as taking an “every
man for himself” attitude. Thus, a weak moral identity diminishes the effect of
ethical leadership on employees’ moral justification. Employees with a strong
moral identity have high internal moral standards that are similar to those of
their leaders, and easily accept and follow their leaders’ moral code; thus, they
are susceptible to the influence of ethical leadership. Therefore, I proposed the
following hypothesis:
Hypothesis 2: Employee moral identity will moderate the relationship between
ethical leadership and employee moral justification, such that this relationship
will be weaker when employee moral identity is weaker.

I further argued that employee moral identity would moderate the indirect
relationship between ethical leadership and unethical employee behavior.
The behavioral standards of employees with a weak moral identity can be
distinguished from those of their ethical leader; therefore, they are less likely
to learn cognitively from their ethical leaders (Bergman, 2002; Reynolds &
Ceranic, 2007). In this condition, the effect of ethical leadership on employee
moral justification is diminished, promoting unethical employee behavior. In
contrast, employees’ strong moral identity will strengthen the effect of ethical
leadership on employee moral justification, further inhibiting unethical employee
behavior. Therefore, I offered the following hypothesis:
Hypothesis 3: The indirect relationship between ethical leadership and unethical
employee behavior through employee moral justification will be conditional on
employee moral identity, such that this indirect relationship will be weaker when
employee moral identity is lower.

Method

Participants and Procedure
I obtained my data from 271 employees of 17 firms in China, collecting

these data at two points in time separated by approximately 3 weeks. The firms
belonged to the construction, traditional manufacturing, financial services, and
advertising industries. A list of people who could be participants was provided to
me by each firm’s human resources department. Each participant was assigned
a unique survey code so that I could match the data from Times 1 and 2. Each
questionnaire was put into an envelope with a clear plastic window to assure
participants’ full confidentiality. I also informed the participants that their data
would be used for academic research purposes only.

Surveys were distributed and collected by my research team. Ethical leadership,
moral identity, and demographic information (age, gender, and level of
education) were assessed at Time 1, and moral justification and the participant’s

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1276

own unethical behavior were assessed at Time 2. At Time 1 423 participants
completed the survey, and 355 participants completed the survey at Time 2.
I discarded some records for unmatched Time 1 and Time 2 survey pairs;
ultimately, complete responses were obtained from 271 employees. Among the
participants, 60% were men and 40% were women, and the average age was 30
years (SD = 8.35).

Measures
The original relevant scales were written in English, so these were translated

into Chinese following Brislin’s (1980) recommendations. That is, the scales
were first translated from English to Chinese by one bilingual speaker. Then,
another translator without access to the original survey back-translated the same
items from Chinese into English. The second translator was asked to comment
on any items that were perceived to be ambiguous. Both translators were native
Chinese speakers who had studied abroad in English-speaking countries and
had scored over 7 points on the International English Language Testing System
(Brislin, 1980). All scales were anchored on a 5-point Likert scale that ranged
from 1 = strongly disagree to 5 = strongly agree.

Ethical leadership. I measured ethical leadership with the 10-item Ethical
Leadership Scale ( = .95) developed and validated by Brown et al. (2005).
Sample items include “My supervisor listens to what employees have to say”
and “My supervisor sets an example of how to do things the right way in terms
of ethics.”

Moral justification. I assessed moral justification ( = .82) with a four-item
scale developed by Bandura, Barbaranelli, Caprara, and Pastorelli (1996).
Sample items include “It is all right to lie to keep yourself out of trouble” and “It
is all right to fight to protect yourself.”

Moral identity. I measured moral identity using Aquino and Reed’s (2002)
five-item internalization subscale ( = .92). Participants were asked to read a
list of nine characteristics (i.e., caring, compassionate, fair, friendly, generous,
helpful, hardworking, honest, and kind) and to visualize a person (which could
be the person him/herself or someone else) who has these characteristics. Once
participants had a clear image of this person, they were asked to rate their level of
agreement with a set of statements. Sample items include “Being someone who
has these characteristics is an important part of who I am” and “I strongly desire
to have these characteristics.”

Unethical behavior. I measured unethical behavior with a 16-item scale
( = .92) developed by Zey-Ferrell, Weaver, and Ferrell (1979). The original
scale contains 17 items. After considering the pilot study results I integrated two
items (“I pad an expense account more than 10%” and “I pad an expense account
up to 10%”) into one item: “I pad an expense account.” Sample items include

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1277

“I do personal business on company time” and “I pass blame for errors to an
innocent coworker.”

Control variables. Following the procedure used in previous research
on unethical employee behavior (O’Fallon & Butterfield, 2011, 2012), I
controlled for three demographic variables to better estimate the effect sizes
of the hypothesized relationships. Specifically, I measured and controlled for
employees’ gender (male = 0, female = 1), age, and level of education (junior
high school degree or lower = 1, high school or technical school degree = 2,
college degree = 3, bachelor’s degree = 4, master’s degree or higher = 5), because
these can influence unethical employee behavior.

Results

Validity of the Measures
I used Amos 17.0 to conduct a series of confirmatory factor analyses to

examine whether the study variables captured distinct constructs (Anderson &
Gerbing, 1988). The four-factor measurement model fitted the data well: chi
square (2) = 1046.18, degrees of freedom (df) = 553, comparative fit index
(CFI) = .92, incremental fit index (IFI) = .92, Tucker–Lewis index (TLI) = .91,
root mean square error of approximation (RMSEA) = .06, and all indicators had
statistically significant factor loadings (p < .01). Compared to the four-factor model, the one-factor measurement model fitted the data poorly: 2 = 3815.55, df = 559, CFI = .44, IFI = .45, TLI = .37, RMSEA = .15. The chi-square difference compared with the four-factor model was significant (Δ2 = 2769.37, p < .01), indicating distinctly different factors. Because data on ethical leadership and employee moral identity were collected from the same source at Time 1, and data on employee moral justification and unethical behavior were collected from the same source at Time 2, I merged ethical leadership and employee moral identity into one factor and employee moral justification and unethical behavior into another factor to further test whether these variables captured distinct constructs. Compared to the four-factor model, the two-factor measurement model fitted the data poorly: 2 = 2163.75, df = 558, CFI = .73, IFI = .73, TLI = .69, RMSEA = .10. The chi-square difference compared with the four-factor model was significant (Δ2 = 1117.57, p < .01). These results provide support for the discriminant validity of my measures.

Descriptive Statistical Analysis
Table 1 presents the descriptive statistics, internal consistency reliabilities, and

correlations among the variables. All correlations were in the expected directions
and provided conditions to further test my hypotheses.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1278

Table 1. Descriptive Statistics and Correlations Among Study Variables

Variable M SD 1 2 3 4 5 6 7

1. Gender 0.40 0.49
2. Age 30.03 8.35 .03
3. Level of education 3.18 1.03 -.01 -.22**
4. Ethical leadership 4.08 0.76 .11 -.11 .10 (.95)
5. Moral justification 2.10 0.97 -.09 .04 .10 -.19** (.82)
6. Moral identity 4.51 0.70 .12 -.03 -.03 .46** -.19** (.92)
7. Unethical employee 1.85 0.59 -.16* -.00 .06 -.29** .25** -.28** (.92)
behavior

Note. N = 271. Cronbach’s alpha coefficients are reported in parentheses.
* p < .05, ** p < .01.

Testing of Hypotheses
Following Muller, Judd, and Yzerbyt (2005), I used multiple regression

analysis to test my moderated mediation model. Further, to reduce
multicollinearity I centered ethical leadership, employee moral justification, and
moral identity, per Aiken and West (1991). As shown in Table 2 (Model 1), after
controlling for employees’ gender, age, and level of education, ethical leadership
was negatively related to unethical employee behavior; this result is consistent
with those of previous researchers. In Hypothesis 1 I predicted that employee
moral justification would mediate the relationship between ethical leadership and
unethical employee behavior. After controlling for the demographic variables,
ethical leadership was significantly and negatively related to employee moral
justification (see Table 2, Model 2), and moral justification was positively and
significantly related to unethical employee behavior (see Table 2, Model 3).
Furthermore, when the effect of ethical leadership was controlled for, moral
justification was still positively related to unethical behavior (see Table 2, Model
4). The results of an additional bootstrapping procedure with 95% confidence
intervals (CI), as outlined by Preacher and Hayes (2008), revealed that ethical
leadership was indirectly related to unethical employee behavior through the
mediator of moral justification: indirect effect = -.03; Sobel z = -2.29, p < .05; 95% CI [-.063, -.008]. These results provide support for Hypothesis 1.

In Hypothesis 2 I predicted that employee moral identity would moderate
the relationship between ethical leadership and unethical employee behavior,
such that this relationship would be weaker when moral identity was weaker.
As shown in Table 2 (Model 6), the interaction term of ethical leadership and
moral identity was significant in predicting employee moral justification, and the
additional proportion of the variance in employee moral justification explained
by this interaction term was also significant (ΔR2 = .02, p < .01; compared to Model 5). To clarify this conditional effect, simple slope tests were conducted

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1279

(Cohen, Cohen, West, & Aiken, 2003). As shown in Figure 2, the relationship
between ethical leadership and unethical employee behavior was significantly
negative for employees with a strong moral identity (M + 1 SD; simple
slope = -.27, p < .01), and for employees with a weak moral identity, the effect of ethical leadership on unethical employee behavior was nonsignificant (M − 1 SD; simple slope = -.04, ns), thus supporting Hypothesis 2.

In Hypothesis 3 I predicted that employee moral identity would moderate
the indirect relationship between ethical leadership and unethical employee
behavior through employee moral justification. As shown in Table 2 (Model 7),
after controlling for demographic variables, ethical leadership, employee moral
identity, and the interaction term of the latter two variables, employee moral
justification was found to be positively and significantly related to unethical
employee behavior. The results of an additional bootstrapping procedure with
95% CI (Preacher & Hayes, 2008) revealed that ethical leadership was indirectly
related to unethical employee behavior through employee moral justification
when employee moral identity was strong (M + 1 SD): indirect effect = -.03;
Sobel z = -1.97, p < .05; 95% CI [-.068, -.004]. For employees with a weak moral identity, the indirect relationship between ethical leadership and unethical employee behavior was not significantly different from zero (M − 1 SD): indirect effect = -.01; Sobel z = -0.63, ns; 95% CI [-.034, .013]. Hypothesis 3 was, therefore, supported.

Figure 2. Moderating role of moral identity on the ethical leadership–employee moral
justification relationship.

M
or

al
j

us
ti

fi
ca

ti
on

5

4.5

4

3.5

3

2.5

2

1.5

1
Low ethical leadership High ethical leadership

Low moral identity
High moral identity

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1280

Ta
b
le

2
.

R
e
su

lts
o

f
M

u
lti

p
le

R
e
g
re

ss
io

n
A

n
a
ly

si
s

V
ar

ia
bl

es

U
ne

th
ic

al

M
or
al

U
ne
th
ic
al

U
ne
th
ic
al

M
or
al

M
or
al

U
ne
th
ic
al

be
ha

vi
or

ju

st
if

ic
at

io
n

be
ha
vi
or

be

ha
vi

or

ju
st

if
ic

at
io

n
ju

st
if
ic
at
io
n
be
ha
vi
or

(M

od
el

1
)

(M
od

el
2

)
(M

od
el

3
)

(M
od

el
4

)
(M
od
el

5
)

(M
od

el
6

)
(M
od
el

7
)

G
en

de
r

-.
13

-.

05

-.
16

*
-.

12

-.
04

-.

03

-.
11

A
ge

.0

0
.0

5
.0

2
-.

01

.0
5

.0
4

.0
0

L
ev

el
o

f
ed

uc
at

io
n

.0
8

.1
2

.0
4

.0
6

.1
1

.1
0

.0
5

E
th

ic
al

l
ea

de
rs

hi
p

-.
33

*

*

-.
21

**

-.

29
**

-.

16
*

-.
15

*
-.

21
**

M
or
al
j
us
ti
fi
ca
ti
on

.2

3*
*

.1
7*

*

.1

6*
*

M
or

al
i

de
nt

it
y

-.
11

-.

19
*

-.
17

*
E
th
ic
al
l
ea
de
rs
hi
p

-.

15
*

.0
0

×

M
or

al
i
de
nt

it
y

R
2

.1
4

.0
5

.0
9

.1
6

.0
7

.0
9

.1
9

F

9.
68

**

4.
11

**

5.
90

**

9.
63

**

3.
77

**

3.
88

**

7.
93

**

N
o
te

.
N

=
2

71
.

S
ta

nd
ar

di
ze

d
co

ef
fi

ci
en

ts
a

re
r

ep
or

te
d.

*

p
<

.
05

,
**

p
<

.
01

.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1281

Discussion

Theoretical Implications
In this study I explored the mediating role of moral justification on the

relationship between ethical leadership and unethical employee behavior, and
examined the moderating effect of employee moral identity on this moderated
mediation model. My theoretical contributions are reflected in the following four
aspects: First, although Bandura (1977) noted the importance of the cognitive
mediating process in social learning, few scholars have empirically explored and
examined cognitive mediating mechanisms in the ethical leadership–unethical
employee behavior relationship. Therefore, I have enriched and extended extant
research on social learning theory by exploring moral justification as a cognitive
mediating mechanism between ethical leadership and unethical employee
behavior. Second, I further investigated the mediating cognitive process in
social learning by exploring employee moral identity as a moderator of this
effect, thereby explaining why not all employees learn from ethical leaders.
Third, few researchers (see, e.g., Mayer et al., 2012) have investigated when
and how ethical leadership is related to unethical employee behavior. I presented
a moderated mediation model that has broadened understanding of the ethical
leadership–unethical employee behavior relationship. Fourth, a majority of
studies on unethical behavior (see, e.g., Gino & Margolis, 2011; Hershfield,
Cohen, & Thompson, 2012) have been conducted in a laboratory and primarily
used cheating as the proxy variable of unethical behavior; thus, my study has a
higher level of external validity and social relevance.

Practical Implications
The study also has some practical implications. My findings indicate that

leaders can limit unethical employee behavior by engaging in normatively
appropriate behaviors, and by demanding that employees be accountable for the
moral consequences of their behavior and care for others’ interests. I also suggest
that the behavioral standards for managers and supervisors should be emphasized
in organizations to ensure that their behaviors are ethical. Furthermore, managers
of organizations should establish the principle of advocating and incentivizing
employees to take responsibility for their behavior. The mediating role of moral
justification in the relationship between ethical leadership and unethical employee
behavior suggests that managers of organizations should pay more attention to
employees’ cognitive judgment and should ensure that their employees have
low levels of moral justification, which can inhibit unethical behavior. Finally,
the moderating role of employee moral identity that I found suggests that moral
identity is activated in organizations through cues in the social environment, such
as posters, slogans, or material symbols that make moral constructs and concerns
salient (Aquino et al., 2009; Mayer et al., 2012).

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1282

Limitations and Directions for Future Research
My research has several limitations. First, all of the variables were collected

from the same source (employees). Although I collected the data at two separate
times, the observed relationships should still be interpreted with caution because
of same-source concerns. For example, employees’ reporting of moral justification
may have biased their ratings of unethical behaviors. Future researchers should
strive to measure all predictors and unethical employee behaviors from different
sources or to utilize manipulations or objective outcomes. Second, although I
used several approaches to ensure confidentiality for the participants, they still
may not have reported their unethical behaviors truthfully because of social
desirability bias. In future studies, researchers should consider utilizing objective
indicators of unethical behavior. Third, I did not control for other styles of
leadership. Future researchers should control for other leadership styles that have
been found to be positively related to ethical leadership, such as transformational
leadership or charismatic leadership.

References

Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions.
Thousand Oaks, CA: Sage.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and
recommended two-step approach. Psychological Bulletin, 103, 411–423. https://doi.org/c76

Aquino, K., Freeman, D., Reed, A., II, Lim, V. K., & Felps, W. (2009). Testing a social-cognitive
model of moral behavior: The interactive influence of situations and moral identity. Journal of
Personality and Social Psychology, 97, 123–141. https://doi.org/dnxdnj

Aquino, K., & Reed, A., II (2002). The self-importance of moral identity. Journal of Personality and
Social Psychology, 83, 1423–1440. https://doi.org/cbq389

Aquino, K., Reed, A., II, Thau, S., & Freeman, D. (2007). A grotesque and dark beauty: How moral
identity and mechanisms of moral disengagement influence cognitive and emotional reactions to
war. Journal of Experimental Social Psychology, 43, 385–392. https://doi.org/c8kzsk

Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (1996). Mechanisms of moral

disengagement in the exercise of moral agency. Journal of Personality and Social Psychology,
71, 364–374. https://doi.org/cw4g7w

Barsky, A. (2011). Investigating the effects of moral disengagement and participation on unethical
work behavior. Journal of Business Ethics, 104, 59–75. https://doi.org/c63brn

Bergman, R. (2002). Why be moral? A conceptual model from developmental psychology. Human
Development, 45, 104–124. https://doi.org/dtb92r

Brislin, R. W. (1980). Translation and content analysis of oral and written material. In H. C. Triandis
& J. W. Berry (Eds.), Handbook of cross-cultural psychology (pp. 349–444). Boston, MA: Allyn
& Bacon.

Brown, M. E., & Treviño, L. K. (2006). Ethical leadership: A review and future directions. The
Leadership Quarterly, 17, 595–616. https://doi.org/b9cc76

Brown, M. E., Treviño, L. K., & Harrison, D. A. (2005). Ethical leadership: A social learning
perspective for construct development and testing. Organizational Behavior and Human Decision
Processes, 97, 117–134. https://doi.org/bkgm7b

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1283

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation
analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.

Detert, J. R., Treviño, L. K., & Sweitzer, V. L. (2008). Moral disengagement in ethical decision
making: A study of antecedents and outcomes. Journal of Applied Psychology, 93, 374–391.
https://doi.org/bfvmvv

Gino, F., & Margolis, J. D. (2011). Bringing ethics into focus: How regulatory focus and risk
preferences influence (un)ethical behavior. Organizational Behavior and Human Decision
Processes, 115, 145–156. https://doi.org/cbrpwb

Green, R. M. (1991). When is “everyone’s doing it” a moral justification? Business Ethics Quarterly,
1, 75–93. https://doi.org/dr6w59

Hershfield, H. E., Cohen, T. R., & Thompson, L. (2012). Short horizons and tempting situations: Lack
of continuity to our future selves leads to unethical decision making and behavior. Organizational
Behavior and Human Decision Processes, 117, 298–310. https://doi.org/fsn4g7

Hunt, S. D., & Vitell, S. (1986). A general theory of marketing ethics. Journal of Macromarketing,
6, 5–16. https://doi.org/bkw8x8

Liu, Y., Long, W. L., & Loi, R. (2012). Ethical leadership and workplace deviance: The role of moral
disengagement. In W. H. Mobley, Y. Wang, & M. Li (Eds.), Advances in global leadership (Vol.
7, pp. 37–56). Bingley, UK: Emerald Group.

Mayer, D. M., Aquino, K., Greenbaum, R. L., & Kuenzi, M. (2012). Who displays ethical leadership,
and why does it matter? An examination of antecedents and consequences of ethical leadership.
Academy of Management Journal, 55, 151–171. https://doi.org/qsz

Moore, C., Detert, J. R., Klebe Treviño, L., Baker, V. L., & Mayer, D. M. (2012). Why employees do
bad things: Moral disengagement and unethical organizational behavior. Personnel Psychology,
65, 1–48. https://doi.org/263

Mulder, L. B., & Aquino, K. (2013). The role of moral identity in the aftermath of dishonesty.
Organizational Behavior and Human Decision Processes, 121, 219–230. https://doi.org/cjs4

Muller, D., Judd, C. M., & Yzerbyt, V. Y. (2005). When moderation is mediated and mediation is
moderated. Journal of Personality and Social Psychology, 89, 852–863. https://doi.org/czt

O’Fallon, M. J., & Butterfield, K. D. (2011). Moral differentiation: Exploring boundaries of the
“monkey see, monkey do” perspective. Journal of Business Ethics, 102, 379–399. https://
doi.org/ffk6rv

O’Fallon, M. J., & Butterfield, K. D. (2012). The influence of unethical peer behavior on observers’
unethical behavior: A social cognitive perspective. Journal of Business Ethics, 109, 117–131.
https://doi.org/dpjwfk

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and
comparing indirect effects in multiple mediator models. Behavior Research Methods, 40,
879–891. https://doi.org/b9b2k3

Reynolds, S. J., & Ceranic, T. L. (2007). The effects of moral judgment and moral identity on moral
behavior: An empirical examination of the moral individual. Journal of Applied Psychology, 92,
1610–1624. https://doi.org/cgvqmp

Stevens, G. W., Deuling, J. K., & Armenakis, A. A. (2012). Successful psychopaths: Are they
unethical decision-makers and why? Journal of Business Ethics, 105, 139–149. https://doi.org/
bmj2t3

Vitell, S. J., Keith, M., & Mathur, M. (2011). Antecedents to the justification of norm violating
behavior among business practitioners. Journal of Business Ethics, 101, 163–173. https://
doi.org/b63vkc

Zey-Ferrell, M., Weaver, K. M., & Ferrell, O. C. (1979). Predicting unethical behavior among
marketing practitioners. Human Relations, 32, 557–569.

https://doi.org/ffk6rv

https://doi.org/bmj2t3

https://doi.org/b63vkc

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1284

Copyright of Social Behavior & Personality: an international journal is the property of
Society for Personality Research and its content may not be copied or emailed to multiple
sites or posted to a listserv without the copyright holder’s express written permission.
However, users may print, download, or email articles for individual use.

Article

What Style o f Mediation Do You Need?
by Richard A. Kaplan

I f you need a m ediator, you probably know there a re reputable
organizations that p rom ote p anels of lawyers an d retired judges
who offer m ediation services an d there is literature available to
help you narrow your search. In Utah, for example, you will find
m ediator biographies (self-interest alert, including m ine) on
utiilradrsenices.com o r law firm websites, and of course you can
w ork with the American Arbitration Association o r other
organizations depending on the subject matter. As for literature,
there is a “checklist” available with thirteen separate criteria to
consider; Negotiating and Settling Tort Cases, § 6.11 (updated 2009),
and there are m any p apers with the title “Selecting a M ediator”
o r w ords to that effect. See, e.g., Weinstein & Chao, Choosing a
Mediator, 5 Bus. & Com . Litig. Fe d . Courts § 51.35 (4th ed).

As I see it, the criteria an d suggestions in the literature boil
down to a few key questions: What do you need most from the
m ediator? Who has that p articu lar capability? Will that person
be acceptable to the parties and opposing counsel?

Deciding what you need from a m ediator obviously requires both
an accurate assessm ent of the case and a good understanding of
the participants an d the p rocess. Determ ining which m ediators
provide what you need in those respects will not likely be possible
based on a typical m ediator biography alone. Gaining acceptance
of the m ediator you select may d epend on how you play it.

SCENARIO ONE:

Strong Case, Form idable Adversary, Zero Progress

Let’s say you rep resen t the plaintiff in w hat you see as a potential
$ 5 – 6 million com m ercial lawsuit. Discovery is com plete, you’ve
developed good facts. On balance, the law favors your client.
You co nsid er yourself a good lawyer and an able negotiator. Still
you’ve gotten now here in settlem ent discussions. Your adversary
has a lot m o re experience than you do, an d a rich resum e of
accom plishm ents at trial. The obstacle to serio us negotiations
an d settlem ent may have som ething to do with your adversary’s
self-confidence, ego, o r both. While you co nsid er yourself “up
an d com ing,” opposing counsel is a long-tim e pillar of the b ar
and knows it. Most importantly, he is highly regarded as a trial
lawyer p e r se, not for settling early, cheaply, o r even at all.

You dem anded $4.5 million an d accom panied that dem and with
a w ritten analysis of why you will win an d why your dam ages
greatly exceed your dem and. He d o e sn ’t seem to u nd erstand
your argum ents, or, if h e does, h e clearly dism isses them. He’s
at zero. The only p ro gress you’ve m ade is that he has agreed to
voluntary m ediation. How will you ap pro ach the pro blem of
choosing a mediator?

It s not ju s t the stre ng th s and w e a kn e sse s of the case

It’s the process and the p sycho d yna m ics you m ust

contend w ith .

I’ve h eard it said, not entirely in jest, that there a re essentially
three kinds of people: people who make things happen; people
who watch things happen; and people who wonder what
happened. At first blush, people who “m ak e” things h appen
may sound som ehow su p erio r to people who “w atch” tilings
happen. Not so fast. That’s not true at all, particularly as it
relates to m ediation. And, as for you an d me, if truth be told, we
m ust admit w e’ve found ourselves w ondering w hat happened
from time to time.

As it relates to m ediators, this simplistic classification of people
is useful in beginning to consider what mediation style you want.
All three describe not just people but skills, styles, and strategies.
1 w ant to explain the choices available to you without resorting
to m ediation jargon, an d then to use the language of mediation
to help you determ ine what type of m ediation style you w’ant for
Scenario One. The discussion of types of m ediation skills, styles,
an d strategies should have b ro ad enough applicability to help
you decide what you need in oth er scenarios as well. I’ll p resent
another, p erhap s m o re typical, scenario tow ard the end.

RICHARD A. KAPLAN is a shareholder at
Anderson & Karrenberg. His practice
focuses on complex civil litigation and
mediation, as well as independent
investigations and risk assessment at the
outset o f commercial ligation.

willing to negotiate. They essentially “w atch” things happen.
This is the style you’re looking for m ost of the time.

B a s i c m e d i a t i o n s t y l e s a n d t h e s t r a t e g i e s t h e y s u p p o r t .

The best m ediators, while open-m inded, are highly skilled at
seeing areas of agreem ent o r m utual interest and even the
b ro ad outlines of a deal before it happens o r w hile it develops.
The focus h ere is on how they handle that knowledge.

Some of the m ost accom plished m ediators view a deal as the
overriding goal of mediation, occasionally requiring abandoning
the p rin ciple of neutrality in favor of raw truth. Always having
the end gam e in m ind, they w ork h ard to get the parties to that
point by design. They may push one side o r the other, o r both,
gently an d forcefully at different times during the day. They try to
“m a k e” things happen.

O ther highly skilled m ediators take a m uch m o re hands-off,
d eliberate approach. They may nudge the p arties from time to
time. But they derive the m ost satisfaction from watching the
parties strike a bargain themselves. Such m ediators view
neutrality d uring the p ro cess not just as ap pro priate but as
fundam ental. They’ve w o rked h ard to develop that skill, to know
w hen to talk, w hen n ot to, and when to stop. They know that
ap p ro ach engenders trust and thus h as great value to parties

I doubt that many m ediators, if any, studiously practice
“w ondering w hat h ap p en e d .” But it’s w orth thinking ab ou t this
idea of “w o n d er” at a d eep e r level. Some m ediators a re simply
b etter than o th ers at feeling an d expressing em pathy and
appreciation. I ap preciate, and you probably do too, people
who are easily able to attribute ideas to others, ra th e r than to
themselves. That quality is especially im portant in m ediators.
W hether you w ant to take the notion of three kinds of people
furth er an d call the third category those who have the capacity
to com m unicate “w o n d er” (like a p aren t encouraging a child)
o r something else, the ability to attribute au thorship of ideas o r
pro po sals to o th ers tends to m ake them feel like they’re
contributing and taking “ow nership.”

Some highly skilled m ediators a re flexible and able to use
multiple strategies and skills successfully, along with dozens of
tactics for working through im passe. They can plan an d intuit
when and how to employ this, that, o r the o th er skill o r tactic in
the co urse of a single day.

CONYERS 0 NIX
CRIMINAL DEFENSE ATTORNEYS

We welcome yo u r referrals a n d
the o p p o rtu n ity to work w ith you.

ALL FELONIES & MISDEMEANORS | DISTRICT COURTS & JUSTICE COURTS

Kate Conyers Jesse Nix

KATE@CONYERSNIX.COM | WWW. CONYERS NI X. COM | JESSE@CONYERSNIX.COM
405 SOUTH MAIN STREET, SUITE 930, SALT LAKE CITY, UTAH 84111 | 801-839-4649

13

mailto:KATE@CONYERSNIX.COM

http://WWW.CONYERSNIX.COM

mailto:JESSE@CONYERSNIX.COM

0 – 3

cl

M e d ia tio n s ty le s , s tra te g ie s , and v o c a b u la r y as
a p p lie d to S c e n a r io One.
Given the state of play in Scenario One, as plaintiff’s counsel you’re
not ready for the subset of m ediation known as ‘ facilitation
A facilitator’s greatest skill in m ediation is assisting the parties
themselves in negotiating resolution. That m eans w orking with
them patiently and supportively, usually observing ra th e r than
talking as each side w orks through its own concerns. Excessive
criticism, optimism, o r pessimism are studiously avoided, because
the substance o r tone of such communications could be interpreted
as favoring one side o r the other. To be sure, m ost facilitators
will ask h ard questions o r m ake serious suggestions, even for a
“c o rrectio n ” if a p ro p o sed move seem s way off base. But by and
large a facilitator d o esn ’t try to influence the p ro cess o r direct
it, because that com prom ises the ap pearance of neutrality. All
m ediators, regardless of style, know that the m ost enduring
settlem ents are the ones the parties negotiate themselves. All
m ediators and m ost lawyers w h o ’ve used mediation know that
facilitation is m ost likely to be effective when the gap to be
bridged is relatively small.

Absent deux machina, a facilitator w on’t help you in Scenario
One b ecau se your adversary d o esn ’t w ant to be facilitated, and,
in truth, you d o n ’t either. The gap couldn’t be larger. The parties
are simply n ot poised to strike a deal essentially on their own.

S u p p re s s th e “g o o d gu y” te m p ta tio n .
Be wary that the m ediator (w hether o r not a facilitator) might
appeal to you to be the “good guy.” That is especially likely in
Scenario One, since y our adversary w on’t touch that role and
som ething’s got to give. Reason doesn’t always trum p em otion,
m uch less always prevail.

Consider what opposing counsel might have to say on the morning
of the mediation regardless of who the m ediator is: “It’s my
position and I ’m sticking to it.” He knows from experience that
despite the m ed iato r’s entreaties (“I n eed you to give me
som ething to w ork with, anything p lease”), the m ediator may be
wrong. T here’s potential success in standing pat, insisting till the
while to the m ediator that you, his opponent, a re the one being
unrealistic. Consider also that you may be the one the m ediator
tests, not your adversary.

The temptation can be quite strong to show the mediator that you’re
in fact reason able by indicating you an d your client are flexible
an d will in fact move in the face of a credible offer. Be aw are of
that trap, know it when you see it, an d d on ‘t let it catch you.

D on’t overplay any natural desire you may have to cooperate
with the mediator, to have h e r like you. It may take very little to
dem onstrate that you’re the reason able one in the room . If you
show too much flexibility, particularly early on, you put your client
at risk of being push ed tow ard a “split-the-baby” com prom ise
you d on ‘t really want.

Be a w a r e th a t w e a k n e s s is sign a le d in u n ex p e cte d w a y s .
Your adversary may not “know ” that you conveyed such
flexibility’ to the mediator. The m ediator c a n ’t tell him w ithout
your consent and you obviously d o n ’t give it.

Still, you m ust figure that your adversary has an acute sense of
w eakness. He will watch the m ediator and listen carefully to h er
when she com es b ack from a caucus with the oth er side. Her
body language, tone of voice, and delivery may suggest your
weakness regardless of her best efforts not to. Opposing counsel
will look beneath her efforts to generate movement, to see what’s
really there. We’ve all h eard (an d all m ediators sometimes
p ro po se o r use) h y p o th e tica l: “S u p p o se .. Or, “I ’m certainly
not saying I can, but what if I can get him off of that num ber?”
When opposing counsel h ears that early on, he h ears it the way
he wants to: th e re ’s a lot of ro o m here. He’ll be tem pted to let
that dynamic play out in his favor.

Rem em ber that despite the “w in/w in” paradigm that dom inates
cu rrent m ediation literature an d training, your adversary wants
to “win” the old-fashioned way, and at every step along the way.
Try to u nd erstand w hat he thinks w ould constitute a “win” in
your case an d have that low-ball n u m b er in m ind as som ething
you m ust avoid as you calibrate your moves. For him, “winning”
in Scenario One probably d o esn ’t m ean paying zero o r anything
close to that. If the fair value of settlem ent is something north of
$4 million, he m ore likely thinks a “w in” would involve paying
som ething south of $2 million, better yet $1.5 million.

So, assume you have given the m ediator the impression that you’re
reason able and w hat you really want is a just a good settlement,
som ew here in the middle. In that case the m ediator will alm ost
certainly apply p re ssu re to move the parties, particularly your
side, in that direction, consistent with the principle of neutrality.
After all, you had said orally and in writing before the m ediation
that you want a lot, but your dem eanor and choice of w ords now
reveal that you’ll take not so much. With a strong sense of that
in hand, the m ediator w on’t violate h e r duty of confidentiality o r
any o th er p rinciple o r duty by helping you get there. In fact, that
would be in keeping with the m ediator’s job.

14

So, what type of mediator do you need?
I think Scenario One requires you to find a mediator who
(1) has first-rate analytical skills and is highly likely to perceive
the value of your case as you see it (that means you’re right and
she agrees); and (2) who will credibly and convincingly,
perhaps brutally, convey that value to opposing counsel. In
mediation jargon, that would entail “evaluative” mediation
skills and perhaps “transformative” mediation skills. What
“evaluative” normally means is a mediator who comes to his oi­
lier own conclusion about the value of the case and discusses it
openly with both sides. All mediators surely offer the ability to
estimate a range of settlement values for a case. Paradoxically
though, few have the credibility to forecast the likely outcome at
trial, particularly when one side is represented by a highly
experienced trial lawyer and one is not. In this scenario, you
need that credibility. While you’re in a separate room, you want
the mediator to look opposing counsel in the eye and to say
something like this: “This case has a lot of jury appeal –
settlement value now in the mid-seven figures. I know you are a
terrific trial lawyer, but you are likely to lose this one and in all
events you have significant trial risk in this case. Here are all the
reasons why that is so….”

(Note: When you ask for “evaluation” you obviously run the risk
that the mediator doesn’t see things the way you do. Be prepared
to hear that your valuation is grossly inflated and wildly off the
mark. Understand that your selection of an evaluative mediator
assumes that you are highly confident in your estimation of the
case and able to defend it persuasively. If you can’t, you need to
be flexible enough to consider resetting your sights. If your
views remain the same despite the mediator’s contradictory
assessment, you need to be willing to walk.)

What “transformative” means is the ability to get the parties and
their lawyers to understand each other’s perspectives and to gain
respect for those opposing perspectives increasingly as the day goes
on. Mutual understanding and respect drive compromises and deals.

Can you narrow this down to the right person(s)
for the job?
Who is the most likely to have these abilities? How about someone
who already enjoys opposing counsel’s respect as a smart and
accomplished trial lawyer and mediator in his or her own right?
Tact, empathy, listening skills, and other such qualities are great,
but they’re not the heart of what you need in this case. You haven’t

o – o

Congratulations to Our Colleague
RUTH A. SHAPIRO

Ruth has been named the Utah Defense Lawyers Association
Legacy Award winner of 20

18

-2019.

Whether she is fiercely defending an insurance client in court or
valiantly competing in a road cycling event, Ruth gives it her all.
We are honored to have Ruth part of our team and proudly
recognize another impressive achievement.

Ruth A. Shapiro
ras@scmlaw.com
801.322.9

16

9

Sn o w
C h r i s t e n s e n
& M a r t in eau

www.scmlaw.com j 801.521.9000

15

mailto:ras@scmlaw.com

http://www.scmlaw.com

CXD

gotten your adversary to take you and your case seriously yet.
You need help making that happen. You want a mediator who
will know, and tell your adversary, that she understands his
dismissiveness is an act, a tactic that he needs to leave behind
now, and that it would serve his client’s interests to get the case
settled as early as possible.

(Note: The mediator knows opposing counsel knows what both
sides’ strengths and weaknesses are. Good lawyers, like opposing
counsel here, listen while pretending not to; they are always
focused on their client’s needs, not them own. So, the task of cutting
through the smokescreen is not nearly as difficult in mediation as
your experience with the guy suggests, provided you find a
mediator who can do it.)

It’s not ju s t up to you. H ow do you get opposing

counsel to a c c e p t the m ediator?

If you believe you have identified just the right person, I know you
realize you can’t dictate who the mediator will be. Your adversary
doesn’t have to agree to engage the mediator you propose and
will usually have other ideas of his own. As we’ve posited, your
adversary wants to “win,” and generally that means at every step
along the way. Winning, for him, means selecting the mediator
he wants, finding reasons to reject the people you propose, and
working you over until you agree with his choice.

So, let me suggest that you surprise opposing counsel by making
die choice he would have made, and he will be hard pressed to
reject. Be bold. Find and propose the mediator/trial lawyer that
he respects and uses more frequently than any other in town,
whether he’s worked with or against her 100 times, and whether
you’ve interacted with her yourself. If he asks, tell him you’re aware
of the prior relationship he has with the mediator and you’re not
concerned about it. You have an abiding conviction she’ll be fair.

You should mean it when you tell him you’re not worried about
bias. You should realize that, if anything, she’s likely to bend
over backwards to be fair to you and your client. Indeed, that
concern may well occur to your adversary as he thinks through
what you’re doing. He may say no. If instead he hesitates, tell
him you’ll let her know you understand the full extent of their
professional relationship and don’t want her to be concerned
about it. You can’t force your adversary to agree, but it’s hard
for him to reject someone he’s used often himself. Regardless,
you’ve now set the bar high and, if you don’t settle on her, you’ll
agree on someone like her.

There are a host of reasons for selecting a mediator “as good
as” opposing counsel, and here’s the main one: In this case, the

mediator is not just evaluating your case on paper. She’s evaluating
you, and your ability to persuade judge and jury. Who better to
tell opposing counsel that you are “for retd” than someone he
himself considers “for real” and thus already respects and
listens to? Who better to persuade him that it would be wise for
him to come down from Mount Olympus and negotiate with us
mortals? That’s what he and his client need to hear from the
mediator to get real themselves.

To be sure, you’ll have to perform at your highest level to pull
this off, but that should go without saying anyway.

THE MODERN CONCEPT OF MEDIATION ADVOCACY

In a leading article published almost twenty years ago in the Ohio
State Journal on Dispute Resolution, James K.L. Lawrence
advocates the concept of “Partnering with the Mediator.” 15
Onto St . J. on D is p. Resol. 425 (2000). The heart of Lawrence’s
argument is that mediations generally have the greatest chance
of success when “the mediation advocate and her client [are]
engaged with the mediator in the problem-solvingprocess: merely
accepting or rejecting proposals from the mediator or the other
side is insufficient ‘engagement.'” James K.L. Lawrence,
Mediation Advocacy: Partnering With the Mediator, 15:2 O hio
St . J. on D is p. Resol. 425, at 426 (2000) (emphasis added).

While 1 strongly agree with Lawrence’s premise, I think the
“partnership” characterization probably takes the idea of
collaboration a bit too far. You represent your client. The
mediator is not your “partner” in achieving your client’s goals,
and it’s dangerous to think about her that way. While sometimes
it might make sense to be open and transparent at the very
outset, most of the time you need to be more strategic in what
you reveal, how much, and when. You want momentum to build,
not wane, over the course of the day. Disclosing everything you
have to say first thing in the morning may leave you with nothing
left to say by noon. For example, you may think it will help the
mediator to know your “walk away” number early in the process.
You may think that will influence her to encourage your adversary
to start well above your “walk away” number and to stay there
throughout the process. On the contrary, that disclosure may be
the first number the mediator actually considers and may therefore
tend to anchor the mediator’s thinking on the low side of the
bargaining range. That early disclosure thus could lead her to
work with the other side and you (despite the best of intentions
and a conscious effort to maintain neutrality) to make moves
toward your walk away number the rest of the day. This “anchor
effect” is well documented in negotiation literature, and it could
well work to your disadvantage during a mediation as well.

16

On the other hand, assuming opposing counsel is reasonably
competent (as you should), he already knows the weaknesses
in your case. In most cases, opposing counsel already knows
them. You will help the mediator and your client’s bargaining
position, at virtually no risk, by being forthright with the mediator
about those weaknesses and by authorizing the mediator to tell
the other side you know exactly what they are and why they
don’t concern you.

Assume you’re about to give the mediator a demand or offer or
counteroffer for her to present to the other side. It is best if the
mediator can communicate at the same time that the dollar amount
(or whatever you’re proposing) takes those weaknesses into
account. She should add that you have carefully considered them
as well as the strengths of your case, and that this proposal gives
those weaknesses what you consider fair weight and value. Give
her your reasons too and let her pass those on at the same time.

Among the advantages of this approach, Lawrence points out, is
that by allowing the mediator to put your weaknesses on the
table, and to explain how you’ve taken them into account in
deciding what to propose, you’ve actually strengthened your
relative bargaining position. The weaknesses you expose now
can’t bite you later. Sure, the mediator can come back to you
with a proposal that says you didn’t give those weaknesses
enough value. But she cannot surprise you with the response
that your proposal didn’t consider your weaknesses at all, and
here they are, outlined with force by the other side. Oops.

SCENARIO TWO:
A Simple Example of Interest-based Negotiations

Let’s say you represent the plaintiff in a lawsuit in which he buys
trucking services from the defendant to pick up merchandise from
his customers’ warehouses and to deliver it to its destination. Assume
both sides have leverage. Yours is the warranties and reps in the
contract. You could likely get summary judgment. Your opponent has
a different kind of leverage. For one tiling, the judgment would be
hard to collect because of defendant’s precarious financial situation.
For another, your client needs defendant’s services because no one
else in the area has ever provided them on a consistent or reliable
basis. In these circumstances, your client and you may want a mediator
ultimately to “facilitate” a business deal between the parties.

Suppose it’s your first communication (a Mediation Brief) or first
caucus with the mediator. You’re not sure about the defendant’s
cash situation so you probably want to take a shot at getting complete
monetary relief. You need to make your point simple and compelling:
the parties entered into a contract for trucking services and the

defendant expressly warranted that he would provide them. He
didn’t. You had to refund $

17

5,000 to your customers, and you
can substantiate another $50,000 in compensable damages. The
defendant took risks in the ordinary course of business, and the
defendant needs to come to grips hilly with your damages.

Suppose though that the mediator comes to you with a small,
unacceptable cash offer and nothing more, and makes clear she
believes the money just isn’t there. You now have a choice where
to direct the negotiations. You can of course respond in kind or
not at all. You have a strong summary judgment argument based
on the warranties. You can walk.

On the other hand, you can introduce the concept of a negotiated
resolution containing both monetary and non-monetary components.
You could tell the mediator truthfully that you understand the dispute
at a deeper level and want to share your insights with her. In
mediation jargon, you propose to help her understand the underlying
“interests” of the parties – their needs, their desires, fears, and
uncertainties. You can help her work toward a “win/win” outcome.

You might explain that the parties have had a business and personal
relationship for a decade that has been fractured by this dispute. Both
sides are terribly frustrated. Your client knows that over the three-dav

Commercial Mediation

Blake D. M ille r
• Seasoned tria l law yer w ith

over 35 years of experience
in high-stakes com m ercial
litig a tio n before federal,
state, and trib a l co u rts

• Form er fo u n d e r and CEO of
several businesses

Direct: 8 0 1 .590.2353
b m ille r@ a kla w firm .co m

Richard A. Kaplan
• Form er A ssista n t U.S. A tto rn e y

• Harvard M.B.A. w ith experience
at a Fortune 500 com pany and
ow ning sm all businesses

• C om plex co m m e rcial litig a tio n
and re g u lato ry m atters practice

Direct: 8 0 1 .639.0954
rkaD lan@ aklaw firm .com

f. ‘ ‘ ANDERSON &
X^y KARRENBERG

50 West Broadway, Suite 700 | Salt Lake City, Utah 84101
801.534.1700 | www.aklawfirm.com

17

mailto:bmiller@aklawfirm.com

mailto:rkaDlan@aklawfirm.com

http://www.aklawfirm.com

0 – 3

period when defendant breached, there was bad luck involved
because most of his drivers called in with the flu. But still, you
had warranties. Your adversary knows that you tried yourself to
secure alternative transportation but couldn’t get it. Your adversary
says that’s his point too; he wasn’t able to get any other drivers
either after trying to do that for hours and hours. Your client might
stand firm on the warranties, or he might consider trying to salvage
the relationship, provided the other part}’ makes your client’s
customers substantially whole. You understand that the defendant
doesn’t have that much cash on hand, but your client might
consider payment terms. That gives the mediator a smorgasbord
of material to work with in developing potential compromises.

There’s no substitute for understanding your client’s underlying
interests – business, personal, economic, and emotional –
before the mediation. Think about how and when you might
best weave his mixed feelings about a renewed business
relationship into the conversation. You also want to research
and think about your adversary’s likely needs and giveaways, as
well as his BATNA. In the end, a reasonable resolution may be
the one that satisfies as many of the parties’ mutual interests as
possible. The absence of any likelihood of re-establishing the
business relationship takes a host of solutions off the table. The
prospect of a renewed relationship is one a facilitator who
understands business deals can work with skillfully.

Getting your client ready.
Mediation requires not just your engagement but your client’s as
well. You need to help your client get emotionally invested in the
process and ready to work at it throughout the day. You and he share
a common purpose. Among other reasons for your client to get
engaged, he’s the one who is going to pay for it, including not just
your fees but half the mediator’s. It is better if he appreciates that
the odds of success improve with his full attention and engagement.

As to subjects to be discussed, most of them are probably obvious
and there’s too much literature on preparation to cite here. Just
a couple of points bear mention. You want to get your client ready
for the mediation process (including what you know about the
mediator and why you chose or agreed to hire her). Make sure
you have agreement beforehand on what your goals are, what
your strategy is for achieving them, and what happens if you
don’t. Focus on what your client needs, what he doesn’t need,
what is essential, and what he can give up to get something else.
What is your client’s best alternative to a negotiated resolution
(aka BATNA)? Quantify that as best you can. What is his w’alk
away number? Explore with your client before the mediation
what his real “interests” are: Is his objective to get as much
money as he can? Or would he prefer a combination of money
and, for example, assurances of priority service. Recognize that
your client’s “interests” include probing the needs, desires, and
fears that motivate him. James K.L. Lawrence, Mediation
Advocacy: Partnering With the Mediator, 15:2 Ohio St. J. on
Disp. Resol. 425, at 426 (2000). Thus, in Scenario Two, for
example, he’s understandably fearful of keeping his relationship
with the defendant, and fearful of losing it too, particularly if he
can’t find an alternative. He’s worried about keeping his own
customers. He had thought before this mess that defendant
valued him a valued customer. Now he thinks that wasn’t true at
all. He doesn’t want a new business deal with defendant unless
the terms somehow reflect his valued status.

Finally, prepare your client for the experience – the slog – of
mediation, which tends not to be fun for clients, especially if
they don’t feel heard and understood. This preparation requires
orientation, including visualization. So talk with your client
about what mediation is and the role of the mediator, what the
floor plan will be, how you and your client will have privacy,
whether or not the mediator is one who first brings the parties
and lawyers together in the same room, what will happen if she
does, how she might spend just a minute with you and your
client to introduce herself and then spend an hour with the
other side (or vice versa), how frustrating that can be, what to
make and not make of that, where the negotiations stand before
mediation, and how to understand and withstand the inevitable
moments when he’ll want to leave or just throw the towel in.

Getting yourself ready.
Indeed, you need to ready yourself for the same experiences
throughout the day. How are you going to handle disappointments
when the mediator comes back again and again with less than
you expected? Will you allow a sigh of resignation? Or will you
smile knowingly? Don’t discount the impact of your own body
language. You need to maintain your energy and resolve. You
need to recognize and avoid the “good guy” trap, especially at
the end of the day when it becomes most alluring.

That means remaining vigilant and positive, and helping yourself
to do that by keeping your purpose squarely in mind. Keep in
mind that mediations can and do flip on a dime, sometimes in
the last ten minutes. To be sure, that may not happen on any
given day. Your best alternative at some point may be to exercise
your greatest leverage, to walk. Meanwhile, your job throughout
the day is to look for and work on developing ways to create
opportunity for your client, or to recognize it when it comes
knocking, unexpectedly or otherwise.

18

Copyright of Utah Bar Journal is the property of Utah State Bar and its content may not be
copied or emailed to multiple sites or posted to a listserv without the copyright holder’s
express written permission. However, users may print, download, or email articles for
individual use.

RESEARCH ARTICLE Open Access

The role of mediation in solving medical
disputes in China
Mengxiao Wang1,2, Gordon G. Liu1*, Hanqing Zhao3, Thomas Butt1, Maorui Yang4 and Yujie Cui5

Abstract

Background

: Medical litigation represents a growing cost to healthcare systems. Mediation, arbitration, and other
alternative dispute resolution (ADR) methods are increasingly used to help solve the disputes and improve
healthcare satisfaction. In China, the increasing number of medical disputes has contributed to concern for the
safety of physicians and mistrust between physician and patients resulting in ADR processes being established in
several provinces in recent years. Our aim was to describe and explain the impact of this new mediation process in
the Chinese healthcare system.

Methods

: Our study investigated mediation practices in China using case-level data from 5614 mediation records
in Guangdong Province between 2013 and 2015. We investigated how the resolution success as well as the
compensations are associated with the case characteristics using regression analysis.

Results

: Among the cases analyzed, 1995 (41%) were solved with agreement through mediation, 1030 were closed
by reconciliation, 559 were closed by referring to court and 1017 cases were withdrawn after mediation. Five
hundred five Yinao cases were solved with the help of mediators on the spot. We find that mediation solved about
90% of medical disputes under present mechanisms, while more police support is needed to cope with Yinao. The
average compensation of mediation is CNY60,200 and average length of mediation is 87 days. Longer time taken to
reach resolution and more money claimed by patients are associated with lower resolution success rate (p < 0.01) and higher compensation levels (p < 0.01).

Conclusion

: Our results show the performance of mediation mechanisms in China to help solve medical disputes.
ADR plays a role in reducing the need for initiating litigation and may ultimately increase satisfaction with the
healthcare system.

Keywords: Medical dispute, Compensation, Duration, Medical mediation, China

Background
Accompanied by the rapid growth of household income,
demand for better healthcare services has been increas-
ing in China, leading to rising expenditure on health-
care. At the same time, a number of organizational
reforms on the supply-side have been implemented to
improve the quality and efficiency of Chinese healthcare
delivery [1, 2]. Despite these improvements, the cost and

volume of medical disputes has grown in recent years.
Mistrust between patients and physicians has grown [3]
and the physician-patient relationship has worsened.
Extreme cases of medical disputes can cause great
physical, mental and reputational damages to physicians
and hospitals, and bring losses and adverse social
influences due to negative media coverage. Workplace
violence has been shown to contribute to higher levels
of occupational stress and lower levels of job satisfaction
[4, 5] and even brought psychological problems to physi-
cians [6].

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article’s Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: gordonliu@nsd.pku.edu.cn
1National School of Development, Peking University, Beijing 100871, China
Full list of author information is available at the end of the article

Wang et al. BMC Health Services Research (2020) 20:225
https://doi.org/10.1186/s12913-020-5044-7

http://crossmark.crossref.org/dialog/?doi=10.1186/s12913-020-5044-7&domain=pdf

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/publicdomain/zero/1.0/

mailto:gordonliu@nsd.pku.edu.cn

Hospital disputes occur in nearly all departments and
most often in tertiary hospitals [7]. One particular case
of hospital violence, commonly known as Yinao in Chin-
ese, describes organized unemployed gangs who are paid
by patient families to create medical disturbances to get
compensation for actual or perceived malpractice from
hospitals [8]. Through this illegal approach, patients in
dispute with hospitals usually expect to get higher com-
pensation compared to other approaches [9]. With the
hospital being surrounded by Yinao gangs, hospital oper-
ations such as diagnosis and treatment are disrupted and
hospital staff, equipment and facilities are at risk.
In one recent example, on July 12 of 2018, a female

doctor was stabbed by three gangsters in Tianjin and
died later in hospital. None of the three gangsters were
patients of the doctor [10].
To tackle this problem, China has made a number of

changes to laws and regulations. Several Provisions of
the Supreme People’s Court on Evidence in Civil Proce-
dures meant that medical damage lawsuits started to be
judged based on reversed onus from 2002. However, this
created more tension between physicians and patients
and made physicians more defensive and spend more
time on medical record keeping [11] with implications
for the quality of medical services while contributing to
increase in health care costs [12]. In 2010, the Tort Li-
ability Law was implemented and the reversed onus was
to some degree reduced for healthcare providers [13].
To deal with more serious disputes, extreme forms of

Yinao like “setting up a mourning hall”, “burning paper
money” (a Chinese traditional way to memorize the dead
at a funeral) to disrupt hospital operations and so forth
can constitute crimes [14]. This was formalized by
Amendment IX to the Criminal Law of China in 2015
whereby crowds who are assembled to disrupt hospital
services will be sentenced with serious consequences.
Most recently, the government has begun to advocate

resolution of disputes by alternative mechanisms. On July
31, 2018, Prime Minister Li Keqiang signed “Medical Dis-
putes Prevention and Treatment Regulations”, which fur-
ther emphasized the prevention and resolution of medical
disputes and protection of both physicians and patients.
Medical disputes can be resolved through litigation

[15, 16] or, where alternative dispute resolution (ADR)
methods exist, through mediation or arbitration [17, 18].
Facing the scarcity of both medical and legal resources,
China has advocated mediation as an ADR approach to
deal with medical disputes. Gradually, mediation has
been adopted in many provinces and cities, such as
Guangdong Province, Tianjin City and Hainan Province.
Studies on approaches to deal with medical disputes are
more focused on litigation [19, 20], a few have analyzed
a small sample of mediation cases without highlighting
the role of mediation in coping with Yinao [21, 22].

The contribution of this study is threefold. First, this
study provides an analysis of mediation practices in
Guangdong, the largest province in China, with 5614
case-level data and a detailed description of the mediation
mechanism. Second, we analyze the medical disputes han-
dled through mediation-based cases from 2013 to 2015 in
China and investigate the factors that influence successful
mediation and levels of compensation. Third, our analysis
deals with not only the common medical disputes but also
Yinao cases. With the information of 505 Yinao cases, this
study investigates the unique features of Yinao.
Our study aims to quantitatively describe and explain

the outcomes of mediation in China to provide policy
recommendations for hospital risk management and
safety enhancement of both patients and physicians.

Methods
Mediation in China
According to the People’s Mediation Law, the People’s
Mediation Committee is under the supervision of the Ju-
dicial Administration department, and mediates disputes
for free. The agreement after mediation has legal effect
but no power of law enforcement. In order to be legally
binding, patients or hospitals must seek judicial confirm-
ation. Without this confirmation, if either side defaults
on the agreement, mediators will advise them to choose
other ways to deal with the dispute.
Guangdong province in China had a GDP of 8.09 tril-

lion CNY and a population of 110 million people in
2016, making it the largest province measured by wealth
and population [23]. Most citizens live in urban areas
(69.2%, 2016). In 2017, the migrant population in
Guangdong reached 41 million, which adds difficulty to
public governance and conflict resolution.
Guangdong People’s Mediation Committee (PMC) was

established in 2010, and its service region has expanded to
16 out of 21 cities by 2015. Guangdong’s PMC operates
according to the principles of “Fairness, Impartiality, Neu-
trality, Timeliness, and Convenience”. It built a mechan-
ism combining mediation, compensation and dispute
prevention, which actively responses to Yinao and aims to
settle medical disputes outside hospitals to maintain nor-
mal diagnosis and treatment order. To mediate disputes
between health care facilities and patients, the mediators
usually have educational backgrounds such as medicine,
law, and psychology and they receive annual training to
improve their mediation skills. The mediation process is
also supported by both medical and legal experts and
most of the medical experts have a job title of deputy dir-
ector or above. Guangdong PMC has 46 branches in
Guangdong Province and its expert group (including doc-
tors, nurses and lawyers) has more than 2000 members.
Figure 1 depicts the mediation procedure in Guangdong.
Usually the patient or medical institution applies for

Wang et al. BMC Health Services Research (2020) 20:225 Page 2 of 10

Fig. 1 The mediation process of People’s Mediation Committee in Guangdong

Wang et al. BMC Health Services Research (2020) 20:225 Page 3 of 10

mediation first. Once accepted, the mediator will start by
collecting evidence (e.g. medical records). The main pro-
cedure of mediation contains four steps: 1), both sides
make statements; 2), medical and legal experts analyze the
case; 3), experts are invited for meetings regarding to the
specific case (e.g. clinic department, treatment or nursing);
and 4), the mediator negotiates with both patient and hos-
pital sides based on the opinions of the experts.
Generally, medical disputes will be handled at the

PMC and follow the procedures described above. While
for Yinao cases, once reported, the mediators will go to
hospitals to handle them on site.

Data sample
We used data from Guangdong PMC, which contains
information on case-level medical disputes, including
cause, duration, and hospital characteristics. Our sample
included 5614 cases settled during 2013–2015 in Guang-
dong Province. Ninety-seven percent of mediation cases
were reported by hospitals and patients (Fig. 2). The
cases referred by health departments or courts account
for only 1 % each. Overall, 4902 out of the 5614 cases
(87%) were registered and 207 cases (4%) were rejected

after preliminary examination. 505 Yinao cases (9%)
were handled on the spot.
The outcomes of the 4902 registered mediation cases

closed during 2013–2015 consisted of four types: 1995
cases reached mediation agreement (41%); 1030 cases re-
sulted in reconciliations between hospitals and patients
(21%); 559 cases proceeded to litigation (11%); 1017
withdrawal of cases (21%). The outcome of 301 (6%)
cases were missing.

Regression method
We used logistic regression to analyze the mediation
success and generalized linear regression to analyze
compensation of mediation. Given the positively
skewed distribution of compensation, we use a general-
ized linear regression model with gamma distribution
[24]. We have investigated the relationship of medi-
ation success and compensation amount with duration
of the case, first claim by patients, age of patients, con-
trolling causes of medical disputes, hospital classifica-
tion (primary, secondary, and tertiary), consequences
and year effects.

Fig. 2 Overview of the approaches and outcomes of mediation. Notes: We analyzed 5614 mediation cases in total, including 5109 cases settled
at the People’s Mediation committee and 505 Yinao cases settled on the spot

Wang et al. BMC Health Services Research (2020) 20:225 Page 4 of 10

Results
Characteristics of mediation cases
The number of medical disputes increased from 1234 in
2013 to 2293 in 2014 and remained above 2000 in 2015
(Table 1). The mean age of patients in our sample was
37.6 and the mean claim amount was CNY357,900. The
top three causes of disputes were: different opinions on
responsibility (for example, patients would ask that hos-
pitals take major or all responsibility for the unsatisfac-
tory medical outcome) (54%), the skill of physicians or
nurses (27%) and discontent with nursing and medicine
(7%). While medical accidents (the unexpected and un-
intentional medical harm, usually out of limited medical
conditions and skills) (5%) and hospital management
issue (4%), and lack of informed consent (3%) accounted
for small proportions of the cases.
More than half of the cases closed with compensation

to patients. The average compensation was CNY60,200,
which is much less than the amount claimed by patients.
Most cases (86%) closed with compensation of less than

CNY100,000. Only 1 % of cases settled with compensa-
tion above CNY500,000.
The average length of time between registration and

closure was 87 days. 1798 (36%) of the medical disputes
were settled within 1 month, 65% of them were solved
within 3 months and 86% were closed within half a year.
140 (3%) of the cases lasted for more than 1 year.
Our sample consists of 2044 tertiary hospitals

(36%), 2627 secondary hospitals (47%) and 943 (17%)
primary hospitals. In China, the hospital accreditation
system posed lower bounds on the number of beds
for each of the hospital grades, of 20, 100, and 500
beds for primary, secondary and tertiary hospitals re-
spectively [25].
General hospitals were more frequently involved in

medical disputes (3792 cases, 68%), followed by township
hospitals (546 cases, 10%). Traditional Chinese medicine
hospitals and maternal and child health hospitals
accounted for 9% of all cases each. Departments like gen-
eral surgery, obstetrics and gynecology, internal medicine

Table 1 Characteristics of 5614 Mediation records, China, 2013–2015

Category Category

Closure date-no.(%) 5614 ≤10 297(10)

2013 1234(22) > 10,≤100 1110(39)

2014 2293(41) > 100,≤500 847(30)

2015 2087(37) > 500 584(21)

Main cause-no.(%) 4927 Hospital grade-no.(%) 5614

Responsibility 2656(54) Primary 943(17)

Skills 1336(27) Secondary 2627(47)

Nursing and Medicine 333(7) Tertiary 2044(36)

Medical Accidents 249(5) Medical Specialties-no.(%) 5072

Management problem 190(4) General surgery 1553(31)

Informed consent 163(3) Obstetrics and gynecology 1290(25)

Time from dispute to closure (days)-no.(%) 4953 Internal medicine 824(16)

Mean 87 Pediatrics 355(7)

≤1 month 1798(36) Ophthalmology and otorhinolaryngology 146(3)

> 1 month,≤3 month 1425(29) Oncology 73(1)

> 3 month,≤6 month 1054(21) Others 831(16)

> 6 month,≤1 year 536(11) Type of medical facility-no.(%) 5614

> 1 year 140(3) General hospital 3792(68)

Closed with compensation-no.(%) 2941 Township hospital 546(10)

Mean (CNY1,000) 60.2 Traditional Chinese Medicine hospital 522(9)

≤10 1203(41) Maternal and Child Health hospital 493(9)

> 10,≤100 1328(45) Specialty hospital 261(5)

> 100,≤500 369(13) Severity-no.(%) 5590

> 500 41(1) Death 2305(41)

First claim by patients-no.(%) 2840 Disability 429(8)

Mean (CNY1,000) 357.9 Others 2856(51)

SOURCE Authors’ analysis of mediation data for 2013–2015

Wang et al. BMC Health Services Research (2020) 20:225 Page 5 of 10

and pediatrics are more often involved with medical dis-
putes, accounting for almost 80% of the cases in total.
About half of the cases were related to injury to pa-

tients. We find that death is the cause of a large propor-
tion (41%) of these cases compared to disability (8%).
Table 2 illustrates the features of Yinao cases that are

reported to People’s Mediation Committee and handled
by mediators on the spot. During 2013–2015, there were
505 cases in total, which is much more than the number
of cases reported by the media. The sudden increase of
Yinao cases from 14 in 2013 to more than 200 in later
years is mainly because People’s Mediation Committee
started to record Yinao cases separately and more clearly
in 2014. Overall, tertiary and secondary hospitals are
more common locations for Yinao. On average, 26
people take part in Yinao and some of them are reported
as “professional Yinao”, which means they were hired by
the patient side and making a living by taking disruptive
actions in hospitals to make hospitals to pay to the pa-
tients. The chaos lasts for 5 hours on average and can be
more than 10 hours in some extreme cases. Most Yinao
cases are related to death of patients, which accounts for
79% of all the Yinao cases.

Regression analysis on success rate of mediation
We use logistic regression model to analyze the outcomes
of mediation. Since ADR is often viewed as complemen-
tary to lawsuits, we classified mediation agreement, recon-
ciliation and withdrawn cases as successful, indicated by
one in our model. Litigation outcome is represented by
zero. Table 3 shows the results of our logistic regression.
Duration has an odds ratio of 0.998 and is significant,

which means for one-day increase in duration the odds
of the case solved through mediation decreases by 0.2%
given other variables are fixed. Compared to

responsibility, the odds ratio of nursing and medicine is
positive and significant, which means its chance to be
solved through mediation is 68.7% higher than responsi-
bility. With tertiary hospitals as reference, we find that
the cases in primary and secondary hospitals are more
likely to be solved through mediation and secondary
hospital cases have the highest chance to be mediated.
After including the consequence of the cases, column

2 shows that other outcomes are more likely to be
solved through mediation compared to cases with death
as results. Since other cases are less severe than death or
disability cases, it is easier to clarify the cases and for the
patients and hospitals to reach agreements.
Considering the trend of mediation, we find that the

chance of successful mediation grows year by year. In
2014, the odds is 1.7 and significant, which means the
chance of success is 70% higher than 2013. In 2015, the
odds is 2.649 and significant, which mean the chance of
success is 165% higher than 2013. We can see the suc-
cess rates of mediation is increasing which may be due
to increased experience.
Finally, the first claim of patients is also included in

our model. Since patients may not be clear about the
compensation amount when they chose mediation, the
sample size dropped to 2525 because of missing values
of first claim amount. Column 4 shows that for CNY10,
000 increase in first claim amount by patients, the odds
decrease by 7% given other factors remain fixed. Com-
pared to responsibility, the odds ratio of nursing and
medicine becomes insignificant, while the odds ratio of
management problem becomes significant at 10% level,
which means its chance to be solved through mediation
is 121% higher than responsibility. The successful medi-
ation trend grows at a higher magnitude compared to
year 2013, reaching 84.8% in 2014 and 167% in 2015.

Table 2 Yinao cases mediated on the spot, China, 2013–2015

Category Category

Yinao date-no.(%) 505 > 20,≤50 168(35)

2013 14(3) > 50 37(8)

2014 237(47) Yinao Duration (hours)-no.(%) 456

2015 254(50) Mean 5

Hospital level-no.(%) 505 ≤5 274(60)

Tertiary 165(33) > 5,≤10 135(30)

Secondary 232(46) > 10 47(10)

Primary 108(21) Severity-no.(%) 505

No. of participants-no.(%) 482 Death 399(79)

Mean 26 Disability 17(3)

≤10 136(28) Others 89(18)

> 10,≤20 141(29)

SOURCE Authors’ analysis of mediation data for 2013–2015

Wang et al. BMC Health Services Research (2020) 20:225 Page 6 of 10

Regression analysis on compensation of mediation
Table 4 shows generalized linear model analysis of com-
pensation. Across all the columns, we can see that the
coefficient of duration is positive and significant. The co-
efficient of duration shows that for one-day increase in

duration, the compensation can increase by 0.2 to 0.3%
(exp (0.002)-1, same below). The coefficient of patients’
age is insignificant all the time.
Compared to responsibility, most other causes of med-

ical disputes are compensated less. In column 1, we can
see that the compensation of nursing and medicine is

Table 3 Logistic regression analysis of the success rate of
mediation, China, 2013–2015

(1) (2) (3) (4)

VARIABLES

Duration of case 0.998*** 0.998*** 0.997*** 0.997***

(0.000) (0.000) (0.000)

(0.001)

First claim by patients 0.993***

(0.002)

Age of patients 1.001 1.002 1.001 1.000

(0.002) (0.002) (0.002) (0.003)

Main Cause

Medical Accidents 1.357 1.515 1.730** 2.738**

(0.339) (0.387) (0.451) (1.154)

Skills 0.948 0.885 1.009 1.119

(0.103) (0.099) (0.116) (0.175)

Nursing and Medicine 1.687** 1.605** 1.693** 1.225

(0.400) (0.382) (0.406) (0.332)

Informed consent 0.713 0.664* 0.655* 0.604*

(0.170) (0.162) (0.162) (0.182)

Management problem 1.376 1.418 1.467 2.210*

(0.417) (0.437) (0.459) (0.956)

Hospital level

Primary 1.836*** 1.835*** 1.844*** 1.484**

(0.271) (0.277) (0.279) (0.292)

Secondary 2.090*** 2.029*** 2.047*** 1.919***

(0.217) (0.214) (0.218) (0.277)

Consequence

Disabled 0.891 0.869 0.926

(0.143) (0.141) (0.203)

Others 2.530*** 2.524*** 1.926***

(0.270) (0.272) (0.297)

Year

Year 2014 1.700*** 1.848***

(0.202) (0.296)

Year 2015 2.649*** 2.672***

(0.348) (0.480)

Constant 5.604*** 3.740*** 2.247*** 3.371***

(0.667) (0.478) (0.339) (0.825)

Observations 4313 4313 4313 2525

Robust se in parenthese
*** p < 0.01, ** p < 0.05, * p < 0.1

Note: Mediation agreement, reconciliation and withdrawn cases are classified
as successful. Litigation cases are classified as unsuccessful

Table 4 Generalized linear regression analysis of the
compensation of mediation, China, 2013–2015

(1) (2) (3) (4)
VARIABLES

Duration of case 0.003*** 0.003*** 0.003*** 0.002***

(0.000) (0.000) (0.000) (0.000)

First claim by patients 0.014***

(0.001)

Age of patients −0.001 −0.002 −0.002 0.001

(0.001) (0.001) (0.001) (0.001)

Main Cause

Medical Accidents 0.124 −0.081 −0.107 − 0.116

(0.147) (0.121) (0.120) (0.119)

Skills −0.167** − 0.016 − 0.042 0.035

(0.077) (0.078) (0.078) (0.077)

Nursing and Medicine −0.785*** −0.749*** − 0.771*** −0.530***

(0.138) (0.103) (0.102) (0.102)

Informed consent −0.776*** −0.716*** − 0.735*** −0.391**

(0.227) (0.199) (0.201) (0.174)

Management problem −0.758*** −0.741*** − 0.758*** −0.464***

(0.220) (0.160) (0.154) (0.117)

Hospital level

Primary −0.167 −0.222** −0.223** − 0.241***

(0.108) (0.096) (0.097) (0.092)

Secondary −0.213*** −0.216*** − 0.217*** −0.191***

(0.075) (0.075) (0.074) (0.070)

Consequence

Disabled 0.016 −0.004 0.097

(0.110) (0.109) (0.098)

Others −1.256*** −1.261*** −0.841***

(0.067) (0.066) (0.072)

Year

Year 2014 0.035 0.055

(0.090) (0.074)

Year 2015 −0.147* −0.069

(0.087) (0.080)

Constant 1.821*** 2.303*** 2.339*** 1.433***

(0.088) (0.090) (0.106) (0.121)

Observations 2764 2764 2764 1726

Robust se in parenthese
*** p < 0.01, ** p < 0.05, * p < 0.1

Wang et al. BMC Health Services Research (2020) 20:225 Page 7 of 10

54.4% less than responsibility, followed by inform con-
sent and management problem. Medical accident is as-
sociated with higher compensation compared to
responsibility, while the result is not significant.
The coefficient of secondary hospital is negative and sig-

nificant in all models. In column 1, we find that secondary
hospitals are compensated 19.2% less than tertiary hospi-
tals on average. After controlling for consequence and
year effects, the compensation is about 19.5% less than
tertiary hospitals.
After including first claim amount by patients, we find

for CNY10,000 increase in the claimed amount, the
compensation will increase by 14.1%. In column 4, we
find that the coefficient of disabled is insignificant, which
means the compensation of disabled is similar to death
cases on average. The coefficient for others category is
negative and significant, which means the less severe
cases will get 56.9% less than death cases on average.
Although the coefficient of year is significant at 10%

level in 2015 in column 3, the year effects becomes in-
significant in column 4. It is reasonable because the
compensation standard doesn’t change that much in dif-
ferent years.

Discussion

Mediation as a buffer
Our data shows that 89% of the cases were successfully
handled through mediation without going to litigation
and the average length of mediation is 87 days, which
are comparable to the ADR practice in America and
Canada [26]. Our result highlights the importance of
communication between different sides. We find that
mediation can help avoid “nuisance lawsuits” by offering
suggestions to patients and hospitals and encourage
them to give up unreasonable demands. 4% of cases
were rejected and 11% of cases were withdrawn, which
reflects that many cases can be settled after clarification
of the medical and legal issues, saving time and re-
sources. Mediation can offer physicians and patients the
chance to talk, negotiate and apologize, which may also
improve the doctor-patient relationship.
More medical and legal knowledge is helpful for patients

to shape a reasonable expectation on the outcome of med-
ical services and compensation amount, which can ease the
conflicts between both sides and help build consensus. Our
regression analysis results show that both longer duration
and higher claim amount are associated with less probabil-
ity to be solved by mediation and higher compensation.
Compared to death cases, less severe cases are more likely
to be settled through mediation, and closed with less com-
pensation. It reflects that patients actually have the right
idea about the severity of medical services’ consequence,
while in general they may initially claim more than the ac-
tual compensation amount received.

PMC, as a third-party, has gradually gained popularity.
More specifically, the more flexible process and easier
access for patients can help save a lot of time and money
in contrast with lawsuits [16, 19]. The neutral stance of-
fers PMC a more objective view compared to mediation
conducted by health administrative departments [27].
Given all these advantages and to avoid the previous

privately and costly settlement between medical institu-
tion and patients [28], several regions have implemented
special regulations on the ways to solve medical disputes
and mediation is almost the first choice for both patients
and hospitals there. In 2013, the government of Guang-
dong Province issued a provincial notice on prevention
and resolution of medical disputes, which stated that ne-
gotiation between hospital and patients is only appropri-
ate for claims below CNY10,000 [29]. Similarly, Baoji
city in Shaanxi Province also set a cap on the compensa-
tion for negotiation approach in 2015 [30].

Hospital management can play a role in dispute
prevention and resolution
Studies have shown the possibility to reduce disputes by
changing the procedure of dealing with medical disputes
[31] and early detection/intervention of medical disputes
[32].
In our analysis, we find that tertiary hospitals and de-

partments like surgery, obstetrics and gynecology, and
internal medicine are more likely to face medical dis-
putes. According to the requirements of the National
Health Commission (previously known as the Ministry
of Health) [33], tertiary hospitals are designed to treat
patients with serious conditions as well as common dis-
eases and supposed to provide higher-quality of services,
so they are more likely to be involved in complex med-
ical disputes with associated higher compensation. Our
results show that cases in primary and secondary hospi-
tals are more likely to be solved through mediation and
end up with less compensation compared to tertiary
hospitals. Surgery, obstetrics and gynecology, and in-
ternal medicine departments are the top three depart-
ments that are involved in medical disputes. The
analysis on main causes show that most of the medical
disputes can be attributed to responsibility and skills, it
is more demanding for tertiary hospitals and the top
three departments to enhance their risk management
practices and help improve the skills of relative services.

More help is needed to handle Yinao cases
The average duration (5 h) and number of people partici-
pating (26) in Yinao depict the chaotic scene faced by hos-
pitals during 2013–2015. In China, legal progress has been
made to prohibit Yinao. However, we find that most Yinao
cases are related with the death or disability of patients
(82%). Consequently, it may be hard to peacefully resolve

Wang et al. BMC Health Services Research (2020) 20:225 Page 8 of 10

Yinao and more support should be given to guarantee the
safety of physicians. In 2016, we conducted interviews
with the mediators and the director of PMC. Based on
their working experiences, we found that Yinao is not well
prohibited and that the police force didn’t go to the spot
to calm down the issue every time.
Although mediators can go and help deal with Yinao

cases, it’s necessary for police department and procurato-
rate to participate and solve Yinao at the time. As the
“Medical Disputes Prevention and Treatment Regulations”
in 2018 has stated that hospitals should report Yinao to
the police immediately, more analysis should be done to
evaluate whether police forces have the necessary re-
sources to handle Yinao cases. The influences of Yinao are
not just limited within one case. In China, extreme Yinao
cases will be on the news and generate social media dis-
cussion. It is necessary for the media to advocate and
emphasize the legal ways to solve disputes instead of just
reporting the fact of Yinao or violence against physicians.

Limitations
Our study has several limitations. First, the finding that me-
diation is time-saving compared to litigation maybe im-
pacted by the observation period. We only included cases
that were settled during the observation period so there
could be some very long ongoing cases that were not re-
solved during this time and so the average length of reso-
lution may be under-estimated. Furthermore, some serious
cases may bypass the mediation method and choose the
legal approach directly. Second, due to data availability, our
analysis focuses on the mechanism and results of cases han-
dled through PMC in Guangdong Province, further analysis
could include data from other provinces to confirm if our
findings in Guangdong apply at the national level. Third,
our data didn’t cover the period after 2015, therefore we
cannot capture the effects of litigation change before and
after Yinao was included in Criminal Law in 2015. Since
this law change is expected to deter patients from taking
illegal actions, future analysis would be valuable to under-
stand its effect on Yinao cases in more detail.

Conclusion
Through the analysis of the mediation committee work
in China, we find that mediation can help substantially
reduce conflicts between physicians and patients to
avoid litigations, thus saving time and money for both
parties. While for illegal actions taken by patients like
Yinao, not only mediators but also the police are needed
to fully handle such cases. By setting up a neutral third-
party institution like the mediation committee together
with an appropriate legal framework, healthcare systems
can also better handle the disputes between hospitals
and patients to improve the patient-provider relation-
ships and public satisfaction.

Abbreviations

ADR: Alternative Dispute Resolution; CNY: China Yuan; GDP: Gross Domestic
Product; PMC: People’s Mediation Committee

Acknowledgements

We wish to thank Min Huang for her extremely helpful suggestions on the
early design of this paper. We are grateful for Xinyue Dong for her help and
comments on the first draft.

Authors’ contributions

MW designed and collected data for the qualitative study. MY cleaned the
data for the quantitative study. MW and TB analyzed the quantitative data.
MW, GL, HZ, TB, MY and YC contributed to the methodological elaboration
of the results. MW wrote the first draft of the paper, the others commented
on subsequent drafts. All authors read and approved the final manuscript.

Funding

Data analysis is supported by China Medical Board, “The PKU-CMB Lab for
Health Economics Research: Health Financing, Delivery, and Economic Pros-
perity” (15–217, 2016–2018) and National Natural Science Foundation of
China (71773002). The funder of the study had no role in study design, data
collection, data analysis, data interpretation, or writing of the report.

Availability of data and materials

The data that support the findings of this study are available from People’s
Mediation Committee in Guangdong but restrictions apply to the availability
of these data, which were used under license for the current study, and so
are not publicly available. Data are however available upon research request
and with permission of People’s Mediation Committee in Guangdong.

Ethics approval and consent to participate

No administrative permissions were required to access and use the
mediation records described in our study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Author details

1National School of Development, Peking University, Beijing 100871, China.
2People’s Mediation Committee in Guangdong, Guangzhou 510095, China.
3Sichuan Development Center for Healthy Aging, Chengdu 610094, China.
4Department of Economics, Louisiana State University, Baton Rouge 70820,
USA. 5China Hospital Development Institute, Shanghai Jiao Tong University,
Shanghai 200025, China.

Received: 26 July 2019 Accepted: 26 February 2020

References

1. Partnership CJS. Deepening health reform in China: building high-quality

and value-based service delivery; 2016.
2. Blumenthal D, Hsiao W. Privatization and its discontents-the evolving

Chinese health care system. N Engl J Med. 2005;353(11):1165–70.
3. Tucker JD, Cheng Y, Wong B, Gong N, Nie JB, Zhu W, McLaughlin MM, Xie

RS, Deng YH, Huang MJ, et al. Patient-physician mistrust and violence
against physicians in Guangdong Province, China: a qualitative study. BMJ
Open. 2015;5(10):e008221.

4. Yao S, Zeng Q, Peng M, Ren S, Chen G, Wang J. Stop violence against
medical workers in China. J Thoracic Disease. 2014;6(6):141–5.

5. Wu D, Wang Y, Lam KF, Hesketh T. Health system reforms, violence
against doctors and job satisfaction in the medical profession: a cross-
sectional survey in Zhejiang Province, Eastern China. BMJ Open. 2014;
4(12):e006431.

6. Saeki K, Okamoto N, Tomioka K, Obayashi K, Nishioka H, Ohara K,
Kurumatani N. Work-related aggression and violence committed by
patients and its psychological influence on doctors. J Occup Health.
2011;53(5):356–64.

Wang et al. BMC Health Services Research (2020) 20:225 Page 9 of 10

7. Pan Y, Yang X, He JP, Gu YH, Zhan XL, Gu HF, Qiao QY, Zhou DC, Jin HM. To
be or not to be a doctor, that is the question: a review of serious incidents
of violence against doctors in China from 2003-2013. J Public Health. 2015;
23(2):111–6.

8. Hesketh T, Wu D, Mao LN, Ma N. Violence against doctors in China. BMJ.
2012;345.

9. Zheng T: Analysis on generation mechanism and regulation approaches of
interests-seeking Yinao. Journal of Yunnan University Law Edition 2016,
29(3):51–57.

10. Tianjin procuratorate approved the arrest of criminal suspects of hospital
violence of the Armed Police Froce Hospital. https://www.guancha.cn/
society/2018_07_17_464513.shtml. Accessed 17 July 2018.

11. Wang H, Shi J, Cheng W. Research review on medical disputes in China.
Med Jurisprud. 2016;12(1):83–7.

12. He AJ, Qian J. Explaining medical disputes in Chinese public hospitals: the
doctor-patient relationship and its implications for health policy reforms.
Health Econ Policy L. 2016;11(4):359–78.

13. Chen B. Analysis of the inversion for responsibility of proof citation in
medical tort actions under the background of Tort Liability Law of P.R.C.
China Health Law. 2010;5:53–5.

14. Ministry of Health, Ministry of Public Security. Notice on maintaining the
order of medical institutions 2012. http://www.nhfpc.gov.cn/yzygj/s7658/2
01205/fb53a6594ef2413ca7ef661859a5b7de. Accessed 17 July 2017.

15. Friedson AI. Medical malpractice damage caps and provider reimbursement.
Health Econ. 2017;26(1):118–35.

16. Li H, Wu X, Sun T, Li L, Zhao X, Liu X, Gao L, Sun Q, Zhang Z, Fan L. Claims,
liabilities, injures and compensation payments of medical malpractice
litigation cases in China from 1998 to 2011. BMC Health Serv Res. 2014;
14(1):181–9.

17. DeVille KA. The jury is out: pre-dispute binding arbitration agreements for
medical malpractice claims – law, ethics, and prudence. J Legal Med. 2007;
28(3):333–95.

18. Nakanishi T. New communication model in medical dispute resolution in
Japan. Bull Yamagata Univ Med Sci. 2013;31:1–8.

19. Shi M, Zhang H, Cheng Q. Analysis on the causes, distribution and
compensation of 5012 medical damage disputes. Med Jurisprud. 2015;
7(6):42–8.

20. Pegalis SE, Bal BS. Closed medical negligence claims can drive patient safety
and reduce litigation. Clin Orthop Relat R. 2012;470(5):1398–404.

21. Sohn DH, Sonny Bal B. Medical malpractice reform: the role of alternative
dispute resolution. Clin Orthop Relat R. 2012;470(5):1370–8.

22. Zhao C, Zhu B. Empirical study and countermeasures discussion on medical
disputes -Based on the analysis on cases accepted by Gulou District
People’s Mediation Committee of medical disputes from 2010 to 2013. Acta
Universitatis Medicinalis Nanjing (Social Sciences). 2015;5:349–54.

23. National Bureau of Statistics of the People’s Republic of China: China
Statistical Yearbook, vol. 36; 2017.

24. Cohen SM, Kim J, Roy N, et al. Factors influencing the health care
expenditures of patients with laryngeal disorders. Otolaryngol Head Neck
Surg. 2012;147(6):1099–107.

25. National Health and Family Planning Commission. Basic Measurements for
Medical Institutions (Pilot Draft) 2017. http://www.ncwjw.gov.cn/html/
faguizhongxin/201708286822.html. Accessed 8 Aug 2018.

26. Szmania SJ, Johnson AM, Mulligan M. Alternative dispute resolution in
medical malpractice: a survey of emerging trends and practices. Conflict Res
Q. 2008;26(1):71–96.

27. Li C. Empirical research on the third-party mediation mechanism in medical
dispute. Chin Health Serv Manag. 2014;31(2):125–7.

28. Niu D, Hou F. Discussing the cause of private remedy on medical dispute
and its countermeasures. Chin Health Serv Manag. 2011;28(6):442–4.

29. Government of Guangdong Province. Approaches for preventing and
dealing with medical disputes in Guangdong province 2013. http://zwgk.gd.
gov.cn/006939748/201304/t20130409_371711.html. Accessed 18 Sept 2014.

30. City Government of Baoji. Interim procedures for mediation of medical
disputes in Baoji city 2014. http://baoji.gov.cn/site/11/html/276/290/271688.
htm. Accessed 8 July 2015.

31. Lu Z. Study on the influence of medical disputes prevention based on the
process reformation of cybernetic. Health Econ Res. 2015;338(6):46–9.

32. Xu P, Fan Z, Li T, et al. Preventing surgical disputes through early detection
and intervention: a case control study in China. BMC Health Serv Res. 2015;
15:5.

33. Ministry of Health. General hospital classification and management
standards (pilot draft) 1989. https://wenku.baidu.com/view/e1c9493710661
ed9ad51f3ec.html. Accessed 19 Oct 2015.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

Wang et al. BMC Health Services Research (2020) 20:225 Page 10 of 10

https://www.guancha.cn/society/2018_07_17_464513.shtml

https://www.guancha.cn/society/2018_07_17_464513.shtml

http://www.nhfpc.gov.cn/yzygj/s7658/201205/fb53a6594ef2413ca7ef661859a5b7de

http://www.nhfpc.gov.cn/yzygj/s7658/201205/fb53a6594ef2413ca7ef661859a5b7de

http://www.ncwjw.gov.cn/html/faguizhongxin/201708286822.html

http://www.ncwjw.gov.cn/html/faguizhongxin/201708286822.html

http://zwgk.gd.gov.cn/006939748/201304/t20130409_371711.html

http://zwgk.gd.gov.cn/006939748/201304/t20130409_371711.html

http://baoji.gov.cn/site/11/html/276/290/271688.htm

http://baoji.gov.cn/site/11/html/276/290/271688.htm

https://wenku.baidu.com/view/e1c9493710661ed9ad51f3ec.html

https://wenku.baidu.com/view/e1c9493710661ed9ad51f3ec.html

BioMed Central publishes under the Creative Commons Attribution License (CCAL). Under
the CCAL, authors retain copyright to the article but users are allowed to download, reprint,
distribute and /or copy articles in BioMed Central journals, as long as the original work is
properly cited.

Abstract
Background
Methods
Results
Conclusion
Background
Methods
Mediation in China
Data sample
Regression method
Results
Characteristics of mediation cases
Regression analysis on success rate of mediation
Regression analysis on compensation of mediation
Discussion
Mediation as a buffer
Hospital management can play a role in dispute prevention and resolution
More help is needed to handle Yinao cases
Limitations
Conclusion
Abbreviations
Acknowledgements
Authors’ contributions
Funding
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
Competing interests
Author details
References
Publisher’s Note

Psychological Methods
Improved Inference in Mediation Analysis: Introducing
the Model-Based Constrained Optimization Procedure
Davood Tofighi and Ken Kelley
Online First Publication, March

1

9, 2020. http://dx.doi.org/10.1037/met0000259

CITATION
Tofighi, D., & Kelley, K. (2020, March 19). Improved Inference in Mediation Analysis: Introducing the
Model-Based Constrained Optimization Procedure. Psychological Methods. Advance online
publication. http://dx.doi.org/10.1037/met0000259

Improved Inference in Mediation Analysis: Introducing the Model-Based
Constrained Optimization Procedure

Davood Tofighi
University of New Mexico

Ken Kelley
University of Notre Dame

Abstract
Mediation analysis is an important approach for investigating causal pathways. One approach used in
mediation analysis is the test of an indirect effect, which seeks to measure how the effect of an independent
variable impacts an outcome variable through 1 or more mediators. However, in many situations the proposed
tests of indirect effects, including popular confidence interval-based methods, tend to produce poor Type I
error rates when mediation does not occur and, more generally, only allow dichotomous decisions of “not
significant” or “significant” with regards to the statistical conclusion. To remedy these issues, we propose a
new method, a likelihood ratio test (LRT), that uses nonlinear constraints in what we term the model-based
constrained optimization (MBCO) procedure. The MBCO procedure (a) offers a more robust Type I error rate
than existing methods; (b) provides a p value, which serves as a continuous measure of compatibility of data
with the hypothesized null model (not just a dichotomous reject or fail-to-reject decision rule); (c) allows
simple and complex hypotheses about mediation (i.e., 1 or more mediators; different mediational pathways);
and (d) allows the mediation model to use observed or latent variables. The MBCO procedure is based on a
structural equation modeling framework (even if latent variables are not specified) with specialized fitting
routines, namely with the use of nonlinear constraints. We advocate using the MBCO procedure to test
hypotheses about an indirect effect in addition to reporting a confidence interval to capture uncertainty about
the indirect effect because this combination transcends existing methods.

Translational Abstract
Mediation analysis has become one of the most important approaches for investigating causal pathways. One
instrument used in mediation analysis is a test of indirect effects that seeks to measure how the effect of an
independent variable impacts an outcome variable through one or more mediators. However, in many
situations the proposed tests of indirect effects, including popular confidence interval-based methods, tend to
produce too few or too many false positives (Type I errors) and are commonly used to make only dichotomous
decisions about acceptance (“not significant”) or rejection (“significant”) of a null hypothesis. To remedy these
issues, we propose a new procedure to test an indirect effect. We call this new procedure the model-based
constrained optimization (MBCO) procedure. The MBCO procedure (a) more accurately controls the false
positive rate than existing methods; (b) provides a p value, which serves as a continuous measure of
compatibility of data with the hypothesized null model (not just a dichotomous reject or fail-to-reject decision
rule); (c) allows simple and complex hypotheses about mediation (i.e., one or more mediators; different
mediational pathways); and (d) allows the mediation model to use observed or latent variables common in
psychological research. We advocate using the MBCO procedure to test hypotheses about an indirect effect
in addition to reporting a confidence interval to capture uncertainty about the indirect effect.

Keywords: indirect effect, mediation analysis, confidence interval, likelihood ratio, model-comparison test

Supplemental materials: http://dx.doi.org/10.1037/met0000259.supp

Mediation analysis has become one of the most important and
widely used methods to study causal mechanisms in psychology,
management, education, and other related fields (e.g., Ato García,

Vallejo Seco, & Ato Lozano, 2014; Boies, Fiset, & Gill, 2015;
Bulls et al., 2017; Carmeli, McKay, & Kaufman, 2014; Deković,
Asscher, Manders, Prins, & van der Laan, 2012; Ernsting, Knoll,
Schneider, & Schwarzer, 2015; Graça, Calheiros, & Oliveira,
2016; Haslam, Cruwys, Milne, Kan, & Haslam, 2016; Koning,
Maric, MacKinnon, & Vollebergh, 2015; MacKinnon, 2008; Mo-
lina et al., 2014). Researchers have proposed several methods to
test for the presence of mediation (MacKinnon, Lockwood, Hoff-
man, West, & Sheets, 2002), including the widely recommended
(a) Monte Carlo confidence interval (CI; MacKinnon, Lockwood,
& Williams, 2004; Tofighi & MacKinnon, 2016), also known as
parametric bootstrap method (Efron & Tibshirani, 1993); (b) per-

X Davood Tofighi, Department of Psychology, University of New
Mexico; X Ken Kelley, Department of Information Technology, Analyt-
ics, and Operations, University of Notre Dame.

Correspondence concerning this article should be addressed to Davood
Tofighi, Department of Psychology, University of New Mexico, Logan Hall,
MSC03-2220, Albuquerque, NM 87131– 0001. E-mail: dtofighi@unm.edu

T
hi

s
do

cu
m

en
t

is
co

py
ri

gh
te

d
by

th
e

A
m

er
ic

an
P

sy
ch

ol
og

ic
al

A
ss

oc
ia

ti
on

or
on

e
of

it
s

al
li

ed
pu

bl
is

he
rs

.
T

hi
s

ar
ti

cl
e

is
in

te
nd

ed
so

le
ly

fo
r

th
e

pe
rs

on
al

us
e

of
th

e
in

di
vi

du
al

us
er

an
d

is
no

t
to

be
di

ss
em

in
at

ed
br

oa
dl

y.

Psychological Methods
© 2020 American Psychological Association 2020, Vol. 2, No. 999, 000
ISSN: 1082-989X http://dx.doi.org/10.1037/met0000259

1

https://orcid.org/0000-0001-8523-7776

https://orcid.org/0000-0002-4756-8360

mailto:dtofighi@unm.edu

http://dx.doi.org/10.1037/met0000259

centile and bias-corrected (BC) bootstrap resampling CI (Bollen &
Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et al., 2004;
Shrout & Bolger, 2002); (c) profile-likelihood CI method (Folmer,
1981; Neale & Miller, 1997; Pawitan, 2001; Pek & Wu, 2015); and
(d) joint significance test (Kenny, Kashy, & Bolger, 1998; MacK-
innon et al., 2002). Henceforth, we will use the term CI-based
methods to emphasize a dual role that CIs have played in the
mediation analysis literature. In addition to providing a range of
plausible values for a population indirect effect, CIs have also
controversially been used to test the null hypothesis of no medi-
ation (i.e., a zero indirect effect) at the � level with a (1-�) 100%
CI. However, testing an indirect effect is complicated because the
indirect effect is the product of the coefficients along the mediation
chain (MacKinnon, 2008). Further, each of the methods mentioned
earlier has key limitations, many of which researchers commonly
encounter. Thus, even though these procedures are currently rec-
ommended, that recommendation in research is questionable.

Many popular testing methods in mediation models are limited
because they lack robustness of the observed Type I error rate.
That is, the empirical probability of falsely rejecting the null
hypothesis about an indirect effect does not equal, and can fre-
quently fall outside of, a robustness interval for the Type I error
rate, such as Bradley’s (1978) liberal interval [0.5�, 1.5�], where
� is the nominal significance level (Biesanz, Falk, & Savalei,
2010; Tofighi & Kelley, 2019). The idea of using .025–.075 as an
acceptable departure from the idealized properties of the statistical
procedures, when � � .05, acknowledges that a null hypothesis
procedure may not work perfectly but is still useful. Bradley’s
(1978) liberal criterion accepts a procedure as robust when violations
of the model’s assumptions are small enough not to fundamentally
change the statistical conclusion. The importance of the conclusion in
the context of mediation, for example, is when sample size is less than
100 and � � .05. Several methods, including widely recommended
CI-based methods, can result in Type I error rates that fall outside of
Bradley’s liberal interval [0.025 .075] (Tofighi & Kelley, 2019) and
are, therefore, unsatisfactory. We propose a method that is more
robust than other procedures because it better satisfies Bradley’s
(1978) liberal criterion and has desirable statistical properties that
allow its wide use, including in situations beyond which existing
methods are applicable.

Currently used methods of testing mediation are limited because
these tests are commonly used to make only dichotomous deci-
sions about acceptance (“not significant”) or rejection (“signifi-
cant”) of a null hypothesis. These dichotomous decisions provide
a false sense of certainty about a model and data and are not
generally recommended (Amrhein, Trafimow, & Greenland, 2019;
Wasserstein, Schirm, & Lazar, 2019). Further, these methods do
not offer either a p value, a continuous measure of compatibility
between data and the null hypothesis (Greenland et al., 2016), or
an additional means of measuring statistical evidence, such as a
likelihood ratio (Blume, 2002). As recommended by Wilkinson
and the Task Force on Statistical Inference, American Psycholog-
ical Association, Science Directorate (1999), reporting exact p
values, CIs, and effect sizes, and not just noting “reject” or
“fail-to-reject” is the ideal for science to progress. None of the
CI-based tests, by their very nature, offers additional ways of
evaluating the strength of evidence against a null hypothesis of no
mediation. A likelihood ratio, however, is a continuous measure
that can compare the goodness of fit of two competing models

under standard assumptions, as measured by the ratio of the
models’ maximized likelihoods. One model, a null model, repre-
sents and satisfies the null hypothesis of no mediation (e.g., zero
indirect effect), and the other model, a full model, shows that
mediation does occur, as posited by a researcher (e.g., the indirect
effect is nonzero). Further, a likelihood value can be used to
compute information fit indices, such as Akaike information cri-
terion (AIC; Akaike, 1974) and Bayesian information criterion
(BIC; Schwarz, 1978); such indices compare the fit of the full and
null models while penalizing each model for a lack of parsimony
(i.e., the number of model parameters).

To address these limitations, we propose a model-based con-
strained optimization (MBCO) method as a new procedure that tests
any function representing an indirect effect1 and that provides a
formal way to recast testing simple and complex hypotheses about an
indirect effect into a model-comparison framework. The MBCO
procedure offers two formal mechanisms to evaluate hypotheses
about indirect effects in the model-comparison framework. First, the
MBCO procedure offers a likelihood ratio test (LRT) statistic, which
we term LRTMBCO, that has a large sample chi-squared distribution
and provides a p value to test hypotheses about an indirect effect.
Second, the MBCO procedure compares the fit of two models in
terms of a likelihood ratio and information fit indices, such as the AIC
and BIC, as well as their generalizations.

In the model-comparison framework, we estimate the null and full
models representing different hypotheses about an indirect effect
under a null and an alternative hypothesis, respectively (see Maxwell,
Delaney, & Kelley, 2018). A null model represents the null hypoth-
esis about the indirect effect. Unlike existing approaches to model
comparisons, the MBCO procedure can estimate the null mediation
model by using nonlinear constraints for testing a null hypothesis in
mediation models. This innovative application of nonlinear con-
straints can be used to test both simple and complex hypotheses in
mediation models. The MBCO procedure also allows the indirect
effect of a full model, posited by a researcher, to be freely estimated.
For example, a full model is a mediation model shown in Figure 1.
Although model comparisons using the likelihood ratio test are com-
mon in techniques such as multilevel modeling (Raudenbush & Bryk,
2002) and structural equation modeling (SEM; Kline, 2016), in almost
all such situations the parameters are subject to linear constraints.
Testing indirect effects is problematic because constraining the indi-
rect effect, H0: �1�2 � 0, is inherently a nonlinear constraint optimi-
zation problem as the constraint is the product of two parameters
(Snyman, 2005).

The MBCO procedure offers several advantages over the exist-
ing, recommended methods. First, the MBCO procedure offers the
test statistic LRTMBCO that has better statistical properties than the
widely recommended approaches in the literature. This test statis-
tic, widely used in statistics, has desirable properties for testing
complex null hypotheses (Cox & Hinkley, 2000). The LRTMBCO
yields more robust Type I error rates than the currently recom-
mended methods when the null hypothesis of no mediation occurs.
Second, the MBCO procedure offers researchers multiple ways of
evaluating a null hypothesis of an indirect effect rather than only

1 More formally, a function of model parameters should be a smooth
function, which is a function that has continuous derivatives up to a desired
order (Weisstein, 2018).

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

2 TOFIGHI AND KELLEY

allowing a dichotomous reject or fail conclusion that has been
criticized by researchers (e.g., Cohen, 1994; Harlow, Mulaik, &
Steiger, 1997; Wasserstein et al., 2019). The MBCO procedure
also allows researchers to test simple and complex hypotheses for
a variety of mediation models involving multiple mediators (ob-
served or latent) in a flexible manner. This flexibility aids re-
searchers interested in complex causal models that are often pos-
ited by theories and can be implemented in an SEM framework.
Such flexibility and desirable statistical performance make the
MBCO procedure a desirable advancement for mediation studies.
To make our proposed method immediately available to research-
ers, we provide detailed instructions along with the necessary R (R
Core Team, 2019) code (see the online supplemental materials).
Nonlinear constraint optimization algorithms to implement the
MBCO procedure and to compute the LRTMBCO are already
implemented in at least one R software package, and we expect
other software packages to soon make such allowances as well.

The Model-Based Constrained Optimization Procedure

We begin by explaining the MBCO procedure conceptually and
then discussing the required steps required to perform this proce-
dure. Without loss of generality, we elaborate on the MBCO
procedure to test the zero indirect effect null hypothesis, H0:
�1�2 � 0, for the single-mediator model example (see Figure 1).
In the Empirical Example of Using the MBCO Procedure on a
Complex Mediation Model section, we discuss extensions of the
MBCO procedure to the more elaborate, parallel two-mediator
model (see Figure 2). In the Simulation Study section, we briefly
extend the MBCO procedure to a sequential two-mediator model
and then compare the performance of the MBCO procedure with
the most commonly used tests of indirect effects for both a single-
mediator model and a sequential two-mediator model.

The MBCO procedure recasts hypotheses about indirect effects
into a model-comparison framework. To illustrate, consider the
single-mediator model (see Figure 1), which can be represented by the
following regression equations:

M � �0M � �1X � εM (1)

Y � �0Y � �2M � �3X � εY , (2)

where �1 is the effect of instruction (X � 1 corresponds to
instruction to use imagery rehearsal; X � 0 corresponds to instruc-
tion to use repetition rehearsal) on the use of imagery (imagery);
�2 is the effect of imagery on the number of words recalled (recall)
controlling for instruction, and �3 is the direct effect of the
instruction on recall controlling for imagery. �0M and �0Y denote
the intercepts while εM and εY are residual terms. The indirect
effect of instruction on recall through imagery is defined as
whether instruction to use imagery increases the use of imagery
and then, in turn, increases the number of words participants
recalled. Under the no-omitted-confounder assumption, which as-
sumes that a variable should not be omitted from the model that
would influence both the mediator and the outcome variables, the
indirect effect equals the product of two coefficients, �1�2 (Imai,
Keele, & Tingley, 2010; Judd & Kenny, 1981; Valeri & Vander-
Weele, 2013; VanderWeele, 2010). Suppose that we are interested
in testing whether the indirect effect of instruction on recall
through imagery is zero. We can use the following null hypothesis
to answer this question:

H0: �1�2 � 0

H1: �1�2 � 0

To recast the hypotheses about the indirect effect into a model-
comparison framework, we estimate two mediation models: a full
(alternative) model and a null (restricted) model (Maxwell et al.,
2018). Which model to estimate first is arbitrary. For our example,
the full model is the single-mediator model in (1)–(2). Note that
the full model does not constrain the value of the indirect effect,
�1�2. Next, for the null model, we estimate a mediation model in
which the indirect effect �1�2 is constrained to equal 0. Both the
full and the null model can be represented by the set of equations
in (1) and (2) with the limitation that �1�2 is constrained to be 0
in the null model. The null model is nested within the full model;

Figure 1. An example of a randomized single-mediator model. Instruction (X) is a random assignment with two
conditions: instruction to use mental imagery rehearsal (X � 1) versus instruction to use repetition (X � 0). Imagery
is a self-reported score of using mental imagery to memorize words, and recall is the number of words out of a total
of 20 words that each student correctly remembered at the end of the experiment. A solid arrow between two variables
indicates a direct effect of the variable on the left on the other variable. Greek letters denote population values, where
�s denote the regression (path) coefficients, with �1 being the treatment effect on imagery, �2 being the partial effect
of imagery on recall controlling for treatment effect, and �3 is the partial effect of instruction on recall controlling for
imagery. Terms εY and εM denote residuals that are assumed to be normally distributed.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

3IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

that is, by setting the indirect effect to zero in the full model, we
would obtain the null model.

The final step of the MBCO procedure is to formally compare
the two models. One such approach compares the fit of the two
models by computing LRTMBCO. To compute the LRTMBCO, we
first calculate the deviance, denoted by D, for each model:

D � �2LL, (3)

where (LL) denotes the log-likelihood. The equation proposes that
deviance equals twice the negative of the maximum value of the
log-likelihood (LL) of the corresponding model. Deviance is a
positive number quantifying the “badness” of fit or the “misfit” of
a model. The larger the value of deviance for the same data set, the
worse the fit of the model to that sample data.

Next, we compute the LRTMBCO as the difference between the
deviance for the null and full models. Under the null hypothesis,
LRTMBCO has a large sample chi-squared distribution (Wilks,
1938) as follows:

LRTMBCO � Dnull � Dfull � �
2(df), (4)

where the degrees of freedom, df, equal the difference in the number
of the free parameters between the two models. The larger values of
the LRTMBCO indicate that the null model, which estimates no indi-
rect effect, fits the data worse than does the full model in which the
indirect effect is freely estimated. To obtain a p-value for the
LRTMBCO, we calculate the upper tail probability from the chi-
squared distribution (i.e., the area larger than the observed chi-squared
statistic) with the specified degrees of freedom. While using a likeli-
hood ratio test is standard in SEM, the LRTMBCO uses the nonlinear

constraint of �1�2 � 0 that has not been previously explored for
testing the null hypothesis of no indirect effect. The nonlinear con-
straint of �1�2 � 0 is, in fact, much more difficult to implement than
is simply fixing a single parameter to a specified value (such as zero).
Because of this nonlinear constraint of �1�2 � 0, specialized algo-
rithms must be used (discussed below), which at present are not
implemented in all programs.

Additionally, the MBCO procedure can compare the fit of the
null and full models in terms of information fit indices such as the
AIC (Akaike, 1974) or BIC (Schwarz, 1978). Such indices con-
sider the fit of the model (as measured by the maximized values of
likelihood for the model) to the data while penalizing the model for
estimating more parameters. To clarify, consider the formulas for
AIC and BIC:

AIC � D � 2 k (5)

BIC � D � 2 k ln(n), (6)

where k is the number of free parameters (e.g., degrees of freedom)
and n is the sample size in a mediation model. Both the AIC and
the BIC penalize estimating more parameters in a model by adding
the term 2 k (for AIC) and 2 k ln(n) (for BIC) to deviance. If two
models have the same deviance, the AIC favors the model with
fewer free parameters (i.e., the more parsimonious model). For the
BIC, if two models have the same deviance and sample size, the
BIC favors the more parsimonious model. A lower AIC or BIC
value indicates a better fit.

The MBCO procedure and LRTMBCO have not previously been
explored for testing the null hypothesis of no mediation for a few
reasons. First, the nonlinear constraints require a sophisticated
optimization algorithm (e.g., Zahery, Maes, & Neale, 2017). To
our knowledge, the only structural equation modeling programs
that are equipped to implement the nonlinear constraint are
OpenMx (Boker et al., 2011; Neale et al., 2016) and SAS PROC
CALIS (SAS Institute Inc., 2016). Second, researchers commonly
rely on the CI-based tests because other researchers recommend
them (e.g., MacKinnon et al., 2004; Preacher & Hayes, 2008;
Shrout & Bolger, 2002). Third, few researchers have realized the
infinite number of ways to realize and test the null hypothesis of no
indirect effect, such as H0: �1�2 � 0. For example, �1 can be zero
but �2 may take on any value; �2 can be zero but �1 may take on
any value; or �1 and �2 are both zero. Thus, in an infinite number
of ways, mediation does not happen. The MBCO procedure offers
flexibility in testing and improvement in evaluating simple and
complex hypotheses about indirect effects, which we demonstrate
next using an empirical example.

Empirical Example of Using the MBCO Procedure on
a Complex Mediation Model

The MBCO procedure can be applied to a mediation model with
two parallel mediators; in fact, the MBCO procedure can test a
variety of simple and complex hypotheses about indirect effects
and can offer flexibility in testing and improvement in evaluating
these hypotheses. Using R (R Core Team, 2019) and the OpenMx
(Boker et al., 2011; Neale et al., 2016) package, we apply and
illustrate parts of the code and output to a sample problem. Full
descriptions of the OpenMx code and output are given in the
online supplemental materials.

Figure 2. A randomized parallel (covarying) two-mediator model. In-
struction (X) is a random assignment with two conditions: instruction to use
mental imagery rehearsal (X � 1) versus instruction to use repetition (X �
0). Repetition is a self-reported score of using a repetition rehearsal to
memorize words. Imagery is a self-reported score of using mental imagery
to memorize words. Recall is the number of words out of a total of 20
words that each student correctly remembered at the end of the experiment
and is the outcome variable of interest. A solid arrow from one variable to
another indicates a direct effect of the variable on the other variable. The
curved double-headed arrow shows covariance between two residual terms
associated with the mediators, εM1 and εM2. The numbers next to each
arrow denote regression (path) coefficient estimates while the numbers in
parentheses denote the respective standard errors. The ε terms denote
residuals, which we assume are normally distributed.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

4 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/met0000259.supp

MacKinnon, Valente, and Wurpts (2018) discussed a study
on using mental imagery to improve the memorization of
words. Table 1 shows descriptive statistics for this empirical
example. Oliver, Bays, and Zabrucky (2016) showed individu-
als who are instructed to create mental images of words appear
to recall more of them than do individuals who were simply
instructed to remember the words. MacKinnon et al. (2018)
conducted another study to test the mediation hypothesis that
instructions to participants to create mental images of words
would increase their using mental imagery and, in turn, would
increase the number of words participants recalled (see Figure
2). The study was replicated eight times, each with a different
group of undergraduate students enrolled in introductory psy-
chology courses; the sample sizes were 77, 43, 24, 79, 22, 45,
35, and 44. At the beginning of each assessment, the instructor
informed the students that they would hear a number of words
and that they were to memorize the words using techniques
described in the instructional sheets that they would receive.
The students then were randomly assigned to receive instruc-
tions on one of two different memorization techniques: repeti-
tion or imagery rehearsal. The students who received repetition
instructions were asked to memorize each word by repeating it
to themselves. The students who received imagery instructions
were asked to memorize each word by forming a mental image
of the word along with other words they had heard during the
experiment. For example, upon hearing the word camel and
after hearing the word women, they might imagine a woman
riding a camel. The students listened to 20 words with a 10-s
interval to rehearse each word. Ten seconds after hearing the
last word, the students were asked to write down as many words
as they remembered. Next, the students were all asked to (a)
rate the degree to which they formed images of the words in
order to memorize the words using a scale of 1 � not at all to
9 � absolutely; and (b) rate the degree to which they memo-
rized each word by repetition on a scale of 1 � not at all to 9 �
absolutely.

For this mediation model, we were interested in testing the
following hypotheses:

Research Question 1: Does the instruction to create mental
images of words increase use of mental imagery that, in turn,
increases the number of words recalled?

Research Question 2: Does the instruction to repeat words
increase use of repetition to memorize the words that, in turn,

increases the number of words recalled over and above using
mental imagery?

Research Question 3: Is the indirect effect of instruction to
create mental images on the number of words recalled through
use of mental imagery greater than the indirect effect of
instruction to repeat words on the number of words recalled
through use of repetition?

To answer each research question, we implemented the MBCO
procedure in three steps:

Step 1: Specify and estimate the full mediation model.
Step 2: Specify and estimate the null mediation model.
Step 3: Compare fit of the full and null models using the MBCO

procedure.
For Research Question 1, we briefly explain OpenMx (Boker et

al., 2011; Neale et al., 2016) syntax for conducting the MBCO
procedure and provide the relevant part of OpenMx code and
output. For Research Questions 2 and 3, we summarize the MBCO
procedure statistical results.

2

Research Question 1

Because we were interested in testing whether the indirect effect
of instruction of recall through imagery is different from zero, we
answered this question by using the single-mediator model (see
Figure 1). We briefly introduced the key parts of OpenMx code
(Boker et al., 2011; Neale et al., 2016) and R (R Core Team, 2019)
and then followed the three steps outlined in the article to imple-
ment the MBCO procedure.

Before reporting the formal modeling process, we show a glimpse
of the data set and its structure using the glimpse function from the
dplyr package (Wickham, François, Henry, & Müller, 2019):

2 All OpenMx scripts for the model along with a detailed explanation are
provided in the online supplemental materials, which can also be found at
https://github.com/quantPsych/mbco. For a comprehensive list of resources
about the OpenMx software, we encourage readers to visit the project
website https://openmx.ssri.psu.edu. For a detailed document on each
function within R, use the help() function. For example, type help
(mxModel) within the R console.

Table 1
Means, Standard Deviations, and Correlations With Confidence Intervals

Variable M SD 1 2 3

1. Repetition 6.08 2.84
2. Recall 12.07 3.4 �.28 [�.37, �.19]
3. Imagery 5.66 2.96 �.56 [�.63, �.49] .51 [.43, .58]
4. Instruction 0.51 0.5 �.67 [�.72, �.61] .32 [.22, .41] .62 [.55, .68]

Note. M and SD are used to represent mean and standard deviation, respectively. Instruction (X) is a random
assignment with two conditions: instruction to use mental imagery rehearsal (X � 1) versus instruction to use
repetition (X � 0). Repetition is a self-reported score of using a repetition rehearsal to memorize words. Imagery
is a self-reported score of using mental imagery to memorize words. Recall is the number of words out of a total
of 20 words that each student correctly remembered at the end of the experiment and is the outcome variable of
interest. Values in square brackets indicate the 95% confidence interval for each correlation.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

5IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

https://github.com/quantPsych/mbco

https://openmx.ssri.psu.edu

Step 1

The MBCO procedure begins by specifying the full single-mediator model (see Figure 1). We fit the full single-mediator model using

the code shown below:

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

6 TOFIGHI AND KELLEY

In the first two lines of the script above, before specifying the
model parts with mxModel, we can simplify the process by
grouping the names of variables as a vector that will be used in
model specification. For example, we specified a vector of the
names of observed variables and saved them as maniVar. Then,
we specified a vector of the names of endogenous variables and
saved them as endVar. Next, we specified the full single-
mediator model.

The main command to specify an SEM is mxModel. The
arguments provided to the mxModel function specified all the
elements of the mediation model. For the single-mediator example,
we specified the manifest (observed) variables, the paths (regres-
sion coefficients), the indirect effect to be tested (or any function
of model parameters), the constraint distinguishing the full and
null models, and the variances and covariances among the vari-
ables. The first argument to mxModel is single_med_full,
which is a name we chose for the single-mediator model. The next
argument, type=”RAM”, specifies that OpenMx uses the retic-
ular action model (RAM; McArdle & McDonald, 1984), a sym-
bolic algebraic notation to specify an SEM. In the argument
manifestVars, we introduced the vector of the names of the
observed (manifest) variables. The variable names must match the
names in the data set memory_df, as shown earlier in the output
from the glimpse function.

Next, we specified the paths between the variables using
mxPath, a function that corresponds to the graphical representa-
tion of paths in an SEM such as the model in Figure 1. For
example, we used mxPath to indicate a path (coefficient) corre-
sponding to an arrow between the two variables specified in the
arguments from (predictors) and to (response variables) in Fig-
ure 1. The argument arrows=1 indicates a unidirectional arrow
that starts from the variable in the argument from and ends at the
variable specified in the argument as to. The argument
arrows=2 indicates a bidirectional arrow representing a covari-
ance between the two variables. The argument free=TRUE
indicates that the parameter is freely estimated; otherwise,

free=FALSE indicates that the parameter is fixed at the values
set by the argument values. If the parameter is freely estimated,
the argument values would provide starting values. The argu-
ment labels provides labels for the coefficients. Because we
specified more than one coefficient in the arguments to and
from, we could provide a vector of labels corresponding to the
stated order of the coefficients. For our example, b1 is the coef-
ficient for X ¡ imagery and b3 is the coefficient for X ¡ recall.

We used the function mxAlgebra to define the indirect effect
that, in general, is a function of model parameters. A function may
include combinations of mathematical operations, such as �, �, �,
and / (e.g., b1�b2/(b1�b2+b3)), exponentials (e.g., exp), and
logarithms (e.g., log). The first argument to mxAlgebra was the
product of two coefficients, b1�b2, in which b1 and b2 had been
defined in mxPath. The argument name=”ind” named the indirect
effect. Next, we specified the data set for the model. The mxData
identified the data set to be analyzed. The argument observed=
memory_df specified the name of the data set in R. The second
argument, type=”raw,” indicated that the data set was in the raw
format, which meant that the data set included observations on the
participants as opposed to being a summary statistic such as a cova-
riance matrix.

Finally, we ran the model using mxRun, where the first argument
was the name of the mxModel that was then saved as fit_single_
med_full. Because we received a warning (not shown here) from
the optimizer that the convergence criterion was not satisfied, we used
the function mxTryHard. This function would either run the model
multiple times until an acceptable solution, according to the conver-
gence criterion set by the estimation algorithm, was found or it would
run a preset maximum number of times until an acceptable solution
was reached. In the subsequent analysis, we first used mxRun and, if
the model had convergence issues, we then used mxTryHard. We
used the function summary to save or print the summary of the
results. We saved the summary of the results as stat_single_
med_full and then printed the summary. Below, is the relevant part
of the summary results.

Below the title Model Statistics, the row that starts with
Model gives the pertinent information for the full model. The output
showed that the full single-mediator model had eight free parameters,
with dfFull � 1099 and DFull � 4305.727. Under Information
Criteria, we present the estimates of the information indices. The
information indices that match the formulas in (5) and (6) are under
the Parameters Penalty column. For the full model, the infor-
mation fit indices were AICFull � 4321.727 and BICFull � 4353.013.

Before proceeding, after fitting a mediation model, we checked
the degree to which the statistical assumptions about normality of
the residuals and the presence of outliers to ensure that the esti-

mates were unbiased and the inference about the parameters was
valid (Cohen, Cohen, West, & Aiken, 2003). We checked for
normality of the residuals using QQ plots, which indicated that the
normality assumption for the residuals was reasonable. Using the
methods described by Fox (2016), we did not find any outliers that
would change the results.

Step 2

We ran the null model (code below), which is the single-mediator
model in which the indirect effect of instruction on recall through
imagery is constrained to zero, �1�2 � 0.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

7IMPROVED INFERENCE IN MEDIATION ANALYSIS

Instead of specifying all parts of the null mediation model using
MxModel in the above code, we modified the full model
single_med_full, as specified in the argument model. Next,
we specified the nonlinear constraint for the indirect effect through
mxConstraint. The first argument, ind = = 0, constrains the

indirect effect defined in the mxAlgebra statement to zero. The
argument name assigns a name to the constraint. Finally, we saved
the null model to single_med_null. Next, we ran the null
model and saved the results to fit_single_med_null. Rele-
vant parts of the summary of the model results are shown below:

For the null model, dfNull � 1100 and DNull � 4481.493, and the
information fit indices were AICNull � 4497.493 and BICNull �
4528.779.

Step 3

We compared the full and null model both in terms of
the LRTMBCO and the information fit indices to evaluate H0:
�1�2 � 0. The LRTMBCO equals the difference between the

deviance of the two models: LRTMBCO � DNull � DFull �
4481.493 � 4305.727 � 175.766. The degrees of freedom for the
LRTMBCO equal the difference in the degrees of freedom associ-
ated with each model; that is, dfLRT � dfNull � dfFull � 1100 �
1099 � 1. Given that the LRTMBCO has a large sample �

2

distribution with dfLRT � 1, we have the p-value � Pr(�
2(1) �

175.766) � 4.073738E � 40. More conveniently, we computed
the LRTMBCO using the mxCompare or anova function where
the arguments are the names of the full and null models:

The first row of the above output shows the results for the full
model, which is the single-mediator model in Figure 1. The col-
umns ep, minus2LL, df, and AIC show the number of estimated
parameters, deviance, degrees of freedom, and AIC, respectively.
The columns diffLL, diffdf, and p represent the difference in
deviance (not the log-likelihoods), the difference in degrees of
freedom, and p-value for the two models being compared. The
second row shows the results for the null model under the columns
ep, minus2LL, df, and AIC. The results of comparing the null
and full model are shown under the columns diffLL, diffdf,

and p. The value for the test statistic LRTMBCO � 175.766 is
located under the column diffLL. The degrees of freedom are
dfLRT � 1 and the p-value � 4.073738E � 40; these amounts are
located under the columns diffdf and p, respectively.

We used the following commands to compute the Monte Carlo
CI for the indirect effect. First, we used the functions coef and
vcov to extract the path coefficients and covariance matrix of the
coefficients, respectively. Next, we used the ci function in the
RMediation package (Tofighi & MacKinnon, 2011, 2016) to com-
pute the Monte Carlo CI. The first argument mu to this function is

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

8 TOFIGHI AND KELLEY

a vector of the coefficient estimates, and the second argument
Sigma is a covariance matrix of the coefficient estimates. The

argument quant accepts a formula for the indirect effect that starts
with the symbol “ ”.

The numbers below 2.5% and 97.5% show the lower and upper
limits of the 95% CI, [1.6, 2.682]; the numbers below Estimate
and SE show the estimates of indirect effect and its standard error,
�̂1�̂2 � 2.121 and SE � 0.276, respectively.

We next obtained R2 for the endogenous variables of the full and
null models using the rsq function available in the online sup-
plemental materials.

The first argument model to rsq specifies an OpenMx
model, and the second argument name specifies names of
endogenous variables (i.e., variables in which a single-headed
arrow enters; namely those that are a function of another
variable).

To summarize, the LRTMBCO result showed that the indirect
effect of instruction on recall through imagery appeared to be
greater than zero, �̂1�̂2 � 2.121 (SE � 0.276), 95% Monte
Carlo CI [1.6, 2.682], LRTMBCO � 175.766, dfLRT � 1, p �
4.073738E � 40. We recommend researchers compute the
difference in R2s between the full and null models and then
examine the change in the effect sizes that occurs as a result of
the indirect effect through imagery. For recall, R2 remained
unchanged to four decimal places while for imagery
R2 �
.3779. Further, the information fit indices supported the asser-
tion that the indirect effect was greater than zero because the
AIC and the BIC for the full single-mediator model were
smaller than those of the null single-mediator model. This result
indicates that constraining the indirect effect to zero worsens
the fit of the full single-mediator model. On average, the
instruction to students to use mental imagery increased use of
mental imagery that, in turn, increased the number of words
they recalled by approximately two words. We have inferential
support that mediation occurred. Note these results are valid if
the no-omitted confounder assumption is met. That is, an omit-
ted variable may not exist that influences the relations between
instruction, imagery, and recall.

Research Question 2

Here, we were interested in whether the instruction to use
repetition increased the use of repetition to memorize the words
that, in turn, improved participants’ memory over and above the
indirect effect of instruction on recall through imagery. In other
words, we wanted to test H0: �4�5 � 0 for the parallel two-
mediator model in Figure 2. We again applied the three-step
MBCO procedure.

Step 1

We estimated the model with two parallel mediators in Figure 2,
which is the full model for this research question. For this model,
the two specific indirect effects associated with imagery and
repetition were freely estimated. Below are the regression equa-
tions for the full parallel two-mediator model:

M1 � �0,M1 � �1X � εM1 (7)

M2 � �0,M2 � �4X � εM2 (8)

Y � �0,Y � �3X � �2M1 � �5M2 � εY , (9)

where �1 is the effect of the instruction (X) on imagery (M1), �4 is
the effect of the instruction on repetition (M2), �3 is the direct
effect of the instruction on recall (Y), �2 is the effect of imagery on
recall controlling for the instruction and repetition, and �5 is the
effect of repetition on recall controlling for instruction and imag-

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

9IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

http://dx.doi.org/10.1037/met0000259.supp

ery; �0,M1, �0,M2, �0,Y are the intercepts; εM1, εM2, and εY are the
residuals.

We fitted the full two-mediator model within OpenMx. The
results showed that the full two-mediator model had 13 free
parameters, with dfFull � 1463 and DFull � 5874.685. For the full
model, the results were that AICFull � 5900.685 and BICFull �
5951.526; the effect sizes were RImagery2 � .38, RRepetition2 � .45, and
RRecall

2 � .26. We computed the specific indirect effects through
imagery and repetition and the 95% CI for each specific indirect
effect using the ci function in the RMediation package. The
results showed that the specific indirect effect through imagery
was �̂1 �̂2 � 2.139 (SE � 0.284), 95% Monte Carlo CI [1.602,
2.716], and the specific indirect effect through repetition was
�̂4 �̂5 � �0.082 (SE � 0.285), 95% Monte Carlo CI [�0.643,
0.477].

Step 2

Recall that we were interested in whether instruction to use
repetition increased the use of repetition to improve memory over
and above the indirect effect of instruction on recall through
imagery. Thus, in the null parallel two-mediator model, we con-
strained the specific indirect effect through repetition to zero while
letting the specific indirect effect through imagery be freely esti-
mated. The null parallel two-mediator model was estimated by
fitting the equations in (7)–(9) subject to the null hypothesis
constraint H0: �4�5 � 0, which fixed the specific indirect effect
through repetition to zero. In OpenMx, we specified the null model
by adding the nonlinear constraint �4 �5 � 0 to the full model in
Step 1. The results showed that the null parallel two-mediator
model had dfNull � 1464 and DNull � 5874.768. The information
fit indices for the null two-mediator model were AICNull �
5900.768 and BICNull � 5951.609. The effect sizes were RImagery

2 �
.38, RRepetition2 � .45, and RRecall2 � .26.

Step 3

We compared the two models and computed LRTMBCO. The
results showed that LRTMBCO � 0.083, dfLRT � 1, and p � .773.
The specific indirect effect through Repetition was, therefore, not
different from zero, �̂4�̂5 � � 0.08 (SE � 0.29), 95% Monte Carlo
CI [�0.64, 0.48]. Further, the R2 for imagery, repetition, and recall
remained unchanged to three decimal places, and the information
fit indices between the two models were roughly the same. These
results indicate that the specific indirect effect through repetition
above and beyond the specific indirect effect through Imagery
does not appear to be different from zero.

Research Question 3

For this question, we were interested in comparing the sizes of
the two specific indirect effects: the indirect effect of instruction
on recall through imagery (i.e., �1�2) and the indirect effect of
instruction on recall through repetition (i.e., �4�5). The null hy-
pothesis for this research question is:

H0: �1�2 � �4�5 (10)

To test this null hypothesis, we again employed the three-step
MBCO procedure.

Step 1

The full model for this research question was the same as the
full model in Research Question 2; that is, they matched the
parallel two-mediator model in (7)–(9). Thus, we used the results
(i.e., the vector of the coefficient estimates, the covariance matrix
of the coefficient estimate, R2, indirect effects estimates, AIC, and
BIC) of the full parallel two-mediator model from Research Ques-
tion 2.

Step 2

The null model was the parallel two-mediator model in (7)–(9)
subject to the null hypothesis constraint in (10). That is, we
estimated the two-mediator model while we constrained the two
specific indirect effects to be equal. Alternatively, we could spec-
ify the contrast of the two specific indirect effects to zero. We fitted
the new null model in OpenMx. The results showed that the null
model had dfNull � 1464 and DNull � 5900.514. The information fit
indices for the null model were AICNull � 5926.514 and BICNull �
5977.354. The effect sizes were RImagery2 � .38, RRepetition2 � .45, and
RRecall

2 � .21.

Step 3

We compared the full parallel two-mediator model in Step 1 and
the null parallel two-mediator model in Step 2. We computed the
LRTMBCO as well as the 95% Monte Carlo CI for the contrast of
the two indirect effects using the ci function in the RMediation
package. The results of the MBCO procedure showed that
LRTMBCO � 25.828, dfLRT � 1, and p � 3.731857E � 07. These
outcomes indicate that the indirect effect through imagery ap-
peared to be larger than the indirect effect through repetition by
�̂1�̂2 � �̂4�̂5 � 2.222 (SE � 0.445) words, 95%

Monte Carlo CI

[1.364, 3.11]. In comparing the R2s for the endogenous variables
obtained from Step 1 and 2 for imagery and repetition, R2s re-
mained unchanged to three decimal places while for recall
R2 �
.05. Comparing the information fit indices of the null model with
the full model also supported the conclusion that the full model
fitted the data better than did the null model.

Comparing Methods of Testing Indirect Effect Via
Monte Carlo Simulations

In this section, we describe simulation studies assessing the
Type I error rates and the statistical power of the LRTMBCO of our
proposed MBCO procedure; we then compare the Type I error
rates and power of LRTMBCO to those of the currently recom-
mended methods of testing indirect effects. Monte Carlo simula-
tion studies are necessary because there are no known ways of
assessing the performance via mathematical proofs or derivations.
In our simulations, we generated data from known population
models, fitted the same models to the data, repeated this process
many times, and assessed the properties of the different methods of
testing indirect effects.

The simulation studies were designed to answer the following
questions:

1. Does the LRTMBCO provide more robust Type I error
rates than the recommended methods? In particular,

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

10 TOFIGHI AND KELLEY

would the LRTMBCO remain robust across the combina-
tion of the parameter values for smaller sample sizes?

2. Is the LRTMBCO as powerful as, if not more powerful
than, the currently recommended methods across differ-
ent effect sizes, sample sizes, and other types of media-
tion models?

To answer these questions, we compared the Type I error rate
and power of the LRTMBCO to five currently recommended meth-
ods of testing indirect effects: (a) the percentile bootstrap CI
(Bollen & Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et
al., 2004); (b) the bias-corrected (BC) bootstrap CI (Bollen &
Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et al., 2004);
(c) the Monte Carlo CI (MacKinnon et al., 2004; Tofighi &
MacKinnon, 2016); (d) the profile-likelihood CI (Folmer, 1981;
Neale & Miller, 1997; Pawitan, 2001; Pek & Wu, 2015); and (e)
the joint significance test (Kenny et al., 1998; MacKinnon et al.,
2002). We describe these methods in the next section.3

For a two-path indirect effect, we did not separately consider
distribution of the product CI because its performance is similar to
that of the Monte Carlo CI (MacKinnon et al., 2004). We also did
not study the Wald (1943) (multivariate delta) z or the causal steps
test because research has shown that these two tests are overly
conservative and do not provide adequate power in smaller sample
sizes and effect sizes (MacKinnon et al., 2002). For completeness
of the simulation study and because of its popularity and endorse-
ments (MacKinnon et al., 2004; Preacher & Hayes, 2008; Shrout
& Bolger, 2002), we studied the bias-corrected (BC) bootstrap CI.
However, we do not recommend using the BC bootstrap CI to test
an indirect effect because it shows inflated Type I error rates for
both small and large samples (Falk & Biesanz, 2015; Koopman,
Howe, Hollenbeck, & Sin, 2015).

In the simulation studies, we considered two types of indirect
effects commonly found in empirical research as well as in sim-
ulation studies (Tofighi & Kelley, 2019): (a) a two-path indirect
effect (e.g., �1�2), which is the product of two coefficients for a

single-mediator chain,

X ¡
�1

M ¡
�2

Y; and (b) a three-path indirect
effect (e.g., �1�2�3), which is the product of three coefficients for

a sequential two-mediator chain, X ¡
�1

M1 ¡
�2

M2 ¡
�3

Y. We chose
these two population models because they have been extensively
discussed in both substantive and methodological literature (To-
fighi & Kelley, 2019).

Percentile and Bias-Corrected Bootstrap CI

We now discuss two methods of computing a CI for nonpara-
metric bootstrap (Bollen & Stine, 1990; Efron & Tibshirani, 1993).
In this technique, many samples (e.g., R � 1,000) with replace-
ment were drawn from the sample data. The hypothesized medi-
ation model was fit to each resampled data set, and indirect effects
were computed for each model. This process resulted in R esti-
mates of indirect effects, which is called the bootstrap sampling
distribution of the indirect effects. To compute a CI, we obtained
quantiles corresponding to 1 � �/2 and �/2 percentiles of the
bootstrap sampling distribution, resulting in what is called a per-
centile CI. As a modified version of the percentile bootstrap, the
BC bootstrap uses adjusted percentiles 1 � �=/2 and �=/2 to obtain
the upper and lower limits of the CI, where �= is computed to

correct for potential bias due to skewness (Efron, 1987). Although
not in the context of mediation analysis, Efron (1987) argued that,
for smaller sample sizes, the BC bootstrap CI yields a more
accurate coverage (i.e., the proportion of times a CI contains true
values of the parameter is closer to the nominal coverage of 1 �
�) than does the percentile CI.

Monte Carlo CI

The Monte Carlo method, also known as the parametric boot-
strap (Efron & Tibshirani, 1993), is another sampling-based tech-
nique used to compute a CI for indirect effects (MacKinnon et al.,
2004; Tofighi & MacKinnon, 2016). In this method, the sampling
distribution for each parameter is estimated by drawing R random
samples from a multivariate normal distribution where the mean of
the distribution is the vector of the coefficient estimates and the
covariance matrix of the distribution is the covariance matrix of
the coefficient estimates. The Monte Carlo method is based on the
theory that the maximum likelihood (ML) estimates of the coef-
ficients in an SEM asymptotically have a multivariate normal
distribution (Bollen, 1989). The population parameters (i.e., mean
vector and covariance matrix) for this multivariate normal distri-
bution are the parameter estimates from the estimated model. The
number of Monte Carlo samples, R, should be large, typically
1,000 or more. The indirect effect is estimated within each sample,
resulting in a total of R estimates. The mean and standard deviation
of the R indirect effect estimates are then used to estimate the
indirect effect and its standard error, respectively. To compute a
(1 � �)100% CI, we obtained the �/2 and 1 � �/2 quantiles of the
Monte Carlo sample of the indirect effects.

Profile Likelihood CI

In its simplest form, the profile-likelihood approach produces a
CI for a single parameter using a profile-likelihood function (Fol-
mer, 1981; Neale & Miller, 1997; Pawitan, 2001). In mediation
analysis, the profile-likelihood method has been extended to pro-
duce a CI for the indirect effect in a single-mediator chain
(Cheung, 2007; Pek & Wu, 2015). Let � be a vector of all the free
parameters in a single-mediator model. The profile-likelihood
function for a single-mediator model, L(� | �1�2), is computed by
assuming that the indirect effect, �1�2, is a known value. Next, the
profile-likelihood function is maximized. In practice, the indirect
effect takes on different values, and the profile-likelihood function
is maximized while the indirect effect is fixed at specific values.

Next, we compared the deviance of the maximized profile-
likelihood function and the deviance of the maximized likelihood
function, where deviance equals negative twice the maximized
log-likelihood function as shown in (3). Asymptotically, the dif-
ference in the deviance of the two functions had a chi-squared
distribution with degrees of freedom equaling the difference in the
number of free parameters between the profile likelihood and
likelihood function for the model (Pawitan, 2001). For the indirect
effect, in general, this difference asymptotically had a chi-squared
distribution with one degree of freedom. The lower and upper

3 We do not name all the existing methods for single-mediator models
because they are considered elsewhere (e.g., Falk & Biesanz, 2015; Mac-
Kinnon et al., 2002).

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

11IMPROVED INFERENCE IN MEDIATION ANALYSIS

limits for the 100 (1 � �)% profile-likelihood CI corresponded to
the minimum and maximum of all values of the indirect effect that
satisfied the following equality:

D(�̂prof | �1�2) � D(�̂) � ��2 (1) (11)

where �̂prof is the ML estimate of the model parameters given that
the indirect effect is fixed and ��2 (1) denotes upper-tail � critical
value of the chi-squared distribution with one degree of freedom.
For a specific mediation model, the values of D��̂� (computed by
estimating the mediation model) and the critical value of the
chi-squared distribution (e.g., �.05

2 (1) � 3.84) were known. A 95%
profile-likelihood CI was computed by solving the expression in
(11) for two values of �1�2 corresponding to the upper and lower
limits.

Joint Significance Test

For a single-mediator model, MacKinnon, Lockwood, Hoffman,
West, and Sheets (2002) proposed a joint significance test to
describe a variation of the causal steps test (Kenny et al., 1998).
This method declares an indirect effect to be significantly different
from zero if every coefficient in the indirect effect is significantly
different from zero. The joint significance test does not produce a
robust Type I error rate for smaller sample sizes, a CI, or a p value
for the indirect effect (MacKinnon et al., 2002). While the joint
significance test can be extended to test indirect effects that are the
products of two or more coefficients in a mediational chain, the
method cannot be used to test a complex function of indirect
effects such as a contrast of two indirect effects (e.g., H0: �1�2 �
�3�4 � 0). The joint significance test has been recommended to
test indirect effects in single-mediator and two-mediator sequential
chains (Falk & Biesanz, 2015; Taylor, MacKinnon, & Tein, 2008;
Yzerbyt, Muller, Batailler, & Judd, 2018).

Simulation Design and Population Values

In the simulation studies, we manipulated three factors: (a)
effect size R2, which quantifies how well the predictors account
for the variance in the endogenous variables; (b) sample size;
and (c) the method of testing the indirect effect(s). Previous
work showed that sample size and the effect size R2 influence
the Type I error and the power of the existing tests of the
indirect effect for single-mediator and sequential two-mediator
chains (MacKinnon et al., 2002, 2004; Williams & MacKinnon,
2008). Based on Cohen’s (1988) guidelines, we specified the
population effect sizes in the simulation to be R2 � .02, .13, and
.26, which leads to the corresponding nonzero population re-
gression coefficients of 0.14, 0.39, and 0.59, respectively (see
the method described by Thoemmes, MacKinnon, & Reiser,
2010, which we used).

Sample size took on the following values: N � 50, 75, 100,
and 200. We chose these values to cover a range of sample sizes
reported in the applied literature of psychology, in related
disciplines, and in the simulation studies of mediation effects
(e.g., MacKinnon et al., 2002; Tofighi & Kelley, 2019). The
smallest sample size we considered, 50, might not be viewed as
a best practice in the SEM literature, and a sample size of 200
is roughly equal to the median sample size in an SEM (Jaccard
& Wan, 1995; MacCallum & Austin, 2000). However, the

studies with a sample size of 50 do appear in the applied
literature (Tofighi & Kelley, 2019). We do not report results
from a sample size greater than 200 because our preliminary
simulation study for the LRTMBCO as well as the previous
simulation studies for the existing methods (e.g., MacKinnon et
al., 2002, 2004; Taylor et al., 2008; Williams & MacKinnon,
2008) found that the performance of the tests did not differ
between methods at larger sample sizes. In these large samples,
all the methods (which are based on large sample theory)
worked effectively.

The method conditions included six commonly used tests of
indirect effects for both the Type I error and power study. These
conditions resulted in 168 conditions for the Type I error and 216
conditions for the power study for the single-mediator model; and
240 conditions for the Type I error and 648 conditions for the
power study for the sequential two-mediator model. We used a
mixed full factorial design for each simulation study where the
factors effect size and sample size were between-subjects factors
and the method was a within-subjects factor.

Data Generation

We generated 5,000 independent data sets for each combina-
tion of the between-subjects factors using a known population

model, either X ¡
�1

M ¡
�2

Y for the single-mediator model or

X ¡
�1
M1 ¡
�2
M2 ¡
�3

Y for the sequential two-mediator model. The
variables X, M1, M2, and Y were observed. Without loss of gen-
erality, the population values for the intercepts were fixed at zero
but were estimated in the simulation study. Values for X were
sampled from a binomial distribution with .5 probability of two
categories (i.e., treatment vs. control). Data for each residual term
were generated from the standard normal distribution.

We chose � � .05 to test the indirect effects, as is commonly
done in psychology and related areas. We used OpenMx Version
2.9.6 (Boker et al., 2011; Neale et al., 2016) to conduct the
LRTMBCO as well as the profile-likelihood CI.

4 For the Monte
Carlo, percentile and BC bootstrap CI, and for the joint signifi-
cance test, we estimated the model using lavaan package Version
0.5–23 (Rosseel, 2012). We then used the model estimates and ci()
function in the RMediation package Version 1.1.4 (Tofighi &
MacKinnon, 2011, 2016) to compute a Monte Carlo CI; both

4 We studied the software manual and references for the optimization
techniques implemented in OpenMx (Boker et al., 2011; Neale et al., 2016;
Zahery et al., 2017), communicated with the authors/programmers, and
conducted an extensive simulation study to examine whether the LRTMBCO
can be implemented using different optimization methods in OpenMx. In
addition, we examined whether the software packages lavaan Version
0.5-23.1097, Mplus Version 8.0, and PROC CALIS (SAS Institute Inc.,
2016) allowed nonlinear constraints to estimate the LRTMBCO. We also
communicated with the authors/programmers of Mplus and lavaan. Based
on our communications, at the time of writing, Mplus and lavaan do not
guarantee obtaining optimal results for the mediation models with nonlin-
ear constraints in the form of null hypothesis about the indirect effect. We
chose the NPSOL (Gill, Murray, Saunders, & Wright, 1986) optimizer
instead of the default CSOLNP (Zahery et al., 2017) optimizer in OpenMx
because the CSOLNP showed convergence problems in our preliminary
simulation studies. Based on our personal communication with OpenMx
developers, they suggested that the SLSQP (Snyman, 2005) optimizer may
be successful as well. However, we did not formally evaluate this option
because it is beyond the scope of the current article.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

12 TOFIGHI AND KELLEY

percentile and BC bootstrap CI were estimated using the facilities
provided in the lavaan package. For the joint significance test, we
used the z test to examine the joint significance of the coefficients
of the indirect effects.5

Results

For the simulation studies assessing the robustness of the Type
I error, the empirical Type I error rate is the proportion of times out
of 5,000 replications that a test incorrectly rejects the null hypoth-
esis of zero indirect effect. For the statistical power studies, the
empirical power is the proportion of times out of 5,000 replications
that a test correctly rejects the null hypothesis of zero indirect
effect. The large sample size of 5,000 within each condition
precluded the use of repeated measures ANOVA for the simulation
results. To determine if a test was robust, as discussed, we used
Bradley’s (1978) liberal interval of .025 and .075 to determine a
Type I error rate that was “good enough” for practical purposes.
We considered a test robust (i.e., good enough) according to
Bradley’s (1978) criteria if its empirical Type I error rate fell
within the robustness interval. Otherwise, a test was considered
conservative or liberal depending on whether the Type I error rate
was smaller or greater than Bradley’s lower or upper bound,
respectively.

Type I error. To facilitate comparison of the Type I error
rates across different combinations of the sample sizes and effect
sizes, we created graphs showing the empirical Type I error rate as
a function of sample size and size of nonzero coefficients. To save
space, we only present a few graphs (Figures 3– 6). More graphs
can be found in the online supplemental materials. For the two-
path indirect effect where both coefficients were zero (see Figure
3), the empirical Type I error rate for the LRTMBCO was robust as
it fell within Bradley’s (1978) robustness interval. The other five
methods were not robust across all sample sizes; that is, their
empirical Type I error rates were consistently below .01. The
LRTMBCO produced more robust Type I error rates because it
estimated the null mediation model and the sampling distribution
of the indirect effect under the null hypothesis.

As can be seen in Figure 4, when one of the coefficients was
zero, the LRTMBCO was the most robust method across all com-
binations of the nonzero coefficients and sample sizes; its Type I
error rate remained close to the nominal .05 value and stayed
within the limits of Bradley’s (1978) robustness interval. The BC
bootstrap showed conservative Type I error rates or inflated Type
I error rates in certain conditions. For example, when �2 � 0.59
and N � 50, and �2 � 0.39 and N � 100, the BC bootstrap showed
an inflated Type I error rate; when �2 � 0.14 and N � 75, the BC
bootstrap showed a conservative Type I error rate. In general, the
Type I error rates for the percentile bootstrap, Monte Carlo,
profile-likelihood, and joint significance methods were all robust
except when one of the nonzero coefficients was small (i.e., 0.14).
More specifically, when the magnitude of the coefficient was small
and the sample size was 100, all four methods showed a con-
servative Type I error rate. The results for CI-based and joint
significance tests, when one of the coefficients was zero, essen-
tially matched the results from previous studies (Biesanz et al.,
2010; Falk & Biesanz, 2015), as would be expected. Finally, which
coefficient was zero did not appear to change the overall conclu-
sion about the Type I error rate robustness of the methods.

For the sequential two-mediator model, the LRTMBCO was the
most robust test across the combinations of the sample sizes and
effect sizes. When all three coefficients were zero (see Figure 5) or
when two of the coefficients were zero (see Figure 6), the
LRTMBCO’s Type I error rate remained robust within the Bradley’s
(1978) limits. The five other methods showed conservative Type I
error rates. Finally, which coefficient was zero did not appear to
change the Type I error rates.

Power. The simulation study showed that the LRTMBCO was
as powerful as the other methods, except for the BC bootstrap
method, for both two-path and three-path indirect effects. The BC
bootstrap method showed slightly more power although that power
was at the expense of yielding inflated Type I error rates. This
result is consistent with previous research (Biesanz et al., 2010;
Falk & Biesanz, 2015; Taylor et al., 2008). The results of power
simulation studies were similar between the single-mediator model
and the sequential two-mediator model. We explain the similar
power between the CI-based methods and the LRTMBCO in the
next section.

Summary

One explanation for why the simulation studies showed that the
LRTMBCO produces more robust empirical Type I error rates than
other CI-based methods is that the sampling distribution of the
indirect effect tends to differ from the null sampling distribution of
the indirect effect estimate. In the null sampling distribution, the
population value of the indirect effect is fixed to zero. The sam-
pling distribution used by the CI-based methods is likely to reflect
an alternative hypothesis of a nonzero population indirect effect
rather than the null hypothesis of zero population indirect effect.
As a result, the CI-based methods tend to produce nonrobust
empirical Type I error rates, especially for smaller sample sizes.
However, the LRTMBCO uses both the null and the full models to
test an indirect effect and, thus, more appropriately maps the
statistical method onto the question of interest when seeking to
determine if a hypothesized mediator mediates the relationship
between X and Y. Hence, the LRTMBCO appears more appropriate
than other methods that have been recommended in the literature
in the conditions we studied.

The simulation results also showed that the LRTMBCO is as
powerful as the other recommended tests except for the BC boot-
strap method; however, we do not recommend the BC bootstrap
because of its inflated Type I error rate. On the other hand, the
CI-based tests have adequate power when the null hypothesis is
false (i.e., nonzero indirect effect) and the sampling distribution is
computed under an estimated alternative hypothesis in which the
indirect effect is nonzero. In other words, the estimated alternative
sampling distribution of CI-based methods is more accurate when
the null hypothesis is false as opposed to when the null hypothesis
is true. In these cases, the CI-based tests and LRTMBCO appear to
exhibit similar power.

5 The simulation study code scripts can be found at https://github.com/
quantPsych/mbco.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

13IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

https://github.com/quantPsych/mbco

https://github.com/quantPsych/mbco

Limitations

In our simulations, we made a few common assumptions for
estimating a mediation model. The data for the simulation studies
were generated assuming that residuals associated with the medi-
ators and outcome variable had a multivariate normal distribution
and that the correct model was fit (e.g., the variables were not
nonlinearly related and there were no omitted confounders).
Though assessing the effectiveness of the LRTMBCO under the
multivariate normality of the simulated data is a limitation, using
the multivariate normal data is necessary to compare performance
of the LRTMBCO to other commonly used methods of testing
mediation in known conditions. Therefore, the results of the sim-
ulation studies apply to situations in which these assumptions are
reasonably met. Assessing the performance of the LRTMBCO for
non-normally distributed residuals remains a topic for future re-
search. We also assumed the simulation study included correct
functional forms of the relationships between the posited variables.

For the empirical study, we evaluated if there were outliers (and
we did not find any in the sample data), and we assumed that no

common method biasing effect of measuring the endogenous vari-
ables existed (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003).
Moreover, we assumed that the observed mediator and outcome
variables were measured without error (Dwyer, 1983; Fritz,
Kenny, & MacKinnon, 2016); otherwise, the results could be
misleading (Cole & Preacher, 2014). In addition, one of the more
stringent of these assumptions, which is untestable, is the no-
omitted-confounder assumption (Imai et al., 2010; Judd & Kenny,
1981; Valeri & VanderWeele, 2013; VanderWeele, 2010). That is,
no variable should be omitted from the model that would affect
both the mediator and the outcome variables, given the indepen-
dent variable (X) and covariates (if they exist and are included in
the model). Even when participants are randomly assigned to a
treatment or control group, making causal claims about an indirect
effect requires the no-omitted-confounder assumption. If research-
ers believe that not all confounders are included in the model, then
the claims about the magnitude and existence of an indirect effect
need to be relaxed. We recommend that researchers conduct a
sensitivity analysis to investigate the biasing impact of a potential

Figure 3. Point and 95% CI estimate of the Type I error rate for six methods of testing a two-path indirect
effect, �1�2, where both parameters were fixed at zero. Horizontal solid lines show the limits of the Bradley’s
(1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s (1978) criterion
if its Type I error rate falls within the criterion’s interval. MBCO � model-based constrained optimization
(LRTMBCO); Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test;
Profile � profile-likelihood CI. See the online article for the color version of this figure.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

14 TOFIGHI AND KELLEY

confounder on the indirect effect point and interval estimate (Cox,
Kisbu-Sakarya, Miočević, & MacKinnon, 2013; Tofighi et al.,
2019; Tofighi & Kelley, 2016). MacKinnon and Pirlott (2015)
provide an excellent discussion of different approaches to sensi-
tivity analyses in mediation analysis. Sensitivity analyses for cer-
tain mediation models can also be conducted in an SEM frame-
work in Mplus software, which may include observed and latent
variables (Muthén & Asparouhov, 2015; Muthén & Muthén,
2018). Conducting sensitivity analyses using the LRTMBCO also
remains a topic for future study.

One requirement of our proposed LRTMBCO and the profile-
likelihood method is that the mediation model be estimated within
the SEM (or other appropriate multivariate) framework. That is,
the equations for the mediation model must be simultaneously
estimated. These methods do not work when using OLS regression
to separately estimate the equations representing a mediation

model, which is the classical way of testing mediation (Baron &
Kenny, 1986).

We discussed applying the LRTMBCO to the models when the
independent variable and mediator do not interact and when
the mediator and outcome are continuous. For such models,
both the SEM and causal inference methods provide the same
estimate of the indirect effect (Muthén & Asparouhov, 2015);
thus, the LRTMBCO can be used to test an indirect effect.
However, when the independent variable and a mediator inter-
act or when either a mediator or the outcome variable is a
categorical variable, researchers need to use the potential out-
come framework to correctly estimate the indirect effect
(VanderWeele, 2015). Extension of the MBCO procedure and
LRTMBCO to the causal inference framework for these scenarios
remains a topic for future study. Finally, we did not discuss
application of the MBCO procedure to multilevel mediation

Figure 4. Point and 95% CI estimate of the Type I error rate for six methods of testing a two-path indirect
effect, �1�2, where only �1 was fixed at zero. The nonzero parameters take on the values: 0.14, 0.39, and 0.59.
The x-axis shows the parameter values. Horizontal solid lines show the limits of the Bradley’s (1978) liberal
interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s (1978) criterion if its Type I error
rate falls within the criterion’s interval. MBCO � model-based constrained optimization (LRTMBCO); Percen-
tile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test; Profile � profile-
likelihood CI. See the online article for the color version of this figure.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

15IMPROVED INFERENCE IN MEDIATION ANALYSIS

analysis (Krull & MacKinnon, 2001; Tofighi, West, & MacK-
innon, 2013), where the data have a multilevel (i.e., clustered,
repeated measures) structure (Snijders & Bosker, 2012). Ex-
tending the MBCO procedure to multilevel data also remains a
topic for future study.

Discussion

This article proposed an MBCO procedure that uses a model-
comparison approach to make inferences about any function rep-
resenting an indirect effect in mediation analysis. An innovation of
our proposed MBCO procedure is using a nonlinear constraint to
test a variety of simple and complex hypotheses about indirect
effects in a model-comparison framework. The MBCO procedure
offers the following advantages compared with the existing meth-
ods. First, the MBCO procedure produces a likelihood ratio test,

termed LRTMBCO, to formally evaluate simple and complex hy-
potheses about indirect effects and produces a p value, a continu-
ous measure of compatibility between data and the null hypothe-
ses. Second, through the model-comparison framework, the
MBCO procedure computes a likelihood ratio, a continuous mea-
sure of comparing goodness of fit of the null and full models. It
also computes information fit indices such as AIC and BIC, used
to compare the null and full models based on both goodness fit and
parsimony.

To assess robustness of the LRTMBCO and the five most rec-
ommended methods of testing indirect effects in a single-mediator
or in a sequential two-mediator model, we conducted a Monte
Carlo simulation study. The results showed that the LRTMBCO is
more robust in terms of the empirical Type I error rate (i.e., it was
within Bradley’s, 1978 liberal interval of .025 and .075 for � �

Figure 5. Point and 95% CI estimates of the Type I error rate for six methods of testing a three-path indirect
effect, �1�2�3, where all three parameters were zero. Horizontal solid lines show the limits of the Bradley’s
(1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s criterion if its Type
I error rate falls within the criterion’s interval. MBCO � model-based constrained optimization (LRTMBCO);
Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test; Profile �
profile-likelihood CI. See the online article for the color version of this figure.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

16 TOFIGHI AND KELLEY

.05) than the other methods when the residuals were generated
under an ideal condition of the multivariate normality. In addition,
the LRTMBCO is as powerful as the currently used tests (except for
the BC bootstrap) and offers more robust empirical Type I error
rates in situations typical of psychology and related fields.

In addition, the MBCO procedure provides a model-comparison
framework to compare one or more alternative mediation models.
This important new feature relieves researchers of the restriction to
test a null hypothesis with a single hypothesized model. Research-
ers can test multiple competing models in addition to fitting a

Figure 6. Point and 95% CI estimate of the Type I error rate for six methods of testing a three-path indirect
effect, �1�2�3, where two out of the three parameters were fixed at zero. The nonzero parameter took on the
values: 0.14, 0.39, and 0.59. The x-axis shows the values of the nonzero parameter. Horizontal solid lines show
the limits of the Bradley’s (1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a
Bradley’s (1978) criterion if its Type I error rate falls within the criterion’s interval. MBCO � model-based
constrained optimization (LRTMBCO); Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS �
joint significance test; Profile � profile-likelihood CI. See the online article for the color version of this figure.

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

17IMPROVED INFERENCE IN MEDIATION ANALYSIS

single hypothesized model and can focus on testing and comparing
one or more indirect effects. More than a significance testing
framework, a model-comparison framework allows researchers to
compare one null model to one or more alternative models. These
comparisons also can address the significance of indirect effects,
the effect sizes associated with the mediators and outcome vari-
ables, and the overall fit and complexity of the models as measured
by various fit indices such as the AIC (Akaike, 1974) and the BIC
(Schwarz, 1978).

We applied the MBCO procedure to an empirical example. The
accompanying computer code in OpenMx (Boker et al., 2011;
Neale et al., 2016), the data set, and detailed analysis results are in
the online supplemental materials. As the empirical example
shows, after conducting the MBCO procedure, researchers should
report the resulting LRTMBCO, exact p value as well as the CIs for
the indirect effects calculated using the Monte Carlo, profile-
likelihood, or percentile bootstrap method. A p value should be
interpreted as a measure of compatibility of a null model with the
sample data and should not be used to make dichotomous deci-
sions about the significance of a null hypothesis. Further, research-
ers should report R2 effect sizes associated with mediators and the
outcome variable and compute differences in the respective R2s
between the competing models. Computing differences in R2s
allows researchers to gauge potential changes in the effect sizes
between the competing models, changes that could be a result of
nonzero indirect effects. In addition, researchers should report the
information fit indices AIC or BIC. Although using CI-based
methods to test the existence of an indirect effect or to make
dichotomous decisions about significance of the indirect effect is
not recommended, researchers should report a CI to convey a
range of plausible values for an indirect effect estimate. These
recommendations are consistent with and enhance the APA rec-
ommendations (APA Publication Manual, 2010; Appelbaum et al.,
2018; Wilkinson & Task Force on Statistical Inference, American
Psychological Association, Science Directorate, 1999) for report-
ing statistical analysis results.

In conclusion, we believe that our proposed MBCO procedure
provides multiple ways to evaluate hypotheses about mediation
effects beyond the methods widely recommended in the literature
(e.g., CI-based approaches). We believe that work using the
MBCO procedure will advance the rich literature on testing and
interpreting mediation models.

References

Akaike, H. (1974). A new look at the statistical model identification. IEEE
Transactions on Automatic Control, 19, 716 –723. http://dx.doi.org/10
.1109/TAC.1974.1100705

Amrhein, V., Trafimow, D., & Greenland, S. (2019). Inferential statistics
as descriptive statistics: There is no replication crisis if we don’t expect
replication. The American Statistician, 73, 262–270. http://dx.doi.org/10
.1080/00031305.2018.1543137

APA Publication Manual. (2010). Publication manual of the American
Psychological Association (6th ed.). Washington, DC: American Psy-
chological Association.

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M.,
& Rao, S. M. (2018). Journal article reporting standards for quantitative
research in psychology: The APA Publications and Communications
Board task force report. American Psychologist, 73, 3–25. http://dx.doi
.org/10.1037/amp0000191

Ato García, M., Vallejo Seco, G., & Ato Lozano, E. (2014). Classical and
causal inference approaches to statistical mediation analysis. Psico-
thema, 26, 252–259.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable
distinction in social psychological research: Conceptual, strategic, and
statistical considerations. Journal of Personality and Social Psychology,
51, 1173–1182. http://dx.doi.org/10.1037/0022-3514.51.6.1173

Biesanz, J. C., Falk, C. F., & Savalei, V. (2010). Assessing mediational
models: Testing and interval estimation for indirect effects. Multivariate
Behavioral Research, 45, 661–701. http://dx.doi.org/10.1080/00273171
.2010.498292

Blume, J. D. (2002). Likelihood methods for measuring statistical evi-
dence. Statistics in Medicine, 21, 2563–2599. http://dx.doi.org/10.1002/
sim.1216

Boies, K., Fiset, J., & Gill, H. (2015). Communication and trust are key:
Unlocking the relationship between leadership and team performance
and creativity. The Leadership Quarterly, 26, 1080 –1094. http://dx.doi
.org/10.1016/j.leaqua.2015.07.007

Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., . . . Fox,
J. (2011). OpenMx: An open source extended structural equation mod-
eling framework. Psychometrika, 76, 306 –317. http://dx.doi.org/10
.1007/s11336-010-9200-6

Bollen, K. A. (1989). Structural equations with latent variables. New
York, NY: Wiley. http://dx.doi.org/10.1002/9781118619179

Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and
bootstrap estimates of variability. Sociological Methodology, 20, 115–
140. http://dx.doi.org/10.2307/271084

Bradley, J. V. (1978). Robustness? British Journal of Mathematical &
Statistical Psychology, 31, 144 –152. http://dx.doi.org/10.1111/j.2044-
8317.1978.tb00581.x

Bulls, H. W., Lynch, M. K., Petrov, M. E., Gossett, E. W., Owens, M. A.,
Terry, S. C., . . . Goodin, B. R. (2017). Depressive symptoms and sleep
efficiency sequentially mediate racial differences in temporal summation
of mechanical pain. Annals of Behavioral Medicine, 51, 673– 682. http://
dx.doi.org/10.1007/s12160-017-9889-x

Carmeli, A., McKay, A. S., & Kaufman, J. C. (2014). Emotional intelli-
gence and creativity: The mediating role of generosity and vigor. The
Journal of Creative Behavior, 48, 290 –309. http://dx.doi.org/10.1002/
jocb.53

Cheung, M. W. L. (2007). Comparison of approaches to constructing
confidence intervals for mediating effects using structural equation
models. Structural Equation Modeling, 14, 227–246. http://dx.doi.org/
10.1080/10705510709336745

Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(2nd ed.). Hillsdale, NJ: Erlbaum.

Cohen, J. (1994). The earth is round (p � .05). American Psychologist, 49,
997–1003. http://dx.doi.org/10.1037/0003-066X.49.12.997

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple
regression/correlation analysis for the behavioral sciences (3rd ed.).
Mahwah, NJ: Erlbaum.

Cole, D. A., & Preacher, K. J. (2014). Manifest variable path analysis:
Potentially serious and misleading consequences due to uncorrected
measurement error. Psychological Methods, 19, 300 –315. http://dx.doi
.org/10.1037/a0033805

Cox, D. R., & Hinkley, D. V. (2000). Theoretical statistics. Boca Raton,
FL: Chapman & Hall/CRC.

Cox, M. G., Kisbu-Sakarya, Y., Miočević, M., & MacKinnon, D. P.
(2013). Sensitivity plots for confounder bias in the single mediator
model. Evaluation Review, 37, 405– 431. http://dx.doi.org/10.1177/
0193841X14524576

Deković, M., Asscher, J. J., Manders, W. A., Prins, P. J. M., & van der
Laan, P. (2012). Within-intervention change: Mediators of intervention
effects during multisystemic therapy. Journal of Consulting and Clinical
Psychology, 80, 574 –587. http://dx.doi.org/10.1037/a0028482

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

18 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/met0000259.supp

http://dx.doi.org/10.1109/TAC.1974.1100705

http://dx.doi.org/10.1109/TAC.1974.1100705

http://dx.doi.org/10.1080/00031305.2018.1543137

http://dx.doi.org/10.1080/00031305.2018.1543137

http://dx.doi.org/10.1037/amp0000191

http://dx.doi.org/10.1037/amp0000191

http://dx.doi.org/10.1037/0022-3514.51.6.1173

http://dx.doi.org/10.1080/00273171.2010.498292

http://dx.doi.org/10.1080/00273171.2010.498292

http://dx.doi.org/10.1002/sim.1216

http://dx.doi.org/10.1002/sim.1216

http://dx.doi.org/10.1016/j.leaqua.2015.07.007

http://dx.doi.org/10.1016/j.leaqua.2015.07.007

http://dx.doi.org/10.1007/s11336-010-9200-6

http://dx.doi.org/10.1007/s11336-010-9200-6

http://dx.doi.org/10.1002/9781118619179

http://dx.doi.org/10.2307/271084

http://dx.doi.org/10.1111/j.2044-8317.1978.tb00581.x

http://dx.doi.org/10.1111/j.2044-8317.1978.tb00581.x

http://dx.doi.org/10.1007/s12160-017-9889-x

http://dx.doi.org/10.1007/s12160-017-9889-x

http://dx.doi.org/10.1002/jocb.53

http://dx.doi.org/10.1002/jocb.53

http://dx.doi.org/10.1080/10705510709336745

http://dx.doi.org/10.1080/10705510709336745

http://dx.doi.org/10.1037/0003-066X.49.12.997

http://dx.doi.org/10.1037/a0033805

http://dx.doi.org/10.1037/a0033805

http://dx.doi.org/10.1177/0193841X14524576

http://dx.doi.org/10.1177/0193841X14524576

http://dx.doi.org/10.1037/a0028482

Dwyer, J. H. (1983). Statistical models for the social and behavioral
sciences. New York, NY: Oxford University Press.

Efron, B. (1987). Better bootstrap confidence intervals. Journal of the
American Statistical Association, 82, 171–185. http://dx.doi.org/10
.1080/01621459.1987.10478410

Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New
York, NY: Chapman & Hall.

Ernsting, A., Knoll, N., Schneider, M., & Schwarzer, R. (2015). The
enabling effect of social support on vaccination uptake via self-efficacy
and planning. Psychology Health and Medicine, 20, 239 –246. http://dx
.doi.org/10.1080/13548506.2014.920957

Falk, C. F., & Biesanz, J. C. (2015). Inference and interval estimation
methods for indirect effects with latent variable models. Structural
Equation Modeling: A Multidisciplinary Journal, 22, 24 –38. http://dx
.doi.org/10.1080/10705511.2014.935266

Folmer, H. (1981). Measurement of the effects of regional policy instru-
ments by means of linear structural equation models and panel data.
Environment & Planning A, 13, 1435–1448. http://dx.doi.org/10.1068/
a131435

Fox, J. (2016). Applied regression analysis and generalized linear models
(3rd ed.). Los Angeles, CA: SAGE.

Fritz, M. S., Kenny, D. A., & MacKinnon, D. P. (2016). The combined
effects of measurement error and omitting confounders in the single-
mediator model. Multivariate Behavioral Research, 51, 681– 697. http://
dx.doi.org/10.1080/00273171.2016.1224154

Gill, P. E., Murray, W., Saunders, M. A., & Wright, M. H. (1986). Fortran
package for nonlinear programming. User’s guide for NPSOL (Version
4. 0). Stanford, CA: Stanford University. Retrieved from https://apps
.dtic.mil/dtic/tr/fulltext/u2/a169115

Graça, J., Calheiros, M. M., & Oliveira, A. (2016). Situating moral disen-
gagement: Motivated reasoning in meat consumption and substitution.
Personality and Individual Differences, 90, 353–364. http://dx.doi.org/
10.1016/j.paid.2015.11.042

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C.,
Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values,
confidence intervals, and power: A guide to misinterpretations. Euro-
pean Journal of Epidemiology, 31, 337–350. http://dx.doi.org/10.1007/
s10654-016-0149-3

Harlow, L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there
were no significance tests? Mahwah, NJ: Erlbaum.

Haslam, C., Cruwys, T., Milne, M., Kan, C.-H., & Haslam, S. A. (2016).
Group ties protect cognitive health by promoting social identification
and social support. Journal of Aging and Health, 28, 244 –266. http://
dx.doi.org/10.1177/0898264315589578

Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal
mediation analysis. Psychological Methods, 15, 309 –334. http://dx.doi
.org/10.1037/a0020761

Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of
interaction effects between continuous predictors using multiple regres-
sion: Multiple indicator and structural equation approaches. Psycholog-
ical Bulletin, 117, 348 –357. http://dx.doi.org/10.1037/0033-2909.117.2
.348

Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating medi-
ation in treatment evaluations. Evaluation Review, 5, 602– 619. http://
dx.doi.org/10.1177/0193841X8100500502

Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social
psychology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The
handbook of social psychology (4th ed., Vol. 1, pp. 233–265). Boston,
MA: McGraw-Hill.

Kline, R. B. (2016). Principles and practice of structural equation mod-
eling (4th ed.). New York, NY: Guilford Press.

Koning, I. M., Maric, M., MacKinnon, D., & Vollebergh, W. A. M. (2015).
Effects of a combined parent-student alcohol prevention program on
intermediate factors and adolescents’ drinking behavior: A sequential

mediation model. Journal of Consulting and Clinical Psychology, 83,
719 –727. http://dx.doi.org/10.1037/a0039197

Koopman, J., Howe, M., Hollenbeck, J. R., & Sin, H.-P. (2015). Small
sample mediation testing: Misplaced confidence in bootstrapped confi-
dence intervals. Journal of Applied Psychology, 100, 194 –202. http://
dx.doi.org/10.1037/a0036635

Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual
and group level mediated effects. Multivariate Behavioral Research, 36,
249 –277. http://dx.doi.org/10.1207/S15327906MBR3602_06

MacCallum, R. C., & Austin, J. T. (2000). Applications of structural
equation modeling in psychological research. Annual Review of Psychol-
ogy, 51, 201–226. http://dx.doi.org/10.1146/annurev.psych.51.1.201

MacKinnon, D. P. (2008). Introduction to statistical mediation analysis.
New York, NY: Erlbaum.

MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., &
Sheets, V. (2002). A comparison of methods to test mediation and other
intervening variable effects. Psychological Methods, 7, 83–104. http://
dx.doi.org/10.1037/1082-989X.7.1.83

Mackinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence
limits for the indirect effect: Distribution of the product and resampling
methods. Multivariate Behavioral Research, 39, 99 –128. http://dx.doi
.org/10.1207/s15327906mbr3901_4

MacKinnon, D. P., & Pirlott, A. G. (2015). Statistical approaches for
enhancing causal interpretation of the M to Y relation in mediation
analysis. Personality and Social Psychology Review, 19, 30 – 43. http://
dx.doi.org/10.1177/1088868314542878

MacKinnon, D. P., Valente, M. J., & Wurpts, I. C. (2018). Benchmark
validation of statistical models: Application to mediation analysis of
imagery and memory. Psychological Methods, 23, 654 – 671. http://dx
.doi.org/10.1037/met0000174

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experi-
ments and analyzing data: A model comparison perspective (3rd ed.).
New York, NY: Routledge.

McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the
Reticular Action Model for moment structures. British Journal of Math-
ematical & Statistical Psychology, 37, 234 –251. http://dx.doi.org/10
.1111/j.2044-8317.1984.tb00802.x

Molina, B. S. G., Walther, C. A. P., Cheong, J., Pedersen, S. L., Gnagy,
E. M., & Pelham, W. E. J. (2014). Heavy alcohol use in early adulthood
as a function of childhood ADHD: Developmentally specific mediation
by social impairment and delinquency. Experimental and Clinical Psy-
chopharmacology, 22, 110 –121. http://dx.doi.org/10.1037/a0035656

Muthén, B. O., & Asparouhov, T. (2015). Causal effects in mediation
modeling: An introduction with applications to latent variables. Struc-
tural Equation Modeling, 22, 12–23. http://dx.doi.org/10.1080/
10705511.2014.935843

Muthén, L. K., & Muthén, B. O. (2018). Mplus user’s guide (7th ed.). Los
Angeles, CA: Author.

Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R.,
Kirkpatrick, R. M., . . . Boker, S. M. (2016). OpenMx 2.0: Extended
structural equation and statistical modeling. Psychometrika, 81, 535–
549. http://dx.doi.org/10.1007/s11336-014-9435-8

Neale, M. C., & Miller, M. B. (1997). The use of likelihood-based confi-
dence intervals in genetic models. Behavior Genetics, 27, 113–120.
http://dx.doi.org/10.1023/A:1025681223921

Oliver, M. C., Bays, R. B., & Zabrucky, K. M. (2016). False memories and
the DRM paradigm: Effects of imagery, list, and test type. The Journal
of General Psychology, 143, 33– 48. http://dx.doi.org/10.1080/00221309
.2015.1110558

Pawitan, Y. (2001). In all likelihood: Statistical modelling and inference
using likelihood. New York, NY: Oxford University Press.

Pek, J., & Wu, H. (2015). Profile likelihood-based confidence intervals and
regions for structural equation models. Psychometrika, 80, 1123–1145.
http://dx.doi.org/10.1007/s11336-015-9461-1

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

19IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1080/01621459.1987.10478410

http://dx.doi.org/10.1080/01621459.1987.10478410

http://dx.doi.org/10.1080/13548506.2014.920957

http://dx.doi.org/10.1080/13548506.2014.920957

http://dx.doi.org/10.1080/10705511.2014.935266

http://dx.doi.org/10.1080/10705511.2014.935266

http://dx.doi.org/10.1068/a131435

http://dx.doi.org/10.1068/a131435

http://dx.doi.org/10.1080/00273171.2016.1224154

http://dx.doi.org/10.1080/00273171.2016.1224154

https://apps.dtic.mil/dtic/tr/fulltext/u2/a169115

https://apps.dtic.mil/dtic/tr/fulltext/u2/a169115

http://dx.doi.org/10.1016/j.paid.2015.11.042

http://dx.doi.org/10.1016/j.paid.2015.11.042

http://dx.doi.org/10.1007/s10654-016-0149-3

http://dx.doi.org/10.1007/s10654-016-0149-3

http://dx.doi.org/10.1177/0898264315589578

http://dx.doi.org/10.1177/0898264315589578

http://dx.doi.org/10.1037/a0020761

http://dx.doi.org/10.1037/a0020761

http://dx.doi.org/10.1037/0033-2909.117.2.348

http://dx.doi.org/10.1037/0033-2909.117.2.348

http://dx.doi.org/10.1177/0193841X8100500502

http://dx.doi.org/10.1177/0193841X8100500502

http://dx.doi.org/10.1037/a0039197

http://dx.doi.org/10.1037/a0036635

http://dx.doi.org/10.1037/a0036635

http://dx.doi.org/10.1207/S15327906MBR3602_06

http://dx.doi.org/10.1146/annurev.psych.51.1.201

http://dx.doi.org/10.1037/1082-989X.7.1.83

http://dx.doi.org/10.1037/1082-989X.7.1.83

http://dx.doi.org/10.1207/s15327906mbr3901_4

http://dx.doi.org/10.1207/s15327906mbr3901_4

http://dx.doi.org/10.1177/1088868314542878

http://dx.doi.org/10.1177/1088868314542878

http://dx.doi.org/10.1037/met0000174

http://dx.doi.org/10.1037/met0000174

http://dx.doi.org/10.1111/j.2044-8317.1984.tb00802.x

http://dx.doi.org/10.1111/j.2044-8317.1984.tb00802.x

http://dx.doi.org/10.1037/a0035656

http://dx.doi.org/10.1080/10705511.2014.935843

http://dx.doi.org/10.1080/10705511.2014.935843

http://dx.doi.org/10.1007/s11336-014-9435-8

http://dx.doi.org/10.1023/A:1025681223921

http://dx.doi.org/10.1080/00221309.2015.1110558

http://dx.doi.org/10.1080/00221309.2015.1110558

http://dx.doi.org/10.1007/s11336-015-9461-1

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003).
Common method biases in behavioral research: A critical review of the
literature and recommended remedies. Journal of Applied Psychology,
88, 879 –903. http://dx.doi.org/10.1037/0021-9010.88.5.879

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling
strategies for assessing and comparing indirect effects in multiple me-
diator models. Behavior Research Methods, 40, 879 – 891. http://dx.doi
.org/10.3758/BRM.40.3.879

R Core Team. (2019). R: A Language and Environment for Statistical
Computing (Version 3.6.0) [Computer software]. Retrieved from http://
www.R-project.org/

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models:
Applications and data analysis methods (2nd ed.). Thousand Oaks, CA:
SAGE.

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling.
Journal of Statistical Software, 48, 1–36. http://dx.doi.org/10.18637/jss
.v048.i02

SAS Institute Inc. (2016). SAS/STAT 14.2 user’s guide. Cary, NC: Author.
Retrieved from http://support.sas.com/documentation/cdl/en/stathpug/
67524/PDF/default/stathpug

Schwarz, G. (1978). Estimating the dimension of a model. Annals of
Statistics, 6, 461– 464. http://dx.doi.org/10.1214/aos/1176344136

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and non-
experimental studies: New procedures and recommendations. Psycho-
logical Methods, 7, 422– 445. http://dx.doi.org/10.1037/1082-989X.7.4
.422

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An intro-
duction to basic and advanced multilevel modeling (2nd ed.). Thousand
Oaks, CA: SAGE.

Snyman, J. A. (2005). Practical mathematical optimization: An introduc-
tion to basic optimization theory and classical and new gradient-based
algorithms. New York, NY: Springer.

Taylor, A. B., MacKinnon, D. P., & Tein, J.-Y. (2008). Tests of the
three-path mediated effect. Organizational Research Methods, 11, 241–
269. http://dx.doi.org/10.1177/1094428107300344

Thoemmes, F., Mackinnon, D. P., & Reiser, M. R. (2010). Power analysis
for complex mediational designs using Monte Carlo methods. Structural
Equation Modeling, 17, 510 –534. http://dx.doi.org/10.1080/10705511
.2010.489379

Tofighi, D., Hsiao, Y.-Y., Kruger, E. S., MacKinnon, D. P., Van Horn,
M. L., & Witkiewitz, K. A. (2019). Sensitivity analysis of the no-omitted
confounder assumption in latent growth curve mediation models. Struc-
tural Equation Modeling, 26, 94 –109. http://dx.doi.org/10.1080/
10705511.2018.1506925

Tofighi, D., & Kelley, K. (2016). Assessing omitted confounder bias in
multilevel mediation models. Multivariate Behavioral Research, 51,
86 –105. http://dx.doi.org/10.1080/00273171.2015.1105736

Tofighi, D., & Kelley, K. (2019). Indirect effects in sequential mediation
models: Evaluating methods for hypothesis testing and confidence in-
terval formation. Multivariate Behavioral Research. Advance online
publication. http://dx.doi.org/10.1080/00273171.2019.1618545

Tofighi, D., & MacKinnon, D. P. (2011). RMediation: An R package for
mediation analysis confidence intervals. Behavior Research Methods,
43, 692–700. http://dx.doi.org/10.3758/s13428-011-0076-x

Tofighi, D., & MacKinnon, D. P. (2016). Monte Carlo confidence intervals
for complex functions of indirect effects. Structural Equation Modeling,
23, 194 –205. http://dx.doi.org/10.1080/10705511.2015.1057284

Tofighi, D., West, S. G., & MacKinnon, D. P. (2013). Multilevel mediation
analysis: The effects of omitted variables in the 1–1-1 model. British
Journal of Mathematical & Statistical Psychology, 66, 290 –307. http://
dx.doi.org/10.1111/j.2044-8317.2012.02051.x

Valeri, L., & Vanderweele, T. J. (2013). Mediation analysis allowing for
exposure-mediator interactions and causal interpretation: Theoretical
assumptions and implementation with SAS and SPSS macros. Psycho-
logical Methods, 18, 137–150. http://dx.doi.org/10.1037/a0031034

VanderWeele, T. J. (2010). Bias formulas for sensitivity analysis for direct
and indirect effects. Epidemiology, 21, 540 –551. http://dx.doi.org/10
.1097/EDE.0b013e3181df191c

VanderWeele, T. J. (2015). Explanation in causal inference: Methods for
mediation and interaction. New York, NY: Oxford University Press.

Wald, A. (1943). Tests of statistical hypotheses concerning several param-
eters when the number of observations is large. Transactions of the
American Mathematical Society, 54, 426 – 482. http://dx.doi.org/10
.1090/S0002-9947-1943-0012401-3

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a
world beyond “p � 0.05”. The American Statistician, 73, 1–19. http://
dx.doi.org/10.1080/00031305.2019.1583913

Weisstein, E. W. (2018). Smooth function. Retrieved from http://
mathworld.wolfram.com/SmoothFunction.html

Wickham, H., François, R., Henry, L., & Müller, K. (2019). dplyr: A
grammar of data manipulation. Retrieved from https://CRAN.R-project
.org/package�dplyr

Wilkinson, L., & Task Force on Statistical Inference, American Psycho-
logical Association, Science Directorate. (1999). Statistical methods in
psychology journals: Guidelines and explanations. American Psycholo-
gist, 54, 594 – 604. http://dx.doi.org/10.1037/0003-066X.54.8.594

Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio
for testing composite hypotheses. Annals of Mathematical Statistics, 9,
60 – 62. http://dx.doi.org/10.1214/aoms/1177732360

Williams, J., & MacKinnon, D. P. (2008). Resampling and distribution of
the product methods for testing indirect effects in complex models.
Structural Equation Modeling: A Multidisciplinary Journal, 15, 23–51.

Yzerbyt, V., Muller, D., Batailler, C., & Judd, C. M. (2018). New recom-
mendations for testing indirect effects in mediational models: The need
to report and test component paths. Journal of Personality and Social
Psychology, 115, 929 –943. http://dx.doi.org/10.1037/pspa0000132

Zahery, M., Maes, H. H., & Neale, M. C. (2017). CSOLNP: Numerical
optimization engine for solving non-linearly constrained problems. Twin
Research and Human Genetics, 20, 290 –297. http://dx.doi.org/10.1017/
thg.2017.28

Received November 27, 2017
Revision received December 3, 2019

Accepted January 4, 2020 �

T
hi
s
do
cu
m
en
t
is
co
py
ri
gh
te
d
by
th
e
A
m
er
ic
an
P
sy
ch
ol
og
ic
al
A
ss
oc
ia
ti
on
or
on
e
of
it
s
al
li
ed
pu
bl
is
he
rs
.
T
hi
s
ar
ti
cl
e
is
in
te
nd
ed
so
le
ly
fo
r
th
e
pe
rs
on
al
us
e
of
th
e
in
di
vi
du
al
us
er
an
d
is
no
t
to
be
di
ss
em
in
at
ed
br
oa
dl
y.

20 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/0021-9010.88.5.879

http://dx.doi.org/10.3758/BRM.40.3.879

http://dx.doi.org/10.3758/BRM.40.3.879

http://www.R-project.org/

http://www.R-project.org/

http://dx.doi.org/10.18637/jss.v048.i02

http://dx.doi.org/10.18637/jss.v048.i02

http://support.sas.com/documentation/cdl/en/stathpug/67524/PDF/default/stathpug

http://support.sas.com/documentation/cdl/en/stathpug/67524/PDF/default/stathpug

http://dx.doi.org/10.1214/aos/1176344136

http://dx.doi.org/10.1037/1082-989X.7.4.422

http://dx.doi.org/10.1037/1082-989X.7.4.422

http://dx.doi.org/10.1177/1094428107300344

http://dx.doi.org/10.1080/10705511.2010.489379

http://dx.doi.org/10.1080/10705511.2010.489379

http://dx.doi.org/10.1080/10705511.2018.1506925

http://dx.doi.org/10.1080/10705511.2018.1506925

http://dx.doi.org/10.1080/00273171.2015.1105736

http://dx.doi.org/10.1080/00273171.2019.1618545

http://dx.doi.org/10.3758/s13428-011-0076-x

http://dx.doi.org/10.1080/10705511.2015.1057284

http://dx.doi.org/10.1111/j.2044-8317.2012.02051.x

http://dx.doi.org/10.1111/j.2044-8317.2012.02051.x

http://dx.doi.org/10.1037/a0031034

http://dx.doi.org/10.1097/EDE.0b013e3181df191c

http://dx.doi.org/10.1097/EDE.0b013e3181df191c

http://dx.doi.org/10.1090/S0002-9947-1943-0012401-3

http://dx.doi.org/10.1090/S0002-9947-1943-0012401-3

http://dx.doi.org/10.1080/00031305.2019.1583913

http://dx.doi.org/10.1080/00031305.2019.1583913

http://mathworld.wolfram.com/SmoothFunction.html

http://mathworld.wolfram.com/SmoothFunction.html

https://CRAN.R-project.org/package=dplyr

https://CRAN.R-project.org/package=dplyr

http://dx.doi.org/10.1037/0003-066X.54.8.594

http://dx.doi.org/10.1214/aoms/1177732360

http://dx.doi.org/10.1037/pspa0000132

http://dx.doi.org/10.1017/thg.2017.28

http://dx.doi.org/10.1017/thg.2017.28

• Improved Inference in Mediation Analysis: Introducing the Model-Based Constrained Optimization P …

The Model-Based Constrained Optimization Procedure
Empirical Example of Using the MBCO Procedure on a Complex Mediation Model
Research Question 1
Step 1
Step 2
Step 3
Research Question 2
Step 1
Step 2
Step 3
Research Question 3
Step 1
Step 2
Step 3
Comparing Methods of Testing Indirect Effect Via Monte Carlo Simulations
Percentile and Bias-Corrected Bootstrap CI
Monte Carlo CI
Profile Likelihood CI
Joint Significance Test
Simulation Design and Population Values
Data Generation
Results
Type I error
Power
Summary
Limitations
Discussion
References