Assignment

ETHICAL LEADERSHIP AND UNETHICAL EMPLOYEE

BEHAVIOR: A MODERATED MEDIATION MODEL

CHENJING GAN

Ningbo University

I explored the role of employee moral justification as a cognitive mediator in the relationship

between ethical leadership and unethical employee behavior, and then investigated employee

moral identity as a moderator of this indirect relationship

.

I based my moderated meditation

model on social learning theory and tested it by analyzing data collected from 271 employees

of 17 firms in China at 2 time points separated by approximately 3 weeks. The results showed

that the negative indirect relationship between ethical leadership and unethical employee

behavior through moral justification was significant when employee moral identity was

strong. Theoretical and practical implications of the findings are discussed.

Keywords: ethical leadership, unethical employee behavior, moral justification, moral

identity, social learning.

Various types of unethical behavior occur in virtually every sector of society,

and ethical issues have become topics of great interest in both mass media and

the academic community (O’Fallon & Butterfield, 2011, 2012). The concept

of ethical leadership, which is defined as “the demonstration of normatively

appropriate conduct through personal actions and interpersonal relationships, and

the promotion of such conduct to followers through two-way communication,

reinforcement, and decision making” (Brown, Treviño, & Harrison, 2005, p. 120),

has received much attention from many scholars in recent years. Researchers

have mainly drawn upon social learning theory (Bandura, 1977) to explain

the influence of ethical leadership on unethical employee behavior, whereby

employees learn what behaviors are expected, rewarded, and punished by

SOCIAL BEHAVIOR AND PERSONALITY, 2018, 46(8), 1271–128

4

© 2018 Scientific Journal Publishers Limited. All Rights Reserved.

https://doi.org/10.2224/sbp.7328

1271

Chenjing Gan, Business School, Ningbo University.

This research was supported by the Natural Science Foundation of Zhejiang Province (LQ18G020002).

Correspondence concerning this article should be addressed to Chenjing Gan, Business School,

Ningbo University, No. 18 Fenghua Road, Ningbo 3125211, People’s Republic of China. Email:

ganchenjing@nbu.edu.cn

ETHICAL LEADERS AND UNETHICAL EMPLOYEES127

2

observing and imitating ethical leaders’ behaviors (Mayer, Aquino, Greenbaum,

& Kuenzi, 2012). However, it remains unclear how employees learn to inhibit

their unethical behavior, or why some employees learn from ethical leaders

and some do not. In social learning theory it is posited that the effects of

reinforcement are cognitively mediated, which means that learning cannot take

place without awareness of what is being reinforced (Bandura, 1977). In other

words, employees learn to eliminate their unethical behaviors only when they

are aware of which behaviors are unacceptable to ethical leaders. Thus, in the

current study I explored employee moral justification as a cognitive mediating

mechanism in the relationship between ethical leadership and unethical employee

behavior, and examined employee moral identity as a moderator of this indirect

relationship.

Moral justification involves cognitive reconstruction of an ethically questionable

behavior and is an attempt by the individual to legitimize this behavior rather

than to assume responsibility for an outcome (Barsky, 2011; Green, 1991; Hunt

& Vitell, 1986; Stevens, Deuling, & Armenakis, 2012; Vitell, Keith, & Mathur,

2011). In other words, moral justification involves the individual reconstructing

harmful behavior to make it appear personally and socially acceptable (Aquino,

Reed, Thau, & Freeman, 2007; Detert, Treviño, & Sweitzer, 2008). When

employees are engaged in moral justification, they cognitively legitimize their

unethical behavior and are not aware that the behavior is forbidden; thus, the

learning effect of ethical leadership on unethical employee behavior is absent.

Therefore, I proposed that moral justification would act as a cognitive mediator

of the relationship between ethical leadership and unethical employee behavior.

According to social learning theory, people self-regulate by setting behavioral

standards for themselves and responding to their own behavior in accordance

with their self-imposed demands (Bandura, 1977). Researchers have posited that

unethical behaviors occur only when individuals’ self-regulatory mechanisms

are deactivated through several cognitive mechanisms, such that they are unable

to anticipate and judge their actions in comparison to a set of internal moral

standards (Bandura, 1986; Detert et al., 2008; Moore, Detert, Klebe Treviño,

Baker, & Mayer, 2012; Stevens et al., 2012). This being so, some employees may

be reluctant to learn from an ethical leader, as the behaviors of that leader are

inconsistent with the employee’s own behavioral standards.

Moral identity refers to the extent to which one’s self-concept incorporates

the importance of being a moral person (Aquino & Reed, 2002), and it affects

individual self-regulatory mechanisms by setting standards for individual

behavior (Aquino, Freeman, Reed, Lim, & Felps, 2009; Bergman, 2002; Mulder

& Aquino, 2013; Reynolds & Ceranic, 2007). The self-imposed demands of

employees with a weak moral identity are quite different from those of ethical

leaders, and these employees are reluctant to learn from their ethical leaders.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 127

3

Therefore, I argued that the negative effect of ethical leadership on unethical

employee behavior through moral justification would be diminished for

employees with a weak moral identity. Moral identity may operate as a moderator

that strengthens the negative relationship between ethical leadership and moral

justification, thereby further inhibiting unethical employee behavior.

Figure 1 illustrates the proposed moderated mediation model and provides an

overview of the study.

Moral identity

Ethical leadership Moral justification

Unethical employee

behavior

Figure 1. Mediated moderation research model.

Theoretical Background and Hypotheses

Ethical Leadership, Moral Justification, and Unethical Behavior

Brown and Treviño (2006) elaborated on a proposition that moral persons and

moral managers are the two components of ethical leadership. As moral people,

ethical leaders have desirable characteristics, such as being fair and trustworthy.

As moral managers, ethical leaders use transactional efforts to hold followers

to acting according to moral standards, such as by communicating ethics and

punishing followers’ unethical behavior. The influence of ethical leadership on

employee moral justification can also be explained through these two aspects

of leadership style (Bandura, 1986; Brown & Treviño, 2006; Liu, Long, & Loi,

2012). First, as moral people, ethical leaders pay attention to, and consistently

align their own behavior with, moral standards. From observing their leaders’

behaviors, employees learn that unethical behavior is morally unacceptable and,

therefore, do not justify it (Brown & Treviño, 2006; Brown et al., 2005). Because

ethical leaders are credible role models for ethical behaviors, employees cannot

dilute their own faulty actions by diffusing responsibility; this inhibits employee

moral justification (Liu et al., 2012). Second, as moral managers, ethical leaders

demand that employees have high ethical standards, are accountable for the moral

consequences of their behaviors, and care for others’ interests (Brown & Treviño,

2006; Brown et al., 2005). However, individuals use moral justification to make

harming others appear acceptable to both themselves and others. Individuals

who engage in moral justification are concerned about their self-interests and

shirk responsibility for their unethical behaviors (Moore et al., 2012; Vitell et al.,

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1274

2011). The demand and influence of ethical leadership are contradictory to the

cognitive process of moral justification and inhibit individuals from engaging in

moral justification. Therefore, I argued that ethical leadership would decrease

employee moral justification.

As already described, in social learning theory it is posited that learning

cannot take place without awareness of what is being reinforced (Bandura,

1977). Employees who engage in moral justification do not sincerely approve

of ethical leaders’ behaviors and do not learn to inhibit their own unethical

behaviors. Furthermore, researchers have described how unethical behaviors are

inhibited when individuals’ self-regulatory mechanisms are working effectively,

as engaging in unethical behaviors is inconsistent with the individual’s internal

moral standard, which can result in feelings of guilt (Bandura, 1986; Moore et

al., 2012, Vitell et al., 2011). However, moral justification deactivates one’s self-

regulatory mechanism by reducing the negative impact of unethical behaviors

on self-concept through finding excuses to legitimize one’s unethical behavior

(Barsky, 2011; Hunt & Vitell, 1986; Stevens et al., 2012; Vitell et al., 2011).

Thus, the inconsistency between an individual behaving unethically and his or

her internal moral standard is reduced through moral justification. Therefore,

I argued that, through the reframing of their cognitive perceptions, moral

justification would allow individuals to engage in unethical behavior without

experiencing distress. In some empirical studies, scholars have also found that

moral justification is positively related to unethical decision making (Stevens et

al., 2012) and unethical behavior (Barsky, 2011; Moore et al., 2012). On the basis

of these arguments and findings, I proposed the following hypothesis:

Hypothesis 1: Employee moral justification will mediate the negative relationship

between ethical leadership and unethical employee behavior, such that ethical

leadership will decrease employee moral justification and employee moral

justification will increase unethical employee behavior.

Moral Identity as a Moderator

As already described, according to social learning theory, people self-regulate

by setting themselves certain behavioral standards. Moral identity influences

an individual’s self-regulatory mechanism by setting the parameters for his or

her moral standard, which creates a need for the individual to act consistently

with his or her identity (Bergman, 2002; Reynolds & Ceranic, 2007). The high

ethical standards set by ethical leaders for their employees are inconsistent with

the self-concept and behavioral standards of employees with a weak moral

identity (Aquino et al., 2009; Mulder & Aquino, 2013; Reynolds & Ceranic,

2007). Therefore, these employees are not susceptible to the influence of ethical

leadership and continue to avoid responsibility for their own unethical behaviors

through moral justification. Additionally, although ethical leaders are moral

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 127

5

persons who behave ethically and appropriately, employees with a weak moral

identity still find ways to justify their unethical behavior, such as taking an “every

man for himself” attitude. Thus, a weak moral identity diminishes the effect of

ethical leadership on employees’ moral justification. Employees with a strong

moral identity have high internal moral standards that are similar to those of

their leaders, and easily accept and follow their leaders’ moral code; thus, they

are susceptible to the influence of ethical leadership. Therefore, I proposed the

following hypothesis:

Hypothesis 2: Employee moral identity will moderate the relationship between

ethical leadership and employee moral justification, such that this relationship

will be weaker when employee moral identity is weaker.

I further argued that employee moral identity would moderate the indirect

relationship between ethical leadership and unethical employee behavior.

The behavioral standards of employees with a weak moral identity can be

distinguished from those of their ethical leader; therefore, they are less likely

to learn cognitively from their ethical leaders (Bergman, 2002; Reynolds &

Ceranic, 2007). In this condition, the effect of ethical leadership on employee

moral justification is diminished, promoting unethical employee behavior. In

contrast, employees’ strong moral identity will strengthen the effect of ethical

leadership on employee moral justification, further inhibiting unethical employee

behavior. Therefore, I offered the following hypothesis:

Hypothesis 3: The indirect relationship between ethical leadership and unethical

employee behavior through employee moral justification will be conditional on

employee moral identity, such that this indirect relationship will be weaker when

employee moral identity is lower.

Method

Participants and Procedure

I obtained my data from 271 employees of 17 firms in China, collecting

these data at two points in time separated by approximately 3 weeks. The firms

belonged to the construction, traditional manufacturing, financial services, and

advertising industries. A list of people who could be participants was provided to

me by each firm’s human resources department. Each participant was assigned

a unique survey code so that I could match the data from Times 1 and 2. Each

questionnaire was put into an envelope with a clear plastic window to assure

participants’ full confidentiality. I also informed the participants that their data

would be used for academic research purposes only.

Surveys were distributed and collected by my research team. Ethical leadership,

moral identity, and demographic information (age, gender, and level of

education) were assessed at Time 1, and moral justification and the participant’s

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1276

own unethical behavior were assessed at Time 2. At Time 1 423 participants

completed the survey, and 355 participants completed the survey at Time 2.

I discarded some records for unmatched Time 1 and Time 2 survey pairs;

ultimately, complete responses were obtained from 271 employees. Among the

participants, 60% were men and 40% were women, and the average age was 30

years (SD = 8.35).

Measures

The original relevant scales were written in English, so these were translated

into Chinese following Brislin’s (1980) recommendations. That is, the scales

were first translated from English to Chinese by one bilingual speaker. Then,

another translator without access to the original survey back-translated the same

items from Chinese into English. The second translator was asked to comment

on any items that were perceived to be ambiguous. Both translators were native

Chinese speakers who had studied abroad in English-speaking countries and

had scored over 7 points on the International English Language Testing System

(Brislin, 1980). All scales were anchored on a 5-point Likert scale that ranged

from 1 = strongly disagree to 5 = strongly agree.

Ethical leadership. I measured ethical leadership with the 10-item Ethical

Leadership Scale ( = .95) developed and validated by Brown et al. (2005).

Sample items include “My supervisor listens to what employees have to say”

and “My supervisor sets an example of how to do things the right way in terms

of ethics.”

Moral justification. I assessed moral justification ( = .82) with a four-item

scale developed by Bandura, Barbaranelli, Caprara, and Pastorelli (1996).

Sample items include “It is all right to lie to keep yourself out of trouble” and “It

is all right to fight to protect yourself.”

Moral identity. I measured moral identity using Aquino and Reed’s (2002)

five-item internalization subscale ( = .92). Participants were asked to read a

list of nine characteristics (i.e., caring, compassionate, fair, friendly, generous,

helpful, hardworking, honest, and kind) and to visualize a person (which could

be the person him/herself or someone else) who has these characteristics. Once

participants had a clear image of this person, they were asked to rate their level of

agreement with a set of statements. Sample items include “Being someone who

has these characteristics is an important part of who I am” and “I strongly desire

to have these characteristics.”

Unethical behavior. I measured unethical behavior with a 16-item scale

( = .92) developed by Zey-Ferrell, Weaver, and Ferrell (1979). The original

scale contains 17 items. After considering the pilot study results I integrated two

items (“I pad an expense account more than 10%” and “I pad an expense account

up to 10%”) into one item: “I pad an expense account.” Sample items include

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1277

“I do personal business on company time” and “I pass blame for errors to an

innocent coworker.”

Control variables. Following the procedure used in previous research

on unethical employee behavior (O’Fallon & Butterfield, 2011, 2012), I

controlled for three demographic variables to better estimate the effect sizes

of the hypothesized relationships. Specifically, I measured and controlled for

employees’ gender (male = 0, female = 1), age, and level of education (junior

high school degree or lower = 1, high school or technical school degree = 2,

college degree = 3, bachelor’s degree = 4, master’s degree or higher = 5), because

these can influence unethical employee behavior.

Results

Validity of the Measures

I used Amos 17.0 to conduct a series of confirmatory factor analyses to

examine whether the study variables captured distinct constructs (Anderson &

Gerbing, 1988). The four-factor measurement model fitted the data well: chi

square (2) = 1046.18, degrees of freedom (df) = 553, comparative fit index

(CFI) = .92, incremental fit index (IFI) = .92, Tucker–Lewis index (TLI) = .91,

root mean square error of approximation (RMSEA) = .06, and all indicators had

statistically significant factor loadings (p < .01). Compared to the four-factor
model, the one-factor measurement model fitted the data poorly: 2 = 3815.55,
df = 559, CFI = .44, IFI = .45, TLI = .37, RMSEA = .15. The chi-square
difference compared with the four-factor model was significant (Δ2 = 2769.37,
p < .01), indicating distinctly different factors. Because data on ethical leadership
and employee moral identity were collected from the same source at Time 1,
and data on employee moral justification and unethical behavior were collected
from the same source at Time 2, I merged ethical leadership and employee moral
identity into one factor and employee moral justification and unethical behavior
into another factor to further test whether these variables captured distinct
constructs. Compared to the four-factor model, the two-factor measurement
model fitted the data poorly: 2 = 2163.75, df = 558, CFI = .73, IFI = .73,
TLI = .69, RMSEA = .10. The chi-square difference compared with the
four-factor model was significant (Δ2 = 1117.57, p < .01). These results provide
support for the discriminant validity of my measures.

Descriptive Statistical Analysis

Table 1 presents the descriptive statistics, internal consistency reliabilities, and

correlations among the variables. All correlations were in the expected directions

and provided conditions to further test my hypotheses.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1278

Table 1. Descriptive Statistics and Correlations Among Study Variables

Variable M SD 1 2 3 4 5 6 7

1. Gender 0.40 0.49

2. Age 30.03 8.35 .03

3. Level of education 3.18 1.03 -.01 -.22**

4. Ethical leadership 4.08 0.76 .11 -.11 .10 (.95)

5. Moral justification 2.10 0.97 -.09 .04 .10 -.19** (.82)

6. Moral identity 4.51 0.70 .12 -.03 -.03 .46** -.19** (.92)

7. Unethical employee 1.85 0.59 -.16* -.00 .06 -.29** .25** -.28** (.92)

behavior

Note. N = 271. Cronbach’s alpha coefficients are reported in parentheses.

* p < .05, ** p < .01.

Testing of Hypotheses

Following Muller, Judd, and Yzerbyt (2005), I used multiple regression

analysis to test my moderated mediation model. Further, to reduce

multicollinearity I centered ethical leadership, employee moral justification, and

moral identity, per Aiken and West (1991). As shown in Table 2 (Model 1), after

controlling for employees’ gender, age, and level of education, ethical leadership

was negatively related to unethical employee behavior; this result is consistent

with those of previous researchers. In Hypothesis 1 I predicted that employee

moral justification would mediate the relationship between ethical leadership and

unethical employee behavior. After controlling for the demographic variables,

ethical leadership was significantly and negatively related to employee moral

justification (see Table 2, Model 2), and moral justification was positively and

significantly related to unethical employee behavior (see Table 2, Model 3).

Furthermore, when the effect of ethical leadership was controlled for, moral

justification was still positively related to unethical behavior (see Table 2, Model

4). The results of an additional bootstrapping procedure with 95% confidence

intervals (CI), as outlined by Preacher and Hayes (2008), revealed that ethical

leadership was indirectly related to unethical employee behavior through the

mediator of moral justification: indirect effect = -.03; Sobel z = -2.29, p < .05;
95% CI [-.063, -.008]. These results provide support for Hypothesis 1.

In Hypothesis 2 I predicted that employee moral identity would moderate

the relationship between ethical leadership and unethical employee behavior,

such that this relationship would be weaker when moral identity was weaker.

As shown in Table 2 (Model 6), the interaction term of ethical leadership and

moral identity was significant in predicting employee moral justification, and the

additional proportion of the variance in employee moral justification explained

by this interaction term was also significant (ΔR2 = .02, p < .01; compared to
Model 5). To clarify this conditional effect, simple slope tests were conducted

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1279

(Cohen, Cohen, West, & Aiken, 2003). As shown in Figure 2, the relationship

between ethical leadership and unethical employee behavior was significantly

negative for employees with a strong moral identity (M + 1 SD; simple

slope = -.27, p < .01), and for employees with a weak moral identity, the effect
of ethical leadership on unethical employee behavior was nonsignificant (M − 1
SD; simple slope = -.04, ns), thus supporting Hypothesis 2.

In Hypothesis 3 I predicted that employee moral identity would moderate

the indirect relationship between ethical leadership and unethical employee

behavior through employee moral justification. As shown in Table 2 (Model 7),

after controlling for demographic variables, ethical leadership, employee moral

identity, and the interaction term of the latter two variables, employee moral

justification was found to be positively and significantly related to unethical

employee behavior. The results of an additional bootstrapping procedure with

95% CI (Preacher & Hayes, 2008) revealed that ethical leadership was indirectly

related to unethical employee behavior through employee moral justification

when employee moral identity was strong (M + 1 SD): indirect effect = -.03;

Sobel z = -1.97, p < .05; 95% CI [-.068, -.004]. For employees with a weak
moral identity, the indirect relationship between ethical leadership and unethical
employee behavior was not significantly different from zero (M − 1 SD): indirect
effect = -.01; Sobel z = -0.63, ns; 95% CI [-.034, .013]. Hypothesis 3 was,
therefore, supported.

Figure 2. Moderating role of moral identity on the ethical leadership–employee moral

justification relationship.

M

or

al

j

us

ti

fi

ca

ti

on

5

4.5

4

3.5

3

2.5

2

1.5

1

Low ethical leadership High ethical leadership

Low moral identity

High moral identity

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1280

Ta

b

le

2

.

R

e

su

lts

o

f

M

u

lti

p

le

R

e

g

re

ss

io

n

A

n

a

ly

si

s

V

ar

ia

bl

es

U

ne

th

ic

al

M

or

al

U

ne

th

ic

al

U

ne

th

ic

al

M

or

al

M

or

al

U

ne

th

ic

al

be

ha

vi

or

ju

st

if

ic

at

io

n

be

ha

vi

or

be

ha

vi

or

ju

st

if

ic

at

io

n

ju

st

if

ic

at

io

n

be

ha

vi

or

(M

od

el

1

)

(M

od

el

2

)

(M

od

el

3

)

(M

od

el

4

)

(M

od

el

5

)

(M

od

el

6

)

(M

od

el

7

)

G

en

de

r

-.

13

-.

05

-.

16

*

-.

12

-.

04

-.

03

-.

11

A

ge

.0

0

.0

5

.0

2

-.

01

.0

5

.0

4

.0

0

L

ev

el

o

f

ed

uc

at

io

n

.0

8

.1

2

.0

4

.0

6

.1

1

.1

0

.0

5

E

th

ic

al

l

ea

de

rs

hi

p

-.

33

*

*

-.

21

**

-.

29

**

-.

16

*

-.

15

*

-.

21

**

M

or

al

j

us

ti

fi

ca

ti

on

.2

3*

*

.1

7*

*

.1

6*

*

M

or

al

i

de

nt

it

y

-.

11

-.

19

*

-.

17

*

E

th

ic

al

l

ea

de

rs

hi

p

-.

15

*

.0

0

×

M

or

al

i

de

nt

it

y

R

2

.1

4

.0

5

.0

9

.1

6

.0

7

.0

9

.1

9

F

9.

68

**

4.

11

**

5.

90

**

9.

63

**

3.

77

**

3.

88

**

7.

93

**

N

o

te

.

N

=

2

71

.

S

ta

nd

ar

di

ze

d

co

ef

fi

ci

en

ts

a

re

r

ep

or

te

d.

*

p

<

.

05

,

**

p

<

.

01

.

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1281

Discussion

Theoretical Implications

In this study I explored the mediating role of moral justification on the

relationship between ethical leadership and unethical employee behavior, and

examined the moderating effect of employee moral identity on this moderated

mediation model. My theoretical contributions are reflected in the following four

aspects: First, although Bandura (1977) noted the importance of the cognitive

mediating process in social learning, few scholars have empirically explored and

examined cognitive mediating mechanisms in the ethical leadership–unethical

employee behavior relationship. Therefore, I have enriched and extended extant

research on social learning theory by exploring moral justification as a cognitive

mediating mechanism between ethical leadership and unethical employee

behavior. Second, I further investigated the mediating cognitive process in

social learning by exploring employee moral identity as a moderator of this

effect, thereby explaining why not all employees learn from ethical leaders.

Third, few researchers (see, e.g., Mayer et al., 2012) have investigated when

and how ethical leadership is related to unethical employee behavior. I presented

a moderated mediation model that has broadened understanding of the ethical

leadership–unethical employee behavior relationship. Fourth, a majority of

studies on unethical behavior (see, e.g., Gino & Margolis, 2011; Hershfield,

Cohen, & Thompson, 2012) have been conducted in a laboratory and primarily

used cheating as the proxy variable of unethical behavior; thus, my study has a

higher level of external validity and social relevance.

Practical Implications

The study also has some practical implications. My findings indicate that

leaders can limit unethical employee behavior by engaging in normatively

appropriate behaviors, and by demanding that employees be accountable for the

moral consequences of their behavior and care for others’ interests. I also suggest

that the behavioral standards for managers and supervisors should be emphasized

in organizations to ensure that their behaviors are ethical. Furthermore, managers

of organizations should establish the principle of advocating and incentivizing

employees to take responsibility for their behavior. The mediating role of moral

justification in the relationship between ethical leadership and unethical employee

behavior suggests that managers of organizations should pay more attention to

employees’ cognitive judgment and should ensure that their employees have

low levels of moral justification, which can inhibit unethical behavior. Finally,

the moderating role of employee moral identity that I found suggests that moral

identity is activated in organizations through cues in the social environment, such

as posters, slogans, or material symbols that make moral constructs and concerns

salient (Aquino et al., 2009; Mayer et al., 2012).

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1282

Limitations and Directions for Future Research

My research has several limitations. First, all of the variables were collected

from the same source (employees). Although I collected the data at two separate

times, the observed relationships should still be interpreted with caution because

of same-source concerns. For example, employees’ reporting of moral justification

may have biased their ratings of unethical behaviors. Future researchers should

strive to measure all predictors and unethical employee behaviors from different

sources or to utilize manipulations or objective outcomes. Second, although I

used several approaches to ensure confidentiality for the participants, they still

may not have reported their unethical behaviors truthfully because of social

desirability bias. In future studies, researchers should consider utilizing objective

indicators of unethical behavior. Third, I did not control for other styles of

leadership. Future researchers should control for other leadership styles that have

been found to be positively related to ethical leadership, such as transformational

leadership or charismatic leadership.

References

Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions.

Thousand Oaks, CA: Sage.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and

recommended two-step approach. Psychological Bulletin, 103, 411–423. https://doi.org/c76

Aquino, K., Freeman, D., Reed, A., II, Lim, V. K., & Felps, W. (2009). Testing a social-cognitive

model of moral behavior: The interactive influence of situations and moral identity. Journal of

Personality and Social Psychology, 97, 123–141. https://doi.org/dnxdnj

Aquino, K., & Reed, A., II (2002). The self-importance of moral identity. Journal of Personality and

Social Psychology, 83, 1423–1440. https://doi.org/cbq389

Aquino, K., Reed, A., II, Thau, S., & Freeman, D. (2007). A grotesque and dark beauty: How moral

identity and mechanisms of moral disengagement influence cognitive and emotional reactions to

war. Journal of Experimental Social Psychology, 43, 385–392. https://doi.org/c8kzsk

Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.

Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.

Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (1996). Mechanisms of moral

disengagement in the exercise of moral agency. Journal of Personality and Social Psychology,

71, 364–374. https://doi.org/cw4g7w

Barsky, A. (2011). Investigating the effects of moral disengagement and participation on unethical

work behavior. Journal of Business Ethics, 104, 59–75. https://doi.org/c63brn

Bergman, R. (2002). Why be moral? A conceptual model from developmental psychology. Human

Development, 45, 104–124. https://doi.org/dtb92r

Brislin, R. W. (1980). Translation and content analysis of oral and written material. In H. C. Triandis

& J. W. Berry (Eds.), Handbook of cross-cultural psychology (pp. 349–444). Boston, MA: Allyn

& Bacon.

Brown, M. E., & Treviño, L. K. (2006). Ethical leadership: A review and future directions. The

Leadership Quarterly, 17, 595–616. https://doi.org/b9cc76

Brown, M. E., Treviño, L. K., & Harrison, D. A. (2005). Ethical leadership: A social learning

perspective for construct development and testing. Organizational Behavior and Human Decision

Processes, 97, 117–134. https://doi.org/bkgm7b

ETHICAL LEADERS AND UNETHICAL EMPLOYEES 1283

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation

analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.

Detert, J. R., Treviño, L. K., & Sweitzer, V. L. (2008). Moral disengagement in ethical decision

making: A study of antecedents and outcomes. Journal of Applied Psychology, 93, 374–391.

https://doi.org/bfvmvv

Gino, F., & Margolis, J. D. (2011). Bringing ethics into focus: How regulatory focus and risk

preferences influence (un)ethical behavior. Organizational Behavior and Human Decision

Processes, 115, 145–156. https://doi.org/cbrpwb

Green, R. M. (1991). When is “everyone’s doing it” a moral justification? Business Ethics Quarterly,

1, 75–93. https://doi.org/dr6w59

Hershfield, H. E., Cohen, T. R., & Thompson, L. (2012). Short horizons and tempting situations: Lack

of continuity to our future selves leads to unethical decision making and behavior. Organizational

Behavior and Human Decision Processes, 117, 298–310. https://doi.org/fsn4g7

Hunt, S. D., & Vitell, S. (1986). A general theory of marketing ethics. Journal of Macromarketing,

6, 5–16. https://doi.org/bkw8x8

Liu, Y., Long, W. L., & Loi, R. (2012). Ethical leadership and workplace deviance: The role of moral

disengagement. In W. H. Mobley, Y. Wang, & M. Li (Eds.), Advances in global leadership (Vol.

7, pp. 37–56). Bingley, UK: Emerald Group.

Mayer, D. M., Aquino, K., Greenbaum, R. L., & Kuenzi, M. (2012). Who displays ethical leadership,

and why does it matter? An examination of antecedents and consequences of ethical leadership.

Academy of Management Journal, 55, 151–171. https://doi.org/qsz

Moore, C., Detert, J. R., Klebe Treviño, L., Baker, V. L., & Mayer, D. M. (2012). Why employees do

bad things: Moral disengagement and unethical organizational behavior. Personnel Psychology,

65, 1–48. https://doi.org/263

Mulder, L. B., & Aquino, K. (2013). The role of moral identity in the aftermath of dishonesty.

Organizational Behavior and Human Decision Processes, 121, 219–230. https://doi.org/cjs4

Muller, D., Judd, C. M., & Yzerbyt, V. Y. (2005). When moderation is mediated and mediation is

moderated. Journal of Personality and Social Psychology, 89, 852–863. https://doi.org/czt

O’Fallon, M. J., & Butterfield, K. D. (2011). Moral differentiation: Exploring boundaries of the

“monkey see, monkey do” perspective. Journal of Business Ethics, 102, 379–399. https://

doi.org/ffk6rv

O’Fallon, M. J., & Butterfield, K. D. (2012). The influence of unethical peer behavior on observers’

unethical behavior: A social cognitive perspective. Journal of Business Ethics, 109, 117–131.

https://doi.org/dpjwfk

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and

comparing indirect effects in multiple mediator models. Behavior Research Methods, 40,

879–891. https://doi.org/b9b2k3

Reynolds, S. J., & Ceranic, T. L. (2007). The effects of moral judgment and moral identity on moral

behavior: An empirical examination of the moral individual. Journal of Applied Psychology, 92,

1610–1624. https://doi.org/cgvqmp

Stevens, G. W., Deuling, J. K., & Armenakis, A. A. (2012). Successful psychopaths: Are they

unethical decision-makers and why? Journal of Business Ethics, 105, 139–149. https://doi.org/

bmj2t3

Vitell, S. J., Keith, M., & Mathur, M. (2011). Antecedents to the justification of norm violating

behavior among business practitioners. Journal of Business Ethics, 101, 163–173. https://

doi.org/b63vkc

Zey-Ferrell, M., Weaver, K. M., & Ferrell, O. C. (1979). Predicting unethical behavior among

marketing practitioners. Human Relations, 32, 557–569.

https://doi.org/ffk6rv

https://doi.org/bmj2t3

https://doi.org/b63vkc

ETHICAL LEADERS AND UNETHICAL EMPLOYEES1284

Copyright of Social Behavior & Personality: an international journal is the property of

Society for Personality Research and its content may not be copied or emailed to multiple

sites or posted to a listserv without the copyright holder’s express written permission.

However, users may print, download, or email articles for individual use.

Article

What Style o f Mediation Do You Need?

by Richard A. Kaplan

I f you need a m ediator, you probably know there a re reputable

organizations that p rom ote p anels of lawyers an d retired judges

who offer m ediation services an d there is literature available to

help you narrow your search. In Utah, for example, you will find

m ediator biographies (self-interest alert, including m ine) on

utiilradrsenices.com o r law firm websites, and of course you can

w ork with the American Arbitration Association o r other

organizations depending on the subject matter. As for literature,

there is a “checklist” available with thirteen separate criteria to

consider; Negotiating and Settling Tort Cases, § 6.11 (updated 2009),

and there are m any p apers with the title “Selecting a M ediator”

o r w ords to that effect. See, e.g., Weinstein & Chao, Choosing a

Mediator, 5 Bus. & Com . Litig. Fe d . Courts § 51.35 (4th ed).

As I see it, the criteria an d suggestions in the literature boil

down to a few key questions: What do you need most from the

m ediator? Who has that p articu lar capability? Will that person

be acceptable to the parties and opposing counsel?

Deciding what you need from a m ediator obviously requires both

an accurate assessm ent of the case and a good understanding of

the participants an d the p rocess. Determ ining which m ediators

provide what you need in those respects will not likely be possible

based on a typical m ediator biography alone. Gaining acceptance

of the m ediator you select may d epend on how you play it.

SCENARIO ONE:

Strong Case, Form idable Adversary, Zero Progress

Let’s say you rep resen t the plaintiff in w hat you see as a potential

$ 5 – 6 million com m ercial lawsuit. Discovery is com plete, you’ve

developed good facts. On balance, the law favors your client.

You co nsid er yourself a good lawyer and an able negotiator. Still

you’ve gotten now here in settlem ent discussions. Your adversary

has a lot m o re experience than you do, an d a rich resum e of

accom plishm ents at trial. The obstacle to serio us negotiations

an d settlem ent may have som ething to do with your adversary’s

self-confidence, ego, o r both. While you co nsid er yourself “up

an d com ing,” opposing counsel is a long-tim e pillar of the b ar

and knows it. Most importantly, he is highly regarded as a trial

lawyer p e r se, not for settling early, cheaply, o r even at all.

You dem anded $4.5 million an d accom panied that dem and with

a w ritten analysis of why you will win an d why your dam ages

greatly exceed your dem and. He d o e sn ’t seem to u nd erstand

your argum ents, or, if h e does, h e clearly dism isses them. He’s

at zero. The only p ro gress you’ve m ade is that he has agreed to

voluntary m ediation. How will you ap pro ach the pro blem of

choosing a mediator?

It s not ju s t the stre ng th s and w e a kn e sse s of the case

It’s the process and the p sycho d yna m ics you m ust

contend w ith .

I’ve h eard it said, not entirely in jest, that there a re essentially

three kinds of people: people who make things happen; people

who watch things happen; and people who wonder what

happened. At first blush, people who “m ak e” things h appen

may sound som ehow su p erio r to people who “w atch” tilings

happen. Not so fast. That’s not true at all, particularly as it

relates to m ediation. And, as for you an d me, if truth be told, we

m ust admit w e’ve found ourselves w ondering w hat happened

from time to time.

As it relates to m ediators, this simplistic classification of people

is useful in beginning to consider what mediation style you want.

All three describe not just people but skills, styles, and strategies.

1 w ant to explain the choices available to you without resorting

to m ediation jargon, an d then to use the language of mediation

to help you determ ine what type of m ediation style you w’ant for

Scenario One. The discussion of types of m ediation skills, styles,

an d strategies should have b ro ad enough applicability to help

you decide what you need in oth er scenarios as well. I’ll p resent

another, p erhap s m o re typical, scenario tow ard the end.

RICHARD A. KAPLAN is a shareholder at

Anderson & Karrenberg. His practice

focuses on complex civil litigation and

mediation, as well as independent

investigations and risk assessment at the

outset o f commercial ligation.

willing to negotiate. They essentially “w atch” things happen.

This is the style you’re looking for m ost of the time.

B a s i c m e d i a t i o n s t y l e s a n d t h e s t r a t e g i e s t h e y s u p p o r t .

The best m ediators, while open-m inded, are highly skilled at

seeing areas of agreem ent o r m utual interest and even the

b ro ad outlines of a deal before it happens o r w hile it develops.

The focus h ere is on how they handle that knowledge.

Some of the m ost accom plished m ediators view a deal as the

overriding goal of mediation, occasionally requiring abandoning

the p rin ciple of neutrality in favor of raw truth. Always having

the end gam e in m ind, they w ork h ard to get the parties to that

point by design. They may push one side o r the other, o r both,

gently an d forcefully at different times during the day. They try to

“m a k e” things happen.

O ther highly skilled m ediators take a m uch m o re hands-off,

d eliberate approach. They may nudge the p arties from time to

time. But they derive the m ost satisfaction from watching the

parties strike a bargain themselves. Such m ediators view

neutrality d uring the p ro cess not just as ap pro priate but as

fundam ental. They’ve w o rked h ard to develop that skill, to know

w hen to talk, w hen n ot to, and when to stop. They know that

ap p ro ach engenders trust and thus h as great value to parties

I doubt that many m ediators, if any, studiously practice

“w ondering w hat h ap p en e d .” But it’s w orth thinking ab ou t this

idea of “w o n d er” at a d eep e r level. Some m ediators a re simply

b etter than o th ers at feeling an d expressing em pathy and

appreciation. I ap preciate, and you probably do too, people

who are easily able to attribute ideas to others, ra th e r than to

themselves. That quality is especially im portant in m ediators.

W hether you w ant to take the notion of three kinds of people

furth er an d call the third category those who have the capacity

to com m unicate “w o n d er” (like a p aren t encouraging a child)

o r something else, the ability to attribute au thorship of ideas o r

pro po sals to o th ers tends to m ake them feel like they’re

contributing and taking “ow nership.”

Some highly skilled m ediators a re flexible and able to use

multiple strategies and skills successfully, along with dozens of

tactics for working through im passe. They can plan an d intuit

when and how to employ this, that, o r the o th er skill o r tactic in

the co urse of a single day.

CONYERS 0 NIX

CRIMINAL DEFENSE ATTORNEYS

We welcome yo u r referrals a n d

the o p p o rtu n ity to work w ith you.

ALL FELONIES & MISDEMEANORS | DISTRICT COURTS & JUSTICE COURTS

Kate Conyers Jesse Nix

KATE@CONYERSNIX.COM | WWW. CONYERS NI X. COM | JESSE@CONYERSNIX.COM

405 SOUTH MAIN STREET, SUITE 930, SALT LAKE CITY, UTAH 84111 | 801-839-4649

13

mailto:KATE@CONYERSNIX.COM

http://WWW.CONYERSNIX.COM

mailto:JESSE@CONYERSNIX.COM

0 – 3

cl

M e d ia tio n s ty le s , s tra te g ie s , and v o c a b u la r y as

a p p lie d to S c e n a r io One.

Given the state of play in Scenario One, as plaintiff’s counsel you’re

not ready for the subset of m ediation known as ‘ facilitation

A facilitator’s greatest skill in m ediation is assisting the parties

themselves in negotiating resolution. That m eans w orking with

them patiently and supportively, usually observing ra th e r than

talking as each side w orks through its own concerns. Excessive

criticism, optimism, o r pessimism are studiously avoided, because

the substance o r tone of such communications could be interpreted

as favoring one side o r the other. To be sure, m ost facilitators

will ask h ard questions o r m ake serious suggestions, even for a

“c o rrectio n ” if a p ro p o sed move seem s way off base. But by and

large a facilitator d o esn ’t try to influence the p ro cess o r direct

it, because that com prom ises the ap pearance of neutrality. All

m ediators, regardless of style, know that the m ost enduring

settlem ents are the ones the parties negotiate themselves. All

m ediators and m ost lawyers w h o ’ve used mediation know that

facilitation is m ost likely to be effective when the gap to be

bridged is relatively small.

Absent deux machina, a facilitator w on’t help you in Scenario

One b ecau se your adversary d o esn ’t w ant to be facilitated, and,

in truth, you d o n ’t either. The gap couldn’t be larger. The parties

are simply n ot poised to strike a deal essentially on their own.

S u p p re s s th e “g o o d gu y” te m p ta tio n .

Be wary that the m ediator (w hether o r not a facilitator) might

appeal to you to be the “good guy.” That is especially likely in

Scenario One, since y our adversary w on’t touch that role and

som ething’s got to give. Reason doesn’t always trum p em otion,

m uch less always prevail.

Consider what opposing counsel might have to say on the morning

of the mediation regardless of who the m ediator is: “It’s my

position and I ’m sticking to it.” He knows from experience that

despite the m ed iato r’s entreaties (“I n eed you to give me

som ething to w ork with, anything p lease”), the m ediator may be

wrong. T here’s potential success in standing pat, insisting till the

while to the m ediator that you, his opponent, a re the one being

unrealistic. Consider also that you may be the one the m ediator

tests, not your adversary.

The temptation can be quite strong to show the mediator that you’re

in fact reason able by indicating you an d your client are flexible

an d will in fact move in the face of a credible offer. Be aw are of

that trap, know it when you see it, an d d on ‘t let it catch you.

D on’t overplay any natural desire you may have to cooperate

with the mediator, to have h e r like you. It may take very little to

dem onstrate that you’re the reason able one in the room . If you

show too much flexibility, particularly early on, you put your client

at risk of being push ed tow ard a “split-the-baby” com prom ise

you d on ‘t really want.

Be a w a r e th a t w e a k n e s s is sign a le d in u n ex p e cte d w a y s .

Your adversary may not “know ” that you conveyed such

flexibility’ to the mediator. The m ediator c a n ’t tell him w ithout

your consent and you obviously d o n ’t give it.

Still, you m ust figure that your adversary has an acute sense of

w eakness. He will watch the m ediator and listen carefully to h er

when she com es b ack from a caucus with the oth er side. Her

body language, tone of voice, and delivery may suggest your

weakness regardless of her best efforts not to. Opposing counsel

will look beneath her efforts to generate movement, to see what’s

really there. We’ve all h eard (an d all m ediators sometimes

p ro po se o r use) h y p o th e tica l: “S u p p o se .. Or, “I ’m certainly

not saying I can, but what if I can get him off of that num ber?”

When opposing counsel h ears that early on, he h ears it the way

he wants to: th e re ’s a lot of ro o m here. He’ll be tem pted to let

that dynamic play out in his favor.

Rem em ber that despite the “w in/w in” paradigm that dom inates

cu rrent m ediation literature an d training, your adversary wants

to “win” the old-fashioned way, and at every step along the way.

Try to u nd erstand w hat he thinks w ould constitute a “win” in

your case an d have that low-ball n u m b er in m ind as som ething

you m ust avoid as you calibrate your moves. For him, “winning”

in Scenario One probably d o esn ’t m ean paying zero o r anything

close to that. If the fair value of settlem ent is something north of

$4 million, he m ore likely thinks a “w in” would involve paying

som ething south of $2 million, better yet $1.5 million.

So, assume you have given the m ediator the impression that you’re

reason able and w hat you really want is a just a good settlement,

som ew here in the middle. In that case the m ediator will alm ost

certainly apply p re ssu re to move the parties, particularly your

side, in that direction, consistent with the principle of neutrality.

After all, you had said orally and in writing before the m ediation

that you want a lot, but your dem eanor and choice of w ords now

reveal that you’ll take not so much. With a strong sense of that

in hand, the m ediator w on’t violate h e r duty of confidentiality o r

any o th er p rinciple o r duty by helping you get there. In fact, that

would be in keeping with the m ediator’s job.

14

So, what type of mediator do you need?

I think Scenario One requires you to find a mediator who

(1) has first-rate analytical skills and is highly likely to perceive

the value of your case as you see it (that means you’re right and

she agrees); and (2) who will credibly and convincingly,

perhaps brutally, convey that value to opposing counsel. In

mediation jargon, that would entail “evaluative” mediation

skills and perhaps “transformative” mediation skills. What

“evaluative” normally means is a mediator who comes to his oi

lier own conclusion about the value of the case and discusses it

openly with both sides. All mediators surely offer the ability to

estimate a range of settlement values for a case. Paradoxically

though, few have the credibility to forecast the likely outcome at

trial, particularly when one side is represented by a highly

experienced trial lawyer and one is not. In this scenario, you

need that credibility. While you’re in a separate room, you want

the mediator to look opposing counsel in the eye and to say

something like this: “This case has a lot of jury appeal –

settlement value now in the mid-seven figures. I know you are a

terrific trial lawyer, but you are likely to lose this one and in all

events you have significant trial risk in this case. Here are all the

reasons why that is so….”

(Note: When you ask for “evaluation” you obviously run the risk

that the mediator doesn’t see things the way you do. Be prepared

to hear that your valuation is grossly inflated and wildly off the

mark. Understand that your selection of an evaluative mediator

assumes that you are highly confident in your estimation of the

case and able to defend it persuasively. If you can’t, you need to

be flexible enough to consider resetting your sights. If your

views remain the same despite the mediator’s contradictory

assessment, you need to be willing to walk.)

What “transformative” means is the ability to get the parties and

their lawyers to understand each other’s perspectives and to gain

respect for those opposing perspectives increasingly as the day goes

on. Mutual understanding and respect drive compromises and deals.

Can you narrow this down to the right person(s)

for the job?

Who is the most likely to have these abilities? How about someone

who already enjoys opposing counsel’s respect as a smart and

accomplished trial lawyer and mediator in his or her own right?

Tact, empathy, listening skills, and other such qualities are great,

but they’re not the heart of what you need in this case. You haven’t

o – o

Congratulations to Our Colleague

RUTH A. SHAPIRO

Ruth has been named the Utah Defense Lawyers Association

Legacy Award winner of 20

18

-2019.

Whether she is fiercely defending an insurance client in court or

valiantly competing in a road cycling event, Ruth gives it her all.

We are honored to have Ruth part of our team and proudly

recognize another impressive achievement.

Ruth A. Shapiro

ras@scmlaw.com

801.322.9

16

9

Sn o w

C h r i s t e n s e n

& M a r t in eau

www.scmlaw.com j 801.521.9000

15

mailto:ras@scmlaw.com

http://www.scmlaw.com

CXD

gotten your adversary to take you and your case seriously yet.

You need help making that happen. You want a mediator who

will know, and tell your adversary, that she understands his

dismissiveness is an act, a tactic that he needs to leave behind

now, and that it would serve his client’s interests to get the case

settled as early as possible.

(Note: The mediator knows opposing counsel knows what both

sides’ strengths and weaknesses are. Good lawyers, like opposing

counsel here, listen while pretending not to; they are always

focused on their client’s needs, not them own. So, the task of cutting

through the smokescreen is not nearly as difficult in mediation as

your experience with the guy suggests, provided you find a

mediator who can do it.)

It’s not ju s t up to you. H ow do you get opposing

counsel to a c c e p t the m ediator?

If you believe you have identified just the right person, I know you

realize you can’t dictate who the mediator will be. Your adversary

doesn’t have to agree to engage the mediator you propose and

will usually have other ideas of his own. As we’ve posited, your

adversary wants to “win,” and generally that means at every step

along the way. Winning, for him, means selecting the mediator

he wants, finding reasons to reject the people you propose, and

working you over until you agree with his choice.

So, let me suggest that you surprise opposing counsel by making

die choice he would have made, and he will be hard pressed to

reject. Be bold. Find and propose the mediator/trial lawyer that

he respects and uses more frequently than any other in town,

whether he’s worked with or against her 100 times, and whether

you’ve interacted with her yourself. If he asks, tell him you’re aware

of the prior relationship he has with the mediator and you’re not

concerned about it. You have an abiding conviction she’ll be fair.

You should mean it when you tell him you’re not worried about

bias. You should realize that, if anything, she’s likely to bend

over backwards to be fair to you and your client. Indeed, that

concern may well occur to your adversary as he thinks through

what you’re doing. He may say no. If instead he hesitates, tell

him you’ll let her know you understand the full extent of their

professional relationship and don’t want her to be concerned

about it. You can’t force your adversary to agree, but it’s hard

for him to reject someone he’s used often himself. Regardless,

you’ve now set the bar high and, if you don’t settle on her, you’ll

agree on someone like her.

There are a host of reasons for selecting a mediator “as good

as” opposing counsel, and here’s the main one: In this case, the

mediator is not just evaluating your case on paper. She’s evaluating

you, and your ability to persuade judge and jury. Who better to

tell opposing counsel that you are “for retd” than someone he

himself considers “for real” and thus already respects and

listens to? Who better to persuade him that it would be wise for

him to come down from Mount Olympus and negotiate with us

mortals? That’s what he and his client need to hear from the

mediator to get real themselves.

To be sure, you’ll have to perform at your highest level to pull

this off, but that should go without saying anyway.

THE MODERN CONCEPT OF MEDIATION ADVOCACY

In a leading article published almost twenty years ago in the Ohio

State Journal on Dispute Resolution, James K.L. Lawrence

advocates the concept of “Partnering with the Mediator.” 15

Onto St . J. on D is p. Resol. 425 (2000). The heart of Lawrence’s

argument is that mediations generally have the greatest chance

of success when “the mediation advocate and her client [are]

engaged with the mediator in the problem-solvingprocess: merely

accepting or rejecting proposals from the mediator or the other

side is insufficient ‘engagement.'” James K.L. Lawrence,

Mediation Advocacy: Partnering With the Mediator, 15:2 O hio

St . J. on D is p. Resol. 425, at 426 (2000) (emphasis added).

While 1 strongly agree with Lawrence’s premise, I think the

“partnership” characterization probably takes the idea of

collaboration a bit too far. You represent your client. The

mediator is not your “partner” in achieving your client’s goals,

and it’s dangerous to think about her that way. While sometimes

it might make sense to be open and transparent at the very

outset, most of the time you need to be more strategic in what

you reveal, how much, and when. You want momentum to build,

not wane, over the course of the day. Disclosing everything you

have to say first thing in the morning may leave you with nothing

left to say by noon. For example, you may think it will help the

mediator to know your “walk away” number early in the process.

You may think that will influence her to encourage your adversary

to start well above your “walk away” number and to stay there

throughout the process. On the contrary, that disclosure may be

the first number the mediator actually considers and may therefore

tend to anchor the mediator’s thinking on the low side of the

bargaining range. That early disclosure thus could lead her to

work with the other side and you (despite the best of intentions

and a conscious effort to maintain neutrality) to make moves

toward your walk away number the rest of the day. This “anchor

effect” is well documented in negotiation literature, and it could

well work to your disadvantage during a mediation as well.

16

On the other hand, assuming opposing counsel is reasonably

competent (as you should), he already knows the weaknesses

in your case. In most cases, opposing counsel already knows

them. You will help the mediator and your client’s bargaining

position, at virtually no risk, by being forthright with the mediator

about those weaknesses and by authorizing the mediator to tell

the other side you know exactly what they are and why they

don’t concern you.

Assume you’re about to give the mediator a demand or offer or

counteroffer for her to present to the other side. It is best if the

mediator can communicate at the same time that the dollar amount

(or whatever you’re proposing) takes those weaknesses into

account. She should add that you have carefully considered them

as well as the strengths of your case, and that this proposal gives

those weaknesses what you consider fair weight and value. Give

her your reasons too and let her pass those on at the same time.

Among the advantages of this approach, Lawrence points out, is

that by allowing the mediator to put your weaknesses on the

table, and to explain how you’ve taken them into account in

deciding what to propose, you’ve actually strengthened your

relative bargaining position. The weaknesses you expose now

can’t bite you later. Sure, the mediator can come back to you

with a proposal that says you didn’t give those weaknesses

enough value. But she cannot surprise you with the response

that your proposal didn’t consider your weaknesses at all, and

here they are, outlined with force by the other side. Oops.

SCENARIO TWO:

A Simple Example of Interest-based Negotiations

Let’s say you represent the plaintiff in a lawsuit in which he buys

trucking services from the defendant to pick up merchandise from

his customers’ warehouses and to deliver it to its destination. Assume

both sides have leverage. Yours is the warranties and reps in the

contract. You could likely get summary judgment. Your opponent has

a different kind of leverage. For one tiling, the judgment would be

hard to collect because of defendant’s precarious financial situation.

For another, your client needs defendant’s services because no one

else in the area has ever provided them on a consistent or reliable

basis. In these circumstances, your client and you may want a mediator

ultimately to “facilitate” a business deal between the parties.

Suppose it’s your first communication (a Mediation Brief) or first

caucus with the mediator. You’re not sure about the defendant’s

cash situation so you probably want to take a shot at getting complete

monetary relief. You need to make your point simple and compelling:

the parties entered into a contract for trucking services and the

defendant expressly warranted that he would provide them. He

didn’t. You had to refund $

17

5,000 to your customers, and you

can substantiate another $50,000 in compensable damages. The

defendant took risks in the ordinary course of business, and the

defendant needs to come to grips hilly with your damages.

Suppose though that the mediator comes to you with a small,

unacceptable cash offer and nothing more, and makes clear she

believes the money just isn’t there. You now have a choice where

to direct the negotiations. You can of course respond in kind or

not at all. You have a strong summary judgment argument based

on the warranties. You can walk.

On the other hand, you can introduce the concept of a negotiated

resolution containing both monetary and non-monetary components.

You could tell the mediator truthfully that you understand the dispute

at a deeper level and want to share your insights with her. In

mediation jargon, you propose to help her understand the underlying

“interests” of the parties – their needs, their desires, fears, and

uncertainties. You can help her work toward a “win/win” outcome.

You might explain that the parties have had a business and personal

relationship for a decade that has been fractured by this dispute. Both

sides are terribly frustrated. Your client knows that over the three-dav

Commercial Mediation

Blake D. M ille r

• Seasoned tria l law yer w ith

over 35 years of experience

in high-stakes com m ercial

litig a tio n before federal,

state, and trib a l co u rts

• Form er fo u n d e r and CEO of

several businesses

Direct: 8 0 1 .590.2353

b m ille r@ a kla w firm .co m

Richard A. Kaplan

• Form er A ssista n t U.S. A tto rn e y

• Harvard M.B.A. w ith experience

at a Fortune 500 com pany and

ow ning sm all businesses

• C om plex co m m e rcial litig a tio n

and re g u lato ry m atters practice

Direct: 8 0 1 .639.0954

rkaD lan@ aklaw firm .com

f. ‘ ‘ ANDERSON &

X^y KARRENBERG

50 West Broadway, Suite 700 | Salt Lake City, Utah 84101

801.534.1700 | www.aklawfirm.com

17

mailto:bmiller@aklawfirm.com

mailto:rkaDlan@aklawfirm.com

http://www.aklawfirm.com

0 – 3

period when defendant breached, there was bad luck involved

because most of his drivers called in with the flu. But still, you

had warranties. Your adversary knows that you tried yourself to

secure alternative transportation but couldn’t get it. Your adversary

says that’s his point too; he wasn’t able to get any other drivers

either after trying to do that for hours and hours. Your client might

stand firm on the warranties, or he might consider trying to salvage

the relationship, provided the other part}’ makes your client’s

customers substantially whole. You understand that the defendant

doesn’t have that much cash on hand, but your client might

consider payment terms. That gives the mediator a smorgasbord

of material to work with in developing potential compromises.

There’s no substitute for understanding your client’s underlying

interests – business, personal, economic, and emotional –

before the mediation. Think about how and when you might

best weave his mixed feelings about a renewed business

relationship into the conversation. You also want to research

and think about your adversary’s likely needs and giveaways, as

well as his BATNA. In the end, a reasonable resolution may be

the one that satisfies as many of the parties’ mutual interests as

possible. The absence of any likelihood of re-establishing the

business relationship takes a host of solutions off the table. The

prospect of a renewed relationship is one a facilitator who

understands business deals can work with skillfully.

Getting your client ready.

Mediation requires not just your engagement but your client’s as

well. You need to help your client get emotionally invested in the

process and ready to work at it throughout the day. You and he share

a common purpose. Among other reasons for your client to get

engaged, he’s the one who is going to pay for it, including not just

your fees but half the mediator’s. It is better if he appreciates that

the odds of success improve with his full attention and engagement.

As to subjects to be discussed, most of them are probably obvious

and there’s too much literature on preparation to cite here. Just

a couple of points bear mention. You want to get your client ready

for the mediation process (including what you know about the

mediator and why you chose or agreed to hire her). Make sure

you have agreement beforehand on what your goals are, what

your strategy is for achieving them, and what happens if you

don’t. Focus on what your client needs, what he doesn’t need,

what is essential, and what he can give up to get something else.

What is your client’s best alternative to a negotiated resolution

(aka BATNA)? Quantify that as best you can. What is his w’alk

away number? Explore with your client before the mediation

what his real “interests” are: Is his objective to get as much

money as he can? Or would he prefer a combination of money

and, for example, assurances of priority service. Recognize that

your client’s “interests” include probing the needs, desires, and

fears that motivate him. James K.L. Lawrence, Mediation

Advocacy: Partnering With the Mediator, 15:2 Ohio St. J. on

Disp. Resol. 425, at 426 (2000). Thus, in Scenario Two, for

example, he’s understandably fearful of keeping his relationship

with the defendant, and fearful of losing it too, particularly if he

can’t find an alternative. He’s worried about keeping his own

customers. He had thought before this mess that defendant

valued him a valued customer. Now he thinks that wasn’t true at

all. He doesn’t want a new business deal with defendant unless

the terms somehow reflect his valued status.

Finally, prepare your client for the experience – the slog – of

mediation, which tends not to be fun for clients, especially if

they don’t feel heard and understood. This preparation requires

orientation, including visualization. So talk with your client

about what mediation is and the role of the mediator, what the

floor plan will be, how you and your client will have privacy,

whether or not the mediator is one who first brings the parties

and lawyers together in the same room, what will happen if she

does, how she might spend just a minute with you and your

client to introduce herself and then spend an hour with the

other side (or vice versa), how frustrating that can be, what to

make and not make of that, where the negotiations stand before

mediation, and how to understand and withstand the inevitable

moments when he’ll want to leave or just throw the towel in.

Getting yourself ready.

Indeed, you need to ready yourself for the same experiences

throughout the day. How are you going to handle disappointments

when the mediator comes back again and again with less than

you expected? Will you allow a sigh of resignation? Or will you

smile knowingly? Don’t discount the impact of your own body

language. You need to maintain your energy and resolve. You

need to recognize and avoid the “good guy” trap, especially at

the end of the day when it becomes most alluring.

That means remaining vigilant and positive, and helping yourself

to do that by keeping your purpose squarely in mind. Keep in

mind that mediations can and do flip on a dime, sometimes in

the last ten minutes. To be sure, that may not happen on any

given day. Your best alternative at some point may be to exercise

your greatest leverage, to walk. Meanwhile, your job throughout

the day is to look for and work on developing ways to create

opportunity for your client, or to recognize it when it comes

knocking, unexpectedly or otherwise.

18

Copyright of Utah Bar Journal is the property of Utah State Bar and its content may not be

copied or emailed to multiple sites or posted to a listserv without the copyright holder’s

express written permission. However, users may print, download, or email articles for

individual use.

RESEARCH ARTICLE Open Access

The role of mediation in solving medical

disputes in China

Mengxiao Wang1,2, Gordon G. Liu1*, Hanqing Zhao3, Thomas Butt1, Maorui Yang4 and Yujie Cui5

: Medical litigation represents a growing cost to healthcare systems. Mediation, arbitration, and other

alternative dispute resolution (ADR) methods are increasingly used to help solve the disputes and improve

healthcare satisfaction. In China, the increasing number of medical disputes has contributed to concern for the

safety of physicians and mistrust between physician and patients resulting in ADR processes being established in

several provinces in recent years. Our aim was to describe and explain the impact of this new mediation process in

the Chinese healthcare system.

: Our study investigated mediation practices in China using case-level data from 5614 mediation records

in Guangdong Province between 2013 and 2015. We investigated how the resolution success as well as the

compensations are associated with the case characteristics using regression analysis.

: Among the cases analyzed, 1995 (41%) were solved with agreement through mediation, 1030 were closed

by reconciliation, 559 were closed by referring to court and 1017 cases were withdrawn after mediation. Five

hundred five Yinao cases were solved with the help of mediators on the spot. We find that mediation solved about

90% of medical disputes under present mechanisms, while more police support is needed to cope with Yinao. The

average compensation of mediation is CNY60,200 and average length of mediation is 87 days. Longer time taken to

reach resolution and more money claimed by patients are associated with lower resolution success rate (p < 0.01)
and higher compensation levels (p < 0.01).

: Our results show the performance of mediation mechanisms in China to help solve medical disputes.

ADR plays a role in reducing the need for initiating litigation and may ultimately increase satisfaction with the

healthcare system.

Keywords: Medical dispute, Compensation, Duration, Medical mediation, China

Background

Accompanied by the rapid growth of household income,

demand for better healthcare services has been increas-

ing in China, leading to rising expenditure on health-

care. At the same time, a number of organizational

reforms on the supply-side have been implemented to

improve the quality and efficiency of Chinese healthcare

delivery [1, 2]. Despite these improvements, the cost and

volume of medical disputes has grown in recent years.

Mistrust between patients and physicians has grown [3]

and the physician-patient relationship has worsened.

Extreme cases of medical disputes can cause great

physical, mental and reputational damages to physicians

and hospitals, and bring losses and adverse social

influences due to negative media coverage. Workplace

violence has been shown to contribute to higher levels

of occupational stress and lower levels of job satisfaction

[4, 5] and even brought psychological problems to physi-

cians [6].

© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,

which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give

appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if

changes were made. The images or other third party material in this article are included in the article’s Creative Commons

licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons

licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain

permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the

data made available in this article, unless otherwise stated in a credit line to the data.

* Correspondence: gordonliu@nsd.pku.edu.cn

1National School of Development, Peking University, Beijing 100871, China

Full list of author information is available at the end of the article

Wang et al. BMC Health Services Research (2020) 20:225

https://doi.org/10.1186/s12913-020-5044-7

http://crossmark.crossref.org/dialog/?doi=10.1186/s12913-020-5044-7&domain=pdf

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/publicdomain/zero/1.0/

mailto:gordonliu@nsd.pku.edu.cn

Hospital disputes occur in nearly all departments and

most often in tertiary hospitals [7]. One particular case

of hospital violence, commonly known as Yinao in Chin-

ese, describes organized unemployed gangs who are paid

by patient families to create medical disturbances to get

compensation for actual or perceived malpractice from

hospitals [8]. Through this illegal approach, patients in

dispute with hospitals usually expect to get higher com-

pensation compared to other approaches [9]. With the

hospital being surrounded by Yinao gangs, hospital oper-

ations such as diagnosis and treatment are disrupted and

hospital staff, equipment and facilities are at risk.

In one recent example, on July 12 of 2018, a female

doctor was stabbed by three gangsters in Tianjin and

died later in hospital. None of the three gangsters were

patients of the doctor [10].

To tackle this problem, China has made a number of

changes to laws and regulations. Several Provisions of

the Supreme People’s Court on Evidence in Civil Proce-

dures meant that medical damage lawsuits started to be

judged based on reversed onus from 2002. However, this

created more tension between physicians and patients

and made physicians more defensive and spend more

time on medical record keeping [11] with implications

for the quality of medical services while contributing to

increase in health care costs [12]. In 2010, the Tort Li-

ability Law was implemented and the reversed onus was

to some degree reduced for healthcare providers [13].

To deal with more serious disputes, extreme forms of

Yinao like “setting up a mourning hall”, “burning paper

money” (a Chinese traditional way to memorize the dead

at a funeral) to disrupt hospital operations and so forth

can constitute crimes [14]. This was formalized by

Amendment IX to the Criminal Law of China in 2015

whereby crowds who are assembled to disrupt hospital

services will be sentenced with serious consequences.

Most recently, the government has begun to advocate

resolution of disputes by alternative mechanisms. On July

31, 2018, Prime Minister Li Keqiang signed “Medical Dis-

putes Prevention and Treatment Regulations”, which fur-

ther emphasized the prevention and resolution of medical

disputes and protection of both physicians and patients.

Medical disputes can be resolved through litigation

[15, 16] or, where alternative dispute resolution (ADR)

methods exist, through mediation or arbitration [17, 18].

Facing the scarcity of both medical and legal resources,

China has advocated mediation as an ADR approach to

deal with medical disputes. Gradually, mediation has

been adopted in many provinces and cities, such as

Guangdong Province, Tianjin City and Hainan Province.

Studies on approaches to deal with medical disputes are

more focused on litigation [19, 20], a few have analyzed

a small sample of mediation cases without highlighting

the role of mediation in coping with Yinao [21, 22].

The contribution of this study is threefold. First, this

study provides an analysis of mediation practices in

Guangdong, the largest province in China, with 5614

case-level data and a detailed description of the mediation

mechanism. Second, we analyze the medical disputes han-

dled through mediation-based cases from 2013 to 2015 in

China and investigate the factors that influence successful

mediation and levels of compensation. Third, our analysis

deals with not only the common medical disputes but also

Yinao cases. With the information of 505 Yinao cases, this

study investigates the unique features of Yinao.

Our study aims to quantitatively describe and explain

the outcomes of mediation in China to provide policy

recommendations for hospital risk management and

safety enhancement of both patients and physicians.

Methods

Mediation in China

According to the People’s Mediation Law, the People’s

Mediation Committee is under the supervision of the Ju-

dicial Administration department, and mediates disputes

for free. The agreement after mediation has legal effect

but no power of law enforcement. In order to be legally

binding, patients or hospitals must seek judicial confirm-

ation. Without this confirmation, if either side defaults

on the agreement, mediators will advise them to choose

other ways to deal with the dispute.

Guangdong province in China had a GDP of 8.09 tril-

lion CNY and a population of 110 million people in

2016, making it the largest province measured by wealth

and population [23]. Most citizens live in urban areas

(69.2%, 2016). In 2017, the migrant population in

Guangdong reached 41 million, which adds difficulty to

public governance and conflict resolution.

Guangdong People’s Mediation Committee (PMC) was

established in 2010, and its service region has expanded to

16 out of 21 cities by 2015. Guangdong’s PMC operates

according to the principles of “Fairness, Impartiality, Neu-

trality, Timeliness, and Convenience”. It built a mechan-

ism combining mediation, compensation and dispute

prevention, which actively responses to Yinao and aims to

settle medical disputes outside hospitals to maintain nor-

mal diagnosis and treatment order. To mediate disputes

between health care facilities and patients, the mediators

usually have educational backgrounds such as medicine,

law, and psychology and they receive annual training to

improve their mediation skills. The mediation process is

also supported by both medical and legal experts and

most of the medical experts have a job title of deputy dir-

ector or above. Guangdong PMC has 46 branches in

Guangdong Province and its expert group (including doc-

tors, nurses and lawyers) has more than 2000 members.

Figure 1 depicts the mediation procedure in Guangdong.

Usually the patient or medical institution applies for

Wang et al. BMC Health Services Research (2020) 20:225 Page 2 of 10

Fig. 1 The mediation process of People’s Mediation Committee in Guangdong

Wang et al. BMC Health Services Research (2020) 20:225 Page 3 of 10

mediation first. Once accepted, the mediator will start by

collecting evidence (e.g. medical records). The main pro-

cedure of mediation contains four steps: 1), both sides

make statements; 2), medical and legal experts analyze the

case; 3), experts are invited for meetings regarding to the

specific case (e.g. clinic department, treatment or nursing);

and 4), the mediator negotiates with both patient and hos-

pital sides based on the opinions of the experts.

Generally, medical disputes will be handled at the

PMC and follow the procedures described above. While

for Yinao cases, once reported, the mediators will go to

hospitals to handle them on site.

Data sample

We used data from Guangdong PMC, which contains

information on case-level medical disputes, including

cause, duration, and hospital characteristics. Our sample

included 5614 cases settled during 2013–2015 in Guang-

dong Province. Ninety-seven percent of mediation cases

were reported by hospitals and patients (Fig. 2). The

cases referred by health departments or courts account

for only 1 % each. Overall, 4902 out of the 5614 cases

(87%) were registered and 207 cases (4%) were rejected

after preliminary examination. 505 Yinao cases (9%)

were handled on the spot.

The outcomes of the 4902 registered mediation cases

closed during 2013–2015 consisted of four types: 1995

cases reached mediation agreement (41%); 1030 cases re-

sulted in reconciliations between hospitals and patients

(21%); 559 cases proceeded to litigation (11%); 1017

withdrawal of cases (21%). The outcome of 301 (6%)

cases were missing.

Regression method

We used logistic regression to analyze the mediation

success and generalized linear regression to analyze

compensation of mediation. Given the positively

skewed distribution of compensation, we use a general-

ized linear regression model with gamma distribution

[24]. We have investigated the relationship of medi-

ation success and compensation amount with duration

of the case, first claim by patients, age of patients, con-

trolling causes of medical disputes, hospital classifica-

tion (primary, secondary, and tertiary), consequences

and year effects.

Fig. 2 Overview of the approaches and outcomes of mediation. Notes: We analyzed 5614 mediation cases in total, including 5109 cases settled

at the People’s Mediation committee and 505 Yinao cases settled on the spot

Wang et al. BMC Health Services Research (2020) 20:225 Page 4 of 10

Results

Characteristics of mediation cases

The number of medical disputes increased from 1234 in

2013 to 2293 in 2014 and remained above 2000 in 2015

(Table 1). The mean age of patients in our sample was

37.6 and the mean claim amount was CNY357,900. The

top three causes of disputes were: different opinions on

responsibility (for example, patients would ask that hos-

pitals take major or all responsibility for the unsatisfac-

tory medical outcome) (54%), the skill of physicians or

nurses (27%) and discontent with nursing and medicine

(7%). While medical accidents (the unexpected and un-

intentional medical harm, usually out of limited medical

conditions and skills) (5%) and hospital management

issue (4%), and lack of informed consent (3%) accounted

for small proportions of the cases.

More than half of the cases closed with compensation

to patients. The average compensation was CNY60,200,

which is much less than the amount claimed by patients.

Most cases (86%) closed with compensation of less than

CNY100,000. Only 1 % of cases settled with compensa-

tion above CNY500,000.

The average length of time between registration and

closure was 87 days. 1798 (36%) of the medical disputes

were settled within 1 month, 65% of them were solved

within 3 months and 86% were closed within half a year.

140 (3%) of the cases lasted for more than 1 year.

Our sample consists of 2044 tertiary hospitals

(36%), 2627 secondary hospitals (47%) and 943 (17%)

primary hospitals. In China, the hospital accreditation

system posed lower bounds on the number of beds

for each of the hospital grades, of 20, 100, and 500

beds for primary, secondary and tertiary hospitals re-

spectively [25].

General hospitals were more frequently involved in

medical disputes (3792 cases, 68%), followed by township

hospitals (546 cases, 10%). Traditional Chinese medicine

hospitals and maternal and child health hospitals

accounted for 9% of all cases each. Departments like gen-

eral surgery, obstetrics and gynecology, internal medicine

Table 1 Characteristics of 5614 Mediation records, China, 2013–2015

Category Category

Closure date-no.(%) 5614 ≤10 297(10)

2013 1234(22) > 10,≤100 1110(39)

2014 2293(41) > 100,≤500 847(30)

2015 2087(37) > 500 584(21)

Main cause-no.(%) 4927 Hospital grade-no.(%) 5614

Responsibility 2656(54) Primary 943(17)

Skills 1336(27) Secondary 2627(47)

Nursing and Medicine 333(7) Tertiary 2044(36)

Medical Accidents 249(5) Medical Specialties-no.(%) 5072

Management problem 190(4) General surgery 1553(31)

Informed consent 163(3) Obstetrics and gynecology 1290(25)

Time from dispute to closure (days)-no.(%) 4953 Internal medicine 824(16)

Mean 87 Pediatrics 355(7)

≤1 month 1798(36) Ophthalmology and otorhinolaryngology 146(3)

> 1 month,≤3 month 1425(29) Oncology 73(1)

> 3 month,≤6 month 1054(21) Others 831(16)

> 6 month,≤1 year 536(11) Type of medical facility-no.(%) 5614

> 1 year 140(3) General hospital 3792(68)

Closed with compensation-no.(%) 2941 Township hospital 546(10)

Mean (CNY1,000) 60.2 Traditional Chinese Medicine hospital 522(9)

≤10 1203(41) Maternal and Child Health hospital 493(9)

> 10,≤100 1328(45) Specialty hospital 261(5)

> 100,≤500 369(13) Severity-no.(%) 5590

> 500 41(1) Death 2305(41)

First claim by patients-no.(%) 2840 Disability 429(8)

Mean (CNY1,000) 357.9 Others 2856(51)

SOURCE Authors’ analysis of mediation data for 2013–2015

Wang et al. BMC Health Services Research (2020) 20:225 Page 5 of 10

and pediatrics are more often involved with medical dis-

putes, accounting for almost 80% of the cases in total.

About half of the cases were related to injury to pa-

tients. We find that death is the cause of a large propor-

tion (41%) of these cases compared to disability (8%).

Table 2 illustrates the features of Yinao cases that are

reported to People’s Mediation Committee and handled

by mediators on the spot. During 2013–2015, there were

505 cases in total, which is much more than the number

of cases reported by the media. The sudden increase of

Yinao cases from 14 in 2013 to more than 200 in later

years is mainly because People’s Mediation Committee

started to record Yinao cases separately and more clearly

in 2014. Overall, tertiary and secondary hospitals are

more common locations for Yinao. On average, 26

people take part in Yinao and some of them are reported

as “professional Yinao”, which means they were hired by

the patient side and making a living by taking disruptive

actions in hospitals to make hospitals to pay to the pa-

tients. The chaos lasts for 5 hours on average and can be

more than 10 hours in some extreme cases. Most Yinao

cases are related to death of patients, which accounts for

79% of all the Yinao cases.

Regression analysis on success rate of mediation

We use logistic regression model to analyze the outcomes

of mediation. Since ADR is often viewed as complemen-

tary to lawsuits, we classified mediation agreement, recon-

ciliation and withdrawn cases as successful, indicated by

one in our model. Litigation outcome is represented by

zero. Table 3 shows the results of our logistic regression.

Duration has an odds ratio of 0.998 and is significant,

which means for one-day increase in duration the odds

of the case solved through mediation decreases by 0.2%

given other variables are fixed. Compared to

responsibility, the odds ratio of nursing and medicine is

positive and significant, which means its chance to be

solved through mediation is 68.7% higher than responsi-

bility. With tertiary hospitals as reference, we find that

the cases in primary and secondary hospitals are more

likely to be solved through mediation and secondary

hospital cases have the highest chance to be mediated.

After including the consequence of the cases, column

2 shows that other outcomes are more likely to be

solved through mediation compared to cases with death

as results. Since other cases are less severe than death or

disability cases, it is easier to clarify the cases and for the

patients and hospitals to reach agreements.

Considering the trend of mediation, we find that the

chance of successful mediation grows year by year. In

2014, the odds is 1.7 and significant, which means the

chance of success is 70% higher than 2013. In 2015, the

odds is 2.649 and significant, which mean the chance of

success is 165% higher than 2013. We can see the suc-

cess rates of mediation is increasing which may be due

to increased experience.

Finally, the first claim of patients is also included in

our model. Since patients may not be clear about the

compensation amount when they chose mediation, the

sample size dropped to 2525 because of missing values

of first claim amount. Column 4 shows that for CNY10,

000 increase in first claim amount by patients, the odds

decrease by 7% given other factors remain fixed. Com-

pared to responsibility, the odds ratio of nursing and

medicine becomes insignificant, while the odds ratio of

management problem becomes significant at 10% level,

which means its chance to be solved through mediation

is 121% higher than responsibility. The successful medi-

ation trend grows at a higher magnitude compared to

year 2013, reaching 84.8% in 2014 and 167% in 2015.

Table 2 Yinao cases mediated on the spot, China, 2013–2015

Category Category

Yinao date-no.(%) 505 > 20,≤50 168(35)

2013 14(3) > 50 37(8)

2014 237(47) Yinao Duration (hours)-no.(%) 456

2015 254(50) Mean 5

Hospital level-no.(%) 505 ≤5 274(60)

Tertiary 165(33) > 5,≤10 135(30)

Secondary 232(46) > 10 47(10)

Primary 108(21) Severity-no.(%) 505

No. of participants-no.(%) 482 Death 399(79)

Mean 26 Disability 17(3)

≤10 136(28) Others 89(18)

> 10,≤20 141(29)

SOURCE Authors’ analysis of mediation data for 2013–2015

Wang et al. BMC Health Services Research (2020) 20:225 Page 6 of 10

Regression analysis on compensation of mediation

Table 4 shows generalized linear model analysis of com-

pensation. Across all the columns, we can see that the

coefficient of duration is positive and significant. The co-

efficient of duration shows that for one-day increase in

duration, the compensation can increase by 0.2 to 0.3%

(exp (0.002)-1, same below). The coefficient of patients’

age is insignificant all the time.

Compared to responsibility, most other causes of med-

ical disputes are compensated less. In column 1, we can

see that the compensation of nursing and medicine is

Table 3 Logistic regression analysis of the success rate of

mediation, China, 2013–2015

(1) (2) (3) (4)

VARIABLES

Duration of case 0.998*** 0.998*** 0.997*** 0.997***

(0.000) (0.000) (0.000)

(0.001)

First claim by patients 0.993***

(0.002)

Age of patients 1.001 1.002 1.001 1.000

(0.002) (0.002) (0.002) (0.003)

Main Cause

Medical Accidents 1.357 1.515 1.730** 2.738**

(0.339) (0.387) (0.451) (1.154)

Skills 0.948 0.885 1.009 1.119

(0.103) (0.099) (0.116) (0.175)

Nursing and Medicine 1.687** 1.605** 1.693** 1.225

(0.400) (0.382) (0.406) (0.332)

Informed consent 0.713 0.664* 0.655* 0.604*

(0.170) (0.162) (0.162) (0.182)

Management problem 1.376 1.418 1.467 2.210*

(0.417) (0.437) (0.459) (0.956)

Hospital level

Primary 1.836*** 1.835*** 1.844*** 1.484**

(0.271) (0.277) (0.279) (0.292)

Secondary 2.090*** 2.029*** 2.047*** 1.919***

(0.217) (0.214) (0.218) (0.277)

Consequence

Disabled 0.891 0.869 0.926

(0.143) (0.141) (0.203)

Others 2.530*** 2.524*** 1.926***

(0.270) (0.272) (0.297)

Year

Year 2014 1.700*** 1.848***

(0.202) (0.296)

Year 2015 2.649*** 2.672***

(0.348) (0.480)

Constant 5.604*** 3.740*** 2.247*** 3.371***

(0.667) (0.478) (0.339) (0.825)

Observations 4313 4313 4313 2525

Robust se in parenthese

*** p < 0.01, ** p < 0.05, * p < 0.1

Note: Mediation agreement, reconciliation and withdrawn cases are classified

as successful. Litigation cases are classified as unsuccessful

Table 4 Generalized linear regression analysis of the

compensation of mediation, China, 2013–2015

(1) (2) (3) (4)

VARIABLES

Duration of case 0.003*** 0.003*** 0.003*** 0.002***

(0.000) (0.000) (0.000) (0.000)

First claim by patients 0.014***

(0.001)

Age of patients −0.001 −0.002 −0.002 0.001

(0.001) (0.001) (0.001) (0.001)

Main Cause

Medical Accidents 0.124 −0.081 −0.107 − 0.116

(0.147) (0.121) (0.120) (0.119)

Skills −0.167** − 0.016 − 0.042 0.035

(0.077) (0.078) (0.078) (0.077)

Nursing and Medicine −0.785*** −0.749*** − 0.771*** −0.530***

(0.138) (0.103) (0.102) (0.102)

Informed consent −0.776*** −0.716*** − 0.735*** −0.391**

(0.227) (0.199) (0.201) (0.174)

Management problem −0.758*** −0.741*** − 0.758*** −0.464***

(0.220) (0.160) (0.154) (0.117)

Hospital level

Primary −0.167 −0.222** −0.223** − 0.241***

(0.108) (0.096) (0.097) (0.092)

Secondary −0.213*** −0.216*** − 0.217*** −0.191***

(0.075) (0.075) (0.074) (0.070)

Consequence

Disabled 0.016 −0.004 0.097

(0.110) (0.109) (0.098)

Others −1.256*** −1.261*** −0.841***

(0.067) (0.066) (0.072)

Year

Year 2014 0.035 0.055

(0.090) (0.074)

Year 2015 −0.147* −0.069

(0.087) (0.080)

Constant 1.821*** 2.303*** 2.339*** 1.433***

(0.088) (0.090) (0.106) (0.121)

Observations 2764 2764 2764 1726

Robust se in parenthese

*** p < 0.01, ** p < 0.05, * p < 0.1

Wang et al. BMC Health Services Research (2020) 20:225 Page 7 of 10

54.4% less than responsibility, followed by inform con-

sent and management problem. Medical accident is as-

sociated with higher compensation compared to

responsibility, while the result is not significant.

The coefficient of secondary hospital is negative and sig-

nificant in all models. In column 1, we find that secondary

hospitals are compensated 19.2% less than tertiary hospi-

tals on average. After controlling for consequence and

year effects, the compensation is about 19.5% less than

tertiary hospitals.

After including first claim amount by patients, we find

for CNY10,000 increase in the claimed amount, the

compensation will increase by 14.1%. In column 4, we

find that the coefficient of disabled is insignificant, which

means the compensation of disabled is similar to death

cases on average. The coefficient for others category is

negative and significant, which means the less severe

cases will get 56.9% less than death cases on average.

Although the coefficient of year is significant at 10%

level in 2015 in column 3, the year effects becomes in-

significant in column 4. It is reasonable because the

compensation standard doesn’t change that much in dif-

ferent years.

Mediation as a buffer

Our data shows that 89% of the cases were successfully

handled through mediation without going to litigation

and the average length of mediation is 87 days, which

are comparable to the ADR practice in America and

Canada [26]. Our result highlights the importance of

communication between different sides. We find that

mediation can help avoid “nuisance lawsuits” by offering

suggestions to patients and hospitals and encourage

them to give up unreasonable demands. 4% of cases

were rejected and 11% of cases were withdrawn, which

reflects that many cases can be settled after clarification

of the medical and legal issues, saving time and re-

sources. Mediation can offer physicians and patients the

chance to talk, negotiate and apologize, which may also

improve the doctor-patient relationship.

More medical and legal knowledge is helpful for patients

to shape a reasonable expectation on the outcome of med-

ical services and compensation amount, which can ease the

conflicts between both sides and help build consensus. Our

regression analysis results show that both longer duration

and higher claim amount are associated with less probabil-

ity to be solved by mediation and higher compensation.

Compared to death cases, less severe cases are more likely

to be settled through mediation, and closed with less com-

pensation. It reflects that patients actually have the right

idea about the severity of medical services’ consequence,

while in general they may initially claim more than the ac-

tual compensation amount received.

PMC, as a third-party, has gradually gained popularity.

More specifically, the more flexible process and easier

access for patients can help save a lot of time and money

in contrast with lawsuits [16, 19]. The neutral stance of-

fers PMC a more objective view compared to mediation

conducted by health administrative departments [27].

Given all these advantages and to avoid the previous

privately and costly settlement between medical institu-

tion and patients [28], several regions have implemented

special regulations on the ways to solve medical disputes

and mediation is almost the first choice for both patients

and hospitals there. In 2013, the government of Guang-

dong Province issued a provincial notice on prevention

and resolution of medical disputes, which stated that ne-

gotiation between hospital and patients is only appropri-

ate for claims below CNY10,000 [29]. Similarly, Baoji

city in Shaanxi Province also set a cap on the compensa-

tion for negotiation approach in 2015 [30].

Hospital management can play a role in dispute

prevention and resolution

Studies have shown the possibility to reduce disputes by

changing the procedure of dealing with medical disputes

[31] and early detection/intervention of medical disputes

[32].

In our analysis, we find that tertiary hospitals and de-

partments like surgery, obstetrics and gynecology, and

internal medicine are more likely to face medical dis-

putes. According to the requirements of the National

Health Commission (previously known as the Ministry

of Health) [33], tertiary hospitals are designed to treat

patients with serious conditions as well as common dis-

eases and supposed to provide higher-quality of services,

so they are more likely to be involved in complex med-

ical disputes with associated higher compensation. Our

results show that cases in primary and secondary hospi-

tals are more likely to be solved through mediation and

end up with less compensation compared to tertiary

hospitals. Surgery, obstetrics and gynecology, and in-

ternal medicine departments are the top three depart-

ments that are involved in medical disputes. The

analysis on main causes show that most of the medical

disputes can be attributed to responsibility and skills, it

is more demanding for tertiary hospitals and the top

three departments to enhance their risk management

practices and help improve the skills of relative services.

More help is needed to handle Yinao cases

The average duration (5 h) and number of people partici-

pating (26) in Yinao depict the chaotic scene faced by hos-

pitals during 2013–2015. In China, legal progress has been

made to prohibit Yinao. However, we find that most Yinao

cases are related with the death or disability of patients

(82%). Consequently, it may be hard to peacefully resolve

Wang et al. BMC Health Services Research (2020) 20:225 Page 8 of 10

Yinao and more support should be given to guarantee the

safety of physicians. In 2016, we conducted interviews

with the mediators and the director of PMC. Based on

their working experiences, we found that Yinao is not well

prohibited and that the police force didn’t go to the spot

to calm down the issue every time.

Although mediators can go and help deal with Yinao

cases, it’s necessary for police department and procurato-

rate to participate and solve Yinao at the time. As the

“Medical Disputes Prevention and Treatment Regulations”

in 2018 has stated that hospitals should report Yinao to

the police immediately, more analysis should be done to

evaluate whether police forces have the necessary re-

sources to handle Yinao cases. The influences of Yinao are

not just limited within one case. In China, extreme Yinao

cases will be on the news and generate social media dis-

cussion. It is necessary for the media to advocate and

emphasize the legal ways to solve disputes instead of just

reporting the fact of Yinao or violence against physicians.

Limitations

Our study has several limitations. First, the finding that me-

diation is time-saving compared to litigation maybe im-

pacted by the observation period. We only included cases

that were settled during the observation period so there

could be some very long ongoing cases that were not re-

solved during this time and so the average length of reso-

lution may be under-estimated. Furthermore, some serious

cases may bypass the mediation method and choose the

legal approach directly. Second, due to data availability, our

analysis focuses on the mechanism and results of cases han-

dled through PMC in Guangdong Province, further analysis

could include data from other provinces to confirm if our

findings in Guangdong apply at the national level. Third,

our data didn’t cover the period after 2015, therefore we

cannot capture the effects of litigation change before and

after Yinao was included in Criminal Law in 2015. Since

this law change is expected to deter patients from taking

illegal actions, future analysis would be valuable to under-

stand its effect on Yinao cases in more detail.

Conclusion

Through the analysis of the mediation committee work

in China, we find that mediation can help substantially

reduce conflicts between physicians and patients to

avoid litigations, thus saving time and money for both

parties. While for illegal actions taken by patients like

Yinao, not only mediators but also the police are needed

to fully handle such cases. By setting up a neutral third-

party institution like the mediation committee together

with an appropriate legal framework, healthcare systems

can also better handle the disputes between hospitals

and patients to improve the patient-provider relation-

ships and public satisfaction.

ADR: Alternative Dispute Resolution; CNY: China Yuan; GDP: Gross Domestic

Product; PMC: People’s Mediation Committee

We wish to thank Min Huang for her extremely helpful suggestions on the

early design of this paper. We are grateful for Xinyue Dong for her help and

comments on the first draft.

MW designed and collected data for the qualitative study. MY cleaned the

data for the quantitative study. MW and TB analyzed the quantitative data.

MW, GL, HZ, TB, MY and YC contributed to the methodological elaboration

of the results. MW wrote the first draft of the paper, the others commented

on subsequent drafts. All authors read and approved the final manuscript.

Data analysis is supported by China Medical Board, “The PKU-CMB Lab for

Health Economics Research: Health Financing, Delivery, and Economic Pros-

perity” (15–217, 2016–2018) and National Natural Science Foundation of

China (71773002). The funder of the study had no role in study design, data

collection, data analysis, data interpretation, or writing of the report.

The data that support the findings of this study are available from People’s

Mediation Committee in Guangdong but restrictions apply to the availability

of these data, which were used under license for the current study, and so

are not publicly available. Data are however available upon research request

and with permission of People’s Mediation Committee in Guangdong.

No administrative permissions were required to access and use the

mediation records described in our study.

Not applicable.

The authors declare that they have no competing interests.

1National School of Development, Peking University, Beijing 100871, China.

2People’s Mediation Committee in Guangdong, Guangzhou 510095, China.

3Sichuan Development Center for Healthy Aging, Chengdu 610094, China.

4Department of Economics, Louisiana State University, Baton Rouge 70820,

USA. 5China Hospital Development Institute, Shanghai Jiao Tong University,

Shanghai 200025, China.

Received: 26 July 2019 Accepted: 26 February 2020

1. Partnership CJS. Deepening health reform in China: building high-quality

and value-based service delivery; 2016.

2. Blumenthal D, Hsiao W. Privatization and its discontents-the evolving

Chinese health care system. N Engl J Med. 2005;353(11):1165–70.

3. Tucker JD, Cheng Y, Wong B, Gong N, Nie JB, Zhu W, McLaughlin MM, Xie

RS, Deng YH, Huang MJ, et al. Patient-physician mistrust and violence

against physicians in Guangdong Province, China: a qualitative study. BMJ

Open. 2015;5(10):e008221.

4. Yao S, Zeng Q, Peng M, Ren S, Chen G, Wang J. Stop violence against

medical workers in China. J Thoracic Disease. 2014;6(6):141–5.

5. Wu D, Wang Y, Lam KF, Hesketh T. Health system reforms, violence

against doctors and job satisfaction in the medical profession: a cross-

sectional survey in Zhejiang Province, Eastern China. BMJ Open. 2014;

4(12):e006431.

6. Saeki K, Okamoto N, Tomioka K, Obayashi K, Nishioka H, Ohara K,

Kurumatani N. Work-related aggression and violence committed by

patients and its psychological influence on doctors. J Occup Health.

2011;53(5):356–64.

Wang et al. BMC Health Services Research (2020) 20:225 Page 9 of 10

7. Pan Y, Yang X, He JP, Gu YH, Zhan XL, Gu HF, Qiao QY, Zhou DC, Jin HM. To

be or not to be a doctor, that is the question: a review of serious incidents

of violence against doctors in China from 2003-2013. J Public Health. 2015;

23(2):111–6.

8. Hesketh T, Wu D, Mao LN, Ma N. Violence against doctors in China. BMJ.

2012;345.

9. Zheng T: Analysis on generation mechanism and regulation approaches of

interests-seeking Yinao. Journal of Yunnan University Law Edition 2016,

29(3):51–57.

10. Tianjin procuratorate approved the arrest of criminal suspects of hospital

violence of the Armed Police Froce Hospital. https://www.guancha.cn/

society/2018_07_17_464513.shtml. Accessed 17 July 2018.

11. Wang H, Shi J, Cheng W. Research review on medical disputes in China.

Med Jurisprud. 2016;12(1):83–7.

12. He AJ, Qian J. Explaining medical disputes in Chinese public hospitals: the

doctor-patient relationship and its implications for health policy reforms.

Health Econ Policy L. 2016;11(4):359–78.

13. Chen B. Analysis of the inversion for responsibility of proof citation in

medical tort actions under the background of Tort Liability Law of P.R.C.

China Health Law. 2010;5:53–5.

14. Ministry of Health, Ministry of Public Security. Notice on maintaining the

order of medical institutions 2012. http://www.nhfpc.gov.cn/yzygj/s7658/2

01205/fb53a6594ef2413ca7ef661859a5b7de. Accessed 17 July 2017.

15. Friedson AI. Medical malpractice damage caps and provider reimbursement.

Health Econ. 2017;26(1):118–35.

16. Li H, Wu X, Sun T, Li L, Zhao X, Liu X, Gao L, Sun Q, Zhang Z, Fan L. Claims,

liabilities, injures and compensation payments of medical malpractice

litigation cases in China from 1998 to 2011. BMC Health Serv Res. 2014;

14(1):181–9.

17. DeVille KA. The jury is out: pre-dispute binding arbitration agreements for

medical malpractice claims – law, ethics, and prudence. J Legal Med. 2007;

28(3):333–95.

18. Nakanishi T. New communication model in medical dispute resolution in

Japan. Bull Yamagata Univ Med Sci. 2013;31:1–8.

19. Shi M, Zhang H, Cheng Q. Analysis on the causes, distribution and

compensation of 5012 medical damage disputes. Med Jurisprud. 2015;

7(6):42–8.

20. Pegalis SE, Bal BS. Closed medical negligence claims can drive patient safety

and reduce litigation. Clin Orthop Relat R. 2012;470(5):1398–404.

21. Sohn DH, Sonny Bal B. Medical malpractice reform: the role of alternative

dispute resolution. Clin Orthop Relat R. 2012;470(5):1370–8.

22. Zhao C, Zhu B. Empirical study and countermeasures discussion on medical

disputes -Based on the analysis on cases accepted by Gulou District

People’s Mediation Committee of medical disputes from 2010 to 2013. Acta

Universitatis Medicinalis Nanjing (Social Sciences). 2015;5:349–54.

23. National Bureau of Statistics of the People’s Republic of China: China

Statistical Yearbook, vol. 36; 2017.

24. Cohen SM, Kim J, Roy N, et al. Factors influencing the health care

expenditures of patients with laryngeal disorders. Otolaryngol Head Neck

Surg. 2012;147(6):1099–107.

25. National Health and Family Planning Commission. Basic Measurements for

Medical Institutions (Pilot Draft) 2017. http://www.ncwjw.gov.cn/html/

faguizhongxin/201708286822.html. Accessed 8 Aug 2018.

26. Szmania SJ, Johnson AM, Mulligan M. Alternative dispute resolution in

medical malpractice: a survey of emerging trends and practices. Conflict Res

Q. 2008;26(1):71–96.

27. Li C. Empirical research on the third-party mediation mechanism in medical

dispute. Chin Health Serv Manag. 2014;31(2):125–7.

28. Niu D, Hou F. Discussing the cause of private remedy on medical dispute

and its countermeasures. Chin Health Serv Manag. 2011;28(6):442–4.

29. Government of Guangdong Province. Approaches for preventing and

dealing with medical disputes in Guangdong province 2013. http://zwgk.gd.

gov.cn/006939748/201304/t20130409_371711.html. Accessed 18 Sept 2014.

30. City Government of Baoji. Interim procedures for mediation of medical

disputes in Baoji city 2014. http://baoji.gov.cn/site/11/html/276/290/271688.

htm. Accessed 8 July 2015.

31. Lu Z. Study on the influence of medical disputes prevention based on the

process reformation of cybernetic. Health Econ Res. 2015;338(6):46–9.

32. Xu P, Fan Z, Li T, et al. Preventing surgical disputes through early detection

and intervention: a case control study in China. BMC Health Serv Res. 2015;

15:5.

33. Ministry of Health. General hospital classification and management

standards (pilot draft) 1989. https://wenku.baidu.com/view/e1c9493710661

ed9ad51f3ec.html. Accessed 19 Oct 2015.

Springer Nature remains neutral with regard to jurisdictional claims in

published maps and institutional affiliations.

Wang et al. BMC Health Services Research (2020) 20:225 Page 10 of 10

https://www.guancha.cn/society/2018_07_17_464513.shtml

https://www.guancha.cn/society/2018_07_17_464513.shtml

http://www.nhfpc.gov.cn/yzygj/s7658/201205/fb53a6594ef2413ca7ef661859a5b7de

http://www.nhfpc.gov.cn/yzygj/s7658/201205/fb53a6594ef2413ca7ef661859a5b7de

http://www.ncwjw.gov.cn/html/faguizhongxin/201708286822.html

http://www.ncwjw.gov.cn/html/faguizhongxin/201708286822.html

http://zwgk.gd.gov.cn/006939748/201304/t20130409_371711.html

http://zwgk.gd.gov.cn/006939748/201304/t20130409_371711.html

http://baoji.gov.cn/site/11/html/276/290/271688.htm

http://baoji.gov.cn/site/11/html/276/290/271688.htm

https://wenku.baidu.com/view/e1c9493710661ed9ad51f3ec.html

https://wenku.baidu.com/view/e1c9493710661ed9ad51f3ec.html

BioMed Central publishes under the Creative Commons Attribution License (CCAL). Under

the CCAL, authors retain copyright to the article but users are allowed to download, reprint,

distribute and /or copy articles in BioMed Central journals, as long as the original work is

properly cited.

- Abstract

Background

Methods

Results

Conclusion

Background

Methods

Mediation in China

Data sample

Regression method

Results

Characteristics of mediation cases

Regression analysis on success rate of mediation

Regression analysis on compensation of mediation

Discussion

Mediation as a buffer

Hospital management can play a role in dispute prevention and resolution

More help is needed to handle Yinao cases

Limitations

Conclusion

Abbreviations

Acknowledgements

Authors’ contributions

Funding

Availability of data and materials

Ethics approval and consent to participate

Consent for publication

Competing interests

Author details

References

Publisher’s Note

Psychological Methods

Improved Inference in Mediation Analysis: Introducing

the Model-Based Constrained Optimization Procedure

Davood Tofighi and Ken Kelley

Online First Publication, March

1

9, 2020. http://dx.doi.org/10.1037/met0000259

CITATION

Tofighi, D., & Kelley, K. (2020, March 19). Improved Inference in Mediation Analysis: Introducing the

Model-Based Constrained Optimization Procedure. Psychological Methods. Advance online

publication. http://dx.doi.org/10.1037/met0000259

Improved Inference in Mediation Analysis: Introducing the Model-Based

Constrained Optimization Procedure

Davood Tofighi

University of New Mexico

Ken Kelley

University of Notre Dame

Abstract

Mediation analysis is an important approach for investigating causal pathways. One approach used in

mediation analysis is the test of an indirect effect, which seeks to measure how the effect of an independent

variable impacts an outcome variable through 1 or more mediators. However, in many situations the proposed

tests of indirect effects, including popular confidence interval-based methods, tend to produce poor Type I

error rates when mediation does not occur and, more generally, only allow dichotomous decisions of “not

significant” or “significant” with regards to the statistical conclusion. To remedy these issues, we propose a

new method, a likelihood ratio test (LRT), that uses nonlinear constraints in what we term the model-based

constrained optimization (MBCO) procedure. The MBCO procedure (a) offers a more robust Type I error rate

than existing methods; (b) provides a p value, which serves as a continuous measure of compatibility of data

with the hypothesized null model (not just a dichotomous reject or fail-to-reject decision rule); (c) allows

simple and complex hypotheses about mediation (i.e., 1 or more mediators; different mediational pathways);

and (d) allows the mediation model to use observed or latent variables. The MBCO procedure is based on a

structural equation modeling framework (even if latent variables are not specified) with specialized fitting

routines, namely with the use of nonlinear constraints. We advocate using the MBCO procedure to test

hypotheses about an indirect effect in addition to reporting a confidence interval to capture uncertainty about

the indirect effect because this combination transcends existing methods.

Translational Abstract

Mediation analysis has become one of the most important approaches for investigating causal pathways. One

instrument used in mediation analysis is a test of indirect effects that seeks to measure how the effect of an

independent variable impacts an outcome variable through one or more mediators. However, in many

situations the proposed tests of indirect effects, including popular confidence interval-based methods, tend to

produce too few or too many false positives (Type I errors) and are commonly used to make only dichotomous

decisions about acceptance (“not significant”) or rejection (“significant”) of a null hypothesis. To remedy these

issues, we propose a new procedure to test an indirect effect. We call this new procedure the model-based

constrained optimization (MBCO) procedure. The MBCO procedure (a) more accurately controls the false

positive rate than existing methods; (b) provides a p value, which serves as a continuous measure of

compatibility of data with the hypothesized null model (not just a dichotomous reject or fail-to-reject decision

rule); (c) allows simple and complex hypotheses about mediation (i.e., one or more mediators; different

mediational pathways); and (d) allows the mediation model to use observed or latent variables common in

psychological research. We advocate using the MBCO procedure to test hypotheses about an indirect effect

in addition to reporting a confidence interval to capture uncertainty about the indirect effect.

Keywords: indirect effect, mediation analysis, confidence interval, likelihood ratio, model-comparison test

Supplemental materials: http://dx.doi.org/10.1037/met0000259.supp

Mediation analysis has become one of the most important and

widely used methods to study causal mechanisms in psychology,

management, education, and other related fields (e.g., Ato García,

Vallejo Seco, & Ato Lozano, 2014; Boies, Fiset, & Gill, 2015;

Bulls et al., 2017; Carmeli, McKay, & Kaufman, 2014; Deković,

Asscher, Manders, Prins, & van der Laan, 2012; Ernsting, Knoll,

Schneider, & Schwarzer, 2015; Graça, Calheiros, & Oliveira,

2016; Haslam, Cruwys, Milne, Kan, & Haslam, 2016; Koning,

Maric, MacKinnon, & Vollebergh, 2015; MacKinnon, 2008; Mo-

lina et al., 2014). Researchers have proposed several methods to

test for the presence of mediation (MacKinnon, Lockwood, Hoff-

man, West, & Sheets, 2002), including the widely recommended

(a) Monte Carlo confidence interval (CI; MacKinnon, Lockwood,

& Williams, 2004; Tofighi & MacKinnon, 2016), also known as

parametric bootstrap method (Efron & Tibshirani, 1993); (b) per-

X Davood Tofighi, Department of Psychology, University of New

Mexico; X Ken Kelley, Department of Information Technology, Analyt-

ics, and Operations, University of Notre Dame.

Correspondence concerning this article should be addressed to Davood

Tofighi, Department of Psychology, University of New Mexico, Logan Hall,

MSC03-2220, Albuquerque, NM 87131– 0001. E-mail: dtofighi@unm.edu

T

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

Psychological Methods

© 2020 American Psychological Association 2020, Vol. 2, No. 999, 000

ISSN: 1082-989X http://dx.doi.org/10.1037/met0000259

1

https://orcid.org/0000-0001-8523-7776

https://orcid.org/0000-0002-4756-8360

mailto:dtofighi@unm.edu

http://dx.doi.org/10.1037/met0000259

centile and bias-corrected (BC) bootstrap resampling CI (Bollen &

Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et al., 2004;

Shrout & Bolger, 2002); (c) profile-likelihood CI method (Folmer,

1981; Neale & Miller, 1997; Pawitan, 2001; Pek & Wu, 2015); and

(d) joint significance test (Kenny, Kashy, & Bolger, 1998; MacK-

innon et al., 2002). Henceforth, we will use the term CI-based

methods to emphasize a dual role that CIs have played in the

mediation analysis literature. In addition to providing a range of

plausible values for a population indirect effect, CIs have also

controversially been used to test the null hypothesis of no medi-

ation (i.e., a zero indirect effect) at the � level with a (1-�) 100%

CI. However, testing an indirect effect is complicated because the

indirect effect is the product of the coefficients along the mediation

chain (MacKinnon, 2008). Further, each of the methods mentioned

earlier has key limitations, many of which researchers commonly

encounter. Thus, even though these procedures are currently rec-

ommended, that recommendation in research is questionable.

Many popular testing methods in mediation models are limited

because they lack robustness of the observed Type I error rate.

That is, the empirical probability of falsely rejecting the null

hypothesis about an indirect effect does not equal, and can fre-

quently fall outside of, a robustness interval for the Type I error

rate, such as Bradley’s (1978) liberal interval [0.5�, 1.5�], where

� is the nominal significance level (Biesanz, Falk, & Savalei,

2010; Tofighi & Kelley, 2019). The idea of using .025–.075 as an

acceptable departure from the idealized properties of the statistical

procedures, when � � .05, acknowledges that a null hypothesis

procedure may not work perfectly but is still useful. Bradley’s

(1978) liberal criterion accepts a procedure as robust when violations

of the model’s assumptions are small enough not to fundamentally

change the statistical conclusion. The importance of the conclusion in

the context of mediation, for example, is when sample size is less than

100 and � � .05. Several methods, including widely recommended

CI-based methods, can result in Type I error rates that fall outside of

Bradley’s liberal interval [0.025 .075] (Tofighi & Kelley, 2019) and

are, therefore, unsatisfactory. We propose a method that is more

robust than other procedures because it better satisfies Bradley’s

(1978) liberal criterion and has desirable statistical properties that

allow its wide use, including in situations beyond which existing

methods are applicable.

Currently used methods of testing mediation are limited because

these tests are commonly used to make only dichotomous deci-

sions about acceptance (“not significant”) or rejection (“signifi-

cant”) of a null hypothesis. These dichotomous decisions provide

a false sense of certainty about a model and data and are not

generally recommended (Amrhein, Trafimow, & Greenland, 2019;

Wasserstein, Schirm, & Lazar, 2019). Further, these methods do

not offer either a p value, a continuous measure of compatibility

between data and the null hypothesis (Greenland et al., 2016), or

an additional means of measuring statistical evidence, such as a

likelihood ratio (Blume, 2002). As recommended by Wilkinson

and the Task Force on Statistical Inference, American Psycholog-

ical Association, Science Directorate (1999), reporting exact p

values, CIs, and effect sizes, and not just noting “reject” or

“fail-to-reject” is the ideal for science to progress. None of the

CI-based tests, by their very nature, offers additional ways of

evaluating the strength of evidence against a null hypothesis of no

mediation. A likelihood ratio, however, is a continuous measure

that can compare the goodness of fit of two competing models

under standard assumptions, as measured by the ratio of the

models’ maximized likelihoods. One model, a null model, repre-

sents and satisfies the null hypothesis of no mediation (e.g., zero

indirect effect), and the other model, a full model, shows that

mediation does occur, as posited by a researcher (e.g., the indirect

effect is nonzero). Further, a likelihood value can be used to

compute information fit indices, such as Akaike information cri-

terion (AIC; Akaike, 1974) and Bayesian information criterion

(BIC; Schwarz, 1978); such indices compare the fit of the full and

null models while penalizing each model for a lack of parsimony

(i.e., the number of model parameters).

To address these limitations, we propose a model-based con-

strained optimization (MBCO) method as a new procedure that tests

any function representing an indirect effect1 and that provides a

formal way to recast testing simple and complex hypotheses about an

indirect effect into a model-comparison framework. The MBCO

procedure offers two formal mechanisms to evaluate hypotheses

about indirect effects in the model-comparison framework. First, the

MBCO procedure offers a likelihood ratio test (LRT) statistic, which

we term LRTMBCO, that has a large sample chi-squared distribution

and provides a p value to test hypotheses about an indirect effect.

Second, the MBCO procedure compares the fit of two models in

terms of a likelihood ratio and information fit indices, such as the AIC

and BIC, as well as their generalizations.

In the model-comparison framework, we estimate the null and full

models representing different hypotheses about an indirect effect

under a null and an alternative hypothesis, respectively (see Maxwell,

Delaney, & Kelley, 2018). A null model represents the null hypoth-

esis about the indirect effect. Unlike existing approaches to model

comparisons, the MBCO procedure can estimate the null mediation

model by using nonlinear constraints for testing a null hypothesis in

mediation models. This innovative application of nonlinear con-

straints can be used to test both simple and complex hypotheses in

mediation models. The MBCO procedure also allows the indirect

effect of a full model, posited by a researcher, to be freely estimated.

For example, a full model is a mediation model shown in Figure 1.

Although model comparisons using the likelihood ratio test are com-

mon in techniques such as multilevel modeling (Raudenbush & Bryk,

2002) and structural equation modeling (SEM; Kline, 2016), in almost

all such situations the parameters are subject to linear constraints.

Testing indirect effects is problematic because constraining the indi-

rect effect, H0: �1�2 � 0, is inherently a nonlinear constraint optimi-

zation problem as the constraint is the product of two parameters

(Snyman, 2005).

The MBCO procedure offers several advantages over the exist-

ing, recommended methods. First, the MBCO procedure offers the

test statistic LRTMBCO that has better statistical properties than the

widely recommended approaches in the literature. This test statis-

tic, widely used in statistics, has desirable properties for testing

complex null hypotheses (Cox & Hinkley, 2000). The LRTMBCO

yields more robust Type I error rates than the currently recom-

mended methods when the null hypothesis of no mediation occurs.

Second, the MBCO procedure offers researchers multiple ways of

evaluating a null hypothesis of an indirect effect rather than only

1 More formally, a function of model parameters should be a smooth

function, which is a function that has continuous derivatives up to a desired

order (Weisstein, 2018).

T

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

2 TOFIGHI AND KELLEY

allowing a dichotomous reject or fail conclusion that has been

criticized by researchers (e.g., Cohen, 1994; Harlow, Mulaik, &

Steiger, 1997; Wasserstein et al., 2019). The MBCO procedure

also allows researchers to test simple and complex hypotheses for

a variety of mediation models involving multiple mediators (ob-

served or latent) in a flexible manner. This flexibility aids re-

searchers interested in complex causal models that are often pos-

ited by theories and can be implemented in an SEM framework.

Such flexibility and desirable statistical performance make the

MBCO procedure a desirable advancement for mediation studies.

To make our proposed method immediately available to research-

ers, we provide detailed instructions along with the necessary R (R

Core Team, 2019) code (see the online supplemental materials).

Nonlinear constraint optimization algorithms to implement the

MBCO procedure and to compute the LRTMBCO are already

implemented in at least one R software package, and we expect

other software packages to soon make such allowances as well.

The Model-Based Constrained Optimization Procedure

We begin by explaining the MBCO procedure conceptually and

then discussing the required steps required to perform this proce-

dure. Without loss of generality, we elaborate on the MBCO

procedure to test the zero indirect effect null hypothesis, H0:

�1�2 � 0, for the single-mediator model example (see Figure 1).

In the Empirical Example of Using the MBCO Procedure on a

Complex Mediation Model section, we discuss extensions of the

MBCO procedure to the more elaborate, parallel two-mediator

model (see Figure 2). In the Simulation Study section, we briefly

extend the MBCO procedure to a sequential two-mediator model

and then compare the performance of the MBCO procedure with

the most commonly used tests of indirect effects for both a single-

mediator model and a sequential two-mediator model.

The MBCO procedure recasts hypotheses about indirect effects

into a model-comparison framework. To illustrate, consider the

single-mediator model (see Figure 1), which can be represented by the

following regression equations:

M � �0M � �1X � εM (1)

Y � �0Y � �2M � �3X � εY , (2)

where �1 is the effect of instruction (X � 1 corresponds to

instruction to use imagery rehearsal; X � 0 corresponds to instruc-

tion to use repetition rehearsal) on the use of imagery (imagery);

�2 is the effect of imagery on the number of words recalled (recall)

controlling for instruction, and �3 is the direct effect of the

instruction on recall controlling for imagery. �0M and �0Y denote

the intercepts while εM and εY are residual terms. The indirect

effect of instruction on recall through imagery is defined as

whether instruction to use imagery increases the use of imagery

and then, in turn, increases the number of words participants

recalled. Under the no-omitted-confounder assumption, which as-

sumes that a variable should not be omitted from the model that

would influence both the mediator and the outcome variables, the

indirect effect equals the product of two coefficients, �1�2 (Imai,

Keele, & Tingley, 2010; Judd & Kenny, 1981; Valeri & Vander-

Weele, 2013; VanderWeele, 2010). Suppose that we are interested

in testing whether the indirect effect of instruction on recall

through imagery is zero. We can use the following null hypothesis

to answer this question:

H0: �1�2 � 0

H1: �1�2 � 0

To recast the hypotheses about the indirect effect into a model-

comparison framework, we estimate two mediation models: a full

(alternative) model and a null (restricted) model (Maxwell et al.,

2018). Which model to estimate first is arbitrary. For our example,

the full model is the single-mediator model in (1)–(2). Note that

the full model does not constrain the value of the indirect effect,

�1�2. Next, for the null model, we estimate a mediation model in

which the indirect effect �1�2 is constrained to equal 0. Both the

full and the null model can be represented by the set of equations

in (1) and (2) with the limitation that �1�2 is constrained to be 0

in the null model. The null model is nested within the full model;

Figure 1. An example of a randomized single-mediator model. Instruction (X) is a random assignment with two

conditions: instruction to use mental imagery rehearsal (X � 1) versus instruction to use repetition (X � 0). Imagery

is a self-reported score of using mental imagery to memorize words, and recall is the number of words out of a total

of 20 words that each student correctly remembered at the end of the experiment. A solid arrow between two variables

indicates a direct effect of the variable on the left on the other variable. Greek letters denote population values, where

�s denote the regression (path) coefficients, with �1 being the treatment effect on imagery, �2 being the partial effect

of imagery on recall controlling for treatment effect, and �3 is the partial effect of instruction on recall controlling for

imagery. Terms εY and εM denote residuals that are assumed to be normally distributed.

T

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

3IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

that is, by setting the indirect effect to zero in the full model, we

would obtain the null model.

The final step of the MBCO procedure is to formally compare

the two models. One such approach compares the fit of the two

models by computing LRTMBCO. To compute the LRTMBCO, we

first calculate the deviance, denoted by D, for each model:

D � �2LL, (3)

where (LL) denotes the log-likelihood. The equation proposes that

deviance equals twice the negative of the maximum value of the

log-likelihood (LL) of the corresponding model. Deviance is a

positive number quantifying the “badness” of fit or the “misfit” of

a model. The larger the value of deviance for the same data set, the

worse the fit of the model to that sample data.

Next, we compute the LRTMBCO as the difference between the

deviance for the null and full models. Under the null hypothesis,

LRTMBCO has a large sample chi-squared distribution (Wilks,

1938) as follows:

LRTMBCO � Dnull � Dfull � �

2(df), (4)

where the degrees of freedom, df, equal the difference in the number

of the free parameters between the two models. The larger values of

the LRTMBCO indicate that the null model, which estimates no indi-

rect effect, fits the data worse than does the full model in which the

indirect effect is freely estimated. To obtain a p-value for the

LRTMBCO, we calculate the upper tail probability from the chi-

squared distribution (i.e., the area larger than the observed chi-squared

statistic) with the specified degrees of freedom. While using a likeli-

hood ratio test is standard in SEM, the LRTMBCO uses the nonlinear

constraint of �1�2 � 0 that has not been previously explored for

testing the null hypothesis of no indirect effect. The nonlinear con-

straint of �1�2 � 0 is, in fact, much more difficult to implement than

is simply fixing a single parameter to a specified value (such as zero).

Because of this nonlinear constraint of �1�2 � 0, specialized algo-

rithms must be used (discussed below), which at present are not

implemented in all programs.

Additionally, the MBCO procedure can compare the fit of the

null and full models in terms of information fit indices such as the

AIC (Akaike, 1974) or BIC (Schwarz, 1978). Such indices con-

sider the fit of the model (as measured by the maximized values of

likelihood for the model) to the data while penalizing the model for

estimating more parameters. To clarify, consider the formulas for

AIC and BIC:

AIC � D � 2 k (5)

BIC � D � 2 k ln(n), (6)

where k is the number of free parameters (e.g., degrees of freedom)

and n is the sample size in a mediation model. Both the AIC and

the BIC penalize estimating more parameters in a model by adding

the term 2 k (for AIC) and 2 k ln(n) (for BIC) to deviance. If two

models have the same deviance, the AIC favors the model with

fewer free parameters (i.e., the more parsimonious model). For the

BIC, if two models have the same deviance and sample size, the

BIC favors the more parsimonious model. A lower AIC or BIC

value indicates a better fit.

The MBCO procedure and LRTMBCO have not previously been

explored for testing the null hypothesis of no mediation for a few

reasons. First, the nonlinear constraints require a sophisticated

optimization algorithm (e.g., Zahery, Maes, & Neale, 2017). To

our knowledge, the only structural equation modeling programs

that are equipped to implement the nonlinear constraint are

OpenMx (Boker et al., 2011; Neale et al., 2016) and SAS PROC

CALIS (SAS Institute Inc., 2016). Second, researchers commonly

rely on the CI-based tests because other researchers recommend

them (e.g., MacKinnon et al., 2004; Preacher & Hayes, 2008;

Shrout & Bolger, 2002). Third, few researchers have realized the

infinite number of ways to realize and test the null hypothesis of no

indirect effect, such as H0: �1�2 � 0. For example, �1 can be zero

but �2 may take on any value; �2 can be zero but �1 may take on

any value; or �1 and �2 are both zero. Thus, in an infinite number

of ways, mediation does not happen. The MBCO procedure offers

flexibility in testing and improvement in evaluating simple and

complex hypotheses about indirect effects, which we demonstrate

next using an empirical example.

Empirical Example of Using the MBCO Procedure on

a Complex Mediation Model

The MBCO procedure can be applied to a mediation model with

two parallel mediators; in fact, the MBCO procedure can test a

variety of simple and complex hypotheses about indirect effects

and can offer flexibility in testing and improvement in evaluating

these hypotheses. Using R (R Core Team, 2019) and the OpenMx

(Boker et al., 2011; Neale et al., 2016) package, we apply and

illustrate parts of the code and output to a sample problem. Full

descriptions of the OpenMx code and output are given in the

online supplemental materials.

Figure 2. A randomized parallel (covarying) two-mediator model. In-

struction (X) is a random assignment with two conditions: instruction to use

mental imagery rehearsal (X � 1) versus instruction to use repetition (X �

0). Repetition is a self-reported score of using a repetition rehearsal to

memorize words. Imagery is a self-reported score of using mental imagery

to memorize words. Recall is the number of words out of a total of 20

words that each student correctly remembered at the end of the experiment

and is the outcome variable of interest. A solid arrow from one variable to

another indicates a direct effect of the variable on the other variable. The

curved double-headed arrow shows covariance between two residual terms

associated with the mediators, εM1 and εM2. The numbers next to each

arrow denote regression (path) coefficient estimates while the numbers in

parentheses denote the respective standard errors. The ε terms denote

residuals, which we assume are normally distributed.

T

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

4 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/met0000259.supp

MacKinnon, Valente, and Wurpts (2018) discussed a study

on using mental imagery to improve the memorization of

words. Table 1 shows descriptive statistics for this empirical

example. Oliver, Bays, and Zabrucky (2016) showed individu-

als who are instructed to create mental images of words appear

to recall more of them than do individuals who were simply

instructed to remember the words. MacKinnon et al. (2018)

conducted another study to test the mediation hypothesis that

instructions to participants to create mental images of words

would increase their using mental imagery and, in turn, would

increase the number of words participants recalled (see Figure

2). The study was replicated eight times, each with a different

group of undergraduate students enrolled in introductory psy-

chology courses; the sample sizes were 77, 43, 24, 79, 22, 45,

35, and 44. At the beginning of each assessment, the instructor

informed the students that they would hear a number of words

and that they were to memorize the words using techniques

described in the instructional sheets that they would receive.

The students then were randomly assigned to receive instruc-

tions on one of two different memorization techniques: repeti-

tion or imagery rehearsal. The students who received repetition

instructions were asked to memorize each word by repeating it

to themselves. The students who received imagery instructions

were asked to memorize each word by forming a mental image

of the word along with other words they had heard during the

experiment. For example, upon hearing the word camel and

after hearing the word women, they might imagine a woman

riding a camel. The students listened to 20 words with a 10-s

interval to rehearse each word. Ten seconds after hearing the

last word, the students were asked to write down as many words

as they remembered. Next, the students were all asked to (a)

rate the degree to which they formed images of the words in

order to memorize the words using a scale of 1 � not at all to

9 � absolutely; and (b) rate the degree to which they memo-

rized each word by repetition on a scale of 1 � not at all to 9 �

absolutely.

For this mediation model, we were interested in testing the

following hypotheses:

Research Question 1: Does the instruction to create mental

images of words increase use of mental imagery that, in turn,

increases the number of words recalled?

Research Question 2: Does the instruction to repeat words

increase use of repetition to memorize the words that, in turn,

increases the number of words recalled over and above using

mental imagery?

Research Question 3: Is the indirect effect of instruction to

create mental images on the number of words recalled through

use of mental imagery greater than the indirect effect of

instruction to repeat words on the number of words recalled

through use of repetition?

To answer each research question, we implemented the MBCO

procedure in three steps:

Step 1: Specify and estimate the full mediation model.

Step 2: Specify and estimate the null mediation model.

Step 3: Compare fit of the full and null models using the MBCO

procedure.

For Research Question 1, we briefly explain OpenMx (Boker et

al., 2011; Neale et al., 2016) syntax for conducting the MBCO

procedure and provide the relevant part of OpenMx code and

output. For Research Questions 2 and 3, we summarize the MBCO

procedure statistical results.

2

Research Question 1

Because we were interested in testing whether the indirect effect

of instruction of recall through imagery is different from zero, we

answered this question by using the single-mediator model (see

Figure 1). We briefly introduced the key parts of OpenMx code

(Boker et al., 2011; Neale et al., 2016) and R (R Core Team, 2019)

and then followed the three steps outlined in the article to imple-

ment the MBCO procedure.

Before reporting the formal modeling process, we show a glimpse

of the data set and its structure using the glimpse function from the

dplyr package (Wickham, François, Henry, & Müller, 2019):

2 All OpenMx scripts for the model along with a detailed explanation are

provided in the online supplemental materials, which can also be found at

https://github.com/quantPsych/mbco. For a comprehensive list of resources

about the OpenMx software, we encourage readers to visit the project

website https://openmx.ssri.psu.edu. For a detailed document on each

function within R, use the help() function. For example, type help

(mxModel) within the R console.

Table 1

Means, Standard Deviations, and Correlations With Confidence Intervals

Variable M SD 1 2 3

1. Repetition 6.08 2.84

2. Recall 12.07 3.4 �.28 [�.37, �.19]

3. Imagery 5.66 2.96 �.56 [�.63, �.49] .51 [.43, .58]

4. Instruction 0.51 0.5 �.67 [�.72, �.61] .32 [.22, .41] .62 [.55, .68]

Note. M and SD are used to represent mean and standard deviation, respectively. Instruction (X) is a random

assignment with two conditions: instruction to use mental imagery rehearsal (X � 1) versus instruction to use

repetition (X � 0). Repetition is a self-reported score of using a repetition rehearsal to memorize words. Imagery

is a self-reported score of using mental imagery to memorize words. Recall is the number of words out of a total

of 20 words that each student correctly remembered at the end of the experiment and is the outcome variable of

interest. Values in square brackets indicate the 95% confidence interval for each correlation.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

5IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

https://github.com/quantPsych/mbco

https://openmx.ssri.psu.edu

Step 1

The MBCO procedure begins by specifying the full single-mediator model (see Figure 1). We fit the full single-mediator model using

the code shown below:

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

6 TOFIGHI AND KELLEY

In the first two lines of the script above, before specifying the

model parts with mxModel, we can simplify the process by

grouping the names of variables as a vector that will be used in

model specification. For example, we specified a vector of the

names of observed variables and saved them as maniVar. Then,

we specified a vector of the names of endogenous variables and

saved them as endVar. Next, we specified the full single-

mediator model.

The main command to specify an SEM is mxModel. The

arguments provided to the mxModel function specified all the

elements of the mediation model. For the single-mediator example,

we specified the manifest (observed) variables, the paths (regres-

sion coefficients), the indirect effect to be tested (or any function

of model parameters), the constraint distinguishing the full and

null models, and the variances and covariances among the vari-

ables. The first argument to mxModel is single_med_full,

which is a name we chose for the single-mediator model. The next

argument, type=”RAM”, specifies that OpenMx uses the retic-

ular action model (RAM; McArdle & McDonald, 1984), a sym-

bolic algebraic notation to specify an SEM. In the argument

manifestVars, we introduced the vector of the names of the

observed (manifest) variables. The variable names must match the

names in the data set memory_df, as shown earlier in the output

from the glimpse function.

Next, we specified the paths between the variables using

mxPath, a function that corresponds to the graphical representa-

tion of paths in an SEM such as the model in Figure 1. For

example, we used mxPath to indicate a path (coefficient) corre-

sponding to an arrow between the two variables specified in the

arguments from (predictors) and to (response variables) in Fig-

ure 1. The argument arrows=1 indicates a unidirectional arrow

that starts from the variable in the argument from and ends at the

variable specified in the argument as to. The argument

arrows=2 indicates a bidirectional arrow representing a covari-

ance between the two variables. The argument free=TRUE

indicates that the parameter is freely estimated; otherwise,

free=FALSE indicates that the parameter is fixed at the values

set by the argument values. If the parameter is freely estimated,

the argument values would provide starting values. The argu-

ment labels provides labels for the coefficients. Because we

specified more than one coefficient in the arguments to and

from, we could provide a vector of labels corresponding to the

stated order of the coefficients. For our example, b1 is the coef-

ficient for X ¡ imagery and b3 is the coefficient for X ¡ recall.

We used the function mxAlgebra to define the indirect effect

that, in general, is a function of model parameters. A function may

include combinations of mathematical operations, such as �, �, �,

and / (e.g., b1�b2/(b1�b2+b3)), exponentials (e.g., exp), and

logarithms (e.g., log). The first argument to mxAlgebra was the

product of two coefficients, b1�b2, in which b1 and b2 had been

defined in mxPath. The argument name=”ind” named the indirect

effect. Next, we specified the data set for the model. The mxData

identified the data set to be analyzed. The argument observed=

memory_df specified the name of the data set in R. The second

argument, type=”raw,” indicated that the data set was in the raw

format, which meant that the data set included observations on the

participants as opposed to being a summary statistic such as a cova-

riance matrix.

Finally, we ran the model using mxRun, where the first argument

was the name of the mxModel that was then saved as fit_single_

med_full. Because we received a warning (not shown here) from

the optimizer that the convergence criterion was not satisfied, we used

the function mxTryHard. This function would either run the model

multiple times until an acceptable solution, according to the conver-

gence criterion set by the estimation algorithm, was found or it would

run a preset maximum number of times until an acceptable solution

was reached. In the subsequent analysis, we first used mxRun and, if

the model had convergence issues, we then used mxTryHard. We

used the function summary to save or print the summary of the

results. We saved the summary of the results as stat_single_

med_full and then printed the summary. Below, is the relevant part

of the summary results.

Below the title Model Statistics, the row that starts with

Model gives the pertinent information for the full model. The output

showed that the full single-mediator model had eight free parameters,

with dfFull � 1099 and DFull � 4305.727. Under Information

Criteria, we present the estimates of the information indices. The

information indices that match the formulas in (5) and (6) are under

the Parameters Penalty column. For the full model, the infor-

mation fit indices were AICFull � 4321.727 and BICFull � 4353.013.

Before proceeding, after fitting a mediation model, we checked

the degree to which the statistical assumptions about normality of

the residuals and the presence of outliers to ensure that the esti-

mates were unbiased and the inference about the parameters was

valid (Cohen, Cohen, West, & Aiken, 2003). We checked for

normality of the residuals using QQ plots, which indicated that the

normality assumption for the residuals was reasonable. Using the

methods described by Fox (2016), we did not find any outliers that

would change the results.

Step 2

We ran the null model (code below), which is the single-mediator

model in which the indirect effect of instruction on recall through

imagery is constrained to zero, �1�2 � 0.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

7IMPROVED INFERENCE IN MEDIATION ANALYSIS

Instead of specifying all parts of the null mediation model using

MxModel in the above code, we modified the full model

single_med_full, as specified in the argument model. Next,

we specified the nonlinear constraint for the indirect effect through

mxConstraint. The first argument, ind = = 0, constrains the

indirect effect defined in the mxAlgebra statement to zero. The

argument name assigns a name to the constraint. Finally, we saved

the null model to single_med_null. Next, we ran the null

model and saved the results to fit_single_med_null. Rele-

vant parts of the summary of the model results are shown below:

For the null model, dfNull � 1100 and DNull � 4481.493, and the

information fit indices were AICNull � 4497.493 and BICNull �

4528.779.

Step 3

We compared the full and null model both in terms of

the LRTMBCO and the information fit indices to evaluate H0:

�1�2 � 0. The LRTMBCO equals the difference between the

deviance of the two models: LRTMBCO � DNull � DFull �

4481.493 � 4305.727 � 175.766. The degrees of freedom for the

LRTMBCO equal the difference in the degrees of freedom associ-

ated with each model; that is, dfLRT � dfNull � dfFull � 1100 �

1099 � 1. Given that the LRTMBCO has a large sample �

2

distribution with dfLRT � 1, we have the p-value � Pr(�

2(1) �

175.766) � 4.073738E � 40. More conveniently, we computed

the LRTMBCO using the mxCompare or anova function where

the arguments are the names of the full and null models:

The first row of the above output shows the results for the full

model, which is the single-mediator model in Figure 1. The col-

umns ep, minus2LL, df, and AIC show the number of estimated

parameters, deviance, degrees of freedom, and AIC, respectively.

The columns diffLL, diffdf, and p represent the difference in

deviance (not the log-likelihoods), the difference in degrees of

freedom, and p-value for the two models being compared. The

second row shows the results for the null model under the columns

ep, minus2LL, df, and AIC. The results of comparing the null

and full model are shown under the columns diffLL, diffdf,

and p. The value for the test statistic LRTMBCO � 175.766 is

located under the column diffLL. The degrees of freedom are

dfLRT � 1 and the p-value � 4.073738E � 40; these amounts are

located under the columns diffdf and p, respectively.

We used the following commands to compute the Monte Carlo

CI for the indirect effect. First, we used the functions coef and

vcov to extract the path coefficients and covariance matrix of the

coefficients, respectively. Next, we used the ci function in the

RMediation package (Tofighi & MacKinnon, 2011, 2016) to com-

pute the Monte Carlo CI. The first argument mu to this function is

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

8 TOFIGHI AND KELLEY

a vector of the coefficient estimates, and the second argument

Sigma is a covariance matrix of the coefficient estimates. The

argument quant accepts a formula for the indirect effect that starts

with the symbol “ ”.

The numbers below 2.5% and 97.5% show the lower and upper

limits of the 95% CI, [1.6, 2.682]; the numbers below Estimate

and SE show the estimates of indirect effect and its standard error,

�̂1�̂2 � 2.121 and SE � 0.276, respectively.

We next obtained R2 for the endogenous variables of the full and

null models using the rsq function available in the online sup-

plemental materials.

The first argument model to rsq specifies an OpenMx

model, and the second argument name specifies names of

endogenous variables (i.e., variables in which a single-headed

arrow enters; namely those that are a function of another

variable).

To summarize, the LRTMBCO result showed that the indirect

effect of instruction on recall through imagery appeared to be

greater than zero, �̂1�̂2 � 2.121 (SE � 0.276), 95% Monte

Carlo CI [1.6, 2.682], LRTMBCO � 175.766, dfLRT � 1, p �

4.073738E � 40. We recommend researchers compute the

difference in R2s between the full and null models and then

examine the change in the effect sizes that occurs as a result of

the indirect effect through imagery. For recall, R2 remained

unchanged to four decimal places while for imagery

R2 �

.3779. Further, the information fit indices supported the asser-

tion that the indirect effect was greater than zero because the

AIC and the BIC for the full single-mediator model were

smaller than those of the null single-mediator model. This result

indicates that constraining the indirect effect to zero worsens

the fit of the full single-mediator model. On average, the

instruction to students to use mental imagery increased use of

mental imagery that, in turn, increased the number of words

they recalled by approximately two words. We have inferential

support that mediation occurred. Note these results are valid if

the no-omitted confounder assumption is met. That is, an omit-

ted variable may not exist that influences the relations between

instruction, imagery, and recall.

Research Question 2

Here, we were interested in whether the instruction to use

repetition increased the use of repetition to memorize the words

that, in turn, improved participants’ memory over and above the

indirect effect of instruction on recall through imagery. In other

words, we wanted to test H0: �4�5 � 0 for the parallel two-

mediator model in Figure 2. We again applied the three-step

MBCO procedure.

Step 1

We estimated the model with two parallel mediators in Figure 2,

which is the full model for this research question. For this model,

the two specific indirect effects associated with imagery and

repetition were freely estimated. Below are the regression equa-

tions for the full parallel two-mediator model:

M1 � �0,M1 � �1X � εM1 (7)

M2 � �0,M2 � �4X � εM2 (8)

Y � �0,Y � �3X � �2M1 � �5M2 � εY , (9)

where �1 is the effect of the instruction (X) on imagery (M1), �4 is

the effect of the instruction on repetition (M2), �3 is the direct

effect of the instruction on recall (Y), �2 is the effect of imagery on

recall controlling for the instruction and repetition, and �5 is the

effect of repetition on recall controlling for instruction and imag-

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

9IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

http://dx.doi.org/10.1037/met0000259.supp

ery; �0,M1, �0,M2, �0,Y are the intercepts; εM1, εM2, and εY are the

residuals.

We fitted the full two-mediator model within OpenMx. The

results showed that the full two-mediator model had 13 free

parameters, with dfFull � 1463 and DFull � 5874.685. For the full

model, the results were that AICFull � 5900.685 and BICFull �

5951.526; the effect sizes were RImagery2 � .38, RRepetition2 � .45, and

RRecall

2 � .26. We computed the specific indirect effects through

imagery and repetition and the 95% CI for each specific indirect

effect using the ci function in the RMediation package. The

results showed that the specific indirect effect through imagery

was �̂1 �̂2 � 2.139 (SE � 0.284), 95% Monte Carlo CI [1.602,

2.716], and the specific indirect effect through repetition was

�̂4 �̂5 � �0.082 (SE � 0.285), 95% Monte Carlo CI [�0.643,

0.477].

Step 2

Recall that we were interested in whether instruction to use

repetition increased the use of repetition to improve memory over

and above the indirect effect of instruction on recall through

imagery. Thus, in the null parallel two-mediator model, we con-

strained the specific indirect effect through repetition to zero while

letting the specific indirect effect through imagery be freely esti-

mated. The null parallel two-mediator model was estimated by

fitting the equations in (7)–(9) subject to the null hypothesis

constraint H0: �4�5 � 0, which fixed the specific indirect effect

through repetition to zero. In OpenMx, we specified the null model

by adding the nonlinear constraint �4 �5 � 0 to the full model in

Step 1. The results showed that the null parallel two-mediator

model had dfNull � 1464 and DNull � 5874.768. The information

fit indices for the null two-mediator model were AICNull �

5900.768 and BICNull � 5951.609. The effect sizes were RImagery

2 �

.38, RRepetition2 � .45, and RRecall2 � .26.

Step 3

We compared the two models and computed LRTMBCO. The

results showed that LRTMBCO � 0.083, dfLRT � 1, and p � .773.

The specific indirect effect through Repetition was, therefore, not

different from zero, �̂4�̂5 � � 0.08 (SE � 0.29), 95% Monte Carlo

CI [�0.64, 0.48]. Further, the R2 for imagery, repetition, and recall

remained unchanged to three decimal places, and the information

fit indices between the two models were roughly the same. These

results indicate that the specific indirect effect through repetition

above and beyond the specific indirect effect through Imagery

does not appear to be different from zero.

Research Question 3

For this question, we were interested in comparing the sizes of

the two specific indirect effects: the indirect effect of instruction

on recall through imagery (i.e., �1�2) and the indirect effect of

instruction on recall through repetition (i.e., �4�5). The null hy-

pothesis for this research question is:

H0: �1�2 � �4�5 (10)

To test this null hypothesis, we again employed the three-step

MBCO procedure.

Step 1

The full model for this research question was the same as the

full model in Research Question 2; that is, they matched the

parallel two-mediator model in (7)–(9). Thus, we used the results

(i.e., the vector of the coefficient estimates, the covariance matrix

of the coefficient estimate, R2, indirect effects estimates, AIC, and

BIC) of the full parallel two-mediator model from Research Ques-

tion 2.

Step 2

The null model was the parallel two-mediator model in (7)–(9)

subject to the null hypothesis constraint in (10). That is, we

estimated the two-mediator model while we constrained the two

specific indirect effects to be equal. Alternatively, we could spec-

ify the contrast of the two specific indirect effects to zero. We fitted

the new null model in OpenMx. The results showed that the null

model had dfNull � 1464 and DNull � 5900.514. The information fit

indices for the null model were AICNull � 5926.514 and BICNull �

5977.354. The effect sizes were RImagery2 � .38, RRepetition2 � .45, and

RRecall

2 � .21.

Step 3

We compared the full parallel two-mediator model in Step 1 and

the null parallel two-mediator model in Step 2. We computed the

LRTMBCO as well as the 95% Monte Carlo CI for the contrast of

the two indirect effects using the ci function in the RMediation

package. The results of the MBCO procedure showed that

LRTMBCO � 25.828, dfLRT � 1, and p � 3.731857E � 07. These

outcomes indicate that the indirect effect through imagery ap-

peared to be larger than the indirect effect through repetition by

�̂1�̂2 � �̂4�̂5 � 2.222 (SE � 0.445) words, 95%

Monte Carlo CI

[1.364, 3.11]. In comparing the R2s for the endogenous variables

obtained from Step 1 and 2 for imagery and repetition, R2s re-

mained unchanged to three decimal places while for recall

R2 �

.05. Comparing the information fit indices of the null model with

the full model also supported the conclusion that the full model

fitted the data better than did the null model.

Comparing Methods of Testing Indirect Effect Via

Monte Carlo Simulations

In this section, we describe simulation studies assessing the

Type I error rates and the statistical power of the LRTMBCO of our

proposed MBCO procedure; we then compare the Type I error

rates and power of LRTMBCO to those of the currently recom-

mended methods of testing indirect effects. Monte Carlo simula-

tion studies are necessary because there are no known ways of

assessing the performance via mathematical proofs or derivations.

In our simulations, we generated data from known population

models, fitted the same models to the data, repeated this process

many times, and assessed the properties of the different methods of

testing indirect effects.

The simulation studies were designed to answer the following

questions:

1. Does the LRTMBCO provide more robust Type I error

rates than the recommended methods? In particular,

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

10 TOFIGHI AND KELLEY

would the LRTMBCO remain robust across the combina-

tion of the parameter values for smaller sample sizes?

2. Is the LRTMBCO as powerful as, if not more powerful

than, the currently recommended methods across differ-

ent effect sizes, sample sizes, and other types of media-

tion models?

To answer these questions, we compared the Type I error rate

and power of the LRTMBCO to five currently recommended meth-

ods of testing indirect effects: (a) the percentile bootstrap CI

(Bollen & Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et

al., 2004); (b) the bias-corrected (BC) bootstrap CI (Bollen &

Stine, 1990; Efron & Tibshirani, 1993; MacKinnon et al., 2004);

(c) the Monte Carlo CI (MacKinnon et al., 2004; Tofighi &

MacKinnon, 2016); (d) the profile-likelihood CI (Folmer, 1981;

Neale & Miller, 1997; Pawitan, 2001; Pek & Wu, 2015); and (e)

the joint significance test (Kenny et al., 1998; MacKinnon et al.,

2002). We describe these methods in the next section.3

For a two-path indirect effect, we did not separately consider

distribution of the product CI because its performance is similar to

that of the Monte Carlo CI (MacKinnon et al., 2004). We also did

not study the Wald (1943) (multivariate delta) z or the causal steps

test because research has shown that these two tests are overly

conservative and do not provide adequate power in smaller sample

sizes and effect sizes (MacKinnon et al., 2002). For completeness

of the simulation study and because of its popularity and endorse-

ments (MacKinnon et al., 2004; Preacher & Hayes, 2008; Shrout

& Bolger, 2002), we studied the bias-corrected (BC) bootstrap CI.

However, we do not recommend using the BC bootstrap CI to test

an indirect effect because it shows inflated Type I error rates for

both small and large samples (Falk & Biesanz, 2015; Koopman,

Howe, Hollenbeck, & Sin, 2015).

In the simulation studies, we considered two types of indirect

effects commonly found in empirical research as well as in sim-

ulation studies (Tofighi & Kelley, 2019): (a) a two-path indirect

effect (e.g., �1�2), which is the product of two coefficients for a

single-mediator chain,

X ¡

�1

M ¡

�2

Y; and (b) a three-path indirect

effect (e.g., �1�2�3), which is the product of three coefficients for

a sequential two-mediator chain, X ¡

�1

M1 ¡

�2

M2 ¡

�3

Y. We chose

these two population models because they have been extensively

discussed in both substantive and methodological literature (To-

fighi & Kelley, 2019).

Percentile and Bias-Corrected Bootstrap CI

We now discuss two methods of computing a CI for nonpara-

metric bootstrap (Bollen & Stine, 1990; Efron & Tibshirani, 1993).

In this technique, many samples (e.g., R � 1,000) with replace-

ment were drawn from the sample data. The hypothesized medi-

ation model was fit to each resampled data set, and indirect effects

were computed for each model. This process resulted in R esti-

mates of indirect effects, which is called the bootstrap sampling

distribution of the indirect effects. To compute a CI, we obtained

quantiles corresponding to 1 � �/2 and �/2 percentiles of the

bootstrap sampling distribution, resulting in what is called a per-

centile CI. As a modified version of the percentile bootstrap, the

BC bootstrap uses adjusted percentiles 1 � �=/2 and �=/2 to obtain

the upper and lower limits of the CI, where �= is computed to

correct for potential bias due to skewness (Efron, 1987). Although

not in the context of mediation analysis, Efron (1987) argued that,

for smaller sample sizes, the BC bootstrap CI yields a more

accurate coverage (i.e., the proportion of times a CI contains true

values of the parameter is closer to the nominal coverage of 1 �

�) than does the percentile CI.

Monte Carlo CI

The Monte Carlo method, also known as the parametric boot-

strap (Efron & Tibshirani, 1993), is another sampling-based tech-

nique used to compute a CI for indirect effects (MacKinnon et al.,

2004; Tofighi & MacKinnon, 2016). In this method, the sampling

distribution for each parameter is estimated by drawing R random

samples from a multivariate normal distribution where the mean of

the distribution is the vector of the coefficient estimates and the

covariance matrix of the distribution is the covariance matrix of

the coefficient estimates. The Monte Carlo method is based on the

theory that the maximum likelihood (ML) estimates of the coef-

ficients in an SEM asymptotically have a multivariate normal

distribution (Bollen, 1989). The population parameters (i.e., mean

vector and covariance matrix) for this multivariate normal distri-

bution are the parameter estimates from the estimated model. The

number of Monte Carlo samples, R, should be large, typically

1,000 or more. The indirect effect is estimated within each sample,

resulting in a total of R estimates. The mean and standard deviation

of the R indirect effect estimates are then used to estimate the

indirect effect and its standard error, respectively. To compute a

(1 � �)100% CI, we obtained the �/2 and 1 � �/2 quantiles of the

Monte Carlo sample of the indirect effects.

Profile Likelihood CI

In its simplest form, the profile-likelihood approach produces a

CI for a single parameter using a profile-likelihood function (Fol-

mer, 1981; Neale & Miller, 1997; Pawitan, 2001). In mediation

analysis, the profile-likelihood method has been extended to pro-

duce a CI for the indirect effect in a single-mediator chain

(Cheung, 2007; Pek & Wu, 2015). Let � be a vector of all the free

parameters in a single-mediator model. The profile-likelihood

function for a single-mediator model, L(� | �1�2), is computed by

assuming that the indirect effect, �1�2, is a known value. Next, the

profile-likelihood function is maximized. In practice, the indirect

effect takes on different values, and the profile-likelihood function

is maximized while the indirect effect is fixed at specific values.

Next, we compared the deviance of the maximized profile-

likelihood function and the deviance of the maximized likelihood

function, where deviance equals negative twice the maximized

log-likelihood function as shown in (3). Asymptotically, the dif-

ference in the deviance of the two functions had a chi-squared

distribution with degrees of freedom equaling the difference in the

number of free parameters between the profile likelihood and

likelihood function for the model (Pawitan, 2001). For the indirect

effect, in general, this difference asymptotically had a chi-squared

distribution with one degree of freedom. The lower and upper

3 We do not name all the existing methods for single-mediator models

because they are considered elsewhere (e.g., Falk & Biesanz, 2015; Mac-

Kinnon et al., 2002).

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

11IMPROVED INFERENCE IN MEDIATION ANALYSIS

limits for the 100 (1 � �)% profile-likelihood CI corresponded to

the minimum and maximum of all values of the indirect effect that

satisfied the following equality:

D(�̂prof | �1�2) � D(�̂) � ��2 (1) (11)

where �̂prof is the ML estimate of the model parameters given that

the indirect effect is fixed and ��2 (1) denotes upper-tail � critical

value of the chi-squared distribution with one degree of freedom.

For a specific mediation model, the values of D��̂� (computed by

estimating the mediation model) and the critical value of the

chi-squared distribution (e.g., �.05

2 (1) � 3.84) were known. A 95%

profile-likelihood CI was computed by solving the expression in

(11) for two values of �1�2 corresponding to the upper and lower

limits.

Joint Significance Test

For a single-mediator model, MacKinnon, Lockwood, Hoffman,

West, and Sheets (2002) proposed a joint significance test to

describe a variation of the causal steps test (Kenny et al., 1998).

This method declares an indirect effect to be significantly different

from zero if every coefficient in the indirect effect is significantly

different from zero. The joint significance test does not produce a

robust Type I error rate for smaller sample sizes, a CI, or a p value

for the indirect effect (MacKinnon et al., 2002). While the joint

significance test can be extended to test indirect effects that are the

products of two or more coefficients in a mediational chain, the

method cannot be used to test a complex function of indirect

effects such as a contrast of two indirect effects (e.g., H0: �1�2 �

�3�4 � 0). The joint significance test has been recommended to

test indirect effects in single-mediator and two-mediator sequential

chains (Falk & Biesanz, 2015; Taylor, MacKinnon, & Tein, 2008;

Yzerbyt, Muller, Batailler, & Judd, 2018).

Simulation Design and Population Values

In the simulation studies, we manipulated three factors: (a)

effect size R2, which quantifies how well the predictors account

for the variance in the endogenous variables; (b) sample size;

and (c) the method of testing the indirect effect(s). Previous

work showed that sample size and the effect size R2 influence

the Type I error and the power of the existing tests of the

indirect effect for single-mediator and sequential two-mediator

chains (MacKinnon et al., 2002, 2004; Williams & MacKinnon,

2008). Based on Cohen’s (1988) guidelines, we specified the

population effect sizes in the simulation to be R2 � .02, .13, and

.26, which leads to the corresponding nonzero population re-

gression coefficients of 0.14, 0.39, and 0.59, respectively (see

the method described by Thoemmes, MacKinnon, & Reiser,

2010, which we used).

Sample size took on the following values: N � 50, 75, 100,

and 200. We chose these values to cover a range of sample sizes

reported in the applied literature of psychology, in related

disciplines, and in the simulation studies of mediation effects

(e.g., MacKinnon et al., 2002; Tofighi & Kelley, 2019). The

smallest sample size we considered, 50, might not be viewed as

a best practice in the SEM literature, and a sample size of 200

is roughly equal to the median sample size in an SEM (Jaccard

& Wan, 1995; MacCallum & Austin, 2000). However, the

studies with a sample size of 50 do appear in the applied

literature (Tofighi & Kelley, 2019). We do not report results

from a sample size greater than 200 because our preliminary

simulation study for the LRTMBCO as well as the previous

simulation studies for the existing methods (e.g., MacKinnon et

al., 2002, 2004; Taylor et al., 2008; Williams & MacKinnon,

2008) found that the performance of the tests did not differ

between methods at larger sample sizes. In these large samples,

all the methods (which are based on large sample theory)

worked effectively.

The method conditions included six commonly used tests of

indirect effects for both the Type I error and power study. These

conditions resulted in 168 conditions for the Type I error and 216

conditions for the power study for the single-mediator model; and

240 conditions for the Type I error and 648 conditions for the

power study for the sequential two-mediator model. We used a

mixed full factorial design for each simulation study where the

factors effect size and sample size were between-subjects factors

and the method was a within-subjects factor.

Data Generation

We generated 5,000 independent data sets for each combina-

tion of the between-subjects factors using a known population

model, either X ¡

�1

M ¡

�2

Y for the single-mediator model or

X ¡

�1

M1 ¡

�2

M2 ¡

�3

Y for the sequential two-mediator model. The

variables X, M1, M2, and Y were observed. Without loss of gen-

erality, the population values for the intercepts were fixed at zero

but were estimated in the simulation study. Values for X were

sampled from a binomial distribution with .5 probability of two

categories (i.e., treatment vs. control). Data for each residual term

were generated from the standard normal distribution.

We chose � � .05 to test the indirect effects, as is commonly

done in psychology and related areas. We used OpenMx Version

2.9.6 (Boker et al., 2011; Neale et al., 2016) to conduct the

LRTMBCO as well as the profile-likelihood CI.

4 For the Monte

Carlo, percentile and BC bootstrap CI, and for the joint signifi-

cance test, we estimated the model using lavaan package Version

0.5–23 (Rosseel, 2012). We then used the model estimates and ci()

function in the RMediation package Version 1.1.4 (Tofighi &

MacKinnon, 2011, 2016) to compute a Monte Carlo CI; both

4 We studied the software manual and references for the optimization

techniques implemented in OpenMx (Boker et al., 2011; Neale et al., 2016;

Zahery et al., 2017), communicated with the authors/programmers, and

conducted an extensive simulation study to examine whether the LRTMBCO

can be implemented using different optimization methods in OpenMx. In

addition, we examined whether the software packages lavaan Version

0.5-23.1097, Mplus Version 8.0, and PROC CALIS (SAS Institute Inc.,

2016) allowed nonlinear constraints to estimate the LRTMBCO. We also

communicated with the authors/programmers of Mplus and lavaan. Based

on our communications, at the time of writing, Mplus and lavaan do not

guarantee obtaining optimal results for the mediation models with nonlin-

ear constraints in the form of null hypothesis about the indirect effect. We

chose the NPSOL (Gill, Murray, Saunders, & Wright, 1986) optimizer

instead of the default CSOLNP (Zahery et al., 2017) optimizer in OpenMx

because the CSOLNP showed convergence problems in our preliminary

simulation studies. Based on our personal communication with OpenMx

developers, they suggested that the SLSQP (Snyman, 2005) optimizer may

be successful as well. However, we did not formally evaluate this option

because it is beyond the scope of the current article.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

12 TOFIGHI AND KELLEY

percentile and BC bootstrap CI were estimated using the facilities

provided in the lavaan package. For the joint significance test, we

used the z test to examine the joint significance of the coefficients

of the indirect effects.5

Results

For the simulation studies assessing the robustness of the Type

I error, the empirical Type I error rate is the proportion of times out

of 5,000 replications that a test incorrectly rejects the null hypoth-

esis of zero indirect effect. For the statistical power studies, the

empirical power is the proportion of times out of 5,000 replications

that a test correctly rejects the null hypothesis of zero indirect

effect. The large sample size of 5,000 within each condition

precluded the use of repeated measures ANOVA for the simulation

results. To determine if a test was robust, as discussed, we used

Bradley’s (1978) liberal interval of .025 and .075 to determine a

Type I error rate that was “good enough” for practical purposes.

We considered a test robust (i.e., good enough) according to

Bradley’s (1978) criteria if its empirical Type I error rate fell

within the robustness interval. Otherwise, a test was considered

conservative or liberal depending on whether the Type I error rate

was smaller or greater than Bradley’s lower or upper bound,

respectively.

Type I error. To facilitate comparison of the Type I error

rates across different combinations of the sample sizes and effect

sizes, we created graphs showing the empirical Type I error rate as

a function of sample size and size of nonzero coefficients. To save

space, we only present a few graphs (Figures 3– 6). More graphs

can be found in the online supplemental materials. For the two-

path indirect effect where both coefficients were zero (see Figure

3), the empirical Type I error rate for the LRTMBCO was robust as

it fell within Bradley’s (1978) robustness interval. The other five

methods were not robust across all sample sizes; that is, their

empirical Type I error rates were consistently below .01. The

LRTMBCO produced more robust Type I error rates because it

estimated the null mediation model and the sampling distribution

of the indirect effect under the null hypothesis.

As can be seen in Figure 4, when one of the coefficients was

zero, the LRTMBCO was the most robust method across all com-

binations of the nonzero coefficients and sample sizes; its Type I

error rate remained close to the nominal .05 value and stayed

within the limits of Bradley’s (1978) robustness interval. The BC

bootstrap showed conservative Type I error rates or inflated Type

I error rates in certain conditions. For example, when �2 � 0.59

and N � 50, and �2 � 0.39 and N � 100, the BC bootstrap showed

an inflated Type I error rate; when �2 � 0.14 and N � 75, the BC

bootstrap showed a conservative Type I error rate. In general, the

Type I error rates for the percentile bootstrap, Monte Carlo,

profile-likelihood, and joint significance methods were all robust

except when one of the nonzero coefficients was small (i.e., 0.14).

More specifically, when the magnitude of the coefficient was small

and the sample size was 100, all four methods showed a con-

servative Type I error rate. The results for CI-based and joint

significance tests, when one of the coefficients was zero, essen-

tially matched the results from previous studies (Biesanz et al.,

2010; Falk & Biesanz, 2015), as would be expected. Finally, which

coefficient was zero did not appear to change the overall conclu-

sion about the Type I error rate robustness of the methods.

For the sequential two-mediator model, the LRTMBCO was the

most robust test across the combinations of the sample sizes and

effect sizes. When all three coefficients were zero (see Figure 5) or

when two of the coefficients were zero (see Figure 6), the

LRTMBCO’s Type I error rate remained robust within the Bradley’s

(1978) limits. The five other methods showed conservative Type I

error rates. Finally, which coefficient was zero did not appear to

change the Type I error rates.

Power. The simulation study showed that the LRTMBCO was

as powerful as the other methods, except for the BC bootstrap

method, for both two-path and three-path indirect effects. The BC

bootstrap method showed slightly more power although that power

was at the expense of yielding inflated Type I error rates. This

result is consistent with previous research (Biesanz et al., 2010;

Falk & Biesanz, 2015; Taylor et al., 2008). The results of power

simulation studies were similar between the single-mediator model

and the sequential two-mediator model. We explain the similar

power between the CI-based methods and the LRTMBCO in the

next section.

Summary

One explanation for why the simulation studies showed that the

LRTMBCO produces more robust empirical Type I error rates than

other CI-based methods is that the sampling distribution of the

indirect effect tends to differ from the null sampling distribution of

the indirect effect estimate. In the null sampling distribution, the

population value of the indirect effect is fixed to zero. The sam-

pling distribution used by the CI-based methods is likely to reflect

an alternative hypothesis of a nonzero population indirect effect

rather than the null hypothesis of zero population indirect effect.

As a result, the CI-based methods tend to produce nonrobust

empirical Type I error rates, especially for smaller sample sizes.

However, the LRTMBCO uses both the null and the full models to

test an indirect effect and, thus, more appropriately maps the

statistical method onto the question of interest when seeking to

determine if a hypothesized mediator mediates the relationship

between X and Y. Hence, the LRTMBCO appears more appropriate

than other methods that have been recommended in the literature

in the conditions we studied.

The simulation results also showed that the LRTMBCO is as

powerful as the other recommended tests except for the BC boot-

strap method; however, we do not recommend the BC bootstrap

because of its inflated Type I error rate. On the other hand, the

CI-based tests have adequate power when the null hypothesis is

false (i.e., nonzero indirect effect) and the sampling distribution is

computed under an estimated alternative hypothesis in which the

indirect effect is nonzero. In other words, the estimated alternative

sampling distribution of CI-based methods is more accurate when

the null hypothesis is false as opposed to when the null hypothesis

is true. In these cases, the CI-based tests and LRTMBCO appear to

exhibit similar power.

5 The simulation study code scripts can be found at https://github.com/

quantPsych/mbco.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

13IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1037/met0000259.supp

https://github.com/quantPsych/mbco

https://github.com/quantPsych/mbco

Limitations

In our simulations, we made a few common assumptions for

estimating a mediation model. The data for the simulation studies

were generated assuming that residuals associated with the medi-

ators and outcome variable had a multivariate normal distribution

and that the correct model was fit (e.g., the variables were not

nonlinearly related and there were no omitted confounders).

Though assessing the effectiveness of the LRTMBCO under the

multivariate normality of the simulated data is a limitation, using

the multivariate normal data is necessary to compare performance

of the LRTMBCO to other commonly used methods of testing

mediation in known conditions. Therefore, the results of the sim-

ulation studies apply to situations in which these assumptions are

reasonably met. Assessing the performance of the LRTMBCO for

non-normally distributed residuals remains a topic for future re-

search. We also assumed the simulation study included correct

functional forms of the relationships between the posited variables.

For the empirical study, we evaluated if there were outliers (and

we did not find any in the sample data), and we assumed that no

common method biasing effect of measuring the endogenous vari-

ables existed (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003).

Moreover, we assumed that the observed mediator and outcome

variables were measured without error (Dwyer, 1983; Fritz,

Kenny, & MacKinnon, 2016); otherwise, the results could be

misleading (Cole & Preacher, 2014). In addition, one of the more

stringent of these assumptions, which is untestable, is the no-

omitted-confounder assumption (Imai et al., 2010; Judd & Kenny,

1981; Valeri & VanderWeele, 2013; VanderWeele, 2010). That is,

no variable should be omitted from the model that would affect

both the mediator and the outcome variables, given the indepen-

dent variable (X) and covariates (if they exist and are included in

the model). Even when participants are randomly assigned to a

treatment or control group, making causal claims about an indirect

effect requires the no-omitted-confounder assumption. If research-

ers believe that not all confounders are included in the model, then

the claims about the magnitude and existence of an indirect effect

need to be relaxed. We recommend that researchers conduct a

sensitivity analysis to investigate the biasing impact of a potential

Figure 3. Point and 95% CI estimate of the Type I error rate for six methods of testing a two-path indirect

effect, �1�2, where both parameters were fixed at zero. Horizontal solid lines show the limits of the Bradley’s

(1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s (1978) criterion

if its Type I error rate falls within the criterion’s interval. MBCO � model-based constrained optimization

(LRTMBCO); Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test;

Profile � profile-likelihood CI. See the online article for the color version of this figure.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

14 TOFIGHI AND KELLEY

confounder on the indirect effect point and interval estimate (Cox,

Kisbu-Sakarya, Miočević, & MacKinnon, 2013; Tofighi et al.,

2019; Tofighi & Kelley, 2016). MacKinnon and Pirlott (2015)

provide an excellent discussion of different approaches to sensi-

tivity analyses in mediation analysis. Sensitivity analyses for cer-

tain mediation models can also be conducted in an SEM frame-

work in Mplus software, which may include observed and latent

variables (Muthén & Asparouhov, 2015; Muthén & Muthén,

2018). Conducting sensitivity analyses using the LRTMBCO also

remains a topic for future study.

One requirement of our proposed LRTMBCO and the profile-

likelihood method is that the mediation model be estimated within

the SEM (or other appropriate multivariate) framework. That is,

the equations for the mediation model must be simultaneously

estimated. These methods do not work when using OLS regression

to separately estimate the equations representing a mediation

model, which is the classical way of testing mediation (Baron &

Kenny, 1986).

We discussed applying the LRTMBCO to the models when the

independent variable and mediator do not interact and when

the mediator and outcome are continuous. For such models,

both the SEM and causal inference methods provide the same

estimate of the indirect effect (Muthén & Asparouhov, 2015);

thus, the LRTMBCO can be used to test an indirect effect.

However, when the independent variable and a mediator inter-

act or when either a mediator or the outcome variable is a

categorical variable, researchers need to use the potential out-

come framework to correctly estimate the indirect effect

(VanderWeele, 2015). Extension of the MBCO procedure and

LRTMBCO to the causal inference framework for these scenarios

remains a topic for future study. Finally, we did not discuss

application of the MBCO procedure to multilevel mediation

Figure 4. Point and 95% CI estimate of the Type I error rate for six methods of testing a two-path indirect

effect, �1�2, where only �1 was fixed at zero. The nonzero parameters take on the values: 0.14, 0.39, and 0.59.

The x-axis shows the parameter values. Horizontal solid lines show the limits of the Bradley’s (1978) liberal

interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s (1978) criterion if its Type I error

rate falls within the criterion’s interval. MBCO � model-based constrained optimization (LRTMBCO); Percen-

tile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test; Profile � profile-

likelihood CI. See the online article for the color version of this figure.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

15IMPROVED INFERENCE IN MEDIATION ANALYSIS

analysis (Krull & MacKinnon, 2001; Tofighi, West, & MacK-

innon, 2013), where the data have a multilevel (i.e., clustered,

repeated measures) structure (Snijders & Bosker, 2012). Ex-

tending the MBCO procedure to multilevel data also remains a

topic for future study.

Discussion

This article proposed an MBCO procedure that uses a model-

comparison approach to make inferences about any function rep-

resenting an indirect effect in mediation analysis. An innovation of

our proposed MBCO procedure is using a nonlinear constraint to

test a variety of simple and complex hypotheses about indirect

effects in a model-comparison framework. The MBCO procedure

offers the following advantages compared with the existing meth-

ods. First, the MBCO procedure produces a likelihood ratio test,

termed LRTMBCO, to formally evaluate simple and complex hy-

potheses about indirect effects and produces a p value, a continu-

ous measure of compatibility between data and the null hypothe-

ses. Second, through the model-comparison framework, the

MBCO procedure computes a likelihood ratio, a continuous mea-

sure of comparing goodness of fit of the null and full models. It

also computes information fit indices such as AIC and BIC, used

to compare the null and full models based on both goodness fit and

parsimony.

To assess robustness of the LRTMBCO and the five most rec-

ommended methods of testing indirect effects in a single-mediator

or in a sequential two-mediator model, we conducted a Monte

Carlo simulation study. The results showed that the LRTMBCO is

more robust in terms of the empirical Type I error rate (i.e., it was

within Bradley’s, 1978 liberal interval of .025 and .075 for � �

Figure 5. Point and 95% CI estimates of the Type I error rate for six methods of testing a three-path indirect

effect, �1�2�3, where all three parameters were zero. Horizontal solid lines show the limits of the Bradley’s

(1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a Bradley’s criterion if its Type

I error rate falls within the criterion’s interval. MBCO � model-based constrained optimization (LRTMBCO);

Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS � joint significance test; Profile �

profile-likelihood CI. See the online article for the color version of this figure.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

16 TOFIGHI AND KELLEY

.05) than the other methods when the residuals were generated

under an ideal condition of the multivariate normality. In addition,

the LRTMBCO is as powerful as the currently used tests (except for

the BC bootstrap) and offers more robust empirical Type I error

rates in situations typical of psychology and related fields.

In addition, the MBCO procedure provides a model-comparison

framework to compare one or more alternative mediation models.

This important new feature relieves researchers of the restriction to

test a null hypothesis with a single hypothesized model. Research-

ers can test multiple competing models in addition to fitting a

Figure 6. Point and 95% CI estimate of the Type I error rate for six methods of testing a three-path indirect

effect, �1�2�3, where two out of the three parameters were fixed at zero. The nonzero parameter took on the

values: 0.14, 0.39, and 0.59. The x-axis shows the values of the nonzero parameter. Horizontal solid lines show

the limits of the Bradley’s (1978) liberal interval of .025 and .075 for � � .05. A test is robust according to a

Bradley’s (1978) criterion if its Type I error rate falls within the criterion’s interval. MBCO � model-based

constrained optimization (LRTMBCO); Percentile � percentile bootstrap; BC � bias-corrected bootstrap; JS �

joint significance test; Profile � profile-likelihood CI. See the online article for the color version of this figure.

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

17IMPROVED INFERENCE IN MEDIATION ANALYSIS

single hypothesized model and can focus on testing and comparing

one or more indirect effects. More than a significance testing

framework, a model-comparison framework allows researchers to

compare one null model to one or more alternative models. These

comparisons also can address the significance of indirect effects,

the effect sizes associated with the mediators and outcome vari-

ables, and the overall fit and complexity of the models as measured

by various fit indices such as the AIC (Akaike, 1974) and the BIC

(Schwarz, 1978).

We applied the MBCO procedure to an empirical example. The

accompanying computer code in OpenMx (Boker et al., 2011;

Neale et al., 2016), the data set, and detailed analysis results are in

the online supplemental materials. As the empirical example

shows, after conducting the MBCO procedure, researchers should

report the resulting LRTMBCO, exact p value as well as the CIs for

the indirect effects calculated using the Monte Carlo, profile-

likelihood, or percentile bootstrap method. A p value should be

interpreted as a measure of compatibility of a null model with the

sample data and should not be used to make dichotomous deci-

sions about the significance of a null hypothesis. Further, research-

ers should report R2 effect sizes associated with mediators and the

outcome variable and compute differences in the respective R2s

between the competing models. Computing differences in R2s

allows researchers to gauge potential changes in the effect sizes

between the competing models, changes that could be a result of

nonzero indirect effects. In addition, researchers should report the

information fit indices AIC or BIC. Although using CI-based

methods to test the existence of an indirect effect or to make

dichotomous decisions about significance of the indirect effect is

not recommended, researchers should report a CI to convey a

range of plausible values for an indirect effect estimate. These

recommendations are consistent with and enhance the APA rec-

ommendations (APA Publication Manual, 2010; Appelbaum et al.,

2018; Wilkinson & Task Force on Statistical Inference, American

Psychological Association, Science Directorate, 1999) for report-

ing statistical analysis results.

In conclusion, we believe that our proposed MBCO procedure

provides multiple ways to evaluate hypotheses about mediation

effects beyond the methods widely recommended in the literature

(e.g., CI-based approaches). We believe that work using the

MBCO procedure will advance the rich literature on testing and

interpreting mediation models.

References

Akaike, H. (1974). A new look at the statistical model identification. IEEE

Transactions on Automatic Control, 19, 716 –723. http://dx.doi.org/10

.1109/TAC.1974.1100705

Amrhein, V., Trafimow, D., & Greenland, S. (2019). Inferential statistics

as descriptive statistics: There is no replication crisis if we don’t expect

replication. The American Statistician, 73, 262–270. http://dx.doi.org/10

.1080/00031305.2018.1543137

APA Publication Manual. (2010). Publication manual of the American

Psychological Association (6th ed.). Washington, DC: American Psy-

chological Association.

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M.,

& Rao, S. M. (2018). Journal article reporting standards for quantitative

research in psychology: The APA Publications and Communications

Board task force report. American Psychologist, 73, 3–25. http://dx.doi

.org/10.1037/amp0000191

Ato García, M., Vallejo Seco, G., & Ato Lozano, E. (2014). Classical and

causal inference approaches to statistical mediation analysis. Psico-

thema, 26, 252–259.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable

distinction in social psychological research: Conceptual, strategic, and

statistical considerations. Journal of Personality and Social Psychology,

51, 1173–1182. http://dx.doi.org/10.1037/0022-3514.51.6.1173

Biesanz, J. C., Falk, C. F., & Savalei, V. (2010). Assessing mediational

models: Testing and interval estimation for indirect effects. Multivariate

Behavioral Research, 45, 661–701. http://dx.doi.org/10.1080/00273171

.2010.498292

Blume, J. D. (2002). Likelihood methods for measuring statistical evi-

dence. Statistics in Medicine, 21, 2563–2599. http://dx.doi.org/10.1002/

sim.1216

Boies, K., Fiset, J., & Gill, H. (2015). Communication and trust are key:

Unlocking the relationship between leadership and team performance

and creativity. The Leadership Quarterly, 26, 1080 –1094. http://dx.doi

.org/10.1016/j.leaqua.2015.07.007

Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., . . . Fox,

J. (2011). OpenMx: An open source extended structural equation mod-

eling framework. Psychometrika, 76, 306 –317. http://dx.doi.org/10

.1007/s11336-010-9200-6

Bollen, K. A. (1989). Structural equations with latent variables. New

York, NY: Wiley. http://dx.doi.org/10.1002/9781118619179

Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and

bootstrap estimates of variability. Sociological Methodology, 20, 115–

140. http://dx.doi.org/10.2307/271084

Bradley, J. V. (1978). Robustness? British Journal of Mathematical &

Statistical Psychology, 31, 144 –152. http://dx.doi.org/10.1111/j.2044-

8317.1978.tb00581.x

Bulls, H. W., Lynch, M. K., Petrov, M. E., Gossett, E. W., Owens, M. A.,

Terry, S. C., . . . Goodin, B. R. (2017). Depressive symptoms and sleep

efficiency sequentially mediate racial differences in temporal summation

of mechanical pain. Annals of Behavioral Medicine, 51, 673– 682. http://

dx.doi.org/10.1007/s12160-017-9889-x

Carmeli, A., McKay, A. S., & Kaufman, J. C. (2014). Emotional intelli-

gence and creativity: The mediating role of generosity and vigor. The

Journal of Creative Behavior, 48, 290 –309. http://dx.doi.org/10.1002/

jocb.53

Cheung, M. W. L. (2007). Comparison of approaches to constructing

confidence intervals for mediating effects using structural equation

models. Structural Equation Modeling, 14, 227–246. http://dx.doi.org/

10.1080/10705510709336745

Cohen, J. (1988). Statistical power analysis for the behavioral sciences

(2nd ed.). Hillsdale, NJ: Erlbaum.

Cohen, J. (1994). The earth is round (p � .05). American Psychologist, 49,

997–1003. http://dx.doi.org/10.1037/0003-066X.49.12.997

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple

regression/correlation analysis for the behavioral sciences (3rd ed.).

Mahwah, NJ: Erlbaum.

Cole, D. A., & Preacher, K. J. (2014). Manifest variable path analysis:

Potentially serious and misleading consequences due to uncorrected

measurement error. Psychological Methods, 19, 300 –315. http://dx.doi

.org/10.1037/a0033805

Cox, D. R., & Hinkley, D. V. (2000). Theoretical statistics. Boca Raton,

FL: Chapman & Hall/CRC.

Cox, M. G., Kisbu-Sakarya, Y., Miočević, M., & MacKinnon, D. P.

(2013). Sensitivity plots for confounder bias in the single mediator

model. Evaluation Review, 37, 405– 431. http://dx.doi.org/10.1177/

0193841X14524576

Deković, M., Asscher, J. J., Manders, W. A., Prins, P. J. M., & van der

Laan, P. (2012). Within-intervention change: Mediators of intervention

effects during multisystemic therapy. Journal of Consulting and Clinical

Psychology, 80, 574 –587. http://dx.doi.org/10.1037/a0028482

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

18 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/met0000259.supp

http://dx.doi.org/10.1109/TAC.1974.1100705

http://dx.doi.org/10.1109/TAC.1974.1100705

http://dx.doi.org/10.1080/00031305.2018.1543137

http://dx.doi.org/10.1080/00031305.2018.1543137

http://dx.doi.org/10.1037/amp0000191

http://dx.doi.org/10.1037/amp0000191

http://dx.doi.org/10.1037/0022-3514.51.6.1173

http://dx.doi.org/10.1080/00273171.2010.498292

http://dx.doi.org/10.1080/00273171.2010.498292

http://dx.doi.org/10.1002/sim.1216

http://dx.doi.org/10.1002/sim.1216

http://dx.doi.org/10.1016/j.leaqua.2015.07.007

http://dx.doi.org/10.1016/j.leaqua.2015.07.007

http://dx.doi.org/10.1007/s11336-010-9200-6

http://dx.doi.org/10.1007/s11336-010-9200-6

http://dx.doi.org/10.1002/9781118619179

http://dx.doi.org/10.2307/271084

http://dx.doi.org/10.1111/j.2044-8317.1978.tb00581.x

http://dx.doi.org/10.1111/j.2044-8317.1978.tb00581.x

http://dx.doi.org/10.1007/s12160-017-9889-x

http://dx.doi.org/10.1007/s12160-017-9889-x

http://dx.doi.org/10.1002/jocb.53

http://dx.doi.org/10.1002/jocb.53

http://dx.doi.org/10.1080/10705510709336745

http://dx.doi.org/10.1080/10705510709336745

http://dx.doi.org/10.1037/0003-066X.49.12.997

http://dx.doi.org/10.1037/a0033805

http://dx.doi.org/10.1037/a0033805

http://dx.doi.org/10.1177/0193841X14524576

http://dx.doi.org/10.1177/0193841X14524576

http://dx.doi.org/10.1037/a0028482

Dwyer, J. H. (1983). Statistical models for the social and behavioral

sciences. New York, NY: Oxford University Press.

Efron, B. (1987). Better bootstrap confidence intervals. Journal of the

American Statistical Association, 82, 171–185. http://dx.doi.org/10

.1080/01621459.1987.10478410

Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New

York, NY: Chapman & Hall.

Ernsting, A., Knoll, N., Schneider, M., & Schwarzer, R. (2015). The

enabling effect of social support on vaccination uptake via self-efficacy

and planning. Psychology Health and Medicine, 20, 239 –246. http://dx

.doi.org/10.1080/13548506.2014.920957

Falk, C. F., & Biesanz, J. C. (2015). Inference and interval estimation

methods for indirect effects with latent variable models. Structural

Equation Modeling: A Multidisciplinary Journal, 22, 24 –38. http://dx

.doi.org/10.1080/10705511.2014.935266

Folmer, H. (1981). Measurement of the effects of regional policy instru-

ments by means of linear structural equation models and panel data.

Environment & Planning A, 13, 1435–1448. http://dx.doi.org/10.1068/

a131435

Fox, J. (2016). Applied regression analysis and generalized linear models

(3rd ed.). Los Angeles, CA: SAGE.

Fritz, M. S., Kenny, D. A., & MacKinnon, D. P. (2016). The combined

effects of measurement error and omitting confounders in the single-

mediator model. Multivariate Behavioral Research, 51, 681– 697. http://

dx.doi.org/10.1080/00273171.2016.1224154

Gill, P. E., Murray, W., Saunders, M. A., & Wright, M. H. (1986). Fortran

package for nonlinear programming. User’s guide for NPSOL (Version

4. 0). Stanford, CA: Stanford University. Retrieved from https://apps

.dtic.mil/dtic/tr/fulltext/u2/a169115

Graça, J., Calheiros, M. M., & Oliveira, A. (2016). Situating moral disen-

gagement: Motivated reasoning in meat consumption and substitution.

Personality and Individual Differences, 90, 353–364. http://dx.doi.org/

10.1016/j.paid.2015.11.042

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C.,

Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values,

confidence intervals, and power: A guide to misinterpretations. Euro-

pean Journal of Epidemiology, 31, 337–350. http://dx.doi.org/10.1007/

s10654-016-0149-3

Harlow, L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there

were no significance tests? Mahwah, NJ: Erlbaum.

Haslam, C., Cruwys, T., Milne, M., Kan, C.-H., & Haslam, S. A. (2016).

Group ties protect cognitive health by promoting social identification

and social support. Journal of Aging and Health, 28, 244 –266. http://

dx.doi.org/10.1177/0898264315589578

Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal

mediation analysis. Psychological Methods, 15, 309 –334. http://dx.doi

.org/10.1037/a0020761

Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of

interaction effects between continuous predictors using multiple regres-

sion: Multiple indicator and structural equation approaches. Psycholog-

ical Bulletin, 117, 348 –357. http://dx.doi.org/10.1037/0033-2909.117.2

.348

Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating medi-

ation in treatment evaluations. Evaluation Review, 5, 602– 619. http://

dx.doi.org/10.1177/0193841X8100500502

Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social

psychology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The

handbook of social psychology (4th ed., Vol. 1, pp. 233–265). Boston,

MA: McGraw-Hill.

Kline, R. B. (2016). Principles and practice of structural equation mod-

eling (4th ed.). New York, NY: Guilford Press.

Koning, I. M., Maric, M., MacKinnon, D., & Vollebergh, W. A. M. (2015).

Effects of a combined parent-student alcohol prevention program on

intermediate factors and adolescents’ drinking behavior: A sequential

mediation model. Journal of Consulting and Clinical Psychology, 83,

719 –727. http://dx.doi.org/10.1037/a0039197

Koopman, J., Howe, M., Hollenbeck, J. R., & Sin, H.-P. (2015). Small

sample mediation testing: Misplaced confidence in bootstrapped confi-

dence intervals. Journal of Applied Psychology, 100, 194 –202. http://

dx.doi.org/10.1037/a0036635

Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual

and group level mediated effects. Multivariate Behavioral Research, 36,

249 –277. http://dx.doi.org/10.1207/S15327906MBR3602_06

MacCallum, R. C., & Austin, J. T. (2000). Applications of structural

equation modeling in psychological research. Annual Review of Psychol-

ogy, 51, 201–226. http://dx.doi.org/10.1146/annurev.psych.51.1.201

MacKinnon, D. P. (2008). Introduction to statistical mediation analysis.

New York, NY: Erlbaum.

MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., &

Sheets, V. (2002). A comparison of methods to test mediation and other

intervening variable effects. Psychological Methods, 7, 83–104. http://

dx.doi.org/10.1037/1082-989X.7.1.83

Mackinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence

limits for the indirect effect: Distribution of the product and resampling

methods. Multivariate Behavioral Research, 39, 99 –128. http://dx.doi

.org/10.1207/s15327906mbr3901_4

MacKinnon, D. P., & Pirlott, A. G. (2015). Statistical approaches for

enhancing causal interpretation of the M to Y relation in mediation

analysis. Personality and Social Psychology Review, 19, 30 – 43. http://

dx.doi.org/10.1177/1088868314542878

MacKinnon, D. P., Valente, M. J., & Wurpts, I. C. (2018). Benchmark

validation of statistical models: Application to mediation analysis of

imagery and memory. Psychological Methods, 23, 654 – 671. http://dx

.doi.org/10.1037/met0000174

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experi-

ments and analyzing data: A model comparison perspective (3rd ed.).

New York, NY: Routledge.

McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the

Reticular Action Model for moment structures. British Journal of Math-

ematical & Statistical Psychology, 37, 234 –251. http://dx.doi.org/10

.1111/j.2044-8317.1984.tb00802.x

Molina, B. S. G., Walther, C. A. P., Cheong, J., Pedersen, S. L., Gnagy,

E. M., & Pelham, W. E. J. (2014). Heavy alcohol use in early adulthood

as a function of childhood ADHD: Developmentally specific mediation

by social impairment and delinquency. Experimental and Clinical Psy-

chopharmacology, 22, 110 –121. http://dx.doi.org/10.1037/a0035656

Muthén, B. O., & Asparouhov, T. (2015). Causal effects in mediation

modeling: An introduction with applications to latent variables. Struc-

tural Equation Modeling, 22, 12–23. http://dx.doi.org/10.1080/

10705511.2014.935843

Muthén, L. K., & Muthén, B. O. (2018). Mplus user’s guide (7th ed.). Los

Angeles, CA: Author.

Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R.,

Kirkpatrick, R. M., . . . Boker, S. M. (2016). OpenMx 2.0: Extended

structural equation and statistical modeling. Psychometrika, 81, 535–

549. http://dx.doi.org/10.1007/s11336-014-9435-8

Neale, M. C., & Miller, M. B. (1997). The use of likelihood-based confi-

dence intervals in genetic models. Behavior Genetics, 27, 113–120.

http://dx.doi.org/10.1023/A:1025681223921

Oliver, M. C., Bays, R. B., & Zabrucky, K. M. (2016). False memories and

the DRM paradigm: Effects of imagery, list, and test type. The Journal

of General Psychology, 143, 33– 48. http://dx.doi.org/10.1080/00221309

.2015.1110558

Pawitan, Y. (2001). In all likelihood: Statistical modelling and inference

using likelihood. New York, NY: Oxford University Press.

Pek, J., & Wu, H. (2015). Profile likelihood-based confidence intervals and

regions for structural equation models. Psychometrika, 80, 1123–1145.

http://dx.doi.org/10.1007/s11336-015-9461-1

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

19IMPROVED INFERENCE IN MEDIATION ANALYSIS

http://dx.doi.org/10.1080/01621459.1987.10478410

http://dx.doi.org/10.1080/01621459.1987.10478410

http://dx.doi.org/10.1080/13548506.2014.920957

http://dx.doi.org/10.1080/13548506.2014.920957

http://dx.doi.org/10.1080/10705511.2014.935266

http://dx.doi.org/10.1080/10705511.2014.935266

http://dx.doi.org/10.1068/a131435

http://dx.doi.org/10.1068/a131435

http://dx.doi.org/10.1080/00273171.2016.1224154

http://dx.doi.org/10.1080/00273171.2016.1224154

https://apps.dtic.mil/dtic/tr/fulltext/u2/a169115

https://apps.dtic.mil/dtic/tr/fulltext/u2/a169115

http://dx.doi.org/10.1016/j.paid.2015.11.042

http://dx.doi.org/10.1016/j.paid.2015.11.042

http://dx.doi.org/10.1007/s10654-016-0149-3

http://dx.doi.org/10.1007/s10654-016-0149-3

http://dx.doi.org/10.1177/0898264315589578

http://dx.doi.org/10.1177/0898264315589578

http://dx.doi.org/10.1037/a0020761

http://dx.doi.org/10.1037/a0020761

http://dx.doi.org/10.1037/0033-2909.117.2.348

http://dx.doi.org/10.1037/0033-2909.117.2.348

http://dx.doi.org/10.1177/0193841X8100500502

http://dx.doi.org/10.1177/0193841X8100500502

http://dx.doi.org/10.1037/a0039197

http://dx.doi.org/10.1037/a0036635

http://dx.doi.org/10.1037/a0036635

http://dx.doi.org/10.1207/S15327906MBR3602_06

http://dx.doi.org/10.1146/annurev.psych.51.1.201

http://dx.doi.org/10.1037/1082-989X.7.1.83

http://dx.doi.org/10.1037/1082-989X.7.1.83

http://dx.doi.org/10.1207/s15327906mbr3901_4

http://dx.doi.org/10.1207/s15327906mbr3901_4

http://dx.doi.org/10.1177/1088868314542878

http://dx.doi.org/10.1177/1088868314542878

http://dx.doi.org/10.1037/met0000174

http://dx.doi.org/10.1037/met0000174

http://dx.doi.org/10.1111/j.2044-8317.1984.tb00802.x

http://dx.doi.org/10.1111/j.2044-8317.1984.tb00802.x

http://dx.doi.org/10.1037/a0035656

http://dx.doi.org/10.1080/10705511.2014.935843

http://dx.doi.org/10.1080/10705511.2014.935843

http://dx.doi.org/10.1007/s11336-014-9435-8

http://dx.doi.org/10.1023/A:1025681223921

http://dx.doi.org/10.1080/00221309.2015.1110558

http://dx.doi.org/10.1080/00221309.2015.1110558

http://dx.doi.org/10.1007/s11336-015-9461-1

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003).

Common method biases in behavioral research: A critical review of the

literature and recommended remedies. Journal of Applied Psychology,

88, 879 –903. http://dx.doi.org/10.1037/0021-9010.88.5.879

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling

strategies for assessing and comparing indirect effects in multiple me-

diator models. Behavior Research Methods, 40, 879 – 891. http://dx.doi

.org/10.3758/BRM.40.3.879

R Core Team. (2019). R: A Language and Environment for Statistical

Computing (Version 3.6.0) [Computer software]. Retrieved from http://

www.R-project.org/

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models:

Applications and data analysis methods (2nd ed.). Thousand Oaks, CA:

SAGE.

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling.

Journal of Statistical Software, 48, 1–36. http://dx.doi.org/10.18637/jss

.v048.i02

SAS Institute Inc. (2016). SAS/STAT 14.2 user’s guide. Cary, NC: Author.

Retrieved from http://support.sas.com/documentation/cdl/en/stathpug/

67524/PDF/default/stathpug

Schwarz, G. (1978). Estimating the dimension of a model. Annals of

Statistics, 6, 461– 464. http://dx.doi.org/10.1214/aos/1176344136

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and non-

experimental studies: New procedures and recommendations. Psycho-

logical Methods, 7, 422– 445. http://dx.doi.org/10.1037/1082-989X.7.4

.422

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An intro-

duction to basic and advanced multilevel modeling (2nd ed.). Thousand

Oaks, CA: SAGE.

Snyman, J. A. (2005). Practical mathematical optimization: An introduc-

tion to basic optimization theory and classical and new gradient-based

algorithms. New York, NY: Springer.

Taylor, A. B., MacKinnon, D. P., & Tein, J.-Y. (2008). Tests of the

three-path mediated effect. Organizational Research Methods, 11, 241–

269. http://dx.doi.org/10.1177/1094428107300344

Thoemmes, F., Mackinnon, D. P., & Reiser, M. R. (2010). Power analysis

for complex mediational designs using Monte Carlo methods. Structural

Equation Modeling, 17, 510 –534. http://dx.doi.org/10.1080/10705511

.2010.489379

Tofighi, D., Hsiao, Y.-Y., Kruger, E. S., MacKinnon, D. P., Van Horn,

M. L., & Witkiewitz, K. A. (2019). Sensitivity analysis of the no-omitted

confounder assumption in latent growth curve mediation models. Struc-

tural Equation Modeling, 26, 94 –109. http://dx.doi.org/10.1080/

10705511.2018.1506925

Tofighi, D., & Kelley, K. (2016). Assessing omitted confounder bias in

multilevel mediation models. Multivariate Behavioral Research, 51,

86 –105. http://dx.doi.org/10.1080/00273171.2015.1105736

Tofighi, D., & Kelley, K. (2019). Indirect effects in sequential mediation

models: Evaluating methods for hypothesis testing and confidence in-

terval formation. Multivariate Behavioral Research. Advance online

publication. http://dx.doi.org/10.1080/00273171.2019.1618545

Tofighi, D., & MacKinnon, D. P. (2011). RMediation: An R package for

mediation analysis confidence intervals. Behavior Research Methods,

43, 692–700. http://dx.doi.org/10.3758/s13428-011-0076-x

Tofighi, D., & MacKinnon, D. P. (2016). Monte Carlo confidence intervals

for complex functions of indirect effects. Structural Equation Modeling,

23, 194 –205. http://dx.doi.org/10.1080/10705511.2015.1057284

Tofighi, D., West, S. G., & MacKinnon, D. P. (2013). Multilevel mediation

analysis: The effects of omitted variables in the 1–1-1 model. British

Journal of Mathematical & Statistical Psychology, 66, 290 –307. http://

dx.doi.org/10.1111/j.2044-8317.2012.02051.x

Valeri, L., & Vanderweele, T. J. (2013). Mediation analysis allowing for

exposure-mediator interactions and causal interpretation: Theoretical

assumptions and implementation with SAS and SPSS macros. Psycho-

logical Methods, 18, 137–150. http://dx.doi.org/10.1037/a0031034

VanderWeele, T. J. (2010). Bias formulas for sensitivity analysis for direct

and indirect effects. Epidemiology, 21, 540 –551. http://dx.doi.org/10

.1097/EDE.0b013e3181df191c

VanderWeele, T. J. (2015). Explanation in causal inference: Methods for

mediation and interaction. New York, NY: Oxford University Press.

Wald, A. (1943). Tests of statistical hypotheses concerning several param-

eters when the number of observations is large. Transactions of the

American Mathematical Society, 54, 426 – 482. http://dx.doi.org/10

.1090/S0002-9947-1943-0012401-3

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a

world beyond “p � 0.05”. The American Statistician, 73, 1–19. http://

dx.doi.org/10.1080/00031305.2019.1583913

Weisstein, E. W. (2018). Smooth function. Retrieved from http://

mathworld.wolfram.com/SmoothFunction.html

Wickham, H., François, R., Henry, L., & Müller, K. (2019). dplyr: A

grammar of data manipulation. Retrieved from https://CRAN.R-project

.org/package�dplyr

Wilkinson, L., & Task Force on Statistical Inference, American Psycho-

logical Association, Science Directorate. (1999). Statistical methods in

psychology journals: Guidelines and explanations. American Psycholo-

gist, 54, 594 – 604. http://dx.doi.org/10.1037/0003-066X.54.8.594

Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio

for testing composite hypotheses. Annals of Mathematical Statistics, 9,

60 – 62. http://dx.doi.org/10.1214/aoms/1177732360

Williams, J., & MacKinnon, D. P. (2008). Resampling and distribution of

the product methods for testing indirect effects in complex models.

Structural Equation Modeling: A Multidisciplinary Journal, 15, 23–51.

Yzerbyt, V., Muller, D., Batailler, C., & Judd, C. M. (2018). New recom-

mendations for testing indirect effects in mediational models: The need

to report and test component paths. Journal of Personality and Social

Psychology, 115, 929 –943. http://dx.doi.org/10.1037/pspa0000132

Zahery, M., Maes, H. H., & Neale, M. C. (2017). CSOLNP: Numerical

optimization engine for solving non-linearly constrained problems. Twin

Research and Human Genetics, 20, 290 –297. http://dx.doi.org/10.1017/

thg.2017.28

Received November 27, 2017

Revision received December 3, 2019

Accepted January 4, 2020 �

hi

s

do

cu

m

en

t

is

co

py

ri

gh

te

d

by

th

e

A

m

er

ic

an

P

sy

ch

ol

og

ic

al

A

ss

oc

ia

ti

on

or

on

e

of

it

s

al

li

ed

pu

bl

is

he

rs

.

T

hi

s

ar

ti

cl

e

is

in

te

nd

ed

so

le

ly

fo

r

th

e

pe

rs

on

al

us

e

of

th

e

in

di

vi

du

al

us

er

an

d

is

no

t

to

be

di

ss

em

in

at

ed

br

oa

dl

y.

20 TOFIGHI AND KELLEY

http://dx.doi.org/10.1037/0021-9010.88.5.879

http://dx.doi.org/10.3758/BRM.40.3.879

http://dx.doi.org/10.3758/BRM.40.3.879

http://www.R-project.org/

http://www.R-project.org/

http://dx.doi.org/10.18637/jss.v048.i02

http://dx.doi.org/10.18637/jss.v048.i02

http://support.sas.com/documentation/cdl/en/stathpug/67524/PDF/default/stathpug

http://support.sas.com/documentation/cdl/en/stathpug/67524/PDF/default/stathpug

http://dx.doi.org/10.1214/aos/1176344136

http://dx.doi.org/10.1037/1082-989X.7.4.422

http://dx.doi.org/10.1037/1082-989X.7.4.422

http://dx.doi.org/10.1177/1094428107300344

http://dx.doi.org/10.1080/10705511.2010.489379

http://dx.doi.org/10.1080/10705511.2010.489379

http://dx.doi.org/10.1080/10705511.2018.1506925

http://dx.doi.org/10.1080/10705511.2018.1506925

http://dx.doi.org/10.1080/00273171.2015.1105736

http://dx.doi.org/10.1080/00273171.2019.1618545

http://dx.doi.org/10.3758/s13428-011-0076-x

http://dx.doi.org/10.1080/10705511.2015.1057284

http://dx.doi.org/10.1111/j.2044-8317.2012.02051.x

http://dx.doi.org/10.1111/j.2044-8317.2012.02051.x

http://dx.doi.org/10.1037/a0031034

http://dx.doi.org/10.1097/EDE.0b013e3181df191c

http://dx.doi.org/10.1097/EDE.0b013e3181df191c

http://dx.doi.org/10.1090/S0002-9947-1943-0012401-3

http://dx.doi.org/10.1090/S0002-9947-1943-0012401-3

http://dx.doi.org/10.1080/00031305.2019.1583913

http://dx.doi.org/10.1080/00031305.2019.1583913

http://mathworld.wolfram.com/SmoothFunction.html

http://mathworld.wolfram.com/SmoothFunction.html

https://CRAN.R-project.org/package=dplyr

https://CRAN.R-project.org/package=dplyr

http://dx.doi.org/10.1037/0003-066X.54.8.594

http://dx.doi.org/10.1214/aoms/1177732360

http://dx.doi.org/10.1037/pspa0000132

http://dx.doi.org/10.1017/thg.2017.28

http://dx.doi.org/10.1017/thg.2017.28

- Improved Inference in Mediation Analysis: Introducing the Model-Based Constrained Optimization P …

The Model-Based Constrained Optimization Procedure

Empirical Example of Using the MBCO Procedure on a Complex Mediation Model

Research Question 1

Step 1

Step 2

Step 3

Research Question 2

Step 1

Step 2

Step 3

Research Question 3

Step 1

Step 2

Step 3

Comparing Methods of Testing Indirect Effect Via Monte Carlo Simulations

Percentile and Bias-Corrected Bootstrap CI

Monte Carlo CI

Profile Likelihood CI

Joint Significance Test

Simulation Design and Population Values

Data Generation

Results

Type I error

Power

Summary

Limitations

Discussion

References