Markingfeedbackform Quantitativeresearchpaper Critiqueguide Qualitativereseachpaper Lecture3Non-experimentalobservationaldesigns Lecture6IntroductiontoQualitativeResearch Lecture9Interpretingandcritiquingqualitativeresearch Lecture8Criticalappraisalquant Lecture5Inferentialstatisticalanalysis Lecture4samplingvalidityandreliability Lecture7Qualitativemethodsofdatacollectionandanalysis Lecture2Experimentaldesigns
Please find the essay′s information in the ″Critique Guide″ file which includes the full question and the format of the essay. The two articles to be critiqued is included in the uploads. Lecture slides of the module are also uploaded which the essay will need to demonstrate understanding of the knowledge.
Lack of instructions
PH2600Research Methods BSc(Hons) Physiotherapy
Critique Marking/ Feedback Form
Student No: Markers Initials: Date:
Quantitative paper section
Student provides a context
for the article that is being
critiqued
/5
Identifies 3 clear features
of the design or reporting
of the study as strengths
or limitation (note: at least
one must relate to the
data analysis or results).
Makes a case for each
feature demonstrating
knowledge of the relevant
principles of research
methods and how that
feature might impact on
the study’s results or
conclusions.
/35
Summarises the main
points of the critique
/5
Quantitative paper section
Student provides a context
for the article that is being
critiqued
/5
Identifies 3 clear features
of the design or reporting
of the study as strengths
or limitation (note: at least
one must relate to the
data analysis or results).
Makes a case for each
feature demonstrating
knowledge of the relevant
principles of research
methods and how that
feature might impact on
the study’s results or
conclusions.
/35
Summarises the main
points of the critique
/5
Presentation/ Referencing
Overall grammar, spelling,
quality of writing
Referencing appropriate
/10
TOTAL MARK /100
General Comments
A
O
D
S
P
r
M
R
c
C
T
©
K
I
t
T
H
t
o
n
I
b
p
[
t
o
i
t
0
d
Physiotherapy
97 (2011) 250–255
Effect of motion control running shoes compared with neutral
shoes on tibial rotation during running
Alice Rose a,∗, Ivan Birch b, Raija Kuisma
a
a
Division of Physiotherapy, School of Health Professions, University of Brighton, UK
b
F
aculty of Health and Human Sciences at Thames Valley University, UK
bstract
bjective To determine whether a motion control running shoe reduces tibial rotation i
n the transverse plane during treadmill running.
esign An experimental study measuring tibial rotation in volunteer participants using a repeated measures design.
etting Human Movement Laboratory, School of Health Professions, University of Brighton.
articipants Twenty-four healthy participants were tested. The group comprised males and females with size 6, 7, 9 and 11 feet. The age
ange for participants was 19 to 31 years.
ain outcome measures The total range of proximal tibial rotation was measured using the Codamotion 3-D Movement Analysis System.
esults A one-tailed paired t-test indicated a statistically significant decrease in the total range of proximal tibial rotation when a motion
ontrol shoe was worn (mean difference 1.38◦, 95% confidence interval 0.03 to 2.73, P = 0.04).
onclusions There is a difference in tibial rotation in the transverse plane between a motion control running shoe and a neutral running shoe.
he results from this study have implications for the use of supportive running shoes as a form of injury prevention.
2010 Chartered Society of Physiotherapy. Publis
hed by Elsevier Ltd. All rights reserved.
m
p
s
t
f
a
w
r
t
I
s
[
a
a
eywords: Tibial rotation; Running; Motion control shoe
ntroduction
In today’s society, one of the greatest challenges faced by
he Department of Health is the growing epidemic of obesity.
he cost of overweight and obese individuals to the National
ealth Service is estimated to be £4.2 billion [1]. This has led
o considerable media attention on the wider health risks of
ur sedentary lifestyles, and has resulted in an increase in the
umber of people participating in recreational running [2].
n order to understand the effect of running shoes on gait, a
asic knowledge of lower limb mechanics is essential.
Within the first 20% of the stance phase, the subtalar joint
ronates to allow solid contact of the foot with the ground
3]. As forward progression continues through the middle of
he stance phase, maximum pronation and ankle dorsiflexion
ccur. Pronation is a normal part of the running cycle because
t allows for shock absorption and accommodation on uneven
errain [3]. However, in some individuals, excessive pronation
∗ Corresponding author.
E-mail address: alice2rose@googlemail.com (A. Rose).
t
i
h
f
a
f
031-9406/$ – see front matter © 2010 Chartered Society of Physiotherapy. Publis
oi:10.1016/j.physio.2010.08.013
ay occur for various biomechanical reasons [4]. Excessive
ronators present with a broad range of pathologies, such as
tress fractures, achilles tendonitis and iliotibial band (ITB)
endonitis [4]. Yates and White studied naval recruits and
ound that those with a pronated foot type were almost twice
s likely to develop medial tibial stress syndrome compared
ith those with a normal or supinated foot posture [5]. A
isk estimate revealed that recruits with a more pronated foot
ype had a higher relative risk (1.70) than injury-free recruits.
n an attempt to minimise the risk of injury, athletes have
tarted to seek specific equipment, particularly running shoes
5].
Neutral cushion shoes are generally best for runners with
n excessive supinatory gait to provide additional shock
bsorption, whereas motion control shoes are better for
he moderate to severe overpronator. Motion control shoes
nclude a reinforced heel counter and a denser midsole to
elp control any excessive pronation [5]. Clarke et al. [6]
ound that shoes with a positive heel flare and a hard midsole
llowed significantly less maximum pronation and total rear-
oot movement compared with shoes with a softer midsole.
hed by Elsevier Ltd. All rights reserved.
dx.doi.org/10.1016/j.physio.2010.08.013
mailto:alice2rose@googlemail.com
dx.doi.org/10.1016/j.physio.2010.08.013
therapy
a
o
a
s
n
M
d
p
i
i
i
fi
t
c
w
s
m
[
i
c
n
I
m
m
i
r
a
D
a
r
i
h
i
a
C
T
c
T
d
n
C
e
i
r
i
w
i
M
t
r
m
i
P
m
v
s
i
t
M
t
s
1
h
d
o
I
I
b
l
L
u
i
p
w
P
p
n
t
t
o
i
b
X
e
c
•
•
•
A. Rose et al. / Physio
Research has shown that motion control shoes also have
n effect on tibial internal rotation [7]. This is a normal part
f the gait cycle and occurs in conjunction with pronation
nd hindfoot eversion due to the mitered hinge effect of the
ubtalar joint [3]. Running can lead to an increase in inter-
al tibial rotation for people with excessively pronated feet.
iller et al. [8] found that runners with a history of ITB syn-
rome demonstrated increased tibial internal rotation. They
redicted that an increase in internal rotation of the tibia could
ncrease the strain in the ITB, and therefore be a contribut-
ng factor to ITB syndrome. However, the critical degree of
nternal tibial rotation leading to injuries has not been con-
rmed, and research is required in this area. It is reasonable
o assume that by reducing rearfoot pronation with motion
ontrol shoes, tibial rotation would consequently decrease
hen running. This may support the use of supportive running
hoes as a form of injury prevention. To date, there is mini-
al evidence on the effects of running shoes on tibial rotation
8–10]. Stacoff et al. [10] used intracortical bone pins with
njury-free participants to measure the effects of shoe sole
onstruction on skeletal motion during running. They found
o statistically significant change in tibiocalcaneal rotations.
t was concluded that the tibiocalcaneal kinematics of running
ay be unique to the individual, and shoe sole modifications
ay not be able to make substantial changes.
Running gait can be analysed using a number of m
ethods
ncluding real-time observational gait analysis, high-
esolution cameras and video-recording devices, force plates
nd computer systems. Computerised three-dimensional (3-
) motion analysis measurements are currently a widespread
nd useful tool for both clinical practice and biomechanical
esearch. The Coda motion 3-D Movement Analysis System
s able to measure locations of active markers in 3-D with
igh resolution and accuracy. Maynard et al. [11] studied the
ntra- and inter-rater reliability of gait measurements using
Cartesian optoelectronic dynamic anthropometer (CODA,
harnwood Dynamics Limited based in Leicestershire).
hey suggested that natural variation of the participant’s gait
ycle may be overcome by capturing at least three gait trials.
hey concluded that they had not shown complete repro-
ucibility of gait measurements using CODA, but this does
ot suggest that CODA is unreliable. Many studies have used
ODA in the past and none have recorded any anomalies or
rrors attributal to CODA itself [12,13].
Published literature suggests that running footwear can
nfluence lower extremity kinematics and kinetics. Currently,
esearch into the effects of running footwear on tibial rotation
s limited. Therefore, the aim of this study was to determine
hether motion control running shoes reduce tibial rotation
n the transverse plane during treadmill running.
ethods
An experimental study was conducted with two condi-
ions: neutral shoes and motion control shoes. The effect on
o
o
I
97 (2011) 250–255 251
ange of tibial rotation in the transverse plane during tread-
ill running in the two styles of running shoes was measured
n degrees.
articipants
Ethical approval was gained from the School Ethics Com-
ittee at the University of Brighton. Thirty-two consecutive
olunteers from the University of Brighton health profes-
ional courses who responded to a recruitment e-mail were
ncluded. This e-mail included an information sheet for par-
icipants to read before volunteering. On arrival at the Human
ovement Laboratory, all details of the study were explained
o participants and they were asked to sign a consent form.
In order to be included, participants were required to have
ize 6, 7, 9 or 11 feet and be able to run comfortably for 5 to
0 minutes. Participants were excluded from the study if they
ad a history of cardiovascular problems, a lower limb con-
ition that was exacerbated by running, a vestibular disorder
r an allergy to hypoallergenic adhesive tape.
nstrumentation
A Biodex RTM 500 treadmill (Biodex Medical Systems,
nc., New York, USA) was used for all participants and for
oth conditions. The treadmill was placed in the middle of the
aboratory with one CODA MPX30 (Charnwood Dynamics
td., Leicester, UK) scanner unit in front and one scanner
nit behind. These units picked up signals emitted by the
nfra-red ‘active’ markers which provided an immediate and
recise 3-D measurement. All information from the scanners
as stored on the CODA computer system.
rocedure
A pilot study was carried out on one individual to test the
roposed methodology. No problems were experienced, so
o changes were made to the methodology. The data from
he pilot study were included in the main results.
The laboratory was set up before participants arrived on
he first day. The origin or zero point of the system’s mutually
rthogonal measurement framework was established by plac-
ng a single marker in the centre of the treadmill, mid-way
etween the two sensor units. The alignment of the system’s
, Y and Z axes was set using two additional pairs of mark-
rs. The orientation of the three axes used throughout the data
ollection was as follows:
positive Y axis – direction of walking;
positive X axis – to the participant’s right;
positive Z axis – up.
All data were collected using the right leg, and the axis
rientation was set in accordance with the recommendations
f the Standardization and Terminology Committee of the
nternational Society of Biomechanics [14].
252 A. Rose et al. / Physiotherapy 97 (2011) 250–255
F
i
c
n
A
g
l
i
m
C
(
u
i
a
T
p
t
c
f
a
m
i
o
[
a
w
s
c
t
a
o
w
t
t
t
M
w
s
F
Data analysis
The first useable set of data of the three sets collected per
participant was used for data analysis. The average range of
ig. 1. Neutral shoe: Mizuno Wave Rider.
Participants were asked to attend a single session last-
ng approximately 30 minutes. Participants were asked to
ome wearing shorts and to put on the neutral pair of run-
ing shoes (Fig. 1) before marker attachment took place.
ctive markers were attached using double-sided hypoaller-
enic adhesive tape to the following landmarks on the right
eg: medial condyle of tibia, head of fibula, tibial tuberos-
ty, two-thirds of tibial shaft, medial malleolus and lateral
alleolus (Fig. 2). Marker placement complied with the Joint
o-ordinate System recommendation [9]. Technical markers
medial condyle of tibia and two-thirds of tibial shaft) were
sed to define the co-ordination of the virtual markers dur-
ng running trials. Batteries were connected to the markers
nd attached to the leg using hypoallergenic adhesive tape.
he safety clip of the treadmill was attached to the partici-
ants and they were given a 5-minute practice run to allow
hem to adapt to the treadmill motion and the shoes. Lav-
anska et al. found that it takes 5 to 6 minutes to become
amiliar with treadmill running; therefore, participants were
cclimatised for this period prior to commencement of
easurements. The treadmill speed was then gradually
ncreased to 8 km/hour [15]. The average running speed
f unimpaired young adults is between 7 and 9 km/hour
15].
Participants were shown the emergency stop button and
ssured that they could stop running at any time. Confirmation
as gained that the participant was happy running at this
peed. On confirmation, lights were switched off to do a ‘test’
ollection of data to ensure that all markers were in view of
he scanners. One researcher carried out all the tests with an
ssistant to help. If all the markers were in view, three sets
f data were collected and saved to the computer. If a marker
ent out of view at any point, the treadmill was stopped and
he markers were adjusted accordingly or retaped. Once the
hree sets of data were saved, participants dismounted the
readmill and changed into the motion control shoes (Fig. 3).
arkers were checked for secure attachment. The procedure
as then repeated for the motion control shoes. All data were
aved to a password-secured computer. F
ig. 2. Front view of active markers.
ig. 3. Motion control shoe: Mizuno Wave Inspire.
A. Rose et al. / Physiotherapy
Table 1
Total range of proximal tibial rotation in the XY plane of the Cartesian
optoelectronic dynamic anthropometer for the two running shoes.
Participant Total range of tibial
rotation (degrees)
Neutral
shoe
Support
shoe
Difference
(◦)
1 10 16 −6
2 4 9 −5
3 5 9 −4
4 10 14 −4
5 12 14 −2
6 7 7 0
7 6 6 0
8 22 21 1
9 17 16 1
10 8 7 1
11 5 4 1
12 9 8 1
13 15 14 1
14 7 5 2
15 23 21 2
16 9 6 3
17 18 15 3
18 18 15 3
19 10 7 3
20 12 8 4
21 11 6 5
22 20 14 6
23 10 4 6
24 28 17 11
M
m
p
c
t
C
w
t
s
R
p
e
m
u
i
T
f
d
P
d
t
c
t
(
w
t
m
1
t
r
t
D
c
p
p
s
d
s
m
r
w
e
c
d
t
s
h
a
b
w
r
p
s
t
s
i
t
1
b
n
s
i
k
s
t
n
reduction in peak rearfoot eversion for runners in the motion
ean Standard deviation 12.3 (6.4) 10.9(5.3) 1.4 (3.8)
otion of the leg in the transverse plane during the contact
hase of running was calculated using CODA. The data were
opied into Microsoft Excel, and graphs were plotted to show
he total range of proximal tibial rotation in the XY plane of
ODA for the two running shoes. A one-tailed paired t-test
as used to test for differences in tibial rotation recorded for
he two experimental conditions (neutral and motion control
hoes), and the P-level was set at 0.05.
esults
All 32 participants took part in the study, but only 24
articipants provided adequate data for analysis. Naturally,
very individual’s running gait is different and sometimes this
eant that the signals from the active markers were not picked
p by the scanner units. This occurred for eight of the partic-
pants; therefore, the data were excluded from the analysis.
able 1 shows the data collected from the 24 participants.
The paired t-test revealed a statistically significant dif-
erence in tibial rotation between the two conditions (mean
ifference 1.38◦, 95% confidence interval 0.03 to 2.73,
= 0.04). This showed that there is a statistically significant
ifference in tibial rotation in the transverse plane during
readmill running when using motion control running shoes
ompared with neutral running shoes.
c
p
r
97 (2011) 250–255 253
The overall trend showed that the total range of proximal
ibial rotation for participants running in motion control shoes
mean 10.9◦, standard deviation 5.3◦) was smaller compared
ith running in neutral shoes (mean 12.9◦, standard devia-
ion 6.4◦). The largest individual difference was 11◦, but the
ean difference between the two conditions was small (mean
.38◦). The tibial rotation in five participants was smaller in
he neutral running shoes (Table 1).
The largest difference was 11◦; two participants had a total
ange of tibial rotation of 10◦ in the neutral shoes and 4◦ in
he motion control shoes.
iscussion
The aim of this study was to determine whether motion
ontrol running shoes reduce tibial rotation in the transverse
lane during treadmill running, with implications for injury
revention. The findings suggest that there is a statistically
ignificant difference in tibial rotation in the transverse plane
uring treadmill running between motion control running
hoes and neutral running shoes. The total range of proxi-
al tibial rotation was generally reduced when participants
an in motion control shoes. Although the mean difference
as rather small, some participants demonstrated a differ-
nce which was half of the total range, and hence could be
linically significant in their cases.
Clarke et al. [6] analysed the effects of different shoe
esign parameters on rearfoot control in running. They found
hat participants displayed an average of approximately 2.7◦
maller maximum pronation when running in shoes with a
ard midsole compared with a soft midsole. If rearfoot motion
nd tibial rotation are taken as a coupled mechanism [3], it can
e assumed that this is the mechanism by which tibial rotation
as reduced in this study. Controlling rearfoot motion with
unning in motion control shoes will subsequently reduce the
articipant’s tibial rotation. The results from the current study
upport this assumption because the total range of proximal
ibial rotation was reduced when running in motion control
hoes compared with running in neutral shoes for the major-
ty of participants (79%). For example, one participant had a
otal range of tibial rotation of 28◦ in the neutral shoes and
7◦ in the motion control shoes (reduction 11◦).
The results from the current study relate to similar research
y Butler et al. [7], which showed that in low arched run-
ers, peak tibial internal rotation decreased in motion control
hoes and was increased in cushion shoes. They found no
nteraction between high arched runners and lower extremity
inematics. Since arch height was not measured in the current
tudy, it is unclear if this factor contributed to the degree of
ibial rotation. To confirm this association, further research
eeds to be undertaken in the area. Butler et al. [7] found no
ontrol shoes. This is unusual because if there was a cou-
led relationship between rearfoot eversion and tibial internal
otation, it would be expected that both of these movements
2 therapy
w
u
i
o
e
p
t
s
s
b
t
S
k
b
t
w
p
l
s
S
b
m
r
i
p
t
n
[
a
m
b
m
m
e
i
a
m
s
a
a
a
b
n
c
g
a
t
p
o
m
f
t
a
m
A
d
t
e
t
t
f
t
t
s
h
e
t
e
p
r
T
p
a
b
n
c
u
f
c
f
p
t
t
t
t
C
t
w
r
d
m
h
r
s
r
54 A. Rose et al. / Physio
ould decrease when running in motion control shoes. These
nusual results may be due to the motion control shoes having
ncreased support in the midfoot; therefore, the positioning
f this support would not have influenced rearfoot motion.
This study does not provide direct support for the hypoth-
sis that motion control running shoes are a form of injury
revention; however, it does have some supporting implica-
ions. Miller et al. [8] found that runners with a history of ITB
yndrome demonstrated more tibial internal rotation, thus
uggesting that an increase in tibial internal rotation could
e a contributing factor to ITB syndrome. The links between
ibial rotation and running injuries are further supported by
tergiou and Bates [16]. They investigated knee and ankle
inematics in five runners, and showed a strong relationship
etween pronation and knee joint function via tibial rotation;
his was identified as a possible mechanism for injury. They
ent on to explain that overpronation could lead to maximum
ronation (and subsequent internal tibial rotation) occurring
ater in the stance phase. This could then lead to soft tissue
tress around the knee or patellofemoral malalignment [16].
ince anyone with a lower limb condition that is exacerbated
y running was excluded from this study, the hypothesis that
otion control shoes could be a form of injury prevention for
unners was not tested. However, the links between increased
nternal tibial rotation and running injuries found to date seem
romising, and there is scope for further research.
The Codamotion 3-D Movement Analysis System is easy
o use, and previous studies that have used CODA have
ot found any anomalies or errors attributal to CODA itself
12,13]. However, there are certain limitations that may have
ffected the outcome of the results. The issue of skin move-
ent artefacts when using skin markers in gait analysis has
een raised [3], although tibial rotation is considered to be
ore accurate than other measurements of knee and thigh
ovement [12]. When comparing bone pins with skin mark-
rs, Reinschmidt et al. [12] found an average error of 1.1◦,
mplying that tibial rotation can be determined with reason-
ble accuracy using skin markers. In this study, the active
arkers were placed over bony structures where underlying
oft tissue was minimal, with the hope that skin movement
rtefacts would be minimised. This is especially the case
round the ankle where the skin is tightly bound and there is
decreased chance of soft tissue and skin movement.
Maynard et al. [11] studied the intra- and inter-rater relia-
ility of gait measurements using CODA, and suggested that
atural variation of the participant’s gait cycle may be over-
ome by capturing at least three gait trials. In this study, three
ait trials were captured for each participant and the first use-
ble set of data was used for data analysis. This improved
he reliability of the measurement technique and minimised
otential error.
Although treadmill running allows for easy, continuous
bservation and monitoring, it may cause variation in move-
ent pattern compared with overground running. Therefore,
or a true representation of running biomechanics, tibial rota-
ion would need to be measured over ground.
A
F
97 (2011) 250–255
Five participants did not follow the trend of results
nd showed an increase in tibial rotation when wearing
otion control running shoes compared with neutral shoes.
number of factors could have contributed to this unusual
ifference. During data collection, it became apparent that
here was inconsistency with the level of treadmill experi-
nce between the participants. This could have resulted in
he less experienced participants becoming fatigued by the
ime they reached the second test condition. Links between
atigue and rearfoot kinematics have been recorded [7,17].
Participants in this study ran first in the neutral shoes and
hen in the motion control shoes. This may have contributed
o some of the increases in tibial rotation in the motion control
hoes. To minimise possible order effects, the shoes should
ave been presented in a randomised order. This would have
liminated a potential confounding variable and strengthened
he internal validity of the study. Due to limited funding, only
ight pairs of shoes were used for the study, which excluded
articipants outside the available shoe sizes. Therefore, the
esults may not be representative of the general population.
his study would need to be reproduced with a larger sam-
le size (including all shoe sizes) to make the results more
pplicable to the wider population.
The results from the study are open to observer bias
ecause the researcher collected the study data. To elimi-
ate this potential confounding variable, an independent data
ollector should have been used.
A number of ideas for further research arose while eval-
ating the current study. As highlighted above, the results
rom this study have implications for the theory that motion
ontrol running shoes could be a form of injury prevention
or runners. The experiment could be expanded to include
articipants with running-related injuries. Researchers could
hen investigate to see if injured participants displayed more
ibial rotation in the neutral shoes, and furthermore if this
ibial rotation decreased for these participants when wearing
he motion control shoes.
onclusions
The results from this study show that there is a difference in
ibial rotation in the transverse plane during treadmill running
hen comparing motion control running shoes with neutral
unning shoes. This is indicated by the statistically significant
ecrease in the total range of proximal tibial rotation when
otion control shoes were worn. Previous research findings
ave demonstrated links between increased tibial rotation and
unning injuries [8,9]. The results from this study therefore
uggest that supportive running shoes may have an important
ole to play in running injury prevention.
cknowledgements
The authors would like to thank Amy Grimadell, Leigh
agan and Nicole Nielsen for assisting with data collection;
therapy
P
f
B
E
E
C
R
[
[
[
[
[
[
[
1997;6:177–85.
[17] Gerlach K, White S, Burton H, Dorn J, Leddy J, Horvath P. Kinetic
A. Rose et al. / Physio
aul Stevens at Mizuno Ltd. for providing the running shoes
or this study; and all the participants from the University of
righton.
thical approval: School of Health Professions Research
thics and Governance Panel at the University of Brighton.
onflict of interest: None declared.
eferences
[1] Department of Health. Obesity. Department of Health, Lon-
don, 2007. Available at: http://www.dh.gov.uk/en/Publichealth/
Healthimprovement/Obesity/DH 6585 (last accessed 14.04.2009).
[2] Buist I, Bredeweg S, Bessem B, Van Mechelen W, Lemmink K, Dier-
cks R. Incidence and risk factors of running-related injuries during
preparation for a four-mile recreational running event. Br J Sports Med
2008;5:1–12.
[3] Dungan S, Bhat K. Biomechanics and analysis of running gait. Phys
Med Rehabil Clin N Am 2005;16:603–21.
[4] Yamashita M. Evaluation and selection of shoe wear and orthoses for
the runner. Phys Med Rehabil Clin N Am 2005;16:801–29.
[5] Yates B, White S. The incidence and risk factors in the development of
medial tibial stress syndrome among naval recruits. Am J Sports Med
2004;32:772–80.
[6] Clarke T, Frederick E, Hamill C. The effects of shoe design parameters
on rearfoot control in running. Med Sci Sport Exerc 1983;15:376–81.
[7] Butler R, Hamill J, Davis I. Effect of footwear on high and low
arched runners’ mechanics during a prolonged run. Gait Posture
2007;26:219–25.
Available online at www.s
97 (2011) 250–255 255
[8] Miller R, Lowry J, Meardon S, Gillette J. Lower extremity mechan-
ics of iliotibial band syndrome during an exhaustive run. Gait Posture
2007;26:407–13.
[9] Eslami M, Begon M, Farahpour N, Allard P. Forefoot-rearfoot coupling
patterns and tibial internal rotation during stance phase of barefoot
versus shod running. Clin Biomech 2007;22:74–80.
10] Stacoff A, Reinschmidt C, Nigg B, Van den Bogert A, Lundberg A,
Denoth J, et al. Effects of foot orthoses on skeletal motion during
running. Clin Biomech 2000;15:54–64.
11] Maynard V, Bakheit A, Oldham J, Freeman J. Intra-rater and inter-rater
reliability of gait measurements with CODA mpx30 motion analysis
system. Gait Posture 2003;17:59–67.
12] Reinschmidt C, Van den Bogert A, Nigg B, Lundberg A, Murphy N.
Effect of skin movement on the analysis of skeletal knee joint motion
during running. J Biomech 1997;30:729–32.
13] Monaghan K, Delahunt E, Caulfield B. Ankle function during gait
in patients with chronic ankle instability compared to controls. Clin
Biomech 2006;21:168–74.
14] Wu G, Siegler S, Allard P, Kirtley C, Leardini A, Rosenbaum D, et al.
ISB recommendation on definitions of joint coordinate system of vari-
ous joints for the reporting of human joint motion – part 1: ankle, hip,
and spine. J Biomech 2002;35:543–8.
15] Lavcanska V, Taylor N, Schache A. Familiarization to treadmill running
in young unimpaired adults. Hum Mov Sci 2005;24:544–57.
16] Stergiou N, Bates B. The relationship between subtalar and knee joint
function as a possible mechanism for running injuries. Gait Posture
changes with fatigue and relationship to injury in female runners. Med
Sci Sport Exerc 2005;37:657–63.
ciencedirect.com
http://www.dh.gov.uk/en/Publichealth/Healthimprovement/Obesity/DH_6585
http://www.dh.gov.uk/en/Publichealth/Healthimprovement/Obesity/DH_6585
- Effect of motion control running shoes compared with neutral shoes on tibial rotation during running
Introduction
Methods
Participants
Instrumentation
Procedure
Data analysis
Results
Discussion
Conclusions
Acknowledgements
References
PH2600Research Methods
Essay: Critique of research articles
Guidance
2019/ 20
The purpose of this assessment is for you to demonstrate your understanding of research
methods by completing a critique of published literature. You will be required to critique
two research articles which have been chosen by the module team. One paper will
involve qualitative methods and one paper quantitative methods. The titles of research
articles will be released on Blackboard Learn in the folder titled Assessment Information.
Your critique essay should be up to 2000 words long and should not exceed this word
limit. The mark awarded for this essay constitutes 100% of the overall module grade.
Guidance for the critique
The quantitative research paper you are being asked to critically appraise is:
Rose, A., Birch, I., Kuisma, R. (2011) ‘Effect of motion control running shoes compared
with neutral shoes on tibial rotation during running.’ Physiotherapy, 97, pp. 250–255
The qualitative paper you are being asked to critically appraise is:
Sharma, H., Bulley, C., van Wijck, F. (2012) ‘Experiences of an exercise referral scheme
from the perspective of people with chronic stroke: a qualitative study.’ Physiotherapy,
98, pp. 336–343
For each paper identify and discuss 3 aspects of the study that you think represent either
a strength or a limitation of the study and explain why you think that is so. These could
include anything from whether the research question is well explained and justified,
whether the overall design is appropriate for the question, how specific aspects of the
design or conduct of the study might impact on validity (internal or external), coherence
or rigour, the approach taken to analysing data or whether the way that the authors of
the paper present and interpret the results is fair and balanced.
For both papers at least one of those 3 aspects must relate to the choice of data analysis
or the actual results section. You should structure your critique in the following manner:
Write it in 2 distinct sections of around 1000 words each. Each section should focus on
one of the papers. The text should read like a fully developed narrative. Bullet point lists
are not acceptable. This is an exercise in academic writing and constructing critical
arguments/ explanations.
For each section follow this format
Introduction. Open with a short section that describes the paper, its topic area and the
research question it seeks to answer. Offer some brief context.
Main body of the critique.
Identify 3 features from each paper that you think represent either strengths or
limitations. For each of these features justify your answers by explaining how these
features have impacted the study or affect our confidence in the conclusions. In this you
will need to show your knowledge of the relevant research methods.
Conclusion. This section should offer a concise and balanced summary of what you have
presented in the main body of the critique. It should summarise the main points that you
have made. You should not be presenting new information here; if something is
important, it should have been mentioned in the main body of the critique.
Wordage. Your essay should be up to 2000 words long with no allowance for extra words.
Your submission may be less than the maximum wordage. If you exceed the upper word
limit, the excess part of your critique will not be read or marked. All content (except for
any tables, charts and references within brackets in the main body of the text) between
the title and reference list is included in the word count.
Referencing.
You need to reference the research article that you are critiquing. Where appropriate you
should also use other referenced resources (i.e. research methods textbooks and journal
articles) to support the points that you are making but remember this is primarily a
critique of the papers themselves, so an extensive and lengthy reference list is not
expected. All references need to be presented in the Harvard style. For detailed guidance
on how to reference see https://libguides.brunel.ac.uk/referencing/overview. The link is
also available on BBL).
Formatting
Please pay attention to the presentation of your work. The work should have a clear title
and your student number should be present. The text should read like a fully developed
narrative. Bullet point lists are not acceptable. This is an exercise in academic writing and
constructing critical arguments/ explanations.
*Do not have your name anywhere on the documents.*
The work should be formatted as follows:
Font: Calibri, Arial or Times New Roman, size 11
Line spacing: 1.5
Margins: Normal (around 2.54cm)
Submission of your assignment.
The submission deadline for the critique is: 12pm midday on Tuesday 21ST April 2020
The submission will be made electronically on WISEFLOW. Further details to follow.
For the electronic submission, please submit your essay as a pdf.
LABEL YOUR FILES AS FOLLOWS:
studentnumber_CRITIQUE (E.G. 98765432_CRITIQUE)
Feedback
https://libguides.brunel.ac.uk/referencing/overview
All students will receive provisional written feedback on this assessment during the week
beginning 11/5/2020.
A
O
c
D
u
r
S
P
s
F
C
w
C
i
©
K
I
a
p
o
d
o
l
m
i
c
(
0
d
Physiothera
py 98 (2012) 336–343
Experiences of an exercise referral scheme from the perspective of people
with chronic stroke: a qualitative study
Helen Sharma a, Cathy Bulley
b
, Frederike M.J. van Wijck c,∗
a Physiotherapy Department, King’s College Hospital, Denmark Hill, London SE5 9RS, UK
b School of Health Sciences, Queen Margaret University, Queen Margaret University Drive, Edinburgh EH21 6UU, UK
c Institute of Applied Health Research and School of Health, Glasgow Caledonian University, Cowcaddens Road, Glasgow GB4 0BA, UK
bstrac
t
bjective To explore stroke survivors’ experiences of undertaking exercise in the context of an exercise referral scheme for people with
hronic stroke.
esign A qualitative design, using semi-structured interviews within a constructivist framework to explore the experiences of individ-
al participants. Verbatim transcripts were thematically analysed. Rigour mechanisms included respondent validation, peer checking, and
eflexivity.
etting An exercise referral scheme, based at a leisure centre in South London.
articipants Nine community-dwelling stroke survivors took part; 5 male and 4 female, mean age 51 years (range 37–61 years); time post
troke 1–4 years, with mixed ethnic backgrounds.
indings Participants described greater physical and psychological well-being following participation in the exercise referral scheme.
ategories that emerged were: improved exercise engagement and confidence, more internalised perceptions of control and enhanced lifestyle,
ork and social roles. Categories linked to form a master theme, labelled: ‘Exercise Referral Scheme as a catalyst for regaining independence.’
onclusions This study supports the value of exercise referral schemes in enabling people with stroke to engage in exercise. For participants
n this study, the scheme seemed influential in the process of regaining independence.
2011 Chartered Society of Physiotherapy. Publis
hed by Elsevier Ltd. All rights reserved.
eywords: Stroke; Exercise; Physical activity; Exercise referral scheme; Community
[
a
r
a
t
c
d
ntroduction
Globally, fifteen million people suffer a stroke each year
nd with five million being left permanently disabled, stroke
laces a considerable burden on family, community and econ-
my [1]. In England, stroke is the largest single cause of adult
isability; over 900,000 people live with the consequences
f stroke, with associated costs estimated at over seven bil-
ion pounds per year [2]. The impact of stroke on individuals
ay include reduced independence, low mood, sensorimotor
mpairment and decreased fitness.
∗ Corresponding author. Tel.: +44 0141 3318967.
E-mail addresses: hrsharma@btinternet.com (H. Sharma),
bulley@qmu.ac.uk (C. Bulley), Frederike.vanWijck@gcu.ac.uk
F.M.J. van Wijck).
i
o
a
fi
e
p
r
031-9406/$ – see front matter © 2011 Chartered Society of Physiotherapy. Publis
oi:10.1016/j.physio.2011.05.004
Reduced physical fitness is common following stroke
3], presenting a risk for recurrent stroke, cardiac disease
nd fall-related fractures [4,5], and a barrier to community
e-integration [6]. With more people surviving strokes [7]
nd government policies shifting health care from hospi-
al towards community [8], improving physical fitness in
ommunity-dwelling stroke survivors is a priority.
A growing body of evidence demonstrates that exercise,
efined as structured and repetitive physical activity (PA) that
s usually planned to enhance fitness [9], can improve a range
f fitness parameters after stroke [10]. However, evidence
lso suggests that PA must be maintained to sustain bene-
ts [11]. UK national clinical guidelines recommend regular
xercise participation and aerobic training where possible, as
art of a long-term strategy after stroke [12–14].
Studies of PA maintenance after stroke are scarce. One
eport demonstrated immediate reductions in PA follow-
hed by Elsevier Ltd. All rights reserved.
dx.doi.org/10.1016/j.physio.2011.05.004
mailto:hrsharma@btinternet.com
mailto:cbulley@qmu.ac.uk
mailto:Frederike.vanWijck@gcu.ac.uk
dx.doi.org/10.1016/j.physio.2011.05.004
iothera
i
l
v
i
t
(
d
p
p
b
E
7
t
i
o
v
w
[
s
a
r
w
i
c
o
p
a
t
t
e
i
a
w
g
e
a
r
t
t
i
o
g
b
H
l
e
f
t
r
c
w
p
p
M
S
c
c
s
a
‘
f
a
o
R
C
e
I
S
t
s
p
s
l
e
t
p
n
t
r
E
t
u
i
s
r
t
b
P
I
i
P
i
s
a
H. Sharma et al. / Phys
ng completion of an exercise programme, and return to
ow levels three months later [15]. General population sur-
eys have found that maintaining sufficient levels of PA
s difficult [16]. To encourage ongoing exercise participa-
ion among different populations, exercise referral schemes
ERS) were developed in the UK [17]. However, four ran-
omised controlled trials (n = 1200 in total) found a return to
re-intervention exercise levels at 12-to-52 weeks after com-
leting an ERS [18–23]. Reasons for this decline in exercise
ehaviour are unclear, mainly due to a lack of qualitative data.
xercise behaviour is complex; one review identified over
0 correlates of exercise participation in adults [24]. With
he ERS model now well established, knowledge of partic-
pants’ experiences with this type of service is essential to
ptimise uptake and ongoing PA participation. Although the
alue of patient experiences in service design and delivery is
ell recognised by the UK National Health Service (NHS)
25] and government policies [26], little is known about
troke survivors’ experiences of exercise in the context of
n ERS.
The first qualitative study to formally explore the expe-
iences of stroke survivors of an exercise intervention [27]
as associated with a clinical trial [11] and found that partic-
pants had enjoyed the classes, felt empowered to take more
ontrol over their recovery, and more motivated to get out
f the house and undertake other activities. Most exercise
articipants continued with an active lifestyle afterwards,
nd felt their quality of life had increased. Another clinical
rial [28] included an exploration of stroke survivors’ percep-
ions of a community-based scheme combining exercise with
ducation [29]. Exercise and goal setting were valued as pos-
tive actions that enabled improvements in physical function
nd confidence. However, these interventions were delivered
ithin the context of clinical trials, and the findings may not
eneralise to routine clinical practice.
To our knowledge, only one study has explored experi-
nces of a community-based ERS for stroke survivors [30],
nd included people with stroke, fitness instructors who
an the scheme and referring physiotherapists. Four main
hemes emerged: the role of the ERS in continuing rehabili-
ation following physiotherapy discharge; concern regarding
nstructors’ level of knowledge about stroke; low levels
f supervision and interaction with instructors; and sug-
ested improvements to the scheme, including closer contact
etween referring physiotherapists and exercise instructors.
owever, perceived impacts of the scheme on participant’s
ives were not explored, and it was not possible to isolate the
xperiences of stroke survivors.
In summary, published literature on exercise after stroke
ocuses primarily on physical impairments and activity limi-
ations, with little information on perceptions and experiences
elating to exercise and any psychosocial impacts of exer-
ise referral schemes. Therefore, the aim of this study
as to explore the experiences of people in the chronic
hase after stroke, who participated in a community-based,
hysiotherapy-led ERS.
s
k
b
w
py 98 (2012) 336–343 337
ethod
tudy design
Individual experiences were explored in depth using a
onstructivist qualitative approach that focuses on socially
onstructed multiple realities [31]; an interpretivist per-
pective that appreciates individuals’ values and meanings;
nd phenomenological methodology, whereby the researcher
brackets’ personal understandings when deriving meaning
rom the data [32].
One-off, one-to-one semi-structured interviews enabled
n individual focus and coverage of important topics with-
ut limiting responses [33]. Approval was granted by the
esearch and Development centre attached to the Primary
are Trust that was responsible for the ERS, and by both
thics committees of the lead researcher’s Higher Education
nstitution and NHS employer.
ample population and recruitment
The study context was an exercise programme provided
hrough an ERS based at a South London leisure centre,
upervised by a chartered physiotherapist and specific to peo-
le with neurological conditions. The ERS is defined in this
tudy as “the referral to, and uptake of, a physiotherapist-
ed exercise programme”. Individually tailored gym-based
xercise took place in a group format, twice weekly, for up
o three months. Participants were generally referred by a
hysiotherapist at least six months after their stroke.
The study population included people with a primary diag-
osis of stroke who had attended the ERS within the previous
wo years, over which recall of experiences was considered
ealistic. To capture varied experiences, attendance levels and
RS completion were not prerequisites for selection. Poten-
ial participants were identified by the lead researcher (HS),
sing NHS patient records and were sent invitation letters,
nformation sheets and consent forms. Those returning con-
ent forms were telephoned to screen for factors that would
ender an in-depth interview unfeasible, including inability
o engage in conversation, a voice potentially incomprehensi-
le on audiotape, or inability to recall their ERS experiences.
articipants were offered interviews at their venue of choice.
t was thought unethical to undertake further measures to
ncrease recruitment rate.
rocedure
The lead researcher (HS) was a female chartered phys-
otherapist with 2 years’ general experience, and 3 years
pecialising in neurology. She was employed at the ERS
s the primary clinician, but only after participants in this
tudy had been discharged, minimising impacts from prior
nowledge or relationships. Participants were aware of her
ackground but the importance of their views and experiences
as emphasised.
338 H. Sharma et al. / Physiotherapy 98 (2012) 336–343
Table 1
Summary of topic guide.
Topic Example question
Demographics Confirm age, time of stroke, when attended ERS
Meaning of exercise to individual What do you think of when I say ‘exercise’?
Past experience of exercise Before your stroke, how active were you? What sorts of things did you do?
Impact of stroke on exercise behaviour How did your stroke affect the exercise that you could do?
Experience before attending ERS Before you went to the ERS, what did you think it would be like? Why do you think you thought that?
Experience of ERS How did you feel about exercising in this way?
Is there anything you didn’t enjoy? Why was that?
Transition from ERS to independent exercising Did you have any plans to carry on exercising? How did that go?
Current status Is there anything from your experience of ERS that affects your life now? Why is that?
The future Do you have any future plans when it comes to exercise?
E
s
s
r
t
b
c
a
c
c
f
i
r
o
i
p
d
b
D
t
i
i
[
a
t
r
F
l
w
y
d
A
t
d
c
c
i
r
[
R
P
w
w
r
f
a
f
I
4
t
C
e
i
c
i
(
a
e
u
E
i
s
E
w
RS: exercise referral scheme.
An interview topic guide was developed to address the
tudy aims, informed by relevant literature. The questions,
ummarised in Table 1, explored experiences of accepting a
eferral to an exercise programme, participating, and percep-
ions of any impacts. Participants’ experiences of exercise
efore stroke, and their understanding of the word ‘exer-
ise’ were explored. Maintenance of exercise participation
fter programme completion was also addressed, based on
oncerns raised by Morgan [34]. Questions were structured
hronologically to aid recall, and phrasing provided scope
or additional areas to emerge [35]. Only the interviewer and
nterviewee were present at each interview.
A video-recorded, role-played pilot interview enabled
efinement of the interview technique and further devel-
pment of the topic guide. Review of this pilot interview
dentified possible influences on the credibility of the partici-
ants’ responses. This reflexive process continued throughout
ata collection through a research diary to minimise effects
iases and presuppositions [32].
ata analysis
Interviews were audiotaped and transcribed verbatim. Par-
icipant verification of initial interpretation took place using
nterview summaries. In-depth analysis used an iterative cod-
ng process, moving towards greater levels of abstraction
36]. Text was first read and re-read by the lead researcher,
nd annotated with labels that described concepts (first level
hemes). Similar labels were grouped where they described
elated ideas, and given another label (second level theme).
urther abstraction led to the creation of categories (third
evel themes), and finally a master theme (Table 2). This
as iterative, as text from all interviews was analysed. Anal-
sis was considered complete when no new themes were
efined and all relevant text was incorporated in a theme.
word processing package was used rather than a qualita-
ive data analysis programme; while the latter is useful for
ata management, it does not perform the thinking, and can
reate distance from the text [37]. A sample of themes and
ontributing text units was reviewed by an experienced qual-
tative researcher (CB); discussion led to consensus between
‘
w
a
W
esearchers, which enhanced dependability of the analysis
38].
esults
articipant characteristics
Of forty-one invited participants, 12 consented, three of
hom could not be included: one was not contactable, one
as not sufficiently fluent in English, and one could not
ecall ERS participation. Interviewed participants included
our female and five male community-dwelling adults (mean
ge 51; range 37–61 years), who had had a stroke one to
our years previously. Four were of White English, one White
rish, and four Black African ethnic origins. Interviews lasted
5–80 minutes. Six occurred in participant’s homes, one at
heir workplace, and two at a hospital office.
ategories
Four categories emerged: exercise engagement (i.e. rel-
vant behavioural changes); improvement (i.e. individuals’
nterpretations of physical and psychological improvements);
ontrol (changing views regarding who was responsible for
ncreased feelings of independence); and finally, confidence
i.e. a growing sense of self-confidence in the individual’s
bility to exercise). The account focuses on varied experi-
nces, rather than consensus views. Pseudonyms have been
sed throughout.
xercise engagement
Between the stroke and ERS, participants recalled slow-
ng down and making adjustments to lifestyle, work and
ocial roles. This pace-change appeared to be challenged by
RS attendance, which facilitated increases in activity levels
ithin sessions, and outside the ERS:
Before I started going [to the exercise referral scheme], I
asn’t thinking about exercise, and I wasn’t thinking about
nything, other than sit at home, eat and watch television.
hen I started, at least they gave me that ability, they gave
H. Sharma et al. / Physiotherapy 98 (2012) 336–343 339
Table 2
Definitions of themes and categories.
Product of data analysis Definition Example
1st level themes An interpretation of descriptive information (age, ethnicity,
etc.) and coding (labelling text according to subject); a
reflection on meaning
Specific text was grouped, according to how participants
viewed relationships with the physiotherapist. These were
further labelled to reflect the interpretation that some text
focused on the ‘personal service’ from the physiotherapist, and
other text described experiences of ‘partnership’ – these were
designated 1st level themes
2nd level themes Relationships between first level themes Further analysis of ‘personal service’ revealed that it was
linked to participants’ observations of the personal qualities of
the physiotherapist. These relationships formed 2nd level
themes, e.g. ‘A personal service linked to the physiotherapist’s
caring quality’
3rd level
themes/categories
Collections of second level themes united by a central idea The idea of ‘confidence’ transcended many 2nd level themes
and their components. ‘Confidence’ linked related ideas,
including ‘the physiotherapist’, ‘the group’, ‘social situations’,
‘the rehabilitation setting’
Master theme A general pattern or overarching theme that is illustrated by “ERS as a catalyst for regaining independence”. Categories
m
h
i
‘
t
m
d
I
t
‘
t
s
m
m
e
‘
w
i
‘
t
g
C
m
t
i
‘
w
w
t
r
m
s
‘
s
s
y
C
i
r
w
c
‘
t
f
c
h
‘
t
s
t
i
a
‘
h
i
several categories
e that push . . . So thereafter, I just cook up something in my
ead, go down the stairs or go down the street.’ [Louise]
Increased activity generated feelings of normality and
ndependence:
Because when I do exercise, when I go out, it puts me back
o normal. And when I see others walking, what would make
e not walk? I am not disabled. The stroke has not made me
isabled, so I walk.’ [Mary]
mprovement
Participants identified improvements following ERS par-
icipation, predominantly in fitness, strength and movement:
If I didn’t go [to the exercise referral scheme] . . . it would
ake a long time for me to be moving around . . . I gained more
trength . . . [the exercise referral scheme] really helped my
oving, my walking, how I am doing everything.’ [Beth]
Several participants also identified ‘immediate’ improve-
ents to mood following the exercise sessions, perceived as
vidence that the stroke was ‘going away’ and that they were
getting better’. Positive feelings of happiness and enjoyment
ere described, and exercise became a source of pleasure in
tself:
When I finish exercising and I feel so good, so content. . . By
he end of the day you feel good, you know, you say ‘I feel
ood, my health is coming back’ in your head.’ [Peter]
ontrol
Interestingly, participants attributed perceived improve-
ents during initial rehabilitation to external factors, such as
he physiotherapist, God, the consultant, or the health service
n general, and perceived themselves as dependent:
When I’d gone walking, I’d always had somebody from the
ard going out with me and holding me up, or I was in the
heelchair and I hated it.’ [Jim]
s
l
w
were seen as personal benefits of the ERS that were connected,
contributing to greater normality and independence
When recalling ERS participation, interviewees expressed
he importance of their own personal qualities to successful
ecovery and increasing independence, attributing improve-
ents to internal factors such as motivation, willpower and
elf-determination:
The physio is something else, the exercise programme is
omething else, you yourself willing to do something for your-
elf, and having that willpower to proceed and progress what
ou want to do is another thing.’ [Louise]
onfidence
Low confidence was described as a barrier in relation to
nitial ERS attendance. Attitudinal barriers in the form of
eservations about the gym environment were described, as
ell as physical barriers en route from home to the leisure
entre.
My own sense of . . . my self-esteem was very low anyway,
he fact that I couldn’t physically do things I used to take
or granted, and I don’t particularly like that kind of macho
ulture anyway. I wouldn’t want to expose myself to it. I’d
ave been worried about people poking fun.’ [Tony]
Initially when I was discharged from hospital, I want to go
o the gym. When I come out from my house and I look at the
tairs, it’s like I want to fall over, I go back inside and shut
he door.’ [Louise]
Confidence that was specifically related to exercise
ncreased within the ERS, facilitated by the physiotherapist
nd group dynamic:
I mean me, all I can say is that I had determination. I didn’t
ave the confidence, but I had determination and they [phys-
otherapists] gave me the confidence. But when I saw I could
tart to do a thing, that was it. Once I got to start doing it I
ooked at my sheet and saw I did that little bit more than last
eek.’ [Jim]
340 H. Sharma et al. / Physiotherapy 98 (2012) 336–343
Table 3
An overview of the master theme and contributing categories and themes.
Master theme Constituent categories and definitions Second level themes contributing to the category
‘ERS as a catalyst for regaining
independence’
1a. Exercise engagement: behavioural changes
regarding exercise
1ai. Links between confidence and exercise engagement
1aii. Links between physical improvement and exercise
engagement
1aiii. Exercise engagement and perceived exercise restrictions
or opportunities
1aiv. Reflections on the self and exercise engagement
1b. Control: participants’ changing views on who/what
was responsible for them gaining independence
1bi. Links between the rehabilitation setting and perceptions of
control
1bii. Varying roles of the physiotherapist
1biii. Perceptions of control and exercise behaviour
1biv. Perceptions of control and the discharge procedure
1c. Confidence: the progression of feelings of
self-confidence and beliefs in exercise ability
1ci. Links between the rehabilitation setting and confidence
1cii. Links between perceptions of the physiotherapist and
confidence
1ciii. Links between perceptions of the group and confidence
1civ. Links between independence, emotions and confidence
1cv. Confidence and social situations
1d. Improvement: Physical and psychological
improvements and their meanings to participants
1di. Links between improvement and beliefs about the exercise
programme
1dii. Links between improvement and beliefs about exercise
generally
o
‘
s
a
r
t
a
c
t
w
‘
a
c
i
M
n
p
o
d
r
w
t
t
D
M
a
t
i
m
f
t
c
a
o
i
o
t
a
s
p
a
c
s
r
m
Increased confidence also affected more general social and
ccupational activities:
I was ashamed, limping all the time and using a walking
tick. But when I got there, I saw people that were even older
nd even younger than me and they were there for numerous
easons. And these are people that say, when we finish using
he gym, ‘I am going to the high street for window shopping’,
nd I tag along. . . At times, I just walk into the shopping
entre with confidence. . ..’ [Louise]
One participant expressed a belief that the ERS con-
ributed to increased confidence, and felt able to return to
ork:
I started work and I was able to start where I left off. . .
nd if I had not gone through this I would not have had the
onfidence. . . It is not the medication that has made me better,
t is the exercise. . .’ [Peter]
aster theme
The categories emerging from the data describe a jour-
ey through engagement with exercise after stroke, yielding
hysical and psychological improvements, increased feelings
f personal control, and raised confidence. These personal
evelopments appeared to be linked in their contributions to
egaining independence. Thus, the master theme that emerged
as entitled: ‘ERS as a catalyst for regaining independence’.
These results are summarised in Table 3, demonstrating
he audit trail from second level themes, through categories,
o the master theme.
fi
s
e
1diii. Links between beliefs and feelings about exercise
1div. Impacts of beliefs and feelings about exercise on exercise
engagement
iscussion
ain findings
The findings from this study suggest that the ERS encour-
ged participants to become more active within and outside
he sessions. Participants reported physical and psycholog-
cal improvements, including fitness, strength, movement,
ood and enjoyment. Locus of control appeared to shift
rom predominantly external during initial rehabilitation,
o more internal during the ERS. Initial barriers to exer-
ise participation had to be overcome, including concerns
bout getting to the venue, low self-confidence, and feelings
f self-consciousness. Generic and PA-specific self efficacy
ncreased, and the group format appeared to induce feelings
f independence and normality.
Reduced independence is a key problem after stroke with
wo thirds of people experiencing limitations in at least one
ctivity of daily living five years after stroke [39]. The current
tudy revealed that participants had been involved in a slow
rocess of regaining independence, in which the ERS acted
s a catalyst, enhancing perceptions of greater normality,
onfidence and independence.
These findings contribute to the literature on exercise after
troke; one systematic review highlighted the lack of research
elating to effects of exercise on disability, dependence,
ood, self-efficacy and locus of control [10]. The current
ndings resonate with other qualitative work, where stroke
urvivors reported that exercise enhanced physical function,
mpowerment and confidence [27,29]. When looking beyond
iothera
l
t
a
e
t
s
f
i
s
t
e
s
o
i
s
c
p
p
t
s
[
I
s
e
i
b
s
t
c
[
t
p
[
u
b
o
c
s
s
p
S
p
r
e
s
b
i
o
r
s
w
t
E
p
s
t
a
p
w
p
[
a
o
a
t
m
a
i
p
c
v
t
e
r
o
m
d
C
o
t
t
i
a
o
e
s
s
i
t
i
e
H. Sharma et al. / Phys
iterature specific to stroke, the significance of ERS participa-
ion was also identified by one study in an ageing population
nd one in people with mental ill-health [40,41]. Recovery of
lements of the self was described, as participants were said
o ‘reconnect with their human potential’ [40] and ‘reclaim a
ense of personhood’ [41]. This may be particularly important
or stroke survivors who have experienced dramatic changes
n their physical, emotional and social worlds. In the current
tudy, one participant reported feeling ‘completely lost . . .
he stroke took my life away from me; everything I could do,
verything that meant anything to me’ [David]. Evidently,
uch qualitative data deepen our understanding of the impact
f exercise on the self as a whole.
All consenting participants in this study experienced pos-
tive changes in their physical and psychological well-being,
ome up to four years post-stroke. This supports UK national
linical guidelines for stroke [12–14], which advocate the
romotion of an active lifestyle where possible. The ERS
rovides one strategy to this effect and further collabora-
ion between physiotherapy and local health and leisure
ervices is required to facilitate access to exercise after stroke
14,42–44].
mplications of findings for exercise after stroke referral
chemes
Service development requires evidence of efficacy, and
xisting ERSs have been criticised for using primarily
mpairment-orientated outcomes (e.g. blood pressure), and
eing insufficiently client-centred [18]. The current study
uggests that ERS evaluation would benefit from addi-
ional measures relating to mood, self-efficacy and locus of
ontrol.
Low recruitment and attendance must also be addressed
34]. A survey found that the three most common barriers
o exercise after stroke were programme cost, lack of trans-
ortation and lack of knowledge about local exercise facilities
45]. In the current study, some participants said they were
ncomfortable around able-bodied exercisers, which could
e exacerbated in a gym setting where the focus is often
n physical perfection. To reduce perceived barriers to exer-
ise after stroke, opportunities to exercise in a peer group and
olutions for environmental barriers (e.g. providing transport)
hould be offered where possible, as recommended in the best
ractice guidelines [44].
tudy limitations
The credibility of the data must be considered. The sam-
le was small, with a low response rate, possibly due to
ecruitment by mail. Although not ideal, this was approved on
thical review. However, the sample was consistent with the
tudy design and aims of increased depth of insight that can
e transferred to similar contexts, rather than generalisabil-
ty of results [46]. Data were extensive, with 45–80 minutes
f interview per participant. While data saturation through
A
b
py 98 (2012) 336–343 341
epeated interviews was not possible within the scope of the
tudy, analytical saturation was achieved [46].
The sample was not necessarily representative of the
ider stroke population, with varied demographic charac-
eristics and a low mean age, which may have affected
RS uptake. Similar positive perspectives on ERS partici-
ation were expressed, which may have been influenced by
ocial desirability and convenience sampling. The views of
hose who did not accept an invitation to the ERS were not
ddressed, and future research should use purposive sam-
ling to explore successful and unsuccessful experiences. It
ould also be valuable to collect data relating to time since
articipation and frequency of participation.
The ethnic diversity was representative of the local area
47], but in some cases, vocabulary and expression may have
ffected interpretation of responses. A field diary was used in
rder to reflect on possible influences during data collection
nd analysis.
There were insufficient resources to provide communica-
ion support, and regretfully, persons with memory impair-
ents or communication difficulties were therefore excluded,
s is frequently the case. The resulting bias is acknowledged.
Five out of nine participants returned and agreed with the
nterview summaries, enhancing credibility of initial inter-
retation. Respondent validation of the final themes was
onsidered less useful because not all themes related to the
iews of every participant [48]. Although the lack of verifica-
ion of four summaries limits credibility, it was not considered
thical to pursue the missing summaries. On balance, the
esults have credibility and were generated through a rigor-
us process. The relevance of these findings to further settings
ust be evaluated by the reader, using the contextual and
emographic information provided [32].
onclusions
This study aimed to explore stroke survivors’ experiences
f exercise in the context of a community-based ERS. Par-
icipants perceived the scheme as an important driver in
he process of regaining independence, as they experienced
mprovements in physical and psychological function. They
lso reported a shift from an external to a more internal locus
f control and improved general and exercise-specific self-
fficacy, which carried over into activity outside the exercise
essions. The findings from this – albeit small-scale – study
upport exercise referral schemes as a method for increas-
ng physical activity after stroke. However, some barriers
o exercise participation also emerged, and further research
s required to explore how uptake and continued exercise
ngagement after stroke can be optimised.
cknowledgements
The following are gratefully acknowledged: the contri-
ution of the study participants, and the assistance of Helle
3 iothera
S
C
E
C
U
C
c
R
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
42 H. Sharma et al. / Phys
ampson (Clinical Lead Physiotherapist, Southwark Primary
are Trust) with the pilot interview.
thical approval: King’s College Hospital Research Ethics
ommittee (Ref. No. 06/Q0703/210) and Queen Margaret
niversity, Subject Area Ethics Panel, no ref. number.
onflict of interest: The authors report that there were no
onflicts of interest.
eferences
[1] Mackay J, Mensah G. Atlas of heart disease and stroke. Geneva:
World Health Organisation; 2004, available at: http://www.who.int/
cardiovascular diseases/resources/atlas/en/ [accessed 2010 November
16].
[2] Saka RO, McGuire A, Wolfe CDA. Economic burden of stroke in Eng-
land. London: University of London; 2003, available at: http://www.
nao.org.uk/publications/0506/reducing brain damage.aspx [accessed
2010 November 16].
[3] Michael KM, Allen JK, Macko RF. Reduced ambulatory activity after
stroke: the role of balance, gait and cardiovascular fitness. Arch Phys
Med Rehabil 2005;86:1552–6.
[4] Roth EJ. Heart disease in patients with stroke: incidence, impact, and
implications for rehabilitation. Part 1: classification and prevalence.
Arch Phys Med Rehabil 1993;74:752–60.
[5] Marigold DS, Eng JJ, Dawson AS, Ingis JT, Harris JE, Gylfadottir S.
Exercise leads to faster postural reflexes, improved balance and mobil-
ity, and fewer falls in older persons with chronic stroke. J Am Geriatr
Soc 2005;53:416–23.
[6] Mayo NE, Wood-Dauphine S, Ahmed S. Disablement following stroke.
Disabil Rehabil 1999;21:258–68.
[7] Truelsen T, Piechowski-Jóźwiak B, Bonita R, Mathers C, Bogous-
slavsky J, Boysen G. Stroke incidence and prevalence in Europe: a
review of available data. Eur J Neurol 2006;13:581–8.
[8] Department of Health. National stroke strategy. London: Depart-
ment of Health; 2007, available at: http://www.dh.gov.uk/en/
Publicationsandstatistics/Publications/PublicationsPolicyAnd
Guidance/DH 081062 [accessed 2010 November 16].
[9] Caspersen CJ, Powell KE, Christenson GM. Physical activity, exer-
cise and physical fitness: definitions and distinctions for health-related
research. Public Health Rep 1985;100:126–30.
10] Saunders DH, Greig CA, Mead GE, Young A. Physical fitness
training for stroke patients. Cochrane Database Syst Rev 2009;(4),
doi:10.1002/14651858.CD003316.pub3 [Art. No.: CD003316].
11] Mead G, Greig CA, Cunningham I, Lewis SJ, Dinan S, Saunders DH,
Fitzsimons C, Young A. STroke: a randomised trial of exercise or
relaxation (STARTER). J Am Geriatr Soc 2007;55:892–9.
12] Royal College of Physicians. Stroke guidelines: national clinical
guidelines for stroke. 3rd ed. London: Royal College of Physicians;
2008.
13] Scottish Intercollegiate Guidelines Network. SIGN guideline 108:
management of patients with stroke or TIA: assessment, inves-
tigation, immediate management and secondary prevention: a
national clinical guideline. Edinburgh: Scottish Intercollegiate
Guidelines Network; 2008, available at: http://www.sign.ac.uk/
guidelines/fulltext/108/index.html [accessed 2010 November 16].
14] Scottish Intercollegiate Guidelines Network. Guideline 118: manage-
ment of patients with stroke: rehabilitation, prevention and management
of complications, and discharge planning: a national clinical guide-
line. Edinburgh: Scottish Intercollegiate Guidelines Network; 2010,
available at: http://www.sign.ac.uk/guidelines/fulltext/118/index.html
[accessed 2010 November 16].
15] Touillet A, Guesdon H, Bosser G, Beis J-M, Paysant J. Assessment
of compliance with prescribed activity by hemiplegic stroke patients
[
py 98 (2012) 336–343
after an exercise programme and physical activity education. Ann Phys
Rehabil Med 2010;53:250–65.
16] The Information Centre [Internet]. Statistics on obesity, physical
activity and diet. UK: National Health Service; 2006, available at:
http://www.ic.nhs.uk/webfiles/publications/opan06/OPAN%20bulletin
%20finalv2 [accessed 2010 November 16].
17] Department of Health. Exercise referral systems: a national
quality assurance framework. London: Department of Health;
2001, available at: http://www.dh.gov.uk/en/Publicationsandstatistics/
Publications/PublicationsPolicyAndGuidance/DH 4009671 [accessed
2010 November 16].
18] Dugdill L, Graham RC, McNair F. Exercise referral: the public health
panacea for physical activity promotion? A critical perspective of exer-
cise referral schemes; their development and evaluation. Ergonomics
2005;48:1390–410.
19] Taylor AH, Doust J, Webborn N. Randomised controlled trial to exam-
ine the effects of a GP exercise referral programme in Hailsham, East
Sussex, on modifiable coronary heart disease risk factors. J Epidemiol
Commun Health 1998;52:595–601.
20] Halbert JA, Silagy CA, Finucane PM, Withers RT, Hamdorf PA. Phys-
ical activity and cardiovascular risk factors: effect of advice from
an exercise specialist in Australian general practice. Med J Aust
2000;173:84–7.
21] Lamb SE, Bartlett HP, Ashley A, Bird W. Can lay-led walk-
ing programmes increase physical activity in middle-aged adults?
A randomised controlled trial. J Epidemiol Commun Health
2002;56:246–52.
22] Harrison RA, Roberts C, Elton PJ. Does primary care referral to an exer-
cise programme increase physical activity 1 year later? A randomised
controlled trial. J Public Health 2004;27:25–32.
23] Williams NH, Hendry M, France B, Lewis R, Wilkinson C. Effective-
ness of exercise-referral schemes to promote physical activity in adults:
systematic review. Br J Gen Pract 2007;57:979–86.
24] Trost SG, Owen N, Bauman AE, Sallis JF, Brown W. Correlates of
adults’ participation in physical activity: review and update. Med Sci
Sports Exerc 2002;34:1996–2001.
25] Department of Health. The expert patients programme. London:
Department of Health; 2007, available at: http://webarchive.
nationalarchives.gov.uk/+/www.dh.gov.uk/en/Aboutus/Ministersand
DepartmentLeaders/ChiefMedicalOfficer/ProgressOnPolicy/Progress
BrowsableDocument/DH 4102757 [accessed 2010 November 16].
26] The Scottish Government. Better together: Scotland’s patient expe-
rience programme: patient priorities for inpatient care report no.
5/2009; 2009, available at: http://www.scotland.gov.uk/Publications/
2009/07/08114927/0 [accessed 2010 November 16].
27] Carin-Levy G, Kendall M, Young A, Mead G. The psychosocial effects
of exercise and relaxation classes for persons surviving a stroke. Can J
Occup Ther 2009;76:73–80.
28] Harrington R, Taylor G, Hollinghurst S, Reed M, Kay H, Wood VA. A
community-based exercise and education scheme for stroke survivors:
a randomized controlled trial and economic evaluation. Clin Rehabil
2010;24:3–15.
29] Reed M, Harrington R, Duggan A, Wood VA. Meeting stroke survivors’
perceived needs: a qualitative study of a community-based exercise and
education scheme. Clin Rehabil 2010;24:16–25.
30] Wiles R, Demain S, Robison J, Kileff J, Ellis-Hill C, McPherson K.
Exercise on prescription schemes for stroke patients post-discharge
from physiotherapy. Disabil Rehabil 2008;30:1966–75.
31] Crotty M. The foundations of social research. London: Sage Publica-
tions; 1998.
32] Grbich C. Qualitative research in health. London: Sage Publications;
1999.
33] DiCicco-Bloom B, Crabtree BF. The qualitative research interview.
Med Educ 2006;40:314–21.
34] Morgan O. Approaches to increase physical activity: reviewing the
evidence for exercise-referral schemes. Public Health 2005;119:
361–70.
http://www.who.int/cardiovascular_diseases/resources/atlas/en/
http://www.nao.org.uk/publications/0506/reducing_brain_damage.aspx
http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_081062
http://www.sign.ac.uk/guidelines/fulltext/108/index.html
http://www.sign.ac.uk/guidelines/fulltext/118/index.html
http://www.ic.nhs.uk/webfiles/publications/opan06/OPAN%20bulletin%20finalv2
http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4009671
http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/Aboutus/MinistersandDepartmentLeaders/ChiefMedicalOfficer/ProgressOnPolicy/ProgressBrowsableDocument/DH_4102757
http://www.scotland.gov.uk/Publications/2009/07/08114927/0
iothera
[
[
[
[
[
[
[
[
[
[
[
[
[
H. Sharma et al. / Phys
35] Britten N. Qualitative research: qualitative interviews in medical
research. BMJ 1995;311:251–3.
36] Richards L. Handling qualitative data: a practical guide. London: Sage
Publications; 2005.
37] Auerbach C, Silverstein L. Qualitative data: an introduction to coding
and analysis. New York: NYU Press; 2003.
38] Mays N, Pope C. Qualitative research: rigour and qualitative research.
BMJ 1995;311:109–10.
39] Wilkinson PR, Wolfe CD, Warburton FG. A long-term follow-up of
stroke patients. Stroke 1997;28:507–12.
40] Stathi A, McKenna J, Fox KR. The experiences of older people
participating in exercise referral schemes. J R Soc Promot Health
2003;124:18–23.
41] Crone D, Smith A, Gough B. ‘I feel totally at one, totally alive
and totally happy’: a psycho-social explanation of the physical
activity and mental health relationship. Health Educ Res 2005;20:
600–11.
42] Mead GE. Exercise after stroke. BMJ 2009:247–8.
[
Available online at www.s
py 98 (2012) 336–343 343
43] Morris JH, Williams B. Optimising long-term participation in physical
activities after stroke: exploring new ways of working for physiother-
apists. Physiotherapy 2009;95:227–33.
44] Best C, van Wijck F, Dinan-Young S, Dennis J, Smith M, Fraser H,
Donaghy M, Mead G. Best practice guidance for the development of
exercise after stroke services in community settings. Edinburgh: Edin-
burgh University; 2010, available at: http://www.exerciseafterstroke.
org.uk/ [accessed 2010 November 16].
45] Rimmer JH, Wang E, Smith D. Barriers associated with exercise and
community access for individuals with stroke. J Rehabil Res Dev
2008;45:315–22.
46] Krefting L. Rigor in qualitative research: the assessment of trustwor-
thiness. Am J Occup Ther 1991;45:214–22.
47] Commission for Racial Equality [Internet]. London, UK; 2007, avail-
able at: http://www.cre.gov.uk/diversity/map/london/southwark.html
[accessed 2007 May 11].
48] Koch T, Harrington A. Reconceptualising rigour: the case for reflexiv-
ity. J Adv Nurs 1998;28:882–90.
ciencedirect.com
http://www.exerciseafterstroke.org.uk/
http://www.cre.gov.uk/diversity/map/london/southwark.html
- Experiences of an exercise referral scheme from the perspective of people with chronic stroke: a qualitative study
Introduction
Method
Study design
Sample population and recruitment
Procedure
Data analysis
Results
Participant characteristics
Categories
Exercise engagement
Improvement
Control
Confidence
Master theme
Discussion
Main findings
Implications of findings for exercise after stroke referral schemes
Study limitations
Conclusions
Acknowledgements
References
Non-experimental
(observational) designs
Neil O’Connell
PH2600 Research Methods
Learning
outcome
s
By the end of the lecture students should
be able to:
Describe basic common observational
study designs
Describe the concept of correlation
Consider some of the biases that we
attempt to control for
It’s a complex world
Clinical phenomena
and human behaviour
are complex
Research seeks to
unravel the
complexity
But experiments are
often not possible or
appropriate…
because…
Many naturally occurring
variables can’t be
manipulated
Many variables shouldn’t
be manipulated (ethics)
The research question
does not require
manipulation of
variables (descriptive/
exploratory)
Experiments are simply
not feasible
DESCRIBE THE
CHARACTERISTICS
OF A CERTAIN
GROUP
DRAW
COMPARISONS
BETWEEN
DIFFERENT
GROUP
S
ANALYSE
RELATIONSHIPS
BETWEEN NATURALLY
OCCURRING VARIABLES
(CORRELATIONAL
DESIGN)
INVESTIGATE THE
INFLUENCE OF
VARIABLES THAT
CANNOT BE
EXPERIMENTALLY
MANIPULATED
INVESTIGATE THE
NATURAL COURSE
OF A CONDITION
INVESTIGATE RISK
FACTORS FOR A
CERTAIN
OUTCOME
Researchers might want to….
Hierarchy of study design
Experimental &
quasi-experimental
studies
Systematic reviews (+/- meta-
analysis)
Randomised Controlled Trial (RCTs)
Controlled clinical trials
Observational
studies
Cohort
Case-control
Cross-sectional
Case-report, case-series
Expert opinion etc.
Generate
hypotheses
Establish
causality
Descriptive
Describe the sample
characteristics,
behaviours,
conditions
Exploratory/ Analytical
Systematically
investigate relationships
between variables
Hypothesis-driven
Correlational designs
Correlation designs
Analysis of relationships:
correlation research to understand:
1. Whether patterns exist within data.
2. Whether and how variables are related.
Correlation
Variables often co-vary
A variable rises or lowers in relation
to the behaviour of another variable
These variables might be described as
“related” or “associated” with one
another
We measure these associations
statistically
The problem of causality
www.xkcd.com
Study
Population
Cross-sectional
Prospective Retrospective
Longitudinal
Basic model
• aim is to measure (quantify) the relationship between
exposure &
outcome
• cannot determine whether causal link – need experimental
design
EXPOSURE
Risk factor
Determinant
Treatment
OUTCOME
Disease
Mortality
Health
Cross-sectional studies
• Simplest form of observational study
• Describes population at one point in time
‘snapshot’
• Exploration of factors associated with a
disease
or condition (risk and/or protective factors)
• Can compare subgroups within a
population
Typical research question
“What proportion of the population have characteristic x?”
Typical cross sectional study
questions
What are the
characteristics of this
population?
What proportion of the
population have
characteristic X?
What variables are
related to
characteristic X
Do people with
characteristic X differ
from those who don’t?
Important to consider the temporal association
between exposure & outcomes
PAST
PRESENT FUTURE
EXPOSURE
+ OUTCOMEEXPOSURE
X sectional study
TIME
Cross-sectional study: example
Elliot et al. Lancet 1999, 354: 1248-52
Research question: what proportion of the general
population in North East Scotland have chronic pain?
Sample:
– random sample of adults registered with GP in
Grampian region (~5000)
– stratified sample by age & sex
Method: Postal self-completion questionnaire, chronic
pain grade questionnaire , standard definitions
Results: 46% popn had chronic pain, major problem in
general practice
Cross-sectional studies
Strengths
– quick to conduct, relatively cheap
– allows exploration of multiple risk factors
– useful to assess health needs of population
– useful first step: hypothesis generating – may
provide clues & justification for further
investigation
Cross-sectional studies
Weaknesses
– selection bias (selection of sample, responders etc)
– cannot prove cause – single point in time
(snapshot)
Cohort studies
• Observes people over time
– can be for weeks, months or years
• Exposures (agents/risk factors) & outcomes are
recorded
Purpose
To understand the natural history of a disease
To evaluate long-term patient health or quality of
life
Analyse the relationship between risk factors &
disease
Cohort studies
Cohort = group of people who share common
exposure (e.g. year of birth)
= Latin legion of soldiers
• E.G. start with people without the disease, document
exposures, follow through time
• Includes those with & without risk factors
• Determines incidence (new cases) of disease
Research question
Whether exposure to factor ‘x’ leads to disease ‘y’?
OUTCOME PRESENT
NO OUTCOME
OUTCOME PRESENT
NO OUTCOME
PO
PU
LATIO
N
O
F IN
TER
EST
PRESENT FUTURE
MEASURE
EXPOSURE (S) AT
BASELINE
EXPOSED
NOT EXPOSED
MEASURE
OUTCOMEOF
INTEREST
SAMPLE
HEALTHY
GROUP
ACUTE
DISEASE
WHO
DEVELOPES
DISEASE?
WHO
RECOVERS?
WHO
BECOMES
CHRONIC?
CHRONIC
DISEASE
WHO
RECOVERS?
INCEPTION COHORT
‘Landmark’ cohort studies
UK Doctors Cohort Study
(1950s)
– 40,000 UK doctors
USA Nurses Cohort Study (mid-1970s) >121,700
female nurses (Oral contraceptive and breast Ca)
UK Oral Contraceptive Pill cohort study(1968 –
present)
– 46,000 women , all cause mortality
women >50 years (1 in 4 women in UK) HRT, Ca,
other disease
UK Doctors Cohort Study
CAUSE‐SPECIFIC
MORTALITY (LUNG CA)
ALIVE/ DEAD OF OTHER
CAUSE
CAUSE‐SPECIFIC
MORTALITY (LUNG CA)
ALIVE/ DEAD OF OTHER
CAUSE
1950 2001
MEASURE
EXPOSURE (S)
TOBACCO
USE
NO TOBACCO
MEASURE
OUTCOMEOF
INTEREST
UK
DOCTORS
Tobacco & mortality
1951: Smoking questionnaire from 40,000 British doctors
(Doll & Hill)
Assessed: 1951, 1957, 1966, 1971, 1978, 1991, 2001….
Analysis
• Excess mortality in smokers
• Reversal of effect in ex-smokers
• Other diseases eg IHD, respiratory disease, CVD, bladder
cancer, etc.
BMJ paper, Doll et al. 2004:
cessation at age 30 gain 10 years life expectancy
age 40 gain 9 yrs
age 50 gain 6 yrs
age 60 gain 3 yrs
Dose-response relationship
– evidence for causality (one of Bradford-Hill’s criteria)
Strengths of cohort design
• Can directly estimate risk and relative risk of disease in
a population
• Can determine the natural history of a condition
• Can calculate incidence
• Can demonstrate temporal sequence
• Analysis of many different exposures/outcomes
• Less prone to bias than case-control and x-sectional
studies
ATTRITION
BIAS
e.g. loss to follow
up (not at
random)
SELECTION BIAS
e.g.
unrepresentative
population
MEASUREMENT OR
CLASSIFICATION
BIAS
e.g. poor
standardisation,
unblinded
assessors, changes
in diagnostic
criteria
Weaknesses of cohort studies
Case-control studies
Start with people with disease [outcome]
Two groups:
– Cases those with disease
– Controls those without disease
Collect data on risk factors [exposures] of
interest
Compare odds of exposure in two groups
Research question
“ Are those with disease “x” more likely, than those without
disease x, to have been exposed to risk factor “y”?
Key features
• Observational design ~ hypothesis generating
• Identification of subjects with disease X (or other
outcome of interest)
• Identification of suitable control group without
disease X
• For both groups, past exposure is then determined
• Retrospective or historical in nature of enquiry
CASES
(with disease)
CONTROLS
(disease free)
RISK FACTOR
NO RISK FACTOR
RISK FACTOR
NO RISK FACTOR
PO
PU
LAT
IO
N
O
F IN
T
ER
EST
PAST PRESENT
DIRECTION OF ENQUIRY (RETROSPECTIVE)
Historical example: case-control study
• 1960—1961 in Hamburg,
obstetricians noted 27 infants
born with severe malformations
• Rare condition, no cases
observed between 1930–1958
• German study recruited
mothers of affected infants &
sample of mothers of healthy
infants
• Detailed investigation of
behaviours & practices during
pregnancy; documentation of
multiple exposures in both
groups of women
Historical example: case-control study
• Identified higher use of drug thalidomide in cases
• Sleeping pill introduced in late 1950s, safer than
barbiturates – did not induce coma
• Prescribed for pregnant women, anti-emetic taken
between 27–40th week of pregnancy
• Drug had been used in 46 countries – led to overhaul of
drug development & licensing, withdrawn 1961
• http://www.guardian.co.uk/society/2012/sep/01/thali
domide-cover-up
Mothers of babies
with birth
abnormalities
(n=100)
Mothers of healthy
babies
(n=200)
Thalidomide
No Thalidomide
Thalidomide
No Thalidomide
SA
M
P
LE
O
F
M
O
T
H
E
R
S
DIRECTION OF ENQUIRY (RETROSPECTIVE)
OUTCOME (DISEASE)EXPOSURE
Selection of cases & controls
Most critical stage in a case-control study
Selection of cases
– clearly define ‘cases’
– specify inclusion & exclusion criteria
– where is the ‘source’ population? e.g. residents from
geographical region, patients recruited from a hospital ward
or clinic.
Selection of controls
– same source as the cases
– random selection of controls from source population
– or matching (age, sex, SE status…..)
– increase comparator group e.g. >2 controls : 1 case
Case-control contd.
Data collection & measurement
Collecting information about the past…
– questionnaires / face to face interviews
– GP or hospital records
– medical examination
– occupational records
– or other previously collated data e.g. blood
tests, biological markers
– May be missing or incomplete
Strengths of case-control design
• Very useful for investigating rare diseases or
diseases with a long induction period e.g. some
cancers
• Time and cost efficient – when compared to
cohort studies
• Existing records /database can be used
• Permits investigation of multiple risk factors
/exposures
Weaknesses: case control
SELECTION BIAS
(poor case
definition, difficult
finding
representative
controls)
RECALL BIAS
(incomplete,
unreliable recall of
past events,
incomplete records,
worse without
blinding)
OBSERVER BIAS
(awareness of
hypothesis,
exposure or
outcome status)
CONFOUNDER BIAS
(controls not
matched on
important
variables, or
unknown variables
not included in
analysis)
VERY PRONE TO
SYSTEMATIC ERROR
EFFICIENT BUT
VULNERABLE
Study designs
Always consider the timeline
temporal association between exposure &
outcomes
past present future
Case control
Cohort
Cross sectional
exposure
exposure
outcome
outcome
Exposure
&
outcome
Confounding:
The weakness of all non-
randomised studies
Confounding may increase/decrease or
totally manufacture any observed effect.
How to control for confounding:
Prevent at design stage:
◦ – restrict, randomly select, match
Deal with at analysis stage:
◦ – stratification, multivariate modelling
◦ but only for things you know about
TEMPORAL
SEQUENCE?
STRENGTH OF
ASSOCIATION?
CONSISTENCY
OF
ASSOCIATION?
BIOLOGICAL
GRADIENT
(DOSE
RESPONSE)?
SPECIFICITY OF
ASSOCIATION?
BIOLOGICAL
PLAUSIBILITY?
COHERENT WITH
EXISTING
KNOWLEDGE?
Experimental
evidence
(RCT?)
Causality: Bradford Hill Criteria
Recommended Reading
http://en.wikipedia.org/wiki/Observ
ational_study
Hicks Chapter9
Carter, Lubinsky and Domholdt
Section3, Chapter 12
Introduction to Qualitative
Research
Meriel Norris
Research Methods PH2600
2020
Learning Outcomes
By the end of the module, you should be able to:
• Define qualitative
research
• Explain the assumptions and values underpinning
qualitative research
• Explain the main approaches underpinning qualitative
research
• Describe qualitative data collection methods
• Explain different strategies for analysing qualitative
data
• Recognise rigour in qualitative research
What this lecture will cover
• What is qualitative research?
– Introduction to induction
• Why do qualitative research?
– Introduction to appropriate questions
• Where does qualitative research come from?
– Introduction to methodologies
What is Qualitative Research?
(Assumptions and Values)
A process of inquiry
Aims to deepen understanding or
insight
“Qualitative researchers are interested in how people
interpret their experiences, how they construct their
worlds, and what meaning they attribute to their
experiences” Merriam (2009:5)
Provides detailed in-depth views of informants
The researcher builds a complex, holistic picture
Usually conducted in a natural setting
Data is generally considered to be ‘co-
constructed’
Analysis is commonly based on words but
pictures and observation can be used
Based on distinct methodological and
philosophical traditions
(Creswell, 1998/2012)
Quantitative (Experimental)
Research is Deductive
Theory
Hypothesis
Measurement
Confirmation
Deduction
‘top down’
Qualitative Research is Generally
Inductive: a process of enquiry or
exploration
Inductive
‘bottom up’
Observation
Patterns or
Themes
Theory
Hypothesis
Qualitative Research Seeks…
• To understand meaning
• To explore perceptions, motivations
• To understand how people make sense of
something that matters to them in their lives
• To understand an experience
• To gain insight , to illuminate the less tangible
intricacies of human existence
Example of insight in rehab post stroke
Theme – Importance of hope
“On my arm? To be honest? They said to me, ‘we can’t do
anything about your arm so concentrate on the leg’. I
know now this is rubbish. You shouldn’t tell ANYONE
that. The way they treated me was good and bad. On the
good side they got me walking again, and then you’re out
(of hospital). I’m happy with that. But to say to someone
you’re never going to get your arm working, mentally,
that’s no good. It’s really bad, because now I can look
back and say they are wrong, but at the time it’s like the
world was coming over me. It’s a mental side. You can
really fuck someone up in their head and there’s no way
back from that.” (George, 48, moderate disability)
Example – therapists talking about
self-management in stroke
Theme- fear of losing control
‘‘At the beginning I used to get quite frustrated and think ‘Oh
they could be doing so much more’…and you’d kind of try
every way to persuade them to change their mind or…you
know ‘What do you think about doing this?’ But then you’d
kind of get to a point where there’s no point because it’s not
what they want…but it’s what you think they should want.
(laughs)…And I think it’s the fear…and we spoke about it in
our team meeting yesterday…but it’s the fear of not doing
anything, it’s the fear of like you know you should do
something because you’re going there to assess them and you
can’t just you know say well they haven’t got goals or they
don’t really want to…because it feels like it’s a cop-out.’’
(Rose)
Qualitative Research does not…
• generally seek to explain (it doesn’t tend to
answer definitive ‘why’ type questions), and it
doesn’t test or measure (e.g. for effectiveness)
• generally generate or use numerical data, it
does not aim to quantify
However…
Like quantitative research, it seeks to develop
new knowledge through the application of
rigorous methods
Why do Qualitative Research?
• To explore previously under-researched areas of
experience – the research is primarily hypothesis (or theory)
generating rather than hypothesis-testing
• To explore experiential issues in ways that are rich and
sensitive to individual and contextual factors (can uncover
unexpected issues)
• In mixed-method research, a qualitative phase may
complement quantitative enquiry (e.g. to supplement
large-scale surveys with more detailed personal views; to
complement objective measures of therapy outcomes; to
explore a field in order to plan more appropriate quantitative
research)
• To give participants power and influence within the
research process
Types of questions and
methodological traditions
Research Question
What is the meaning attached to
this phenomenon?
What is life like for this group?
What is happening?
Why is it happening?
What are they communicating?
How are they communicating?
What is the trajectory of their
story?
Qualitative Approach
(Methodology)
Phenomenology (philosophy)
Ethnography (anthropology), Case study
Grounded theory (sociology)
Discourse
analysis
(sociology, linguistics)
Narrative
Qualitative Approaches for this
Module
•
Phenomenology
• Grounded theory
• Ethnography
NB there are many others
Phenomenology
• Grounded in existential-
humanistic traditions
• Focus on the individual’s
‘lived experience’
• Small samples (n= 1-10)
are typical
Phenomenology
• Grounded in personal experience
• Is not focussed on sociological structures or
contexts but may comment on how an
individual interprets their life with respect to
broader sociological concepts
• Have an acquaintance with the literature
before undertaking the study
• Does not aim for saturation
Understanding individual experiences
of phenomenon
Hermeneutic Phenomenology
– Strongly interpretative
– Would not bracket
– Acknowledges researcher’s involvement
Descriptive Empirical Phenomenology
– Describes the essence of a phenomenon
– It does not draw on external theory
– Attempt to ‘bracket’ past knowledge
Interpretative Phenomenological Analysis
– Strongly focussed on individual sense
making
– Interpretative, does not bracket
– Reflexive
Grounded Theory
• Sociological origins (Glaser and Strauss)
• Seeks to identify social influences on experience
and behaviour and build theory
• Aims to explain a process through generation of
theory
• Does not test or verify an existing theory
• Generally seek samples of 20-30 to gain
‘saturated’ data (from which no new themes
emerge). Technically sample size cannot be
confirmed in advance
Grounded Theory: Key Features
Data collection
(first case)
Initial coding and
categorisation
Concurrent (iterative)
data collection and
analysis
Memo writing
Theoretical (purposive)
sampling
Constant comparison
Theoretical sensitivity
Intermediate coding
Selecting a core
category
Theoretical saturation
Theoretical integration
Explaining the existence of a
phenomenon within social influences
Example from GT study: The development and maintenance of trust in health care providers (Hupcey, Penrod, &
Morse, 2000).
Ethnography
• Origins in anthropology
• The central aim of ethnography is to provide rich,
holistic insights, through extensive observation, of
social interactions, behaviours and perceptions that
occur in groups, communities, teams, organisations
• Interested in understanding cultural norms and
practices, to examine ‘taken-for-granted’ practices,
to make the ‘familiar’ unfamiliar
• Interested in people’s views and actions, as well as
the nature (e.g. sights, sounds) of the location they
inhabit
• Multiple forms of data collection
Understanding shared phenomena
within a group within context
Social
Environmental
Key feature – participant observation
• A fine balance between being ‘in’ and
therefore experiencing in a naturalistic way
and being ‘out’ so you can look and see as
much as possible
Note
There are many methodological approaches we
have not discussed
Not all qualitative research have a named
methodological approach
BUT the approach does give you a framework to
consider the rest of the study
What do Qualitative Methodologies have
in Common?
• They typically take an inter-subjective perspective on
knowledge – that the data are created / co-constructed /
generated during the research process
• The process is value laden; take a social constructivist
perspective on the development of knowledge
Quant – work from the
assumption that there is an
absolute
truth
, a reality that
can be discovered and
measured
Qual – work from the
assumption that the reality
we perceive is constructed by
our social, cultural, historical
and individual contexts
Knowledge is objective and
neutral
Knowledge is subjective, there
can be no absolute shared
truth
How do qualitative researchers
recruit participants: Sampling
• Most qualitative research uses small samples (1-30) –
not only because data collection, transcription and
analysis are intense and time-consuming – but also to
allow in-depth exploration of individual accounts
http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjA-6u02v7KAhVsOJoKHYwVDRwQjRwIBw&url=http://mind42.com/public/18bcc4ac-215b-4a45-abfe-d92e05c73a78&bvm=bv.114195076,d.d24&psig=AFQjCNG6MSESH_SZ_IvaOOnXzo–ZmUGYw&ust=1455795296960728
http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjA-6u02v7KAhVsOJoKHYwVDRwQjRwIBw&url=http://mind42.com/public/18bcc4ac-215b-4a45-abfe-d92e05c73a78&bvm=bv.114195076,d.d24&psig=AFQjCNG6MSESH_SZ_IvaOOnXzo–ZmUGYw&ust=1455795296960728
http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjA-6u02v7KAhVsOJoKHYwVDRwQjRwIBw&url=http://mind42.com/public/18bcc4ac-215b-4a45-abfe-d92e05c73a78&bvm=bv.114195076,d.d24&psig=AFQjCNG6MSESH_SZ_IvaOOnXzo–ZmUGYw&ust=1455795296960728
What can I do with Qualitative Research?
Generalisability vs transferability
• Generalizability is multidimensional (Johnson, 1997)
• Horizontal generalizability: across settings and samples within
the original population (nomothetic), large n, quant
• Vertical generalizability: provoke a reappraisal of what is
known (idiographic), theoretical generalizability, small n
Quantitative:
Horizontal Generalizability
Qualitative:
Vertical Generalizability/ transferability
Findings new
insight
Summary
• Qualitative research is generally focused on
exploring subjective experience
• Different methodological approaches
• General small samples
• Not aiming for ‘the truth’ or population level
generalisability
Interpreting and Critiquing
Qualitative Research
Meriel Norris
PH2600
Research Methods
Seminar 1
Research ethics
Seminar 2
Research questions (FINER) – qual and quant
Hypotheses – quant
Variables – quant
Study designs – discussed in relation to quant
Seminar 3
Validity, reliability, recruitment
and levels of measurement
Methods
section of a
quantitative
paper
Applies to all papers with
human participants
Seminar 4
Central tendency & variance, SD
Measures of dispersion
Statistical methods and techniques
Type I and type II errors
One and two tailed hypotheses
Seminar 5
Understanding qualitative methods and
methodology – qual
Seminar 6
Critiquing qualitative research and analysis –
qual
Quant
Critiquing Qualitative Research
• In this lecture we will look at how to critically
evaluate qualitative research, taking account of:
– Credibility, confirmability,
transferability
,
dependability, trustworthiness
– Rigour
– Sample size
– Member checking/respondent validation
– Negative case analysis
– Triangulation
– Saturation
Critique
• Systematic analysis and evaluation
• Careful, considered (plusses and minuses)
• Detailed
• Concrete and evidenced
• Summarise and justify your considered opinion
• Consider what has been reported
• Consider whether what has been reported and
what has been left out or glossed over is
reasonable or whether it weakens the credibility
of the paper
• Referenced
What is Qualitative
Research?
• Qualitative research is based in experience
• It is about how people make sense of their world
• The meanings people attach to their experience
How can we judge if this has been achieved??????
Critical Appraisal Tool
http://www.casp-uk.net/#!casp-tools-
checklists/c18f8
CASP qualitative checklist
Why use a critical appraisal tool?
http://www.casp-uk.net/#!casp-tools-checklists/c18f8
Why the CASP?
Systematic ComprehensiveGood for novices
BUT
• Deals very broadly with some of the
principles and assumptions
• Not a definitive guide
http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRw&url=http://xplodemag.co.uk/reading-pick-book/&ei=6kWEVNmUHOyu7Aa8kYGABQ&bvm=bv.80642063,d.ZGU&psig=AFQjCNFD2libuc-5L5z3RrORi1XQ7bsQ1Q&ust=1418041141936538
Spotting a Good Qualitative Paper
• There may be a lot of BAD stuff out
there
• Some of the MOST IMPORTANT
qualities can be the MOST
DIFFICULT to evaluate
(Dixon-Woods et al, 2004)
• The subjectivity bothers people
• How do you judge the creative
contribution?
Telling Good from Bad
‘The thorough assessment of qualitative research is
an interpretative act and requires informed
reflective thought rather than the simple
application of a scoring system.’
Kuper et al, (2008:687).
Undertaking a Critical Review
Start Asking Questions: Read Critically
Lorraine H. De Souza, Centre for Research in Rehabilitation, Brunel University, UK
Andrew O. Frank, Northwick Park Hospital and Institute for Medical Research,
Middlesex, UK
Disability and Rehabilitation 2007, 29(7), pp587-596
Title: does it tell you what you need to
know?
• Topic
• Participants
• Method
• If you were doing a database search for papers in
this field, would this paper come up if your search
was based on title alone?
• If you were interested in reading about this topic
would a quick scan of the title tell you whether this
paper was worth investing in?
Authors
• Who are they?
• Think about what sort of assumptions,
prejudices, preconceptions the authors might
bring to the research
• Look (elsewhere) for information that tells you
something about them (incl. experience doing
qual)
• Ask yourself what you might expect to see in a
section of the paper on reflexivity
Abstract
• Purpose
• Participants
• Methods (incl analysis)
• Findings (results)
• Conclusions
Is there a good fit (purpose, sample, methods, analysis)?
Does it have utility – does it provide an effective and accurate
summary of what was done and what was found?
Go back – check what reported in the main text, does it match?
Format set by journal
Ok but how analysed
What type of qualitative study?
Introduction
• What’s the point of the study, are you convinced?
• Is the rationale persuasive, compelling?
• Have the authors established that the study is worth
doing, important, relevant, timely?
• Arguments based on relevant (recent) literature?
• Theoretical and empirical literature used to good
effect?
• Strong narrative thread?
Are the Research Aims Clearly Stated?
• Does the study seek to illuminate and deepen
understanding of a concrete ‘lived’ subjective
experience?
• Is a qualitative approach appropriate?
• Is the research question clearly articulated?
Or…
http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjG6IC2xYLaAhWGvhQKHUYrCKoQjRx6BAgAEAU&url=http://persuasionpursuit.com/bandwagon-effect-marketing-use-sell-storm/&psig=AOvVaw1pvZmeT15K3wmfYC6qSiQm&ust=1521897844749835
Methodology
• Named, explained and justified
• If no named methodology why not, is this
justified?
• Theoretical position clarified (e.g. realist /
interpretivist), explained and justified?
• Would a different methodology have been
more suitable?
Name it
Claim it
Frame it
Recruitment: How Were Participants
Approached and Recruited?
• Is the recruitment strategy clear and appropriate ?
• Is it justified and named e.g. purposive?
• Is the sample size appropriate and justified?
• Did the participants have adequate experience/
knowledge to contribute to the research (and
address this research question)?
• Who volunteered? Who refused?
• Might there have been better recruitment
methods and settings?
• Look at the demographic data – what does it tell
you?
Recruitment 2
For example –
How might a study of the experience of back pain
be affected by recruiting from a pain clinic?
Becomes important in
relation to
transferability
Data Collection
• What data were collected, and how? Justified?
• Is the method likely to elicit detailed accounts?
• Was the topic guide / SSI guide provided?
– Were the questions appropriate?
– How much structuring did the interview have – consider +/- of tight and
loose structuring
• How long were the interviews?
• If the authors claim saturation is this justified? (thought to apply
to grounded theory)
• Was the influence of the setting/context considered? (e.g. how
might interviewing in a hospital setting influence accounts? Or in
a focus group compared with a one-to-one interview?)
Qualitative data analysis: 1
Phenomenological Analysis
Descriptive phenomenological research, the researchers
may ‘bracket’ – to set aside/ heighten awareness of their
assumptions;
Create a general or essential phenomenological description
Interpretative (hermeneutic) phenomenology – bracketing
theoretically questioned, instead use assumptions as a
springboard for interpretation. Would not claim to identify
‘essences’. Reflexivity expected.
Do these align with the methodology described & results?
Qualitative data analysis: 2
• Grounded theory – aim to build a theory from the data
(usually a visually represented ‘box and arrow’ model, with
theorised social ‘causes’ and conditions) constant comparative
method
• Ethnography – more mixed – probably thematic based. May
have specific perspective. Should discuss how triangulation of
methods was managed
• Narrative analysis – look for story structure, turning points,
rhetoric, characters, metaphors
• Content analysis – search for pre-selected words/ phrases –
usually only in open questions in brief questionnaire answers
(but NB there are different content analyses)
Do these align with the methodology described and data
collection process & results? And the research question?
Judging the Analysis
• How well is the data analysis described? Audit trail
• Do you get a firm/clear idea about the processes
involved, how decisions were taken e.g. about the
development of themes? Transparency
• Who did the analysis? Analytical triangulation –
appropriate?
• Was any ‘checking’ process appropriate? Peer
debriefing/respondent validation – appropriate?
• Do you understand what was done?
• If not, how might this impact on the
quality/trustworthiness of the findings?
Judging the Findings
• Is there adequate quotation and commentary to
build a convincing case for the proposed themes?
• Appropriate evidence of interpretation (if
relevant)?
• Is there adequate attention to differing
experiences/ viewpoints – not only shared
themes? negative case analysis/ balanced voices
• Small fragments of illustrative quotations in
reports make it difficult for readers to check the
trustworthiness of the analysis
• Has the researcher discussed credibility?
The concept of credibility is preferred to the
quantitative concept of ‘internal validity’
– Systematic and transparent data analysis
– Audit trail of decisions and changing perspectives in
relation to the data (not nec. available)
– Data saturation – carrying out interviews and analysis
until no new information emerges (NB mostly a
grounded theory concept)
– Prolonged engagement
– Reflexivity
– Structural coherence
– Member checking/ respondent validation
– Negative Case Analysis
– Triangulation
– Peer debriefing / review
Member Checking /
Respondent Validation
• The researcher shows all or part of a study’s findings to
participants to determine if they are in accord with their
experiences
• May help the researchers to determine whether their
interpretations make sense and are acceptable to
participants
• Not always possible (e.g. when participants have limited
cognitive ability)
• Not always appropriate – why?
Negative Case Analysis
• Search for ‘deviant’ cases is recommended –
accounts which do not fit themes/ patterns identified
in the analysis.
• This action helps to reassure reader that the
quotations have not been ‘cherry-picked’ to suit the
researcher’s preconceptions.
• Can indicate the limits of transferability of the
themes
• Honours individual difference in experience
• Can help to build more complex theory
– Example – 8 participants report that exercise makes them
feel happier and more energetic and 2 say that they feel
inadequate/ unhappy when exercising
– Should the focus be on the views of 6/8? Why? Why not?
Triangulation
• Are the accounts of participants/ interpretations of
researchers corroborated in any way?
• Methodological triangulation vs analytical
triangulation
• Avoid viewing triangulation as a ‘check’ on ‘facts’ –
NOT a check on ‘truth’.
• Also, differing interpretations by different
researchers may reveal something about the
influence of professional concepts and expectations
– not simply ‘bias’ or ‘lack of reliability’
“The findings were checked by someone else”
What do the researchers mean?
Be careful with this term, not necessarily a good
thing:
A researcher brought in to verify ‘themes’ may rely
far more on PERSONAL PRECONCEPTIONS and
GUESSWORK than the main fieldworker
But another person could be used to help the
researcher engage in ‘critical dialogue’
Peer Debriefing and Review
Confirmability
Discussion and Conclusions
• Have the findings been placed in context
of what was already known?
• Are the findings discussed in relation to
the research questions?
• Does the researcher discuss the potential
contribution of the study? Are the claims
reasonable?
Reflexivity 1
• Recognising the influence of the researcher in the
research process (dependability – being accountable)
• How experience, gender, profession, social status,
ethnicity & culture influence the choices made in the
study (from RQ to data collection, interpretation and
discussion)
Reflexivity 2
• Participants’ accounts are not transparent windows on
feelings/experiences – but products of the social
situation
• Have the researchers reflected on their role and
relationship with participants, their pre-conceptions,
how they handled unexpected events?
• Papers should not be ‘trashed’ just because a
researcher has declared a particular cultural
perspective (feminist stance) or personal involvement
with the research participants
• Subjectivity (bias) should be accounted for but can’t be
eliminated
• If it is not accounted for you should state why/how a
lack of reflexivity may have impacted on the findings
Research Ethics: Operates Throughout
Procedural Ethics
• Usually state REC approval and informed consent
• Usually note anonymity – use of pseudonyms and
blurring of other identifying details
Situational Ethics – may discuss management of distress
Relational Ethics
• Researcher has the power to shape what is understood
but also to distort – your judgement
Exiting Ethics
• Publication can unintentionally disrespect the
participants who choose to take part in the
understanding that their lives will be better understood
your judgement
Transferability
(generalisability/ external validity)
• ‘Generalisability’ is valued in quantitative research
but inappropriate for most qualitative studies
• ‘Transferability’ is the preferred qualitative concept –
can findings be transferred to other contexts or
settings? Primarily judged by the person doing the
transfering! Enhanced by detailed description of
findings and setting/context
• Notion of ‘vertical generalisability’ – via theoretical
understandings – how much does the study help to
develop theory/ new insights?
Final Comments
• Issues around trusting the researchers to have
analysed all their data and to have presented the
findings without bias or selection also occur in
quantitative research.
• Not everything can be reported in an article (typically
4000-7000 words long). So focus your critique on
what was/was not reported rather than what was/
was not done by the researchers.
• In your essay pick some of the major strengths and
weaknesses.
• Support with references
Further Reading
Tracey, SJ (2010) Qualitative Quality: eight ‘big-
tent’ criteria for excellent qualitative research.
Qualitative Inquiry, 16(10), pp837-851.
Ryan, F, Coughlan, M, Cronin, P (2007) Step-by-
step guide to critiquing research. Part 2:
qualitative research. British Journal of Nursing,
16(12), pp738-744.
References
• Dixon-Woods, M., Agarwal, S., Young, B., Jones, D., &
Sutton, A. (2004). Integrative approaches to qualitative
and quantitative evidence. London: Health
Development Agency, 181.
• Silverman, D. (2000). Doing qualitative research. A
practical guide. London: Sage.
• Smith, J. A. (2011). Evaluating the contribution of
interpretative phenomenological analysis. Health
psychology review, 5(1), 9-27.
• Yardley, L. (2008). Demonstrating validity in qualitative
psychology. Qualitative psychology: A practical guide to
research methods, 2, 235-251.
Critical appraisal of
quantitative research.
Neil O’Connell 2019
PH2600 Research Methods
Learning outcomes
By the end of the lecture students
should:
Understand the basic principles of
critical appraisal of quantitative
papers
https://www.pedro.org.au/english/downloads/pedro-statistics/
https://www.pedro.org.au/english/downloads/pedro-statistics/
https://www.pedro.org.au/english/dow
nloads/pedro-statistics/
Zoldan et al. Arch Phys Med Rehab 2018; 99: 1: 129-136
Critical Appraisal Skills
Is the method of research
appropriate for the question?
Is the research of good quality?
Are the results relevant to your
group/individual?
Is the research generalisable or
sample specific?
What is the size of the
beneficial/negative effect?
Are these results
reliable – can I trust
them?
Do these results apply
to my practice?
What do they mean
for my practice?
APPROPRIATE
DESIGN?
APPROPRIATE
ANALYSIS?
RISKS OF BIAS?FAIR
INTERPRETATION?
CLEARLY
PRESENTED?
Internal
validity
RELEVANT
POPULATION?
RELEVANT
INTERVENTION?
EFFECT SIZE
MEANINGFUL?
SHOULD IT
CHANGE MY
PRACTICE?
External
validity
Know thyself: You’re biased when
you read the evidence too!
Death Penalty evidence experiment
Republican/ Democrat supporter experiment
Irrationality and bias is the norm – it is a
constant struggle to outwit the inner zombie!
Things you might (will) do
Cherry pick only the evidence that supports
your world view
Ignore evidence that doesn’t
Selectively criticise research you don’t like
(Rescue Bias)
Ignore research and wallow in comfortable
“truths”!
What are my
biases?
Am I being fair?
Am I prepared to
accept any
outcome?
Title /Abstract
Is the title clear? Does
it reflect the content?
Is the abstract clear
– does it fairly
summarise the
study and results?
Spinning a better story
“spin” in more than one section in 40%
of negative RCT reports (Boutron et al. JAMA.
2010;303(20):2058-2064).
Rheumatology RCTs – 23% judged as
having misleading conclusions in the
abstract.
The strongest predictor of misleading
conclusions was a negative trial! (Mathieu
Joint Bone Spine 79 (2012) 262–267)
The question
Do the authors give a fair
and balanced summary of
the preceding literature?
Is the question clear and
well justified?
Is the choice of design
appropriate for that
question?
The study design
Why was this design
chosen?
Is this the best design to
answer this question?
Is this the best design to
answer this question?
What are the broad
strengths and weaknesses
of this design?
My question is… Look for this….
Does this intervention work? RCT,
Systematic review/ meta-analysis of
RCTs
Diagnosis/ Screening tests….
• Is it accurate?
• Does it improve outcomes?
Cross-sectional studies where subjects
get the test & a gold standard
reference.
RCTs
What is the prognosis/ natural history of
a condition?
Longitudinal cohort study
Is this risk factor important? Cohort study
Case-control study
Cross sectional study (v exploratory)
Describe this population and the
relationships within it.
Cohort study
Cross-sectional study
Sampling and recruitment
What sampling strategy did they use?
Probabilistic? Convenience? Is it
reasonable?
What were the inclusion exclusion
criteria? (reasonable? Too tight? Too
loose?)
What were the inclusion exclusion
criteria? (reasonable? Too tight? Too
loose?)
Is the final sample representative of
the relevant broader population? If
not how do they differ?
Is the sample size adequate?
The study process
Are the methods rigorous? Are
potential biases well controlled?
(details depend on the design)
Can you think of any variables that
might confound the results?
Did many participants drop out? What
do we know of the missing data?
Is the analysis fair and appropriate
given the type of data?
THE
INTERVENTION(S)
THEORY
?CLEAR
?PLAUSIBLE
CONTENT
?specific details
education
Exercises
Modalities
Clinicians
Tailored
DELIVERY
SETTING
DOSING
FIDELITY
ADHERENCE
The results and interpretation
Are the results clearly and fairly
presented (can you see the necessary
data?)
Is any important information not
reported?
Is the authors’ interpretation a fair/
measured reflection of the evidence?
Where do these results sit in the
wider evidence base?
How big is the effect? Useful/ relevant outcomes?
Appropriate/ achievable
intervention? Similar Patient Group?
Do I even have
enough
information?
Conclusions
Assessing the quality and relevance of
evidence requires a broad range of skills
It is a challenge of measured judgement
This is the challenge set by the assignment
You can start practising now.
Further Reading
APPENDIX 4, PAGE 371 -390 Hicks, C. (2009)
Research methods for clinical therapists. Applied
project design and analysis. 5th Edition. Churchill
Livingstone. UK
Guide to Reading Research Articles – In the
assessment folder on Blackboard.
(www.casp-uk.net)
INFERENTIAL STATISTICAL
ANALYSIS
Dr Eve Corner
PhD, MRes, BSc (hons)
PLAN…
• Sample size calculations
• One and two tailed hypotheses
• Parametric and non-
parametric
tests
• Comparing groups
• Correlations
SAMPLE SIZE CALCULATIONS
How many is enough?
Get enough
people to get a
representative
sample and to
find an effect
Don’t have time
or money to
recruit every
person in the
population
Point one:
Sample needs to be sufficiently
large to represent the population in
which it was derived/represents.
Running speed in seconds (100 metres)
9.23 10.5 10.
5
8.98 10.8 10.8
10.56 11.1 11.9
17.5 15.3 25.5
18.12 13.6 26
12.3 14.8 17.5
16.5 12.1 20.1
14.3 11.7 8.7
9.2 11.3 11.2
Mean 13.7 seconds (4.54 SD)
Running speed in seconds (100 metres)
9.23 10.5 10.5
8.98 10.8 10.8
10.56 11.1 11.9
17.5 15.3 25.5
18.12 13.6 26
12.3 14.8 17.5
16.5 12.1 20.1
14.3 11.7 8.7
9.2 11.3 11.2
Mean: 17.32 (SD 6.75)
Running speed in seconds (100 metres)
9.23 10.5 10.5
8.98 10.8 10.8
10.56 11.1 11.9
17.5 15.3 25.5
18.12 13.6 26
12.3 14.8 17.5
16.5 12.1 20.1
14.3 11.7 8.7
9.2 11.3 11.2
Mean: 15.0 SD (6.44)
• Population mean (n=24): 13.7 seconds (SD 4.53)
• Sample means (n=3): 17.32 seconds (SD 6.75)
• Sample mean (n=6): 15.0 seconds (SD 6.44)
• Sample mean (n=9): 13.9 seconds (SD 5.35)
SAMPLE SIZE
The larger the sample sizes the less variability between
the sample means i.e. the smaller the standard error
What if we had two samples,
and we want to know the
difference between the
groups?
Get enough
people to get a
representative
sample and to
find an effect
Don’t have time
or money to
recruit every
person in the
population
Is there a difference in 100m sprint time between those who exit
first and the remaining candidates?
Increased sample size = decreased
variability
= greater power to detect an effect if one exists
SAMPLE SIZE CALCULATIONS
• We can calculate the sample size required to find a true difference (or effect) using
a sample size calculation
• Trying to minimise the chance of a type I or type II err
or
• Type I error: Finding a difference (effect) when there isn’t actually one (false
positive)
• Type II error: Not finding a difference (effect) when there is actually one (false
negative)
TYPE 1 ERROR
(false positive)
TYPE 2 ERROR
(false negative)
SAMPLE SIZE
• The ability of a statistical test to detect a true difference (effect) is known as it’s power.
• The power of a test depends on the size of the sample and size of the difference (effect)
• The bigger the sample size the better chance of it finding a true difference (effect) if it’s
there
• If the sample size is big it can detect a small difference (effect)
• If the sample size is small a test could still be powerful if the difference (effect size) is very
big
EFFECT SIZE AND SAMPLE SIZE
100 metre sprint time
Case B
Case A
CALCULATING SAMPLE SIZE: T-TEST.
NB The method used to calculate the sample size depends on the statistical test that
will be used to analyse the primary outcome
If using a t-test, to calculate a sample size required to find a difference between
groups you need:
1. To know the size of the difference (effect) and an estimate of the variability within
the population
2. To decide what risk are you willing to take that you’ll get a type I or type II error
SAMPLE SIZE CALCULATION FOR GROUP
COMPARISONS
Info needed Description/Reason
α level (i.e. p < ?) Set the cut-off for the probability of a type I error.
Conventionally set at α = 0.05 (p < 0.05). This tells
you the probability of getting a false positive is 5%.
Probability of sending an innocent man to jail.
Power Set the cut-off for the probability of avoiding a type
II error. Often set at 80% or 90% (this is the
probability that you’ll detect a true effect if there is
one).
Probability of a guilty man walking free.
Predicted difference
(effect
size)
How large do you expect it to be?
Predicted sample
variability
What’s the variability within the population?
Predicted difference (effect
size)
How large do you expect it to be?
How do you know this??
Clinical judgement of what’s important
Changes observed in previous studies
Minimal Detectable change or Minimal
Clinically Important Difference
Predicted sample variance What’s the variation within the population?
i.e. what’s the standard
deviation
How do you know this??
Standard deviation from previous studies
Pilot studies
What if I the information required isn’t in the literature?
• Pilot study
IMPORTANT POINT!
• 80% power means 80% chance of finding a
true difference if it exists.
• BUT…. 20% chance of not finding it….
• AND if you under recruit, your power drops.
You may let a guilty man walk…
i.e. it may be a false negative i.e. no difference found, when a
difference exists.
HOW DO THESE EFFECT MY SAMPLE SIZE?
Info needed Change Impact on sample size
α = 0.05 e.g. α = 0.01
Power = 80% e.g. power = 90%
Effect size
e.g.
difference =
2 seconds
e.g. Difference = 5 secs
Standard
deviation
e.g. 3 s
e.g. SD = 6 seconds
ONE OR TWO TAILED TESTS
Which do I use?
ONE OR TWO TAILED TESTS
Trying to detect a difference in either
direction
Trying to detect a difference in
one direction
One-tailed test
Two-tailed test
ONE-TAILED OR TWO-TAILED?
• Do a two-tailed test if you think the difference between the groups could be positive
or negative
e.g. Do 6MWT, do an exercise intervention, repeat 6MWT
The 6MWT results in the second test could be better or worse than in the first
test…..therefore need a two-tailed test
NB: Unless it’s physically impossible to have change in either direction (i.e. positive or
negative) use a two-tailed test
(If you choose a one-tailed test, you’re more likely to find a positive effect, even if it’s
not there, i.e. type I error)
STATISTICAL TESTS
For this module: compare groups & examining associations
COMPARING GROUPS
Within group Between groups
2 conditions 3 or more
conditions
2 different groups 3 or more
different groups
Parametric Paired t-test One-way
repeated
measures Analysis
of Variance
(ANOVA)
Independent t-test One-way Analysis
of Variance
(ANOVA)
Non-
parametric
Wilcoxon test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
McNemar’s test
(nominal
data)
Friedman test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Mann-Whitney U
test (ordinal data
or ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Kruskal Wallis test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
ASSESSING
CORRELATIONS
2 variables
Parametric test Pearson test
Non-parametric test Spearman test (ordinal
data or ratio/interval
data with a skewed
distribution)
Cohen’s Kappa
coefficient (nominal
data)
5 STEPS TO STATS
Step 1: What type of data is it and is
it normally distributed?
Step 2: State the null and alternative
hypothesis
Step 3: Set a significance level
Step 4: Choose a test statistic and
conduct the test
Step 5: Interpret the result
PARAMETRIC OR NON-PARAMETRIC
Which do I use?
PARAMETRIC VERSUS NON-
PARAMETRIC
• A parametric statistical test makes assumptions about the parameters
(defining properties) of the population distribution(s) from which the
data are drawn:
• That data is normally distributed.
• That data is ratio or interval.
• Parametric tests are more sensitive or “powerful” than non-parametric tests
• Therefore more likely to detect a true effect if there is one
Parametric
Non-
parametric
ALTERNATIVES…
Within group Between groups
2 conditions 3 or more
conditions
2 different groups 3 or more
different groups
Parametric Paired t-test One-way
repeated
measures Analysis
of Variance
(ANOVA)
Independent t-test One-way Analysis
of Variance
(ANOVA)
Non-
parametric
Wilcoxon test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
McNemar’s test
(nominal data)
Friedman test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Mann-Whitney U
test (ordinal data
or ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Kruskal Wallis test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
EXAMPLES
• 1. Comparing 6MWT results (metres) between two groups (positive skew)?
• Ratio but skew
• Two groups
• = Mann Whitney
• 2. Comparing Oxford scale grading of strength between two time points?
• Ordinal data
• One group
• = Wilcoxon test
• 3. Difference in number of men versus women categorised as overweight?
• Nominal
• Two groups
• = Chi-squared test
COMPARING GROUPS
Two groups of people, one time point
COMPARING GROUPS
Within group Between groups
2 conditions 3 or more
conditions
2 different groups 3 or more
different groups
Parametric Paired t-test One-way
repeated
measures Analysis
of Variance
(ANOVA)
Independent t-test One-way Analysis
of Variance
(ANOVA)
Non-
parametric
Wilcoxon test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
McNemar’s test
(nominal data)
Friedman test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Mann-Whitney U
test (ordinal data
or ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Kruskal Wallis test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Is there a difference in 100m sprint time between those who exit
first and the remaining candidates?
• Step one: What type of data?
• Time in seconds
• RATIO
• Is it normally distributed?
Categorical
data
Nominal Ordinal
Continuous
data
Interval Ratio
• Step two: State the null and alternative hypothesis:
• Null (H0): There is no difference between running speed and half way survival in
adults in the Hunger Games.
• Alternative (H1): There is a difference between running speed and half way
survival in adults in the Hunger Games. (two tailed)
• Step 3: Set a significance level?
• Probability of type 1 error
i.e. how willing are you to accept that you may find
a difference when no difference exists?
• < 5% ?
• <1% ?
• α<.05
“Send an innocent man to jail”
STEP 4: CHOOSE A STATISTICAL
TEST- PARAMETRIC
• To test for differences between groups we generally calculate a statistic known as
“t”
• Paired t-test comparing one group that are measured on two
occasions (e.g. pre and post)
• Independent t-test comparing two separate groups (e.g.
intervention versus control)
• Which would you use?
Two groups = independent
T-TEST
Difference between means for group 1 and group 2
Standard error
t =
Standard error: SD of the original
distribution divided by the
square route of N
Mean first 12: 15 seconds
Mean remaining 12: 9 seconds
Difference = 6 seconds
t = 0.238, p = 0.813
STEP 5: INTERPRETING THE RESULTS
p = 0.813
p is not < 0.05 therefore do not reject the null hypothesis
There is no real difference
COMPARING GROUPS
One group of people, two time points
COMPARING GROUPS
Within group Between groups
2 conditions 3 or more
conditions
2 different groups 3 or more
different groups
Parametric Paired t-test One-way
repeated
measures Analysis
of Variance
(ANOVA)
Independent t-test One-way Analysis
of Variance
(ANOVA)
Non-
parametric
Wilcoxon test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
McNemar’s test
(nominal data)
Friedman test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Mann-Whitney U
test (ordinal data
or ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Kruskal Wallis test
(ordinal data or
ratio/interval
data with a
skewed
distribution)
Chi-squared test
(nominal data)
Does the percentage of Love Island contestants in a
relationship change as a result of the ‘intervention’
from 2012-2019?
TIME POINT ONE: Entry To Love
Island
Single
Single
Single
Single
Single
Single
Single
In a
relationship
Single Single Single
1/11 in a relationship = 9.1%
TIME POINT TWO: Exit To Love
Island
Single
6/11 in a relationship = 54.5%
COMPARING PERCENTAGES
Year Pre Post Difference
2018 9.1% 54.5% 45.40%
2017 0% 45.5% 45.50%
2016 0% 63.6% 63.60%
2015 9.1 % 54.5% 45.40%
2014 0% 54.5% 54.50%
2013 9.1 % 72.7% 63.60%
2012 0% 45.5% 45.50%
Mean (SD) 3.9 % (0.05) 55.83% (0.10) 51.93% (7.99%)
• Step one: What type of data?
• Percentages
• Ratio
• Is it normally distributed?
• Step two: State the null and alternative hypothesis:
• Null (H0): There is no difference in proportion of Love Island Contestants who are
in a relationship before and after they appear on the show between 2012 and
2018
• Alternative (H1): There is a difference in the in proportion of Love Island
Contestants who are in a relationship before and after they appear on the show
between 2012 and 2018 (two tailed)
• Alternative (H1): There is a higher proportion of Love Island Contestants who are
in a relationship before and after they appear on the show between 2012 and
2018 (one tailed)
• Step 3: Set a significance level?
• Probability of type 1 error
i.e. how willing are you to accept that you may find
a difference when no difference exists?
• < 5% ?
• <1% ?
• α<.05
“Send an innocent man to jail”
STEP 4: CHOOSE A STATISTICAL
TEST- PARAMETRIC
• To test for differences between groups we generally calculate a statistic known as
“t”
• Paired t-test comparing one group that are measured on two
occasions (e.g. pre and post)
• Independent t-test comparing two separate groups (e.g.
intervention versus control)
• Which would you use?
One group = Paired
Mean difference is 51.93 %
Standard deviation is 7.99 %
Year Pre Post Difference
2018 9.1% 54.5% 45.40%
2017 0% 45.5% 45.50%
2016 0% 63.6% 63.60%
2015 9.1 % 54.5% 45.40%
2014 0% 54.5% 54.50%
2013 9.1 % 72.7% 63.60%
2012 0% 45.5% 45.50%
Mean (SD) 3.9 % (0.05) 55.83% (0.10) 51.93% (7.99%)
STEP 4: CHOOSE A STATISTICAL
TEST
• To test our null hypothesis the test-statistic we use is:
𝑡 =
ҧ𝑑 −
0
𝑠
𝑛
Is the mean difference different to zero
Standard error (variability of sample means)
t =
p = <.05
STEP 5: INTERPRET THE RESULTS
Ask yourself:
1. Is the difference between groups statistically significant?
2. What is the effect size? What is the mean difference between the groups
relative to the group means?
In other words…
1. Is p <0.05?
2. If p <0.05 is the mean difference of 51.93% between groups big?
What’s meaningful change?
CORRELATIONS
ASSESSING CORRELATIONS
2 variables
Parametric test Pearson test
Non-parametric test Spearman test (ordinal
data or ratio/interval
data with a skewed
distribution)
Cohen’s Kappa
coefficient (nominal
data)
CORRELATIONS
• You may want to look at how two variables are associated with another
• You have two measures (or variables) within one group of people
• If the aim is to investigate the relationship between two variables need to calculate a
correlation coefficient
• -1.0 (perfect negative association between two variables)
• +1.0 (perfect positive association between two variables)
• 0 (no association)
• Written as r
• e.g. r= 0.4, r = -0.7
• Research question:
• Is there an association between 100m speed and position on the leaderboard?
• Step one: What type of data and is it normally distributed?
• Time in seconds
• RATIO
• Ranking
• ORDINAL
STEP TWO: STATE THE NULL AND
ALTERNATIVE HYPOTHESIS:
• Null (H0): There is no association between running speed and ranking on the
leaderboard in adults in the Hunger Games.
• Alternative (H1): There is an association between running speed and ranking
on the leaderboard in adults in the Hunger Games. (two tailed)
• Alternative (H1): There is an association between faster running speed and
better ranking on the leaderboard in adults in the Hunger Games. (one
tailed)
• Step 3: Set a significance level?
• Probability of type 1 error
i.e. how willing are you to accept that you may find
an association between running speed and ranking
when no association exists?
• α<.05 “Send an innocent man to jail”
STEP 4: CHOOSE THE TEST PARAMETRIC
OR NON- PARAMETRIC?
• Is the data ratio or interval?
• If so, is it normally distributed?
• Is the data ratio or interval?
• If so, is it normally distributed?
Parametric test is Pearson’s test a.k.a. Pearson’s
correlation coefficient (r)
Non-parametric equivalent is Spearman
test to calculate Spearman’s rho (r or rs)
0
5
10
15
20
25
30
35
0.00 10.00 20.00 30.00
L
e
a
d
e
rb
o
a
rd
p
o
si
ti
o
n
Running speed
0
5
10
15
20
25
30
0.00 10.00 20.00 30.00
L
e
a
d
e
rb
o
a
rd
p
so
it
o
n
Running speed
STEP 5: INTERPRET THE RESULTS
Perfect positive r= 1, p<.05 Perfect negative r= -1, p <.o5)
The faster you run, the more likely you are to win
or
The faster you run, the more likely you are to lose
0
5
10
15
20
25
30
0.00 5.00 10.00 15.00 20.00 25.00 30.00
L
e
a
d
e
rb
o
a
rd
p
so
it
o
n
Running speed
CORRELATION BETWEEN RUNNING
SPEED AND POSITION ON LEADER
BOARD
No association between running speed and leader
board position
r= ?
p= ?
CONFOUNDING
SUMMARY
• To answer the aim of your study you usually need to use conduct a statistical test
Step 1: Identify type of data and distribution (if interval/ratio)
Step 2: Develop a null hypothesis and alternative hypothesis
Step 3: Decide on a significance level
Step 4: Decide on a statistical test (parametric or non-parametric) and conduct the
statistical test
Step 5: Interpret the results; both effect size (mean difference or r) and p value
QUIZ: WHAT TEST SHOULD I USE?
1. Investigating association between weight and pain in people with OA. Both are ratio
data and data is normally distributed.
2. Investigating the difference in QoL between people with stroke and people without
stroke. QoL is ordinal data.
3. Investigating the change in participation after a stretching programme in people with
MS. Participation is ratio data and is not normally distributed.
4. Investigating the association between height and running speed in athletes. Both are
ratio data and data is not normally distributed.
5. Investigating the difference in the prevalence of diabetes between people with and
without obesity. Data is nominal.
USEFUL RESOURCES
• Martin Bland (2000) An Introduction to Medical Statistics. 3rd edition.
Oxford: Oxford University Press.
• https://www.youtube.com/user/how2stats
• https://www.youtube.com/user/ProfAndyField
• Andy Field (2013) Discovering statistics using IBM SPSS Statistics: and sex
and drugs and rock ‘n’ roll. 4th edition. Sage.
https://www.youtube.com/user/how2stats
Populations and Samples
Validity, Reliability, Accuracy
What do they actually mean?
Neil O’Connell
PH2600 Research Methods
2018
Learning Outcomes
By the end of this lecture you should be able to:
Define the terms population and sample
Discuss some of the issues related to participant recruitment
Describe the various types of sampling methods
Discuss the importance of sample size and power
Define the terms Validity and Reliability and Accuracy
Discuss the issues of validity and reliability with regards to outcome measures
Discuss the issues of internal and external validity with regards to studies
TARGET POPULATION
SAMPLE
ACCESSIBLE
POPULATION
Populations and samples
Population: Every person in the group of
interest (target population)
Accessible population: Everyone in that group
who you have access to
Sample: A subgroup of the population who you
actually recruit.
You will only ever have a sample. The goal is to
make it as representative of the population as
possible
Inclusion/ Exclusion criteria
Sampling Methods
What people do I have access to?
What’s my time period for
recruitment?
Sampling Method
Can introduce known and unknown biases through the
methods used to recruit sample
Selection bias: The way you select people may mean your
sample isn’t representative of your population
Example: Sampling obese people from people attending a
weight-loss clinic – obese people attending a weight-loss
clinic may be severely obese and/or they may be motivated
to lose weight
Volunteer bias: People who volunteer to participate in
research may be different to those who refuse
Sampling Method Definition Advantages Disadvantages
Simple random
sampling
Each member of the
population has an
equal chance of
being selected.
(e.g.use a random
numbers table)
Should result in a
truly representative
sample – bias
minimised
Pragmatically
difficult, particularly
in large populations
Systematic sampling Select every nth
person in the
accessible population
More efficient than
random sampling
The order of the list
can introduce a
systematic bias
Stratified sampling Sampling ensures a
proportion of
representation across
specific
characteristics
Ensures balance on
characteristics that
are known to be
important
Getting info difficult,
time consuming and
sometimes arbitrary
Probability Sampling Methods
Stratified Sampling
Consider: 270 children with CP on a database
– Want to randomly select participants
Random sampling could result in an unequal number of boys and girls
Randomly select participants from each strata e.g. 10 boys and 10 girls
Non probability sampling
methods
Sampling
Methods
Description Advantages Disadvantages
Convenience
(incidental)
sampling
Take who you
can get!
Easy and
efficient
Easily affected
by bias- sample
may not be
representative
Snowball
sampling*
Get those you
have to ask
around!
Purposive
sampling*
Handpick those
who meet your
needs
* More common in qualitative research
SIZE MATTERS
How big a sample do you
need?
Bigger generally= better
Larger samples more
representative
Larger samples give more
precision
More sensitive to detect
differences between
groups/
conditions
For a survey you need enough
people to be sure that your sample
is representative.
For an experiment you need enough
to be sensitive to detect a change
The larger the true effect the easier
it is to detect and so the smaller the
required sample
TYPE 1 ERROR TYPE 2 ERROR
FALSE POSITIVE:
DETECTING AN EFFECT
THAT ISN’T ACTUALLY
THERE
(FAILURE TO ACCEPT THE
NULL HYPOTHESIS)
FALSE NEGATIVE:
FAILING TO DETECT AND
EFFECT THAT IS
ACTUALLY THERE
(ERRONEOUSLY
ACCEPTING THE NULL
HYPOTHESIS)
The two errors
Sample Size Calculation
Finger in the air method common and profoundly
dodgy
Better to perform a formal sample size calculation
But for this you need to establish some background
data
The method varies depending on the study design
This will be discussed in detail in the lecture on
inferential statistics
Measurement
Selecting an outcome measu
re
All outcome measures should be
accurate and feasible
What does accurate mean?
Valid
Reliable
How feasible is it for me to use?
Validity & Reliability of specific
measures
We need to know whether our
outcome measures are valid and
reliable
We can only be sure that we are
accurately quantifying something if
both have been established
Validity
‘Measures what it
intends to measure’
………..a ruler cannot
measure the weight
of an object!
(Hicks, 2009)
FACE VALIDITY
CRITERION VALIDITY
CONSTRUCT VALIDITY
CONTENT VALIDITY
Construct validity- What is a
construct?
An artificial framework that is not
directly observable….
Abstract ideas that explain
observable behaviours
E.G: depression, wellness, IQ…
Construct validity
Do scores on the test accurately reflect
the “construct” being measured?
Is the test a consistent reflection of the
underlying theory of the “construct”
Construct Validity
Can be established by reviewing the
literature of the theory that pertains to the
topic being researched.
Can also be established by examining the
convergent and divergent validity of a
measure.
Construct validity
Convergent Validity
Divergent
Validity
compares the target test with
other measures believed to
measure the same construct.
The results should correlate
highly if the same construct is
reflected in both tests.
compares the target test with
other measures believed to
measure different
characteristics or traits.
A low correlation is expected
in this case.
Convergent and Divergent
Validity
M
cG
il
l
Q
oL
Q
ue
st
io
nn
ai
re
Ro
la
nd
M
or
ri
s
di
sa
bi
li
ty
qu
es
ti
on
na
ir
e
SF-36 SF-36
Content Validity
Does the test measure a range of
behaviours that it purports to measure
based on the theoretical concepts on
which the test is founded?
Typically refers to questionnaires
Content Validity
A wide-range of observable, quantifiable
behaviours should be contained in the
measure.
The content should logically follow on from
the literature reviewed for the construct
validity stage as well as personal and
professional experience.
Can be established by using expert or user
review, or by generating and testing
hypotheses.
Content Validity
Example: Assessing ability of people with COPD to
do ADLs using a questionnaire
Conduct a literature review to establish the ADLs
that are important to people with COPD
Develop the questionnaire based on the literature
review
Ask people with COPD to review the questionnaire
and decide if it covers to topics that are
important to them and if there’s anything missing
Criterion Validity
Criterion validity is established by comparing the new
measure with an accepted gold standard of measurement
a.k.a. a criterion measure
kcal per minute
kcal per minute
Indirect calorimetry
Type Definition
Face Validity Does the test look as though it is measuring
what it is supposed to?
Construct
Validity
Does the test measure the theory relating to
the topic under investigation?
Content Validity Does the test measure a full range of
behaviours expected to emerge from the
theory?
Criterion
validity
Does the test show good levels of agreement
with an accepted gold standard?
Adapted from Hicks 2009
p.269
Reliability
The consistency
or repeatability
of a measure
NOTHING is
100% reliable
INTRA RATER
RELIABILITY
• Degree of consistency
of a set of measures
taken by the same
person
INTER
RATER
RELIABILITY
• Degree of
consistency in the
measures taken by
2 or more people
INSTRUMENT
RELIABILITY
INTER/ INTRA
• The degree of
consistency associated
with the specific
measurement tool
Threats to Reliability
What threats to
reliability might
arise during
goniometry?
Intra-rater
Inter-rater
Instrument
….intrasubject?
TRUTH
Unreliable and not
valid
Valid but not reliable
Reliable but not valid Valid and Reliable
Test
Accuracy
How accurate is my test?
Classification Accuracy: how well a test correctly
identifies or excludes a condition
e.g. does the Lachman test correctly identify people who have
and don’t have an ACL rupture
Sensitivity: how well a test correctly identifies a condition
e.g. does the Lachman test correctly identify people who have
an ACL rupture
Specificity: how well a test correctly excludes a
condition
e.g. does the Lachman test correctly identify people who don’t
have an ACL rupture
For example……
ACL
rupture
No ACL
rupture
total
Positive Test 5 1 6
Negative
Test
5 9 14
total
10 10 20
A physio does the Lachman test on 20 people to
test for ACL rupture
Arthroscopy indicated that 10 people had an
ACL rupture and 10 people didn’t
For example……
ACL
rupture
No ACL
rupture
Positive Test 5 1 6
Negative
Test
5 9 14
10 10 20
A physio does the Lachman test on 20 people to
test for ACL rupture
Arthroscopy indicated that 10 people had an
ACL rupture and 10 people didn’t
ACL
rupture
No ACL rupture
Positive Test 5 1
Negative Test 5 9
10 10
True positives False positives
False negatives
True negatives
ACL
rupture
No ACL rupture
Positive Test 5 1
Negative Test 5 9
10 10
True positives
Sensitivity = Number of people correctly identified as
having an ACL rupture/all people with an ACL rupture
i.e. Sensitivity = True positives/(true positives + false
negatives)
Sensitivity = 5/10 = 50%
ACL rupture No ACL rupture
Positive Test 5 1
Negative Test 5 9
10 10
True negatives
Specificity = People correctly identified as not having an ACL
rupture correctly identified/All people without an ACL
rupture
i.e. Specificity = True negatives/(false positives + true
negatives)
Specificity = 9/10 = 90%
ACL rupture No ACL rupture
Positive Test 5 1
Negative Test 5 9
10 10
True positives
Classification accuracy = People with and without ACL rupture
correctly identified/All people who were tested
i.e. Classification Accuracy = True positives + true
negatives/(positives + negatives)
Accuracy = 5+9/20 = 70%
True negatives
Validity at the “study level”
Just to confuse you, the term
“validity” is also often used in the
critical appraisal of whole research
studies.
In this case it means something a little
different
When we ask “Is this study valid?” we
are asking 2 overall questions…..
CAN I TRUST THESE
RESULTS?
ARE THE METHODS
ROBUST?
WHAT IS THE RISK OF
BIAS?
ARE THE RESULTS
GENERALIZABLE ?
INTERNAL VALIDITY EXTERNAL VALIDITY
Bias
Bias – the presence of systematic
error in a study
Brings you further away from the
truth
Should I believe the
results?
Internal validity
Refers to how well controlled a
study is. Internal validity is high
when:
◦ The risk of bias is low
◦ The risk of confounding is low
◦ The methods are tightly controlled
Threats to internal validity
Experimenter effects
Participant effects
History/ Maturation
Regression to the mean
Unreliable/ invalid measurements
Selection/ Assignment
Confounding
Attrition/ dropout
External Validity
Refers to how far the results can be
generalized to wider populations
External validity is high when:
◦ Sampling is broad and representative
◦ The study conditions mimic real world
conditions
Threats to external validity
Non random sampling (recruitment bias)
Restrictive inclusion/ exclusion criteria
Tightly controlled experimental conditions
Study conducted in a highly unique
environment
Small study size
How big is the
effect?
Useful/ relevant
outcomes?
Appropriate/
achievable
intervention?
Similar Patient
Group?
Do I even
have enough
information?
A Tension
Internal validity
Tight control of all
variables
Strict sampling
Rigorous measurement
Experimental manipulation
External validity
Broad and inclusive
Looser control – variability
allowed
Reflects real world practice
and measures
Summary
How you select your sample is a critical
methodological issue
Acceptable validity and reliability are vital
properties for a good outcome measure
Understanding what is meant by both and
threats to both is important
At the study level validity (internal/
external) refers to how rigorous a study is
and how generalizable its results are to a
broader population
Recommended reading
On selection and assignment of participants:
Chapter 9 Carter RE, Lubinsky J, Domholdt (2011)
Rehabilitation Research (4th edition). USA: Elsevier
On validity and reliability of a measure:
Pages 237-244 Carter RE, Lubinsky J, Domholdt (2011)
Rehabilitation Research (4th edition). USA: Elsevier
Pages 267 and 268 Hicks, C. (2009)
On study validity: Chapter 8 Carter RE, Lubinsky J,
Domholdt (2011) Rehabilitation Research (4th edition). USA:
Elsevier
Qualitative methods of data
collection and analysis
Meriel Norris
Research Methods PH2600
2020
What this lecture will cover
• How is qualitative research conducted?
– Introduction to
methods
• What is the role of the researcher?
– Introduction to reflexivity/ co-construction
• Analysis
– Introduction to thematic analysis
How do Qualitative Researchers
Collect Data?
Research Question Qualitative Approach
(Methodology)
Data Collection
(Methods)
What is the meaning
attached to this
phenomenon?
Phenomenology
(philosophy)
Semi-structured
interviews (
SSI
)
Diaries
What is life like for this
group?
Ethnography
(anthropology)
Biography/Case study
Participant observation
SSI, FG, visual methods,
document analysis,
video…
What is happening?
Why is it happening?
Grounded theory
(sociology)
SSI, focus groups (FG)
What are they
communicating?
How are they
communicating?
Discourse analysis
(sociology, linguistics)
SSI
Interviews
• A conversation with a purpose
• Probably the most widely used method in
qualitative research (Whiting 2008)
• Often personal and intimate encounters
But also
• “complex, labour intensive and uncertain
business, fraught with tricky issues…”
(Bannister 1994:49)
Interview types
• Structured (not really qualitative)
• Semi-structured
• Unstructured/open
• Dyad
Example of SSI Extract: Living with
Ataxia
• What’s it like to live with ataxia?…
• ‘I’m aware that my general gait when I’m walking makes
you look like you’re drunk…and yeah, sometimes you do
stumble and fall down. … part of my job involves working
in big offices and when you walk about there or when
you’re talking to somebody you can realise very simply
there are loads of pairs of eyes going chu-chu-chu –
looking at you and thinking ‘What’s wrong with him?’ I
think that is probably the biggest problem that anybody
with I suppose any disability has coming to terms with it,
accepting that you are different.’ (Jim)
Some strengths and limitations
Strengths
• Flexible enough to explore
experiences in depth
• Ability for participants to
reflect on complex
experiences
• Emic perspective
• Can gain large volumes of
rich data
Weaknesses
• Relevance of data
• Participants self-editing
• Reliant on skill of interviewer
• Transcription and data
analysis time-consuming
Examples of use of photographs/images
in qualitative research
Arti selected the photograph entitled ‘Arti standing’ (Fig 1)
and commented on its significance.
I: Why did you select this picture? What does it mean to you?
Arti: I am very happy to stand. Makes me think I can walk
independently, even though I know I can’t … give me hope that
I will be able to do it eventually. I am happy when I stand. I ask
myself, when will I be able to walk?
‘I view the world, the way the world’s set up is,
if … you’re up walking around, even though you’re not
perfect, but visually you’re perfect …’
Focus groups
• Discussion amongst a group – ‘added value’
• Able to gather multiple views in less time
• May address power issues (may also create
some)
• Can vary in level of structure
BUT
• May impede discussion of sensitive subjects
Example of focus group
• Participant 5: ‘Whatever you do don’t medicalise it … I think one of the key
benefits of this is that it’s not another bloody appointment. You know it’s
not the hospital … It’s also a community facility … it introduces you and
makes other things accessible. There is a gym just around the corner,
there is a pool at the other end, and it brings you to a place where,
suddenly … ‘ (36-year-old female).
• Participant 6: ‘ … It’s encouraged you to do stuff’ (81-year-old female).
• Participant 5: ‘It has … ‘
• Participant 7: ‘It has made this facility accessible to me. And it’s important
that it’s in the community because it brings people that perhaps are stuck
out of the community into the whole community’ (56-year-old male).
Topic guide
• List of indicative areas of interest
• Prompts to encourage depth
• Start general (accessible) and move to more
sensitive subjects
– Open
– Not leading
– Freedom to go ‘off piste’
(Participant) Observation
• Vary in structure, location, duration
• Observation vs immersion
• Can help refine focus
• Usually used in combination with other
methods
BUT
• Ethics can be a concern
• Being observed
• Challenges with documentation
Field notes
Researcher co-produces the data in
qualitative research
‘When I first got diagnosed [with MS], I think I was 24
or 25, that was really difficult then because I was
single then as well and it’s not exactly a starting point
of a conversation, and it did stop my doing a lot of
things that my friends did, I couldn’t go to a night-
club because the strobe lights completely, I can’t
walk, when strobe lights are going on, and (pause) so
it did stop me doing a lot of things, but now I don’t
think of what I can’t do, I think of what I can do.’
• WHAT WOULD BE YOUR NEXT QUESTION?
Data from Dr Frances Reynolds
Researcher co-produces the data in
qualitative research
• H When I first got diagnosed [with MS], I think I was
24 or 25, that was really difficult then because I was
single then as well and it’s not exactly a starting point
of a conversation, and it did stop my doing a lot of
things that my friends did, I couldn’t go to a night-club
because the strobe lights completely, I can’t walk,
when strobe lights are going on, and (pause) so it did
stop me doing a lot of things, but now I don’t think of
what I can’t do, I think of what I can do
• F Mmm you try to look at it differently?………
• H Yes it’s no good thinking of what I can’t do
compared with what I used to be able to do, because
you’d just go mad (pause) or really depressed so I don’t
really dwell on it.
Reflexivity 1
• Recognising the influence of the researcher in the
research process
• How experience, gender, profession, social status,
ethnicity & culture influence the choices made in the
study (from RQ to data collection, interpretation and
discussion)
Reflexivity: acknowledging the
researcher’s position in the research
Example of reflexivity in a qualitative research study (a study of
women’s experiences after surgery for obesity (Ogden et al., 2006;
p277):
• “In the case of the present study it is acknowledged that whilst
none of the researchers are either obese or have had surgery all
had gained sufficient experience of obese patients through clinical
and research work to believe that severe obesity was not a
desirable condition. …As both the quantitative and qualitative
studies progressed (Ogden et al., 2005) all researchers became
increasingly impressed with the effectiveness of surgery …It is
therefore possible that such views influenced the nature of the
analysis. However, … the impact [of surgery] upon the individual’s
broader psychological state and their sense of control … was
surprising”.
• Embodied roles: being a student and daughter
• Embodied gender, marital status, ethnicity and age
• Embodying emotion
• Embodied presences: being the body
What do Qualitative Methods have in Common?
• They typically take a social constructionist
perspective on knowledge – that the data are
created/ co-constructed during the research
process, and the process is value laden
What Else Do Qualitative Methods
have in Common?
• They take the ‘emic’ perspective (insiders’
viewpoints are valued, rather than relying on
‘expert’ outsiders)
• They value subjectivity – participants’ own
experiences, meanings, interpretations (‘lived
experience’).
• They are naturalistic – qualitative methods seek
to minimise ‘demand characteristics’,
intervention, deception, dehumanising
activities, artificial settings
Ethics and Qualitative Research: Harm and Burden
• Kvale (1996, p109) argues:
“An interview inquiry is a moral enterprise: the personal
interaction in the interview affects the interviewee, and the
knowledge produced by the interview affects our
understanding of the human situation”
• Ethical issues arise throughout the qualitative research
process, from initial planning to writing up and
dissemination (especially as most interviews concern
private, often troubling experiences) (Kvale, 2007)
Some ethical considerations
• Pseudonyms
• Removal of personal details (?photographs)
• Consent to quote
• Appropriate representation
• Behaviour in the ‘field’
• Management of potential distress
• Limits of confidentiality
• Fine line of intrusion
How Do Qualitative Researchers
Analyse Data?
• Generally through a type / variation of
thematic analysis
• Thematic analysis is a method for identifying
analysing and reporting patterns (themes) within
data
• Organizes the data set and retains rich detail
Braun, V and Clarke, V (2006) Using thematic analysis in
psychology. Qualitative Research in Psychology, 3, pp77-101
Basic process
• Transcribe the data
• Read multiple times (immersion)
• ‘Code’ the data, assign a label to categorise or
organise data
• Collect codes to create themes (crystalisation)
• Use direct quotations to ground the analysis in
the data
• Check back for similarities and differences (-ve
case analysis)
Variations within Thematic Analysis
• Thematic analysis can be descriptive or interpretative
• Essentialist/realist/descriptive: based on the
experiences, meanings and the reality of participants
• Interpretative: looks for deeper meaning; the
experiential claims of participants
• Constructionist: examines the ways in which events,
realities, meanings, experiences and so on are the
effects of a range of discourses operating within
society
Interpreting Interview Data
Realist versus interpretative approaches to
understanding interview data:
• Terry is explaining why she has become
involved in textile art…
• “I like to create things. And I love colour
and I love learning new things…My
mother, she used to paint as well. And my
father was an architect… I had several
aunts who used to paint as well. An uncle
who was a musician. It’s definitely there
[in the family] somewhere. We’ve never
had any interest in sport. Bottom of the list
always!” (Terry, mid-60s, rheumatoid
arthritis)
Data from Dr Frances Reynolds
Realist approach –
Terry is listing the
family members who
were interested in art
More interpretative
approach to analysis –
Terry is making an
identity claim – she
has an artistic
‘pedigree’; despite her
arthritis which creates
‘difference’, she firmly
belongs to her family
traditions; she is much
more than her
arthritis ; she belongs
to a lineage of artists
Other Data Analysis Methods
Constant Comparative Method (GT)
– Sample is set on theoretical grounds; purposive and added to
– the researcher moves back and forth between the data and the
emergent themes
– the researcher checks emergent themes/theory against further
interviews (and will also return to earlier interviews), simultaneous data
collection and analysis; helps to confirm or disconfirm emergent theory
– themes are refined – to achieve a better representation of participants’
meanings
Other methods of analysis
– Narrative analysis may search for story structures, story type (heroic or
‘road of trials’ etc), characters in the stories
– Framework analysis – systematic – deductive and inductive themes
– Content analysis may count units of meanings or specific words or
phrases (NB many types of content analysis – some like thematic)
Basic process (2)
Regardless of the type of qualitative
research, data analysis requires a ‘deep
dive’
– Sustained engagement
– Immersion
– Dwelling
– Iterative
What do Qualitative Methods have in Common?
• They typically take a social constructionist
perspective on knowledge – that the data are
created/ co-constructed during the research
process
• Researchers don’t tend to take a realist position
(that the data describe ‘real’ events or ‘facts’)
• May take a more strongly interpretative position
What is Qualitative Research?
(Assumptions and Values)
Qualitative research:
Is a process of inquiry
Aims to deepen understanding or insight
It generally doesn’t seek to explain phenomena
Based on distinct methodological and philosophical traditions
The researcher builds a complex, holistic picture
Analysis is commonly based on words but pictures can be
used
Data is generally considered to be ‘co-constructed’
Provides detailed in-depth views of informants
Usually conducted in a natural setting
(Creswell, 1998, p15)
Learning Outcomes
By the end of the module, you should be able to:
• Define qualitative research
• Explain the assumptions and values underpinning
qualitative research
• Explain the main approaches underpinning
qualitative research
• Describe qualitative data collection methods
• Explain different strategies for analysing
qualitative data
• Recognise rigour in qualitative research
Critiquing Qualitative Research
In the next lecture we will look at how to critically
evaluate qualitative research, taking account of:
– Credibility, confirmability, transferability,
dependability, trustworthiness
– Rigour
– Sample size
– Member checking/ respondent validation
– Negative case analysis
– Triangulation
– Saturation
References
Bannister, P. (1994) Qualitative Methods in Psychology: A Research Guide. Philadelphia: Open University Press
Cassidy, E, Reynolds, F, Naylor, S and De Souza, L (2011) Using Interpretative Phenomenological Analysis to
inform physiotherapy practice: an introduction with reference to the lived experience of cerebellar ataxia.
Physiotherapy Theory and Practice, 27 (4), 263-277.
Cresswell, J. (2012). Qualitative inquiry and research design: Choosing among five approaches. London: Sage
Cross, K, Kabel, A. & Lysack, c. (2006). Images of self and spinal cord injury: exploring drawing as a visual
method in disability research. Visual Studies, 21(2), 183-193
Dickson, A., Allan, D., O’Carroll, R. (2008). Biographical disruption and the experience of loss following spinal
cord injury: An interpretative phenomenological analysis. Psychology and Health, 23(4), 407-425
Gibson & Martin (2003) Qualitative Research and Evidence-based Physiotherapy Practice. Physiotherapy 89,
6: 350–358
Kvale, S and Brinkman, S (2009) InterViews: learning the craft of qualitative research. Sage, London
Merriam, S. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey Bass
Norris, M, Allotey, P and Barrett, G (2010) ‘I feel like half my body is clogged up’. Lay models of stroke in Central
Aceh, Indonesia. Social Science and Medicine, 71, pp. 1576-1583.
Norris, M. (2015) The complexities of ‘otherness’: reflections on embodiment of a young White British woman
engaged in cross-generation research involving older people in Indonesia. Ageing and Society 35(5):986-1010
Norris, M, Jones, F, Kilbride, C, Victor, C 2014. Exploring the experience of facilitating self-management with
minority ethnic stroke survivors: a qualitative study of therapists’ perceptions. Disability and Rehabilitation
ISSN (print) 0963-8288
Norris, M, Kilbride, C, Mohagheghi, A, Victor, C. 2013. Exercise-instructor led functional training programme for
community dwelling stroke survivors: A qualitative study. International Journal of Therapy and Rehabilitation
20(12): 597-605
Introduction to experimental
designs
PH2600 2019
N
eil O’Connell
Learning outcomes
By the end of the lecture students should
be able to:
Describe basic common experimental
study designs
Consider some of the biases that
attempt we control for
Describe the basic purpose and
structure of a systematic review
A bottom line
The choice of design should
arise from the research
question – not the other way
around.
Experimental design – definition
In which one (or more) variable(s) is
manipulated and the effect of this
manipulation is observed in other
variables.
It aims to control all other variables.
It allows us to infer causality
Causality
If there is change to A does a change
in B result?
◦ Cause must precede the effect
◦ The cause and effect must co-vary
◦ If the cause does not occur then neither
does the effect
Inferring causation – problems
Confounding
Regression to the mean
Natural recovery
Placebo/ non-specific effects
Hawthorne Effect (Observer)
Rosenthal Effect (Experimenter
expectancy)
Time
itself is a
confounder
Se
ve
ri
ty
Time
Se
ve
ri
ty
Time
Control group
By including a group who undergo the
same conditions (except…) as the
experimental group we control for
numerous possible confounders
For within-subjects designs this might
be a control condition
Blinding
Why conceal the identity of
the experimental condition?
◦ A function of placebo groups –
‘sham’ interventions
◦ Single blind
◦ Double-blind
◦ Triple Blind
◦ What confounders might
blinding control for?
Who can
we blind
in trials
of PT?
Group designs – within or
between subjects
Within Group design
One group of
participants receives
all experimental
conditions (including
control)
Offers paired data
Between-Group
Design
Different groups
receive the different
experimental
conditions
Offers unpaired data
Designs
Randomised controlled experiment.
Parallel, cross-over, factorial
Controlled experiment
Quasi experimental study
Single group pre-test post-test
design
Group before
Same group
after
I
N
T
ERV
EN
T
IO
N
Time series design
IN
T
ER
V
EN
T
IO
N
measure measure
Basic parallel experimental design (pre
test-post test)
Experimental
group
INTERVENTION
CONTROL
Follow up
Follow up
Control group
Pre
test
Post test
SAME
POPULATION
TAKE
BASELINE
MEASURES
INTERVENTION
CONTROL
FOLLOW UP
FOLLOW UP
Pre
test
Post test
How to ensure the groups are
the same?
Matching groups
Or
Use the same group for the different
conditions
Or
Randomisation
RANDOMISATION
The beauty of randomisation
It solves all your problems (maybe)!
In NRS you can only control for known
confounders
Successful randomisation controls for
all
Even imbalances at baseline occur at
random and are unsystematic biases.
RA Fisher (1935)
“Randomisation
relieves the
experimenter from the
anxiety of considering
and estimating the
magnitude of the
innumerable causes by
which his data may be
disturbed”
Concealed allocation (because all
people…)
Think of this like protection for
your randomisation
The person admitting patients
to the trial must not know the
allocation schedule
Because they might cheat
(remember Bill Silverman’s
nurse)
Ask “could the allocation have
been fixed?”
SAME
POPULATION
TAKE
BASELINE
MEASURES
INTERVENTION
CONTROL FOLLOW UP
FOLLOW UP
Pre
test
Post test
R
A
N
D
O
M
ISAT
IO
N
Drop Out and Protocol
Violation
Participants often do not
complete the treatment
or withdraw from trials
Often this is not at
random, so the benefits of
randomisation are lost
Clear reporting and
appropriate management
of violation and dropout is
vital
For example…
In a trial of Constraint induced
therapy versus usual rehab the CIT
group experience high drop-out
because:
◦ Unable to tolerate constraint
◦ Don’t feel they are making progress
How might that skew results?
Questions to ask
How many people
dropped out and
when?
Why did they drop
out?
Was there a big
difference in drop-
out between
groups?
How
were any missing
data managed?
Intention to treat versus per
protocol
Per protocol: only participants who
complete the trial/ treatment to
which they were randomised are
analysed
Intent to treat: (ITT) All participants
who were randomised are entered
into the analysis regardless of
violation or withdrawal
IN
T
E
R
V
E
N
T
IO
N
P
E
R
IO
D
Repeated measures design
baseline
Condition
1
measure Condition
2
measure
Order and carry over effects
Time remains a confounder
Treatments might have lasting
effects (positive or negative)
So each treatment might start
from a different baseline
How can we fix that?
Randomised Cross-Over Design
Hulley et al. Designing Clinical Research. 2ndEdition. Lippincott Williams & Wilkins,
2001
Factorial Design
E.G.
Does spinal
manipulation work
better than CBT or
is giving both the
best?
More than one IV
that may interact?
For example 2
treatments that
might be delivered
together
SMT NO SMT
CBT Group 1 Group 2
NO CBT Group 3 Group 4
Quasi experimental?
An odd term – usually used to denote
that allocation was not randomised.
Can mean a design that lacks one of
these:
◦ Randomisation
◦ Control group
◦ Pre-test – post test design
Non-specific
Natural history
Regression to the mean
Placebo / interaction
effects
Hawthorne effect
Resentful demoralisation/
“frustrebo”/ nocebo
Specific
The effect of the
proposed “active
ingredient” or
mechanism of the
therapy
Specific and non-specific effects
ACTIVE
INTERVENTION
SHAM
INTERVENTION
NO
INTERVENTION
The active
ingredient
+ non
specific interaction
effects
Non-specific
effects
The active
ingredient
Experimenting on the individual:
Single subject designs, n of 1
studies
A-B design
A-B-A design
A-B-A-B design
A-B1-A-B2-A……
Test intervention
and its withdrawal
Can allow
randomisation
Clinically achievable
RCTs are great but at the top of the
evidence based tree….
RCT
COHORT
STUDIES
CASE CONTROL
STUDIES
CASE STUDIES, ANECDOTE,
OPINION, LAB STUDIES
Systematic reviews of
RCTs
Systematic Reviews
Each individual trial represents an estimate
of the true effect of a treatment
Seek to pool all of the available evidence for
the most accurate estimate
Take a very deliberate systematic approach
Allows pooling of results (meta-analysis)
What it isn’t
Formulation of a specific question to
be addressed –
(PICO structure)
Retrieval of ALL relevant literature;
systematic seach, selection of studies
based upon pre-defined criteria
Tabulation of study characteristics/
assessment of study quality/ risk of
bias
Synthesis/numerical aggregation of
study findings
Hypothesis testing? Presentation of
effect size? Narrative synthesis?
Systematic
Review
Meta-Analysis – What’s the
point?
Meta-analysis measures size & consistency
of treatment effect across >1 study
By pooling data from studies we can
increase the accuracy and power of our
estimates
We can estimate both the treatment effect
and the consistency between studies
Meta-Analysis – What’s the
point?
Less chance of rejecting
an intervention that
actually works
Less chance of being
fooled by falsely positive
data…although….
Quality of original studies
is vital
Rubbish in = Rubbish out!
Meta-analysis how does it work?
Estimates the effect size for
each study
Weights each study (usually on
size & precision)
Pools each study to generate
one overall effect size
Done properly this should be
more precise than that from
individual studies
Don’t get lost in
the forest (plot)
The line of no
effect
Confidence
intervals
Point estimate
Pooled estimate of
effect
WHAT EVIDENCE IS OUT
THERE?
IS IT ANY GOOD, SHOULD
WE TRUST IT?
DOES THE TREATMENT
CONSISTENTLY SEEM TO
“WORK”
DOES IT CAUSE HARM?
HOW IMPORTANT/
LARGE ARE THE
EFFECTS?
SO A GOOD SYSTEMATIC REVIEW MIGHT TELL
YOU:
Recommended Reading
Hicks Chapter 6&7
Carter, Lubinsky and Domholdt
Section3, Chapters 10 and 11