7860-U1D2 - Describe the type of quantative research, the key concepts, what is meant by scientific merit, ect. See details below.

Purposeful Sampling in Qualitative

Research Synthesis

Harsh Suri

THE UNIVERSITY OF MELBOURNE

ABSTRACT

Informed decisions about sampling are critical to improving the quality of research synthesis. Even

though several qualitative research synthesists have recommended purposeful sampling for synthes-

izing qualitative research, the published literature holds sparse discussion on how different strategies

for purposeful sampling may be applied to a research synthesis. In primary research, Patton is fre-

quently cited as an authority on the topic of purposeful sampling. In Patton’s original texts that are

referred to in this article, Patton does not make any suggestion of using purposeful sampling for re-

search synthesis. This article makes a unique contribution to the literature by examining the adapt-

ability of each of Patton’s 16 purposeful sampling strategies to the process of qualitative research

synthesis. It illuminates how different purposeful sampling strategies might be particularly suited to

constructing multi-perspectival, emancipatory, participatory and deconstructive interpretations of

published research.

Keywords: Purposeful sampling, qualitative research synthesis, meta-synthesis, meta-study, qualit-

ative meta-analysis.

Research synthesis isaspecial type of research review that isnot only descriptive, informative

and evaluative, but also connective (Mays, Pope, and Popay, 2005). ‘Synthesis refers to

making awhole into something more than the parts alone imply’ (Noblit & Hare, 1988,

p. 28). The purpose of research synthesis isto produce new knowledge by making explicit

connections and tensions between individual study reports that were not visible before. It

involves purposeful selection, review, analysis and synthesis of primary research reports on

asimilar topic. In arigorous synthesis, readers are provided with sufficient information

about the synthesis process so that they can make informed decisions about the extent to

which the synthesized findings may be adapted to their own contexts.

With the growth of research activity in recent years, each topic tends to be examined by

different researchers in diverse contexts, employing awide range of methods, invariably

resulting in disparate findings on the same topic. Making useable sense of such complex

bodies of research can be an overwhelming experience for most stakeholders. These stake-

holders include policymakers, administrators, educators, health professionals, funding

agencies, researchers, students, patients, various advocacy groups and the wider community.

Research syntheses can play an important role in disseminating research knowledge and in

shaping further research, practice and public perception. Hence, issues of ethical represent-

ations (Suri, 2008) and methodological rigour in research syntheses are as crucial as they are

in primary research (Petticrew & Roberts, 2006).

Much of the growing body of literature on research synthesis methods has been dominated

by quantitative researchers. In the last two decades, however, an increasing number of re-

searchers from education and healthcare have recognised the importance of synthesizing

qualitative research. These scholars acknowledge that the emphasis of qualitative research

on particularities and complexities of individual contexts isatodds with any synthetic effort.

Yet they assert that some form of synthesis is essential to enhance the practical value of

Suri, Harsh, 2011, 'Purposeful Sampling in Qualitative Research Synthesis', Qualitative Research Journal , vol. 11, no. 2, pp. 63-75. DOI 10.3316/QRJ1102063. This is a peer-reviewed article. qualitative research in policymaking and informing practice atabroader level (Suri & Clarke,

2009). Stressing that any effort of synthesizing qualitative research should be essentially in-

terpretive rather than aggregative, they have proposed interpretive methods of research syn-

thesis under various names, such as meta-ethnography (Noblit & Hare, 1988), cross-case

analysis (Miles & Huberman, 1994), meta-analysis of qualitative research (Jensen & Allen,

1994), qualitative meta-synthesis (Sandelowski, Docherty, and Emden, 1997; Zimmer,

2006), qualitative systematic review (Booth, 2001), meta-study (Paterson, Thorne, Canam,

and Jillings, 2001), and qualitative research synthesis (Major & Savin-Baden, 2010).

Several other methodologists argue that acomprehensive synthesis of research should

include both quantitative and qualitative studies. They reason that quantitative methods are

inappropriate to synthesize methodologically diverse research as quantitative research syn-

thesis methods assume acertain degree of methodological and contextual homogeneity across

studies, which isimpractical. They assert that only qualitative synthesis methods are suitable

for synthesizing methodologically diverse quantitative and qualitative research (Suri &

Clarke, 2009). Examples of qualitative methods proposed for synthesizing methodologically

diverse research include qualitative meta-analysis (Kasworm, 1990), exploratory case-study

oriented review (Ogawa & Malen, 1991), interpretivist-oriented review (Eisenhart, 1998),

meta-synthesis (Bair, 1999) and realist synthesis (Pawson, 2006).

There isyet another group of methodologists, for example Lather (1999) and Schwandt

(1998), who make acase for post-structural reviews of research which they insist are not

research syntheses per se as they focus on identifying the cracks, or the gaps, in afield rather

than producing ameta-narrative. Unlike Lather (1999) and Schwandt (1998), Ihave delib-

erately used the term ‘research synthesis’ as ablanket term, which includes critically oriented

post-structural reviews, to reclaim its usage for an inclusive context rather than being limited

to only those syntheses that produce meta-narratives. Ihave retained the term research syn-

thesis ‘to both circulate and break with the signs that code it’ (Lather, 1993, p. 674) by

rupturing the exclusive notion of research synthesis as an objective and reductionist aggreg-

ation of research findings.

Ihave used the term qualitative research synthesis as an umbrella term for all qualitative

methods of synthesizing research which are informed by interpretive, critical, emancipatory

and/or postmodern sensibilities. Primary research included in aqualitative research synthesis

may be qualitative and/or quantitative, depending on the purpose of the synthesis.

METHODOLOGICAL UNDERPINNINGS

This article isbased on alarger project in which amethodologically inclusive research syn-

thesis (MIRS) framework was conceptualised (Suri, 2007) by distilling and synthesizing di-

verse ideas, theories, and strategies from the extensive bodies of literature on research syn-

thesis methods and primary research methods. The MIRS framework was developed to address

the following overarching question: 'Given that contemporary educational research ismarked

by diversity, complexity, and richness of purposes, methods, and perspectives, how can such

variety and complexity be accommodated and reflected atthe level of synthesizing educational

research?' In developing the MIRS framework, acombination of purposeful sampling

strategies were employed, some of which are described later in this article to illustrate the

applicability of various purposeful sampling strategies.

The goal of this article isto contest the notion that exhaustive sampling isthe only legit-

imate form of sampling for research synthesis. The question that isaddressed here isnot

64 | Qualitative Research Journal , vol. 11, no. 2, 2011 ‘what sampling strategies are typically employed by qualitative research synthesists?' Rather,

by drawing upon hypothetical examples, the question that isbeing addressed is‘how might

different purposeful sampling strategies be adapted to expand possibilities within research

syntheses?’ It ishoped that the readers will use this discussion as adeparture point to syn-

thesize research for awide range of purposes, many of which are typically not attempted by

contemporary research synthesists.

Ibegin this article by building acase for purposeful sampling in research synthesis. Then

Idraw on the concepts of data saturation and data sufficiency for guiding decisions related

to enacting closure when searching for relevant evidence in research synthesis. Iconclude

this article by outlining key questions which must be considered in making strategic decisions

in relation to sampling in research synthesis. In the literature on primary research methods,

Patton (1990, 2002) has provided acomprehensive discussion of purposeful sampling and

isfrequently cited as an authority on purposeful sampling:

The logic and power of purposeful sampling lie in selecting information-rich cases for study

in depth. Information-rich cases are those from which one can learn agreat deal about issues

of central importance to the purpose of the inquiry, thus the term purposeful sampling.

Studying information-rich cases yields insights and in-depth understanding rather than em-

pirical generalizations (Patton, 2002, p. 230, emphasis in original).

Patton has suggested 16 strategies for purposeful sampling in qualitative research, each

of which isintended to serve adifferent purpose. In Patton’s original texts that Irefer to,

Patton does not make any suggestion of using purposeful sampling in research syntheses.

Patton’s original texts exclusively discuss utilisation of purposeful sampling in primary re-

search. In this article, Iexplore the potential adaptation of Patton’s concept of purposeful

sampling to the process of aresearch synthesis by using three techniques. First, the terms

that Iuse here are the verbatim terms that Patton (2002) has used to distinguish between

the 16 strategies under the broad umbrella of purposeful sampling. Second, Iseamlessly in-

tegrate Patton’s ideas in my discussion of purposeful sampling in research syntheses. In many

instances, Ihave adapted Patton’s quotes to the process of aresearch synthesis beyond their

originally intended context of primary research. Third, Iillustrate with examples how each

of Patton’s 16 strategies may be utilised in syntheses with varied purposes.

BUILDING A CASE FOR PURPOSEFUL SAMPLING IN RESEARCH

SYNTHESIS

Research syntheses on the same topic conducted for different purposes can have different

sampling strategies, each being equally legitimate but tailored to serve the different purposes.

Synthesists must carefully identify sampling strategies that are conceptually aligned with the

synthesis purpose, that credibly and sufficiently address the synthesis purpose, and that are

feasible, ethical and efficient (Kemper, Stringfield, and Teddlie, 2003). Synthesists must

also delineate the caveats associated with their sampling strategies and speculate on how

these caveats may impact upon the synthesis findings. In doing so, itisworth reflecting on

the politics of publishing: that is, who/what ismore likely to get published. This involves

being aware of various publication biases, that is, outcome bias, confirmatory bias, funding

bias and methodological bias (for amethodologically inclusive discussion of these biases, see

Suri, 2008). Synthesists must be reflexive about how these potential biases might impact

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 65 upon their synthesized product as well as how their synthesis might reify/contest some of

these biases.

A rigorous research synthesis makes much more demand on time and resources when

compared with ad hoc reviews (Elmore, 1991; Ogawa & Malen, 1991; Stock, Benito, and

Lasa, 1996). Just as the perfect primary research study has never been conducted, neither

has the perfect synthesis. The issue that confronts asynthesist isoften ‘how to maximise the

quality of the synthesis within the available resources’, rather than ‘how to do the most rig-

orous synthesis’. Several primary researchers discuss various pragmatic constraints which

must be kept in mind when identifying asuitable purpose for study (e.g. Blaxter, Hughes,

and Tight, 2001). Research synthesists are also bound by the pragmatic constraints of time,

resources and access to information and expertise (for adetailed discussion of these pragmatic

constraints, see Suri, 2007). Often decision-makers and stakeholders want relevant, under-

standable, and accurate information which they can use soon. In many practical situations,

synthesists may find that ahighly rigorous approach isoverly formalistic, too time consuming,

and unnecessarily expensive within the available resources and deadlines (Patton, 1991, pp.

287-289).

Many research syntheses which attract large funding, such as the systematic reviews of

the EPPI-centre, cost about ₤75,000 (Oakley, 2003, p. 28). Often these reviews tend to

address the questions of politicians or other decision-making bodies who can provide adequate

funds. Systematic reviewers typically aim for extensive sampling and assume sufficient access

to financial resources, information and expertise. Unless synthesists strategically design

syntheses within various pragmatic constraints, they may inadvertently contribute to the si-

lencing of concerns of certain groups of stakeholders who cannot commission syntheses that

require large scale funding.

Many qualitative research synthesists also question the viability of holistically synthesizing

alarge number of qualitative reports in away that adequately attends to the intersections

between the findings, contexts, epistemologies, ontologies and methodologies of individual

studies. These qualitative synthesists recommend that an in-depth synthesis of purposefully

selected studies ismore desirable than asuperficial synthesis of alarge number of studies

(e.g. Bondas & Hall, 2007; Booth, 2001; Lloyd Jones, 2004; Noblit & Hare, 1988; Pawson,

Greenhalgh, Harvey, and Walshe, 2005). Even though several qualitative research synthesists

have recommended purposeful sampling in research syntheses, the published literature holds

sparse discussion on how different strategies for purposeful sampling might be applied to

research synthesis. In the next section, Iattempt to partially fill this gap in the literature by

discussing how different purposeful sampling strategies may be adapted to synthesize research

for facilitating understanding, participation, emancipation and/or deconstruction.

EXAMINING THE ADAPTABILITY OF PATTON’S PURPOSEFUL

SAMPLING STRATEGIES TO QUALITATIVE RESEARCH SYNTHESES

Purposeful sampling requires access to key informants in the field who can help in identifying

information-rich cases. Qualitative research synthesists can draw upon the ‘invisible college’

technique frequently utilised by quantitative researchers. Atraditional invisible college involves

agroup of central figures investigating aparticular field along with the numerous researchers

who are in touch with any of these key researchers. Research retrieved through this channel

islikely to be biased towards the beliefs prevalent among these key researchers (Rosenthal,

1994). Electronic invisible colleges include listservs or newsgroups, some of which might

66 | Qualitative Research Journal , vol. 11, no. 2, 2011 focus on research while others might focus on contemporary practices (Cooper, 1998).

Listservs with aresearch focus can be useful in identifying primary research studies or previous

syntheses. Practice-focused listservs can be useful in identifying reports that are particularly

valued among practitioners or in identifying the synthesis questions that might be of partic-

ular interest to practitioners. Browsing through listservs can also help identify researchers

and practitioners who have expertise in the substantive domain of interest. The synthesist

can later contact these experts directly to request references to the specific studies on which

their claims and opinions are based.

Extreme or Deviant Case Sampling

The extreme or deviant case sampling in aresearch synthesis would involve selecting ‘illu-

minative cases’ (Patton, 2002, p. 232) that exemplify contexts where an innovation was

perceived notably as asuccess or afailure. The main weakness of extreme case sampling is

its lack of generalisability through representativeness. This weakness isof less concern for

synthesists who focus on how things should be or could be rather than how things are .This

strategy would be particularly suitable for ‘realist syntheses’, proposed by Pawson (2006),

which investigate how aprogram islikely to work under particular circumstances by examin-

ing successful as well as unsuccessful implementations of the program.

Intensity Sampling

Intensity sampling in aresearch synthesis would involve selecting studies that are ‘excellent

or rich examples of the phenomenon of interest, but not highly unusual cases… cases that

manifest sufficient intensity to illuminate the nature of success or failure, but not at the ex-

treme’ (Patton, 2002, p. 234). To develop acomprehensive understanding of many educa-

tional changes, it is crucial to examine cases where these changes have been embedded

thoroughly in the system over asufficient period of time. However, such intense manifestation

of an innovation can be cost-intensive and/or associated with high risk-factors. As aresult,

the innovation might be implemented with sufficient intensity in only afew studies. Many

other studies might examine the implementation of the innovation over short durations of

time with minimal interventions. As an example, alarge number of studies have been con-

ducted to investigate how students learn in collaborative environments. Given the individu-

alistic nature of most high stake testing, most studies use collaborative learning strategies as

an add-on to regular teaching and learning activities. Many students in these studies might

engage with collaborative tasks superficially. In asmall number of studies, the learning

activities as well as assessment tasks have been revised intensely to reward collaboration. An

in-depth synthesis of the latter type of studies would be particularly useful in illuminating

arange of opportunities, challenges, advantages and disadvantages associated with curricula

driven by an ethos of collaborative learning.

Maximum Variation (Heterogeneity) Sampling

Amaximum variation sample isconstructed by identifying key dimensions of variations and

then finding cases that vary from each other as much as possible. This sampling yields: ‘(1)

high-quality, detailed descriptions of each case, which are useful for documenting uniqueness,

and (2) important shared patterns that cut across cases and derive their significance from

having emerged out of heterogeneity’ (Patton, 2002, p. 235). Employing maximum variation

sampling, research synthesists can identify essential features and variable features of aphe-

nomenon as experienced by diverse stakeholders among varied contexts to facilitate informed

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 67 global decision-making. Presuming that different study designs illuminate different aspects

of aphenomenon, maximum variation sampling can be utilised to construct an holistic

understanding of the phenomenon by synthesizing studies that differ in their study designs

on several dimensions. In conceptualising the MIRS framework, Iused acombination of

sampling strategies, one of which ismaximum variation sampling. Ideliberately drew ideas

from methods of primary research and research synthesis that were markedly different from

each other on many dimensions.

Homogenous Sample

‘In direct contrast to maximum variation sampling isthe strategy of picking asmall, homo-

genous sample, the purpose of which is to describe some particular subgroup in depth’

(Patton, 2002, p. 235). Research synthesists are frequently criticised for ‘mixing apples and

oranges’. Research synthesists can overcome this problem to some extent by selecting studies

that are relatively homogenous in their study designs and conceptual scope. Homogenous

samples can facilitate meaningful comparisons across studies. Underscoring the epistemolo-

gical incommensurability of different qualitative methods, some qualitative research synthesists

recommend acertain level of methodological homogeneity among primary research studies

which are included in aqualitative research synthesis (e.g. Eastabrooks, Field, and Morse,

1994; Paterson etal., 2001). Homogenous samples are particularly suitable for participatory

syntheses in which the synthesist co-synthesizes research with practitioners about aphenomen-

on that has direct implications for their practice (for adetailed discussion of participatory

synthesis, see Suri, 2007). For instance, agroup of secondary math teachers intending to

introduce collaborative learning activities into their classroom might benefit more from co-

synthesizing collaborative learning research in secondary math rather than collaborative

learning research across all grade-levels and different disciplines.

Typical Case Sampling

The purpose of typical case sampling ‘is to describe and illustrate what istypical to those

unfamiliar with the setting’. Typical cases are selected ‘with the cooperation of key informants’

or using ‘statistical data… to identify “average-like” cases’. When employing typical case

sampling, itiscrucial ‘to attempt to get broad consensus about which cases are typical–and

what criteria are being used to define typicality’ (Patton, 2002, p. 236). Research synthesists

can select typical primary research studies employed in the field with the cooperation of key

researchers in the field to describe typical methodologies and study designs employed to

examine the phenomenon. This would be particularly useful for studying how common

themes recurring in the published literature might be related to the relative strengths and

weaknesses of the typical methodologies or theories underpinning the typical studies.

Critical Case Sampling

Critical case sampling can facilitate ‘logical generalizations’ with the reasoning ‘that “if it

happens there, itwill happen anywhere,” or, vice versa, “if itdoesn’t happen there, itwon’t

happen anywhere”’ (Patton, 2002, p. 236). Critical case sampling in aresearch synthesis

might be employed to assist stakeholders in making informed decisions about the viability

of an educational program. For example, consider an innovation that produces desirable

outcomes, but isbeing rejected by many practitioners as they believe that its implementation

requires substantial resources. Asynthesis of primary research studies which describe in detail

successful implementation of the innovation with minimal resources might be useful to al-

68 | Qualitative Research Journal , vol. 11, no. 2, 2011 leviate the practitioners’ resistance towards the innovation. Alternatively, consider an innov-

ation which requires substantial financial resources. However, the proponents of the innov-

ation assert that the innovation iscost-effective provided sufficient resources are invested in

its implementation. In such an area, aresearch synthesist can selectively synthesize cases re-

ported in primary research studies that were sufficiently endowed with resources to logically

verify, or challenge, the claims made by those advocating the innovation.

Snowball or Chain Sampling

Snowball sampling involves seeking information from key informants about details of other

‘information-rich cases’ in the field. ‘The chain of recommended informants would typically

diverge initially as many possible sources are recommended, then converge as afew key

names get mentioned over and over’ (Patton, 2002, p. 237). Snowball sampling can be

utilised by seeking information from various listservs to identify primary research reports

that are frequently referred to by various stakeholders interested in the phenomenon. Even

though snowball sampling can introduce an ‘expert bias (e.g. preferences for large samples

or frequently cited studies)’ (Light & Pillemer, 1984, p. 40), it is particularly useful for

capitalising on expert wisdom, identifying studies that are highly valued by different stake-

holders and identifying studies outside the academic mainstream. Another way in which

snowball sampling may be utilised in aresearch synthesis isbased on the assumption that

the most cited primary research reports are the most information-rich cases. The synthesist

might identify the most cited primary research reports by ‘footnote chasing’, which involves

searching the citation indices, or by browsing through the bibliographies of selected reports

on the relevant topic: previous research syntheses, primary research reports, policy documents,

papers written by practitioners and papers written for practitioners. Unlike footnote chasing

for exhaustive sampling, footnote chasing for snowball sampling would involve locating the

most cited papers. However, this approach can reinforce confirmatory bias where studies

that agree with the prevalent wisdom are more likely to be published and cited while studies

that contest the conventional wisdom are less likely to be published or cited (Suri, 2008).

To overcome this problem, Booth (2001) suggests that we regard akey article as one which

references abroad selection of key articles to demonstrate that the study isinformed by a

wide range of perspectives. However, Booth’s suggestion can also be problematic as itmay

exclude methodologically rigorous and relevant studies which focus deeply, rather than

broadly, on acertain aspect of the phenomenon studied.

Criterion Sampling

Criterion sampling involves reviewing and studying ‘all cases that meet some predetermined

criterion of importance’ (Patton, 2002, p. 238). This approach isfrequently employed by

research synthesists to construct acomprehensive understanding of all the studies that meet

certain pre-determined criteria. Most research synthesists employ criterion sampling by

stating explicit inclusion/exclusion criteria which includes specifications for methodological

rigour. Itiscrucial to reflect critically and realistically on the criteria being used, especially

the criteria for methodological rigour. Very strict criteria for methodological rigour can

result in inclusion of such asmall number of studies that the transferability of synthesis

findings becomes questionable. At the same time, including methodologically weak studies

can also result in the synthesis findings being based on questionable evidence. Rather than

looking for methodologically perfect studies, the synthesist must reflect on how the method-

ological specifications of the study might have influenced its report.

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 69 Theory-Based Sampling, Operational Construct Sampling, and Theoretical

Sampling

Theory-based sampling involves selecting cases that represent important theoretical constructs

about the phenomenon of interest. This issimilar to operational construct sampling in which

one selects cases that represent ‘real-world examples (i.e. operational examples) of the con-

structs in which one is interested’ (Patton, 2002, pp. 238-239, emphasis in original).

Grounded-theorists define theoretical sampling as the sampling that isbased on the concepts

emerging from the data for the purpose of exploring ‘the dimensional range or varied condi-

tions along which the properties of concepts vary’ (Strauss & Corbin, 1998, p. 73). Research

synthesists who employ constant comparative methods or grounded-theory approaches can

fruitfully utilise theoretical sampling to systematically elucidate and refine the ‘variations in,

manifestations of, and meanings of aconcept as itisfound’ (Patton, 1978, p. 238) in the

selected primary research studies. Many qualitative synthesists recommend theoretical

sampling as asuitable option for research syntheses (Dixon-Woods, Agarwal, Jones, Young,

and Sutton, 2005; Mays et al., 2005). For example, in their meta-study, Paterson and her

colleagues (2001) draw on theory-based sampling or operational construct sampling by setting

out operational definitions of the key constructs about the phenomenon of interest. The

boundaries of these operational definitions are further articulated by explicitly stating inclu-

sion/exclusion criteria in relation to selecting primary research reports for the synthesis.

Confirming and Disconfirming Cases

‘Confirmatory cases are additional examples that fit already emergent patterns; these cases

confirm and elaborate the findings, adding richness, depth, and credibility’ (Patton, 2002,

p. 239). Confirmatory cases may be sought in an openly ideological synthesis when the

synthesist wishes to advocate aparticular stance for ethical, moral and/or political reasons.

Asynthesist may also seek confirming cases in order to validate the perceptions of aparticular

group of marginalised stakeholders. Disconfirming cases ‘are the examples that don’t fit.

They are asource of rival interpretations as well as away of placing boundaries around

confirmed findings’ (Patton, 2002, p. 239). To shake our complacent acceptance of popular

myths and/or generalisations in afield, synthesists can exclusively seek primary research

studies that disconfirm generalisations proposed in policy documents, previous syntheses or

primary research studies.

Stratified Purposeful Sampling

‘Stratified samples are samples within samples’ where each stratum is‘fairly homogenous’.

The purpose of stratified purposeful sampling is‘to capture major variations’ even though

‘acommon core… may also emerge in the analysis’ (Patton, 2002, p. 240). Stratified pur-

poseful sampling isuseful for examining the variations in the manifestation of aphenomenon

as any key factor associated with the phenomenon is varied. In aresearch synthesis, this

factor may be contextual, methodological, or conceptual. It isparticularly useful to study

different models of implementing aparticular teaching and learning strategy, such as distinct

models of cooperative learning that are commonly used by teachers. Often, traditional re-

viewers tacitly draw on stratified purposeful sampling by clustering studies according to a

key dimension of variation and then discussing each cluster in-depth. In developing the

MIRS framework, Iemployed stratified purposeful sampling to select key publications from

many distinct qualitative research traditions. By seeking input from qualitative researchers

with diverse methodological orientations and reading general qualitative research methods

70 | Qualitative Research Journal , vol. 11, no. 2, 2011 texts, Iidentified distinct strata of qualitative research methodologies and clusters of key

texts within each stratum.

Opportunistic or Emergent Sampling

‘Opportunistic, emergent sampling takes advantage of whatever unfolds as itunfolds’ by

utilising ‘the option of adding to asample to take advantage of unforeseen opportunities

after fieldwork has begun’ (Patton, 2002, p. 240, emphasis in original). Opportunistic or

emergent sampling can be useful for synthesizing aresearch area which isat its exploratory

stage, such as mobile learning, or when the synthesist does not have an emic or insider status

in the relevant field of research. Emergent sampling isalso suited to participatory syntheses

where the synthesis purpose evolves in response to the changing needs of the participant co-

synthesists (Suri, 2007). For instance, the purpose of asynthesis in the area of mobile

learning might be guided by the key questions or concerns of agroup of professors who are

teaching with mobile technologies. The synthesist might then enter the field and search for

reports to address these questions. When the synthesist feeds this information back to the

professors, their questions might also change. In response to their changing questions, the

synthesist might seek further studies with adifferent set of criteria. While pursuing these

searches, the synthesist isalso likely, serendipitously, to find primary research reports that

will provide useful insights into the phenomenon of mobile learning. Given the exploratory

nature of the process of developing the MIRS framework, Iemployed opportunistic sampling

at the broadest level.

Purposeful Random Sampling

‘For many audiences, random sampling, even of small samples, will substantially increase

the credibility of the results’ (Patton, 2002, pp. 240-241). In theory, research synthesists

can employ exhaustive searches to locate most of the primary research reported on atopic

and then randomly select afew reports from this pool of reports for in-depth discussion.

However, given the resources required for locating all primary research reports on atopic,

itwould be cost-ineffective to randomly discard studies from further consideration. Hence,

this sampling has little appeal in practice.

Sampling Politically Important Cases

Sampling politically important cases involves ‘selecting (or sometimes avoiding) apolitically

sensitive site or unit of analysis’ (Patton, 2002, p. 241). Like most scholarly endeavours in

education, research syntheses are essentially political. Asynthesist might consciously select

politically important reports so that the synthesis gains the attention of different stakeholders

and the synthesis findings get used. For instance, in asynthesis of key criticisms of educa-

tional research published in the 1990s, Oancea (2005) illustrated her key observations

through adetailed analysis of three politically important documents that were frequently

cited in the newspapers. Syntheses of hot topics, in which several stakeholders are interested,

are also likely to attract appropriate funding and more impact (Elmore, 1991).

Convenience Sampling

Convenience sampling is‘probably the most common sampling strategy–and the least desir-

able’. It would involve selecting reports that are ‘easy to access and inexpensive to study’.

This form of sampling is‘neither purposeful nor strategic’ (Patton, 2002, pp. 241, emphasis

in original). As in many primary research studies, convenience sampling isalso employed

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 71 in many research reviews. Both primary researchers and research synthesists must resort to

convenience sampling as the last option. First, synthesists must reflect on various other

purposeful sampling strategies to identify astrategy that ismost suitable for their purpose

and is also viable within the pragmatic constraints associated with the synthesis. When

convenience sampling has been employed in aresearch synthesis, the nature of its use and

associated caveats must be clearly described.

Combination or Mixed Purposeful Sampling

Synthesists often employ acombination of two or more sampling strategies to select evidence

that adequately addresses their purpose. Mixed purposeful sampling can facilitate triangulation

and flexibility in meeting the needs of multiple stakeholders (Patton, 2002). For example,

synthesists may strategically utilise extensive sampling to draw generalisations at ahigher

level of abstraction. Then, they may employ typical case sampling to provide readers with

an immediacy of typical studies that contributed towards informing the more abstract gen-

eralisations. When selecting acombination of sampling strategies, synthesists must reflect

on how those strategies complement each other.

SAMPLE SIZE AND ENACTING CLOSURE TO FURTHER SEARCHES

Decisions associated with enacting closure to further searches for evidence must be guided

by the purpose of the synthesis, the overarching logic of sampling, and pragmatic constraints.

There are two main logics associated with these decisions in primary research as well as re-

search synthesis: data saturation and data sufficiency.

Data Saturation

Data saturation may be associated with the stage when further collection of evidence provides

little in terms of further themes, insights, perspectives or information in aqualitative research

synthesis. The concept of data saturation isdependent on the nature of the data source as

well as the synthesis question. There isahigher likelihood of reaching data saturation ifthe

data collection ispurposeful. The more precise aquestion, the quicker ittends to reach data

saturation. Progressive refinement of asynthesis question islikely to bring an earlier stage

of data saturation. With open ended questions, every new report islikely to offer additional

information. Abroad question, like ‘what does research tell us about virtual classrooms?’, is

not likely to bring about asense of closure or data saturation. On the other hand, the syn-

thesist islikely to reach the data saturation stage earlier with afocused question like ‘what

are the key methodologies being employed to examine gender differences in math achievement

on standardised tests among middle school students?’

Data Sufficiency

Most research synthesists refrain from rigidly prescribing aminimum or maximum number

of primary research studies to be included in asynthesis. Some methodologists recommend

their methods are suitable for synthesizing even asmall number of qualitative research

studies: for example, meta-ethnography for three studies (Noblit & Hare, 1988), aggregated

analysis for four studies (Eastabrooks etal., 1994) and meta-study for twelve studies (Paterson

et al., 2001, p. 38). Many qualitative research synthesists who synthesize methodologically

diverse research tend to conduct extensive searches and include alarge number of studies.

For example, Wideen and colleagues (1998) included 93 studies and Kasworm (1990) in-

72 | Qualitative Research Journal , vol. 11, no. 2, 2011 cluded 96 documents. Paterson and her colleagues identify the following principles that

ought to guide the sample size of the synthesis: ‘the data should be sufficient to permit

comparisons among selected dimensions and constructs’; ‘the reports should reflect the work

of several distinct and independent investigators’; and ‘the data should be sufficient to answer

the research question’ (Paterson etal., 2001, p. 37). These principles can be usefully applied

to most qualitative research syntheses. Aresearch synthesist, like aprimary researcher, isoften

confronted with various pragmatic constraints of time and resources as well as access to ex-

pertise and information. The stage of data saturation is not frequently reached in either

primary research or research synthesis projects. The logic of data sufficiency isguided by

the synthesist’s perception of what constitutes sufficient evidence for achieving the synthesis

purpose. The synthesist must repeatedly ensure that the claims made in the synthesis are

sufficiently grounded in the evidence collected for the synthesis.

USING THIS DISCUSSION AS A DEPARTURE POINT

In general, synthesists must leave an ‘interpretive trail’ of the different ways in which studies

have been used or omitted (Pawson et al., 2005, p. 31). In leaving an interpretive trail of

their searches, research synthesists must critically reflect and report on the following questions:

• What sampling logic isamenable to meet the synthesis purpose, within the available re-

sources and pragmatic constraints, efficiently and sufficiently?

• What logic will guide the decision to cease searching for further evidence?

• What are the justifications for these decisions?

• What are the caveats associated with these decisions?

In this article, Ihave illustrated how different, purposeful sampling strategies may be

suited for research syntheses conducted for diverse purposes. By drawing on arange of hy-

pothetical examples, Ihave illuminated how various purposeful sampling strategies might

be particularly suitable for syntheses geared to facilitate understanding, participation,

emancipation and deconstruction. Ihave discussed how research synthesists can draw upon

the concepts of data saturation and data sufficiency to inform their decisions to cease searches

for further studies. In my discussion of strategic sampling in research syntheses, Iurge syn-

thesists to carefully identify sampling strategies which address the synthesis purpose efficiently,

credibly, sufficiently and ethically .Imake no attempt to prescribe certain sampling strategies

for research synthesis. Rather, the intention here isto expand methodological possibilities

within research syntheses by proposing new ways of thinking about the methodology of

synthesis. Ihope the users and producers of research synthesis will use this article as adepar-

ture point to think creatively and critically about purposes and amenable sampling strategies

for aresearch synthesis.

REFERENCES

Bair, C. R. (1999, Nov 18). Meta-synthesis .Paper presented atthe 24th annual meeting ofthe Asso-

ciation for the Study of Higher Education: San Antonio (ED437866).

Blaxter, L., Hughes, C. &Tight, M. (2001). How toresearch (2nd ed.). Buckingham: Open University

Press.

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 73 Bondas, T. & Hall, E. O. C. (2007). Adecade of metasynthesis research in health sciences: Ameta-

method study. International Journal ofQualitative Studies on Health and Well-Being ,2(2), 101-

113. [CrossRef]

Booth, A. (2001, May 14). Cochrane or cock-eyed? How should we conduct systematic reviews of

qualitative research? Paper presented at the Qualitative Evidence-based Practice Conference,

Taking aCritical Stance .Coventry University.

Cooper, H. M. (1998). Synthesizing research: Aguide for literature reviews. (3rd ed.). Thousand

Oaks, CA: Sage.

Dixon-Woods, M., Agarwal, S., Jones, D., Young, B. & Sutton, A. J.(2005). Synthesising qualitative

and quantitative evidence: Areview ofpossible methods. Journal ofHealth Services Research and

Policy ,10(1), 45-53. [CrossRef]

Eastabrooks, C. A., Field, P.A. & Morse, J.M. (1994). Aggregating qualitative findings: An approach

to theory development. Qualitative Health Research ,4(4), 503-511. [CrossRef]

Eisenhart, M. (1998). On the subject ofinterpretive reviews. Review ofEducational Research ,68(4),

391-399.

Elmore, R. F.(1991). Comment on ‘Towards rigor inreviews ofmultivocal literatures: Applying the

exploratory case study method’. Review of Educational Research ,61(3), 293-297.

Jensen, L.A. & Allen, M. N. (1994). Asynthesis ofqualitative research on wellness-illness. Qualitative

Health Research ,4(4), 349-369. [CrossRef]

Kasworm, C. E. (1990). Adult undergraduates inhigher education: Areview ofpast research perspect-

ives. Review of Educational Research ,60(3), 345-372.

Kemper, E. A., Stringfield, S. & Teddlie, C. (2003). Mixed methods sampling strategies in social

science research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social

& behavioral research ,Thousand Oaks, CA: Sage, 273-296.

Lather, P. (1993). Fertile obsession: Validity after poststructuralism. The Sociological Quarterly ,

34(4), 673-693. [CrossRef]

Lather, P. (1999). To be of use: The work of reviewing. Review of Educational Research ,69(1), 2-

Light, R. J.& Pillemer, D. B. (1984). Summing up: The science of reviewing research. Cambridge:

Harvard University Press.

Lloyd Jones, M. (2004). Application of systematic review methods to qualitative research: Practical

issues. Journal of Advanced Nursing ,48(3), 271-278. [CrossRef]

Major, C. H. & Savin-Baden, M. (2010). An introduction toqualitative research synthesis: Managing

the information explosion in social science research. London: Routledge.

Mays, N., Pope, C. & Popay, J.(2005). Systematically reviewing qualitative and quantitative evidence

toinform management and policy making inthe health field. Journal ofHealth Services Research

and Policy ,10(1), 6-20. [CrossRef]

Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd

ed.). Thousand Oaks, CA: Sage.

Noblit, G. W. & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury

Park: Sage.

Oakley, A. (2003). Research evidence, knowledge management and educational practice: Early lessons

from asystematic approach. London Review of Education ,1(1), 21-33. [CrossRef]

Oancea, A. (2005). Criticisms of educational research: Key topics and levels of analysis. British

Educational Research Journal ,31(2), 57-183. [CrossRef]

Ogawa, R. T. & Malen, B. (1991). Towards rigor in reviews of multivocal literatures: Applying the

exploratory case study method. Review of Educational Research ,61(3), 265-286.

Paterson, B. L., Thorne, S. E., Canam, C. & Jillings, C. (2001). Meta-study of qualitative health

research: Apractical guide to meta-analysis and meta-synthesis. Thousand Oaks, CA: Sage.

Patton, M. Q. (1978). Utilization-focused evaluation .Thousand Oaks, CA: Sage.

Patton, M. Q. (1990). Qualitative evaluation and research methods .Thousasnd Oaks, CA: Sage.

74 | Qualitative Research Journal , vol. 11, no. 2, 2011 Patton, M. Q. (1991). Towards utility in reviews of multivocal literatures. Review of Educational

Research ,61(3), 287-292.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA:

Sage.

Pawson, R. (2006). Evidence-based policy: Arealist perspective. London: Sage.

Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. (2005). Realist review: Anew method of

systematic review designed for complex policy interventions. Journal ofHealth Services Research

and Policy ,10(1), 21-34. [CrossRef]

Petticrew, M. & Roberts, H. (2006). Systematic reviews in the social sciences: Apractical guide.

Malden, MA: Blackwell. [CrossRef]

Rosenthal, M. C (1994). The fugitive literature. In H. M. Cooper & L. V. Hedges (Eds.), The

handbook of research synthesis ,New York: Sage, 85-94.

Sandelowski, M., Docherty, S.& Emden, C. (1997). Qualitative metasynthesis: Issues and techniques.

Research in Nursing and Health ,20(4), 365-371. [CrossRef]

Schwandt, T. A. (1998). The interpretive review of educational matters: Isthere any other kind?.

Review of Educational Research ,68 (4), 409-412.

Stock, W. A., Benito, J.G. & Lasa, N. B. (1996). Research synthesis: Coding and conjectures.

Evaluation and the Health Professions ,19(1), 104-117. [CrossRef]

Strauss, A. & Corbin, J.(1998). Basics of qualitative research: Techniques and procedures for devel-

oping grounded theory. (2nd ed.). Thousand Oaks, CA: Sage.

Suri, H. (2007). Expanding possibilities within research syntheses: Amethodologically inclusive re-

search synthesis framework. Unpublished PhD thesis. Melbourne: The University ofMelbourne.

Suri, H. (2008). Ethical considerations insynthesising research: Whose representations?. Qualitative

Research Journal ,8(1), 62-73. [CrossRef]

Suri, H. & Clarke, D. J.(2009). Advancements in research synthesis methods: From amethodolo-

gically inclusive perspective. Review of Educational Research ,79(1), 395-430. [CrossRef]

Wideen, M. F., Mayer-Smith, J.A. & Moon, B. J.(1998). Acritical analysis of the research on

learning toteach: Making the case for an ecological perspective on inquiry. Review ofEducational

Research ,68(2), 130-178.

Zimmer, L. (2006). Qualitative meta-synthesis: Aquestion of dialoguing with texts. Journal of Ad-

vanced Nursing ,53(3), 311-318. [CrossRef]

ABOUT THE AUTHOR

Harsh Suri (PhD) is a Lecturer at the Centre for the Study of Higher Education, The University of

Melbourne, Australia. She has developed a methodologically inclusive research synthesis (MIRS)

framework for designing and evaluating research syntheses from distinct methodological orienta-

tions. She has written several papers on contemporary methods of research synthesis. Two of her

early papers were recognised as outstanding presentations: Early Career Researcher Award at the

Mathematics Education Research Group of Australasia (MERGA) conference in 1997 and Best

Graduate Presentation at the AQR conference in 1999. She is cited in most current publications on

research synthesis methods which are inclusive of qualitative research.

Email: [email protected]

Harsh Suri,'Purposeful Sampling in Qualitative Research Synthesis' | 75