U7A1-60 - Scientific Knowledge, Contributions and Methodology - See Details

CHAPTER 9 - Qualitative Research Methods

To answer some research questions, we cannot skim across the surface. We must dig deep to get a complete understanding of the phenomenon we are studying. In qualitative research, we indeed dig deep: We collect various forms of data and examine them from various angles to construct a rich and meaningful picture of a complex, multifaceted situation.

9.1 Identify several situations in which a qualitative methodology might be especially useful.

9.2 Describe general characteristics and purposes of (a) case studies, (b) ethnographies, (c) phenomenological studies, (d) grounded theory studies, and (e) content analyses. Also, describe effective strategies you might use in each of these five research methodologies.

9.3 Identify effective strategies for collecting data in a qualitative study. As you do so, explain how you can address issues related to (a) validity and reliability, (b) sampling, (c) making observations, and (d) conducting interviews.

9.4 Describe several general criteria that are often used in evaluating qualitative studies.

The term qualitative research encompasses a number of methodologies that are in some respects quite different from one another. Yet all qualitative approaches have two things in common. First, they typically focus on phenomena that are occurring or have previously occurred in natural settings—that is, in the “real world.” And second, they involve capturing and studying the complexity of those phenomena. Qualitative researchers rarely try to simplify what they observe. Instead, they recognize that the issue they are studying has many dimensions and layers, and they try to portray it in its multifaceted form.

Qualitative research can be found in many academic disciplines, including anthropology, sociology, psychology, biology, history, political science, education, and medicine. In fact, it could be argued that inquiry in any discipline begins in a qualitative form (e.g., Lauer & Asher, 1988). When little information exists on a topic, when variables are unknown, when a relevant theory base is inadequate or missing, a qualitative study can help define what is important—that is, what needs to be studied. For example, the field of medicine makes extensive use of qualitative methods when unique or puzzling cases are first observed. Biologists’ efforts to classify newly observed species, create taxonomies, and describe the social behaviors of primates and certain other animal species are largely qualitative efforts. Many analyses of historical data are almost entirely qualitative. And social scientists often look subjectively for patterns in the complex phenomena they observe, sometimes using qualitative methods exclusively and sometimes combining qualitative and quantitative methods into a mixed-methods design (details to follow in Chapter 12).

In this chapter we give you a general idea of what qualitative research is and what it strives to accomplish, with a particular focus on studies of human beings and their creations. Included in the chapter are descriptions of five kinds of qualitative studies: case studies, ethnographies, phenomenological studies, grounded theory studies, and content analyses. We describe a sixth kind, historical research, in Chapter 10.

As you proceed through the chapter, you will find several strategies—sampling, making observations, interviewing—that you previously encountered in the discussion of descriptive quantitative studies in Chapter 6. These are old news, you might think. On the contrary, such strategies can take on very different forms when we want them to yield qualitative data.

Qualitative research can be quite different from quantitative research in another important way as well. In discussions of quantitative designs and strategies in the preceding three chapters, we imply—intentionally—that data collection comes first, with data analysis to follow in a separate step. In qualitative research, however, the methodology often involves an iterative process in which the researcher moves back and forth between data collection and data analysis in what is sometimes called the constant comparative method. For example, the researcher might (a) collect some preliminary data in a natural setting; (b) inspect the data for possible patterns; (c) return to the setting to collect additional data that might substantiate, clarify, or contradict those patterns; and (d) conduct a more thorough, detailed analysis of the data—possibly repeating Steps c and d through additional iterations. Accordingly, if you are planning a qualitative study you should read both this chapter and the discussion of qualitative data analysis in Chapter 11 before beginning data collection.

In Chapter 2 we emphasized the importance of pinning down the research problem with utmost precision. We sometimes find an exception in qualitative research. Some qualitative researchers often formulate only general research problems and ask only general questions about the phenomenon they are studying. For example, they might ask, “What is the nature of the culture of people living in Samoa?” or “What is it like to live with someone who has Alzheimer’s disease?” Such research problems and questions don’t remain so loosely defined, however. As a study proceeds, the qualitative researcher gains increasing understanding of the phenomenon under investigation and thus becomes increasingly able to ask more specific questions—and occasionally can begin to formulate and test specific hypotheses as well.

When qualitative researchers ask only open-ended research questions at the beginning of an investigation, they may have trouble identifying—at the outset—the exact methods they will use. Initially, they may select only a general approach suitable for their purpose, perhaps choosing a case study, ethnography, or content analysis. As they learn more about what they are studying and can therefore ask more specific questions, so, too, can they better specify what strategies they will use to answer those questions.

In some instances, then, the methodology of a qualitative study may continue to evolve over the course of the investigation. Despite this fact, we must emphasize that qualitative research requires considerable preparation and planning. Qualitative researchers must be well trained in observation techniques, interview strategies, and whatever other data collection methods are likely to be necessary to address their research problem. They must have a firm grasp of previous research related to the problem so that they know what to look for and can separate important information from unimportant details in what they observe (some grounded theory studies are exceptions, for reasons you will discover shortly). And they must be adept at wading through huge amounts of data and finding a meaningful order in what, to someone else, might appear to be chaos. For these reasons, a qualitative study can be a challenging task indeed. It is definitely not the approach to take if you’re looking for quick results and easy answers.

Qualitative research studies typically serve one or more of the following purposes:

• ▪ Exploration. They can help you gain initial insights into what has previously been a little-studied topic or phenomenon.

• ▪ Multifaceted description. They can reveal the complex, possibly multilayered nature of certain situations, settings, processes, relationships, systems, or people.

• ▪ Verification. They allow you to test the validity of certain assumptions, claims, theories, or generalizations within real-world contexts.

• ▪ Theory development. They can enable you to develop new concepts or theoretical perspectives related to a phenomenon.

• ▪ Problem identification. They can help you uncover key problems, obstacles, or enigmas that exist within the phenomenon.

• ▪ Evaluation. They provide a means through which you can judge the effectiveness of particular policies, practices, or innovations.

As a general rule, however, qualitative studies do not allow you to identify cause-and-effect relationships—to answer questions such as What caused what? or Why did such-and-such happen? You will need quantitative research, especially experimental studies, to answer questions of this kind.

Qualitative Research Designs

In this section, we describe five commonly used qualitative research designs. We give you enough information to help you determine whether one of these approaches might be suitable for your research question, and we briefly describe the specific nature of each methodology. Later in the chapter, we discuss data collection strategies that are more broadly applicable to qualitative research. But our space is limited here. Should you choose to conduct a qualitative study, we urge you to take advantage of the resources listed in the “For Further Reading” section at the end of the chapter.

Remember, too, that of all the designs we describe in this book, qualitative research methods are the least prescriptive. There are no magic formulas, no cookbook recipes for conducting a qualitative study. This book, as well as any others you may read, can give you only general guidelines based on the experiences of those qualitative researchers who have gone before you. In a qualitative study, the specific methods you use will ultimately be constrained only by the limits of your imagination.

Case Study

In a case study—sometimes called idiographic research—a particular individual, program, or event is studied in depth for a defined period of time. For example, a medical researcher might study the nature, course, and treatment of a rare illness for a particular patient. An educator might study and analyze the instructional strategies that a master teacher uses to teach high school history. A political scientist might study the origins and development of a politician’s campaign as he or she runs for public office. Case studies are common not only in medicine, education, and political science, but also in law, psychology, sociology, and anthropology.

Sometimes researchers focus on a single case, perhaps because its unique or exceptional qualities can promote understanding or inform practice for similar situations. At other times researchers study two or more cases—often cases that are either similar or different in certain key ways—to make comparisons, build theory, or propose generalizations; such an approach is called a multiple or collective case study.

In a typical case study, a researcher collects extensive data on the individual(s), program(s), or event(s) on which the investigation is focused. These data often include observations, interviews, documents (e.g., newspaper articles), past records (e.g., previous test scores), and audiovisual materials (e.g., photographs, videotapes, audiotapes). In many case studies, the researcher spends an extended period of time on site and regularly interacts with the person or people being studied. The researcher also records details about the context surrounding the case or cases of focus, including information about the physical environment and any historical, economic, and social factors that have bearing on the situation. By portraying such contexts, the researcher helps others who later read the research report to draw conclusions about the extent to which the study’s findings might be generalizable to other situations.

A case study may be especially suitable for learning more about a little known or poorly understood situation. It can also be appropriate for investigating how an individual or program changes over time, perhaps as the result of certain conditions or interventions. In either circumstance, it tends to be most useful for generating or providing preliminary support for one or more hypotheses regarding the phenomenon being investigated. Its major limitation is that, especially when only a single case is involved, we cannot be sure that the findings are generalizable to other situations.

Ethnography

In a case study, a researcher looks in considerable depth at a particular person, program, or event. In contrast, in an ethnography, a researcher looks in depth at an entire group—more specifically, a group that shares a common culture. (The word ethnography comes from ethnos, Greek for “a nation or other close-knit group of people,” and graph, “something written or recorded.”) The ethnographic researcher studies a group in its natural setting for a lengthy time period, often several months or several years. The focus of investigation is on the everyday behaviors of the people in the group (e.g., interactions, language, rituals), with an intent to identify cultural norms, beliefs, social structures, and other patterns. Ideally, the ethnographic researcher identifies not only explicit cultural patterns—those readily acknowledged by group members or easily observable in objects or behaviors—but also implicit patterns—those beliefs and assumptions that have such a below-the-surface, taken-for-granted quality that even group members aren’t always consciously aware of them.

Ethnographies were first used in cultural anthropology, but they are now seen in sociology, psychology, education, and marketing research as well. The conception of the type of “culture” that can be studied has also changed over time: Whereas ethnographies once focused on long-standing cultural groups (e.g., people living on the island of Samoa), more recently they have been used to study such “cultures” as those of adult work environments, elementary school classrooms, exclusive social cliques in adolescence, violence-prone adolescent groups, and Internet-based communities1 (e.g., Bender, 2001; Kozinets, 2010; McGibbon, Peter, & Gallop, 2010; Mehan, 1979; Merten, 2011).

1 See Kraut and colleagues (2004) for a good discussion of the research possibilities, potential pitfalls, and ethical issues related to studying people’s postings on the Internet.

The group chosen for in-depth study should, of course, be appropriate for answering a researcher’s general research problem or question. Ideally, it should also be one in which the researcher is a “stranger” and has no vested interest in the study’s outcome. A group that the researcher knows well (perhaps one that involves close acquaintances) might be more accessible and convenient, but by being so close to the situation, the researcher may have trouble looking at it with sufficient detachment to gain a balanced perspective and portray an accurate picture of the processes observed (Creswell, 2013).

Site-based fieldwork is the sine qua non—the essence—of any ethnography. Prolonged engagement in a group’s natural setting gives ethnographic researchers time to observe and record processes that would be almost impossible to learn about by using any other approach. Thus, an essential first step in an ethnographic study is to gain legitimate access to the site. Often researchers must go through a gatekeeper, a person who can smooth the way for their entrance into the situation. This individual might be a tribal chief in a community in a developing country, a principal or teacher in a school or classroom, or a program director at a homeless shelter. Then, after gaining entry into the site, researchers must establish rapport with and gain the trust of the people being studied. At the same time, they must be open about why they are there. The principle of informed consent described in Chapter 4 is just as essential in an ethnography as it is in any other type of research.

Initially, researchers cast a broad net, intermingling with everyone and getting an overall sense of the social and cultural context. Gradually, they identify key informants who can provide information and insights relevant to their research question and can facilitate contacts with other helpful individuals.

In some ethnographic studies, researchers engage in participant observation, becoming immersed in the daily life of the people. In fact, over the course of the study, their role may gradually change from “outsider” to “insider.” The advantage here is that they might gain insights about the group and its behaviors that could not be obtained in any other way. The disadvantage is that they may become so emotionally involved as to lose the ability to assess the situation accurately. In some situations, they may even “go native,” joining the group and therefore becoming unable to complete the study (Creswell, 2013).

Throughout their fieldwork, ethnographic researchers are careful observers, interviewers, and listeners. Furthermore, they take extensive field notes (written either on site at the time or in private later in the day) in the forms of dialogues, diagrams, maps, and other written materials. Lengthy conversations and significant events can be recorded using audiotapes and videotapes. Researchers may also collect artifacts (e.g., tools, ritualistic implements, artistic creations) and records (e.g., accounting ledgers, personal journals, lesson plans) from the group. In order to test hypotheses about a group’s unconsciously shared beliefs or assumptions, some ethnographic researchers occasionally conduct breaching experiments—that is, they intentionally behave in ways they suspect might violate an unspoken social rule—and observe people’s reactions (Mehan & Wood, 1975).

We must caution you that conducting a good ethnography requires both considerable patience and considerable tolerance. One experienced ethnographer has described the process this way:

It requires a great patience under any circumstances for me to “sit and visit.” A rather inevitable consequence of being inquisitive without being a talker is that my conversational queries usually prompt others to do the talking. During fieldwork, I make a conscious effort to be sociable, thus providing opportunities for people to talk to me. . . . I never confront informants with contradictions, blatant disbelief, or shock, but I do not mind presenting myself as a bit dense, someone who does not catch on too quickly and has to have things explained. . . . (Wolcott, 1994, p. 348)

An ethnography is especially useful for gaining an understanding of the complexities of a particular sociocultural group. It allows considerable flexibility in the methods used to obtain information, which can be either an advantage (to an experienced researcher who knows what to look for) or a disadvantage (to a novice who may be overwhelmed and distracted by unimportant details). Hence, if you decide that an ethnography is the approach most suitable for your research problem, we urge you to get a solid grounding in cultural anthropology before you venture into the field (Creswell, 2013).

Phenomenological Study

In its broadest sense, the term phenomenology refers to a person’s perception of the meaning of an event, as opposed to the event as it exists external to the person. A phenomenological study is a study that attempts to understand people’s perceptions and perspectives relative to a particular situation. In other words, a phenomenological study tries to answer the question What is it like to experience such-and-such? For instance, a researcher might study the experiences of people caring for a chronically or terminally ill relative, living in an abusive relationship, or home-schooling a child.

In some cases, the researcher has had personal experience related to the phenomenon in question and wants to gain a better understanding of the experiences of others. By looking at multiple perspectives on the same situation, the researcher can then make some generalizations of what something is like from an insider’s perspective.

Phenomenological researchers depend almost exclusively on lengthy interviews (perhaps 1 to 2 hours in length) with a small, carefully selected sample of participants. A typical sample size is from 5 to 25 individuals, all of whom have had direct experience with the phenomenon being studied.

The actual implementation of a phenomenological study is as much in the hands of the participants as in the hands of the researcher. The phenomenological interview is often a relatively unstructured one in which the researcher and participants work together to “arrive at the heart of the matter” (Tesch, 1994, p. 147). The researcher listens closely as participants describe their everyday experiences related to the phenomenon; the researcher must also be alert for subtle yet meaningful cues in participants’ expressions, pauses, questions, and occasional sidetracks. A typical interview looks more like an informal conversation, with the participant doing most of the talking and the researcher doing most of the listening.

Throughout the data collection process, phenomenological researchers try to suspend any preconceived notions or personal experiences that may unduly influence what they “hear” participants saying. Such suspension—sometimes called bracketing or epoché—can be extremely difficult for researchers who have personally experienced the phenomenon under investigation. Yet it is essential if they are to gain an understanding of the typical experiences that people have had. The ultimate goal of a phenomenological study should be—not only for the researcher but also for readers of the final research report—to provide a sense that “I understand better what it is like for someone to experience that” (Polkinghorne, 1989, p. 46).

Of all the research designs described in this book, a grounded theory study is the one least likely to begin from a particular theoretical framework. On the contrary, the major purpose of a grounded theory approach is to begin with the data and use them to develop a theory. The term grounded refers to the idea that the theory that emerges from the study is derived from and rooted in data that have been collected in the field rather than taken from the research literature. Grounded theory studies are especially helpful when current theories about a phenomenon are either inadequate or nonexistent.2

2 Some researchers associate the term grounded theory with a particular method of data analysis—in particular, that of Corbin and Strauss (2008; Strauss & Corbin, 1990)—and suggest the term emergent theory as a broader, less prescriptive label for this approach (e.g., Jaccard & Jacoby, 2010).

Typically, a grounded theory study focuses on a process related to a particular topic—including people’s actions and interactions—with the ultimate goal of developing a theory about the process. The approach has its roots in sociology (Glaser & Strauss, 1967) but is now also used in such fields as anthropology, geography, education, nursing, psychology, and social work. It has been used effectively for a wide range of topics—for instance, to study children’s eating habits, college students’ thoughts and feelings during classroom discussions, and workers’ stress levels in public service agencies (Do & Schallert, 2004; Kime, 2008; Skagert, Dellve, Eklöf, Pousette, & Ahlborg, 2008).

As is true for the qualitative designs previously described, data collection in a grounded theory study is field-based, flexible, and likely to change over the course of the investigation. Interviews typically play a major role in data collection, but observations, documents, historical records, videotapes, and anything else of potential relevance to the research question might also be used. The only restriction is that the data collected must include the perspectives and voices of the people being studied (Charmaz, 2002, 2014; Corbin & Strauss, 2008).

More so than in any other qualitative methodology, data analysis in a grounded theory study begins almost immediately, at which point the researcher develops categories to classify the data. Subsequent data collection is aimed at saturating the categories—in essence, learning as much about them as possible—and at finding any disconfirming evidence that point to possible revisions in the categories identified or in interrelationships among them. The theory that ultimately evolves is one that includes numerous concepts and interrelationships among those concepts; in other words, it has conceptual density (Schram, 2006).

Virtually all experts agree that grounded theory researchers should have a firm grasp of general concepts and theoretical orientations in their discipline as a whole; hence, an in-depth literature review early in the process is essential. However, experts disagree about whether researchers should look closely at previous findings directly related to the present research problem before collecting and analyzing data. For example, Glaser (1978) has argued that too much advance knowledge of earlier research regarding a topic may limit a researcher’s ability to be open-minded about how to analyze and interpret the data collected. In contrast, many others suggest that the advantages of conducting a relatively thorough literature review outweigh the disadvantages; in particular, previous works and writings about a topic can often help a researcher think more clearly and insightfully about the collected data (e.g., Hesse-Biber, 2010; Jaccard & Jacoby, 2010). Our own advice is to learn as much as you can about your research topic through a thorough review of the related literature but to refrain from forming specific hypotheses about what you yourself might find.

A content analysis is a detailed and systematic examination of the contents of a particular body of material for the purpose of identifying patterns, themes, or biases. Content analyses are typically performed on forms of human communication, including books, newspapers, personal journals, legal documents, films, television, art, music, videotapes of human interactions, transcripts of conversations, and Internet blog and bulletin board entries.3 For example, a researcher might use a content analysis to determine what religious symbols appear in works of art, how middle school science texts portray the nature of science, or what attitudes are reflected in the speeches or newspaper articles of a particular era in history. As you might infer from these examples, content analyses are found in a wide variety of disciplines, including the fine arts, education, history, psychology, journalism, and political science.

3 Again, we refer you to Kraut and colleagues (2004) regarding ethical issues related to studying people’s postings on the Internet.

Of the five designs described in this chapter, a content analysis is apt to involve the greatest amount of planning at the front end of the project. The researcher typically defines a specific research problem or question at the very beginning (e.g., “Do contemporary children’s books reflect traditional gender stereotypes?”, “What religious symbols appeared in early Byzantine architecture, and with what frequency, during the years 527–867?”). Furthermore, the researcher takes measures to make the process as objective as possible. The following steps are typical:

1. The researcher identifies the specific body of material to be studied. If this body is relatively small, it is studied in its entirety. If it is quite large (e.g., if it consists of all newspaper articles written during a particular time period), a sample (perhaps a random sample) is selected.

2. The researcher defines the characteristics or qualities to be examined in precise, concrete terms. The researcher may identify specific examples of each characteristic as a way of defining it more clearly.

3. If the material to be analyzed involves complex or lengthy items (e.g., works of literature, transcriptions of conversations), the researcher breaks down each item into small, manageable segments that are analyzed separately.

4. The researcher scrutinizes the material for instances of each characteristic or quality defined in Step 2. When judgments are objective—for instance, when the study involves looking for the appearance of certain words in a text—only one judge, or rater, is necessary. When judgments are more subjective—for instance, when the study involves categorizing discrete sections of textbooks as conveying various messages about the nature of science—two or three raters are typically involved, and a composite of their judgments is used.

Content analyses are not necessarily stand-alone designs. For example, a systematic content analysis might be an integral part of the data analysis in a phenomenological study (e.g., see Wennick, Lundqvist, & Hallström, 2009). A content analysis might also be used to flesh out the complex, multidimensional aspects of a descriptive or experimental study, resulting in a mixed-methods design with both qualitative and quantitative elements.

Even when a content analysis is the sole research methodology, it’s apt to have a quantitative component. In many instances, quantification may involve simply counting the frequencies with which various characteristics are observed in the body of data being examined. But alternatively, a researcher might conduct one or more statistical analyses on the numbers obtained—for instance, comparing the numbers obtained from two or more distinct subsets of the materials being analyzed (e.g., see Bergman, 2010).

Table 9.1 summarizes the nature of the five designs described in the preceding sections, including a brief description of general data analysis strategies. Keep in mind, however, that the five designs aren’t necessarily as distinctly different as the table might indicate. Any particular study may include elements of two or more qualitative designs. Remember, much qualitative research is, by its very nature, somewhat flexible and may continue to evolve over the course of a project. To the extent that your research question leads you to believe that two or more designs are equally relevant to your purpose, think creatively about how you might combine them into a single study.

▪ Distinguishing Characteristics of Different Qualitative Designs

Such flexibility should not, however, lead you to believe that you can conduct a qualitative research project in a sloppy, poorly thought-through manner. On the contrary, the flexible nature of a qualitative study makes it just that much more challenging, especially for a novice researcher. For anything you do in a qualitative study, you must have a definite rationale and a distinct purpose, and you must keep your overall goal—to answer your research question—clearly in sight at all times.

Following are brief summaries of five potential research projects. Identify the qualitative methodology that is probably most appropriate for each project. The answers appear after the “For Further Reading” section at the end of the chapter.

1. In an effort to learn the nature and appeal of long-standing men’s social groups, a researcher plans to spend a 9-month period with a local chapter (“lodge”) of the Benevolent and Protective Order of Elks. By observing and interacting with the Elks, he hopes to observe the chapter’s meetings, rituals, and charitable activities and to discover the chapter’s beliefs, values, goals, and interpersonal dynamics.

2. A researcher wants to determine to what degree and in what ways television commercials might portray men and women in traditionally gender-stereotypical ways (e.g., how often men versus women are shown cleaning house, how often men versus women are shown making important business decisions).

3. In order to learn how grassroots political parties emerge and develop over time, a researcher wants to study the origins and evolution of three recently established “Tea Party” groups, one in her own state and two in neighboring states.

4. A researcher is intrigued by Asperger syndrome, a cognitive disability in which people have average or above-average intelligence and language skills but poor social skills and little or no ability to interpret other people’s nonverbal social cues (e.g., body language). The researcher wants to find out what it is like to be an adolescent with this syndrome—how a teenager is apt to feel about having few or no friends, being regularly excluded from classmates’ social activities, and so on.

5. A researcher wants to determine how doctors, nurses, and other hospital staff members coordinate their actions when people with life-threatening traumatic injuries arrive at the emergency room. The researcher can find very little useful research on this topic in professional journals.

As you have seen, qualitative researchers often use multiple forms of data in any single study. They might use observations, interviews, objects, written documents, audiovisual materials, electronic entities (e.g., e-mail messages, Internet websites), and anything else that can help them answer their research question. Potential sources of data are limited only by a researcher’s open-mindedness and creativity. For example, in a school setting, a researcher might consider where various students are seated in the lunch room, what announcements are posted on the walls, or what messages are communicated in graffiti (Eisner, 1998). In an ethnographic study of a cultural group, a researcher might ask one or more participants to keep a daily journal or to discuss the content and meaning of photographs and art objects (Creswell, 2013).

While collecting data, many qualitative researchers also begin jotting notes—sometimes called memos—about their initial interpretations of what they are seeing and hearing. Some of these “notes-to-self” might involve emerging themes in people’s actions and statements. Others might make note of initial hunches and intuitions to pursue through further observations or interview questions. Still others might be preliminary theories about possible underlying dynamics within a social group.

Many qualitative studies are characterized by an emergent design, in which data collected early in the investigation influence the kinds of data the researcher subsequently gathers. The flexibility of qualitative methodologies is an advantage for experienced researchers but often a disadvantage for novices, who may not have sufficient background or training to determine how best to adjust data collection strategies midway through a study. Thus, many experts suggest that a novice researcher set forth a definite, fairly structured plan for data collection—a strategy that can minimize the degree to which the researcher wanders off into intriguing but ultimately unproductive diversions.

A predetermined, well-thought-out plan is also essential when submitting a qualitative research proposal to an internal review board (IRB). Most importantly, data collection methods must be consistent with the ethical principles presented in Chapter 4. The researcher must take precautions not to expose people (or animals) to unnecessary physical or psychological harm—as could happen, say, if the researcher were to inquire about highly personal and emotionally charged topics. The people being studied must know the nature of the study and be willing participants in it (this is informed consent), and any data collected should not be traceable back to particular individuals (thus maintaining participants’ right to privacy). One common way of keeping personal data confidential is to assign various pseudonyms to different participants and to use those pseudonyms both during data collection and in the final research report.

As you should recall, Chapter 4 includes a section called “Validity and Reliability in Measurement.” Qualitative researchers don’t necessarily measure things—at least not in the numerical sense of the word. Nevertheless, they must be concerned about both the validity and the reliability of the data they collect. In particular, the data they collect must be both (a) reasonably accurate with regard to the characteristics and dynamics of the entities or situation being studied (this is validity) and (b) consistent in the patterns and dynamics they reflect (this is reliability).

A particular strength of qualitative methods is that a perceptive researcher might discern underlying patterns and dynamics in social interactions or cultural artifacts that a standardized, quantitative measurement instrument would never illuminate. In a sense, the researcher is an instrument in much the same way that an oscilloscope, questionnaire, or multiple-choice achievement test is an instrument. The potential downside of this instrument—the human mind—is that it can be biased by its preconceived theories and expectations, and such biases can adversely affect the quality of the data obtained.

Qualitative researchers use a variety of strategies to enhance the validity and reliability—and hence the credibility—of the data they collect. Following are five important strategies during the data collection phase of a qualitative study (we identify strategies related to data analysis and interpretation in Chapter 11):

▪ Reflexivity. Good qualitative researchers actively try to identify personal, social, political, or philosophical biases that are likely to affect their ability to collect and interpret data—this self-reflection is known as reflexivity—and take whatever steps they can to reduce such influences.

▪ Triangulation. Many qualitative researchers use a strategy called triangulation: They collect multiple forms of data related to the same research question, with the goal of finding consistencies or inconsistencies among the data. For example, imagine that a researcher wants to study the behaviors of an especially exclusive group of snobbish but so-called “popular” girls at a public high school. This researcher might not only interview both members and nonmembers of the group but also observe the girls in action in various locations in and around school—for instance, observing seating patterns in the cafeteria, group clusters in the hallways and school yard, and verbal interaction patterns during class sessions. The researcher might also scan school records regarding which students are members (and possibly officers or captains) of various extracurricular clubs and sports teams.

▪ Clearly distinguishing between data and memos. Right from the get-go, a qualitative researcher must keep interpretations separate from actual observations. For example, consider the ethnographic researcher who decides to take only handwritten notes in the field, perhaps as a way of blending in better with the social environment than would be possible with, say, a laptop or video camera. This researcher might draw a vertical line down the middle of each page, recording observations, interview responses, and any helpful graphics (e.g., maps, diagrams) in the left column and jotting memos about these things in the right column. Only in this way can the researcher separate fact (what the researcher is actually seeing and hearing) from what could possibly be fiction (what the researcher currently thinks might be going on).

▪ Seeking of exceptions and contradictory evidence. By nature, human beings seem to be predisposed to look for and identify patterns and consistencies in their physical worlds (e.g., see Mandler, 2007; Rakison & Oakes, 2003). Furthermore, once they have zeroed in on their conclusions about these patterns and consistencies, they’re often reluctant to revise their beliefs (recall the discussion of confirmation bias in Figure 1.3 in Chapter 1). A good qualitative researcher actively fights such mental predispositions, in part by continually asking the questions “Might I be wrong?” and “What disconfirming evidence can I find?” and then intentionally seeking out the answers.

▪ Spending considerable time on site. Many qualitative studies require extensive data collection in the field; such is true for virtually any ethnography and for many case studies, phenomenological studies, and grounded theory studies. Just a brief visit to the site under investigation—popping in and popping out, as it were—is unlikely to yield the quantity and quality of data (including potentially contradictory observations) essential for drawing accurate, multifaceted understandings of any complex phenomenon.

In planning for data collection, qualitative researchers must also identify one or more appropriate samples from which to acquire data. Furthermore, they are apt to rely heavily on observations and/or interviews as sources of data. We offer suggestions related to each of these three topics in the three Practical Application sections that follow. Some of our suggestions can, in one way or another, enhance the validity and reliability of the data obtained.

Qualitative researchers might draw their data from a variety of sources—not only from people but perhaps also from objects, text materials, and audiovisual and electronic records. The particular entities they select for analysis comprise their sample.

Only rarely—for instance, when a study involves a content analysis of a small number of items—can qualitative researchers look at everything that has potential relevance to a research problem. More typically, they must be choosy about the data they gather and analyze and, as a result, will get an incomplete picture of the phenomenon in question. One experienced qualitative researcher has described the situation this way:

Whether observing, interviewing, experiencing, or pursuing some combination of strategies, you cannot be everywhere at once or take in every possible viewpoint at the same time. Instead . . . you develop certain perspectives by engaging in some activities or talking to certain people rather than others. . . . You build assertions toward the never-quite-attainable goal of “getting it right,” approximating realities but not establishing absolutes.

Your task, both derived from and constrained by your presence, is thus inherently interpretive and incomplete. The bottom line is that there is no bottom line: It is not necessary (or feasible) to reach some ultimate truth for your study to be credible and useful. (Schram, 2006, p. 134)

How you identify your sample must depend on the research question(s) you want to answer. If you want to draw inferences about an entire population or body of objects, you must choose a sample that can be presumed to represent that population or body. Ideally, this sample is chosen through a completely random selection process or through a process that incorporates appropriate proportions of each subgroup within the overall group of people or objects. For possible ways of choosing such a sample, return to the discussion of probability sampling in Chapter 6. (Remember, truly effective researchers often draw on methodologies from diverse research traditions.)

In other circumstances, however, you might need to be intentionally nonrandom in your selection of data sources. In particular, your sampling would be selective, or purposive: You would choose those individuals or objects that will yield the most information about the topic under investigation. For example, grounded theory researchers tend to engage in theoretical sampling, choosing data sources that are most likely to help them develop a theory of the process in question. Later, they may employ discriminant sampling, returning to particular data sources that can help them substantiate the theory. (As you should recall from Chapter 6, some descriptive quantitative researchers also engage in purposive sampling.)

A novice qualitative researcher might ask How large should my sample be? How much is enough? There are no easy, cut-and-dried answers to these questions, but we offer several suggestions to guide decision making:

▪ Be sure that the sample includes not only seemingly “typical” but also seemingly “nontypical” examples.

▪ When a power hierarchy exists—as it does in the workplace and in many clubs and communities—sample participants from various levels in the hierarchy. For example, in the workplace, you might interview both bosses and employees; in a club or community, you might interview not only highly active, influential members but also less involved individuals (e.g., see Becker, 1970).

▪ Actively look for cases that can potentially discredit emerging hypotheses and theories.

▪ If appropriate for your research problem, sample from diverse contexts or situations.

Ideally, the sample should provide information not only about how things are on average but also about how much variability exists in the phenomenon under investigation.

In some instances, a research problem is best addressed by sampling from a large geographical area, perhaps one that includes diverse cultural groups. For example, in a dissertation project involving the experiences of White women who were raising biological children of mixed or other races,4 doctoral student Jennifer Chandler (2014) wanted to interview mothers from diverse locations across the United States—locations that would differ in demographic makeup and possibly also in attitudes regarding multiracial families. To obtain such a sample, she created an “Invitation to Participate” letter that described the purpose of her study, the characteristics of desired participants, and the general nature of the interviews she would conduct. Many individuals across the country helped her distribute the invitation, including (a) personal friends and colleagues; (b) people she met at several professional conferences; (c) officers in parent-teacher organizations in numerous public school districts (e.g., Los Angeles, Houston, Denver, New York); and (d) people who had contributed to Internet blogs about topics related to interracial parenting. The resulting sample included 30 mothers from towns and cities in more than a dozen states across the country. It was certainly not a random sample, but it helped Chandler capture the diversity in experiences that mothers living in various geographical and cultural settings were likely to have had.

4 More precisely, the sample included mothers who (a) identified themselves as being non-Hispanic White women and (b) identified their children as being of mixed or other races. Chandler’s capitalization of “White” when referring to a racial group is consistent with APA style (2010).

In the observation studies described in Chapter 6, observations typically have a limited, prespecified focus, and procedures are set in place in advance for quantifying the observations in some way, perhaps with a rating scale. In contrast, observations in a qualitative study are intentionally unstructured and free-flowing: The researcher shifts focus from one thing to another as new and potentially significant objects and events present themselves. The primary advantage of conducting observations in this manner is flexibility: The researcher can take advantage of unforeseen data sources as they surface. Observations are often recorded in great detail, perhaps with field notes or videotapes that capture the wide variety of ways in which people or other animal species act and interact. From these data, the researcher can construct a complex yet integrated picture of how certain humans or nonhumans spend their time.

Such an approach has its drawbacks, of course. A researcher (especially a novice researcher) won’t always know what things are most important to look for, especially at the beginning, and so may waste considerable time observing and recording trivialities while overlooking entities that are more central to the research question. A second disadvantage is that by his or her very presence, the researcher may influence what people say and do or may change how significant events unfold (recall the discussion of reactivity in Chapter 4).

Recording events can be problematic as well. Written notes are often insufficient to capture the richness of what one is observing. Yet audiotapes and videotapes aren’t always completely dependable either. Background noises may make tape-recorded conversations only partially audible. A video camera can capture only the events happening in a small, focused area. And the very presence of tape recorders and video cameras may make some participants uncomfortable.

If you decide to conduct observations as part of a qualitative study, we offer these recommendations:

1. Before you begin your study, experiment with various data recording strategies (field notes, audiotapes, videotapes), identify the particular methods that work best for you, and practice using them in diverse contexts.

2. When you first enter a research site, have someone introduce you to the people you hope to observe. This is the time to briefly describe your study and get potential participants’ informed consent.

3. As you observe, remain relatively quiet and inconspicuous, yet be friendly to anyone who approaches you. You certainly don’t want to discourage people from developing relationships with you and—perhaps later—taking you into their confidence.

Also remember a strategy alluded to earlier: Clearly distinguish between your actual observations (data) and your interpretations (memos). This strategy is important for two reasons. First, you need to be as objective as you can in the records you keep of what might otherwise be only subjective impressions. And second, your interpretations of what you have seen and heard may very well change over the course of the study.

Interviews can often yield a rich body of qualitative information. A researcher might ask questions related to any of the following (Silverman, 1993):

1. Facts (e.g., biographical information)

2. People’s beliefs and perspectives about the facts

3. Feelings

4. Motives

5. Present and past behaviors

6. Standards for behavior (i.e., what people think should be done in certain situations)

7. Conscious reasons for actions or feelings (e.g., why people think that engaging in a particular behavior is desirable or undesirable)

Interviews in a qualitative study tend not to be as tightly prescribed and structured as the interviews conducted in a quantitative study. A second difference is the general “feel” of the interview: It tends to be informal and friendly in a qualitative study but more formal and emotionally neutral in a quantitative one. Participants in a qualitative interview may feel as if they’re simply engaging in a friendly chat with the researcher, who is often someone they have come to know and trust. In contrast, participants in survey research are continually aware that, yes, this is an interview and that any temporary relationship they’ve formed with the researcher will end once the interview is complete.

In some cases, a qualitative researcher may want to interview several participants simultaneously in a focus group. To conduct a focus group, the researcher gathers several people (usually no more than 10 or 12) to discuss a particular issue for 1 to 2 hours. A moderator—someone who may or may not be the researcher—introduces the issues to be discussed, makes sure that no one dominates the discussion, and keeps people focused on the topic. Focus groups are especially useful when time is limited, group members feel comfortable sharing their thoughts and feelings with one another, and the group interaction might be more informative than individually conducted interviews (Creswell, 2013; Neuman, 2011).

GUIDELINES Conducting a Productive Interview

Conducting an informative interview isn’t as easy as it might seem. The following suggestions are based partly on our own experiences and partly on guidance offered by experts in qualitative research (Creswell, 2013; Eisner, 1998; Shank, 2006; Silverman, 1993).

▪ Example of How a Researcher Might Align Interview Questions with Research Questions

Source:From Uncovering the Conceptual Representations of Students With Reading Disabilities (pp. 140–142) by D. Zambo, 2003, unpublished doctoral dissertation, Arizona State University, Tempe. Reprinted with permission.

1. Identify general interview questions and possible follow-up subquestions in advance. Some experienced qualitative researchers are quite skillful at conducting open-ended, unstructured interviews. As a result, they can gain intriguing information and perspectives they hadn’t planned to ask for. However, a major disadvantage of an unstructured interview is that a researcher might ask different questions of different participants and thus may not be able to make cross-participant comparisons. Furthermore, the researcher must be alert to instances when a conversation is drifting in an unproductive direction and gently guide it back on course.

Novice researchers typically have greater success when they prepare their general interview questions in advance, along with possible follow-up questions that probe for details, and make sure that all questions are addressed at some point during the interview. Such planning increases the odds that a researcher can compare the responses of different participants in the event that certain comparisons are desired. Furthermore, IRB approval of a research project may in some instances require that questions be explicitly laid out in the initial research proposal.

Obviously, interview questions should be related to the research questions and overall research problem. As an example, in a qualitative study she conducted for her doctoral dissertation, Debby Zambo examined how children with reading disabilities believe their minds work when they read. She worked with and extensively studied 11 children in grades 5 through 9, interviewing them 10 to 15 times over the course of her investigation. Figure 9.1 presents an excerpt from her dissertation, in which she showed how her interview questions aligned with her research questions.

For any single interview, limit your list of questions to a small number, perhaps five to seven of them. (Although Debby Zambo had many more questions than this, she spread them throughout a dozen or so interviews with each child.) You will find that you won’t necessarily need to ask every question explicitly, as the answers to some may emerge while a participant is responding to others.

Ideally, interview questions encourage people to talk about a topic without hinting that they should give a particular answer. In other words, avoid leading questions. Questions such as “What is going on now?” “What is it like to work here?” and “What’s a typical day like?” can stimulate an informative conversation without suggesting that one kind of response is somehow more desirable than another (Shank, 2006).

1. Consider how participants’ cultural backgrounds might influence their responses. In an effort to ascertain men’s beliefs about ideal family size for a research project in what is now Bangladesh, Howard Schuman (1967) asked a seemingly simple question: “Suppose you had no children. How many would you like to have?” Most men responded, “As many as God wills.” This response reflected a widespread cultural tradition at the time: to leave one’s fate in the hands of God, or at least to say that one’s fate is in God’s hands. Wisely, Schuman revised the question: “Suppose you had no children. If God wished to give you as many children as you wished, how many would you wish for?” (p. 22). This revision yielded responses that were far more useful in addressing Schuman’s research question.

As Schuman discovered, participants’ cultural backgrounds can influence their interview responses in ways you haven’t necessarily anticipated. For instance, if you are interviewing people from Asian cultures, you should be aware that they are less likely to brag about their individual accomplishments than Westerners are (Heine, 2007). A naive researcher might erroneously conclude that Asian individuals are less productive than Western individuals, when in reality Asian individuals are merely less boastful than their Western counterparts. On average, people from Asian cultures also tend to be more tentative in expressing their opinions than is true for Westerners—for example, they might begin a sentence by saying “I’m not sure, but perhaps . . .”— and they aren’t as likely to reveal their emotions during conversations (Morelli & Rothbaum, 2007; Ward, Bochner, & Furnham, 2001).

Various cultural groups differ, too, in their general verbal interaction patterns—for instance, in how talkative and assertive they are and in how much physical distance (personal space) they prefer when conversing with another person (e.g., see Tyler et al., 2008; Ward et al., 2001). Given such diversity across cultural groups—and often among certain subgroups of a cultural group—we can give you only general, nonprescriptive advice here: Be sensitive to the fact that culture may play a significant role in how your participants interpret and respond to your questions, and experiment with multiple ways of asking for the kinds of information you ultimately want to obtain.

1. Make sure your sample includes people who will give you the kinds of information you are seeking. You should, of course, choose people whom you expect to give you typical perceptions and perspectives. But as noted in the earlier discussion of sampling, you may also intentionally pick a few “extremists” or other exceptional individuals who might give you unique insights related to your research problem. When you do so, however, you should identify them as such in your notes.

2. Find a suitable location. In theory, you can conduct an interview anywhere that people are willing to talk to you. But you will probably have a more successful interview if you find a quiet place where you and your interviewee are unlikely to be distracted or interrupted.

3. Get written permission. Explain the nature of the study and your plans for using the results. Ask the participant (or, in the case of a child, the participant’s parent or legal guardian) to sign an informed consent form. Offer to provide an abstract or copy of the research report once you have completed the study.

4. Establish and maintain rapport. Begin the conversation with small talk that can break the ice. Be courteous and respectful at all times. Show genuine interest in what the person has to say. Ideally, you should try to motivate people to want to participate in your study. For example, in a dissertation research project with Chinese mothers who had immigrated to the United States, doctoral student Christy Leung and her research team began interviews by following this general script:

As you know, we are doing this project because we really want to help Chinese families succeed in the U.S., and to have the children be happy and successful. So, we are trying to learn from the different experiences of the families, in order to find patterns of things that can help the transition, and also things that can be negative for the transition and ways to avoid those negative things. That is why we really appreciate your participation and sharing.

After we started this project, we also found that there are aspects of the families’ experiences that we were not able to understand in detail through some of the questionnaires that we are using. Many parents really wanted to talk about and share their experiences regarding how they and their children are doing in the U.S. We feel that these rich experiences (both positive and negative) can add so much to our understanding of the issues that Chinese families in the U.S. are facing. (Leung, 2012, p. 295)

Because interviews in qualitative studies tend to be rather informal, they might appear similar to casual conversation. There is one critical difference between a qualitative interview and normal dialogue, however: The researcher wants to gain information from the interviewee without also revealing his or her own perspectives. In other words, a critical element of most intimate conversations—disclosure of one’s thoughts, beliefs, and feelings—is lopsided, with only one member of the pair doing the disclosing. To maintain rapport and general feelings of closeness and trust, therefore, you must show compassion and interest in other ways, perhaps through body language (smiling, maintaining eye contact, leaning forward) and such neutral encouragements as “Go on” and “What do you mean?” (Shank, 2006).

1. Focus on the actual rather than on the abstract or hypothetical. You are more likely to get revealing information if you ask what a person does or would do in a specific situation—that is, if you ask about actual behaviors. For example, if you are interviewing a teacher, ask questions about specific teaching strategies rather than about educational philosophy. Otherwise, you might get nothing more than what Eisner (1998) has described as “pious, canned proclamations that seem as though they had been snatched from a third-rate philosophy of education text” (p. 183).

2. Don’t put words in people’s mouths. Let people choose their own way of expressing their thoughts. A good interviewer is, above all, a good listener who lets people say what they want to say in the way they want to say it. Furthermore, a good interviewer recognizes that people may reveal inconsistencies in their recollections, attitudes, and logic: Their perceptions won’t necessarily all fit together in a neat little package (Kvale & Brinkmann, 2009).

3. Record responses verbatim. Whether you use handwritten notes, shorthand, or a tape recorder, smartphone application, or laptop computer, capture everything the person says, especially if interview questions are fairly open-ended. If you suspect that an interviewee may have said something other than what he or she intended to communicate, read or play back the response and ask if it accurately reflects his or her thoughts.

4. Keep your reactions to yourself. Although you won’t necessarily want to maintain a continual “poker face,” you’re more likely to get accurate information if you don’t show surprise, agreement, or disapproval of what someone tells you.

5. Remember that you are not necessarily getting the facts. As confident and convincing as some of your participants may be, you should always treat their responses as perceptions and opinions rather than as facts.

6. When conducting a focus group, take group dynamics into account. Whenever you gather two or more people into a single interview, these individuals will rarely act as true equals. Some participants are likely to dominate the conversation. Others may be reluctant to express their views, perhaps because they’re shy or feel uncertain about the validity of their perspective. In most cases, you will get more representative data—and hence more useful data—if you make sure that everyone in the group has a chance to answer each question. Accordingly, you should keep your list of questions for a focus group quite short. And if you are recording the focus group session, ask participants to identify themselves by name at the beginning of the session; having them do so will help you identify different speakers when you transcribe the session later on.

A student researcher wanted to interview certain United Nations personnel to get their opinions concerning issues related to his study. He planned to travel to New York City for a series of interviews and, to conserve both time and expense, wanted to schedule them as tightly as possible. His procedure was organized and logical.

Approximately 6 weeks before his trip, the student wrote the United Nations representatives with whom he wished to confer; he told them when he would be in New York and requested an interview that would last 30 minutes at most. He asked each prospective interviewee for an indication of several time slots when the interview might be scheduled. In his letter, he clearly explained what information he was seeking and why he was seeking it. His reasons were mature and meaningful and were phrased to pique the interest for those he wanted to interview. (Not among his reasons was the fact that he was writing a thesis! If you must reveal that you are collecting data for a thesis, use the word study instead of thesis. Aside from the student and the graduate advisor, theses hold very little glamour in the everyday world. “Studies” are much more acceptable.)

With the letter, the student enclosed a separate sheet containing the questions he intended to ask during the interview, arranged in the order he would ask them. He also suggested that if the interviewee had no objections, he would tape the interview in order to conserve time and lessen the distraction of handwritten notes. He provided a check box on a return postcard for the interviewee to indicate whether he or she had any objection to recording the interview.

After receiving potential interviewees’ replies, he created a master chart of list appointments and, by letter, immediately confirmed each interviewee’s appointment time and thanked the interviewee for his or her cooperation. When a time conflict arose, he sought to resolve it by suggesting alternative times that were still open.

Ten days before the scheduled interviews, the student mailed reminders along with another copy of the interview questions. He also enclosed his full interview schedule so that the interviewees might appreciate the time constraints under which he was working.

The student arrived promptly for each scheduled interview, introduced himself, asked whether the interviewee wanted a copy of the questions he had previously sent, and began with the first question. He tried to guide the interview, always keeping to his agenda of questions and seeking to preserve a relaxed, friendly, yet also professional atmosphere. He wrapped up each interview by thanking the interviewee for the courtesy of giving his or her time. In 3½ days, he interviewed 35 United Nations representatives and had more than four-fifths of his data on tape.

The student transcribed the substance of the interviews and, within 10 days of his visit, sent each interviewee a typed, double-spaced transcript accompanied by a thank-you letter for granting the interview. He asked each individual to read the transcript carefully and, if it was correct, to sign a statement that it was a correct record of the interview. If the person found it inexact or incorrect in any place, he or she could correct the script as desired. In the same mailing, the researcher included a request for permission to use any quotations from the interview in his final report, with the understanding that he would again send the interview content for the interviewee’s approval. In his final thesis, the researcher acknowledged his interviewees and noted that they had inspected and approved all of their quoted statements. With the use of such strategies, the researcher and the readers of his report could all be confident that the participants’ thoughts and opinions were accurately represented.

In summary, the researcher’s use of the following steps led to a highly productive research effort:

1. Set up the interview well in advance.

2. Send the agenda of questions to ask the interviewee.

3. Ask for permission to tape the conference.

4. Confirm the date immediately in writing.

5. Send a reminder, together with another copy of the questions, 10 days before the interview.

6. Be prompt; follow the agenda; offer a copy of the questions in case the original copy has been mislaid.

7. After the interview, submit a transcript of the interview, and get from the interviewee either a written acknowledgment of its accuracy or a corrected copy.

8. After incorporating the material into a semifinal draft of the research report, send that section of the report to the interviewee for final approval and written permission to use the data in the report.

With appropriate software, most laptops and many smartphones can serve as audio recorders. And, of course, videos recorded on a camcorder can be easily downloaded to a personal computer. Meanwhile, transcription software (e.g., HyperTRANSCRIBE) lets you mark key points in a videotaped or audiotaped interview, retrieve desired pieces of information quickly, and slow down what you have recorded so that you can transcribe it more easily. Other software programs (e.g., Dragon Naturally Speaking) will even do your transcribing for you.

USING TECHNOLOGY

In some cases, you can conduct qualitative interviews long-distance through various Internet mechanisms, including e-mail, Skype, or video conferencing. Focus groups might also be conducted online, perhaps through Internet-based chat rooms or bulletin boards (e.g., see Krueger & Casey, 2009, for suggestions). Keep in mind, however, that ethical standards don’t fly out the window simply because you’re conversing with people in cyberspace rather than in the same room. You must still seek participants’ (or parents’) informed consent, and you must protect participants’ privacy. Furthermore, you must ensure that participants have appropriate characteristics and qualifications for your investigation—something that may be difficult to determine if you never see these individuals in the flesh.

How do readers, reviewers, and practitioners assess the worth of a qualitative proposal or research study? What characteristics are essential to a good study? What makes one study “excellent” and another study only “marginal”?

Experienced qualitative researchers have offered a variety of standards that might be used to evaluate a qualitative study (Altheide & Johnson, 1994; Creswell, 2013; Eisner, 1998; Gall, Gall, & Borg, 2007; Glaser, 1992; Howe & Eisenhardt, 1990). We have boiled down their suggestions to nine general criteria:

1. Purposefulness. The research question drives the methods used to collect and analyze data, rather than the other way around.

2. Explicitness of assumptions and biases. The researcher identifies and communicates any assumptions, beliefs, values, and biases that may influence data collection and interpretation.

3. Rigor. The researcher uses rigorous, precise, and thorough methods to collect, record, and analyze data. The researcher also takes steps to remain as objective as possible throughout the project.

4. Open-mindedness. The researcher shows a willingness to modify hypotheses and interpretations when newly acquired data conflict with previously collected data.

5. Completeness. The researcher depicts the object of study in all of its complexity. The researcher spends sufficient time in the field to understand all nuances of a phenomenon; describes the physical setting, behaviors, and perceptions of participants; and ultimately gives readers an in-depth, multifaceted picture of the phenomenon (i.e., thick description).

6. Coherence. The data yield consistent findings, such that the researcher can present a portrait that “hangs together.” Multiple data sources converge onto consistent conclusions (triangulation), and any contradictions within the data are reconciled.

7. Persuasiveness. The researcher presents logical arguments, and the weight of the evidence suggests one interpretation to the exclusion of others.

8. Consensus. Other individuals, including the participants in the study and other scholars in the discipline, agree with the researcher’s interpretations and explanations.

9. Usefulness. The project yields conclusions that promote better understanding of the phenomenon, enable more accurate predictions about future events, or lead to interventions that enhance the quality of life.

In this chapter we have addressed issues related to only some of these criteria—especially issues related to purposefulness, rigor, and open-mindedness. We address issues related to other criteria in discussions of data analysis in Chapter 11 and report writing in Chapter 13.

As should be clear by now, a qualitative research project is not something to be entered into casually. One key consideration is that, regardless of the kinds of data involved, data collection in a qualitative study takes a great deal of time. The researcher should record any potentially useful data thoroughly, accurately, and systematically, using field notes, sketches, photographs, audio recordings, videos, or some combination of these. And as you will discover in Chapter 11, data organization and analysis must be equally meticulous and time-intensive.

If you think a qualitative approach might be suitable for your purposes, you may want to do a pilot study first to find out whether you feel comfortable with the ambiguity and relative lack of structure in the process. We urge you, too, to learn as much as you can about qualitative research strategies, perhaps by reading some of the sources listed in the “For Further Reading” section at the end of this chapter. Once you have determined that you have both the time and skills to conduct a qualitative study, you may find the following checklist helpful in your planning.

          1. What is the current status of knowledge pertaining to the question?

          2. Why is the study important?

          3. Will the focus be on individuals, groups, cultures, experiences, processes, or content?

          4. Will the design be a case study, ethnography, phenomenological study, grounded theory study, content analysis, a combination of two or more of these, or none of these?

          5. Will you need to gain access to one or more sites in the field? If so, how will you do it?

          6. How much time will you need?

          7. What special resources are needed and available?

          8. Are there any existing constraints on data collection?

          9. How will the participants or materials be sampled?

          10. What role will you, as the researcher, assume?

          11. How will you ensure anonymity and confidentiality for the participants?

          12. What procedures will you follow, and in what order?

          13. What technological tools can assist you in data collection?

Practice Thinking Like a Researcher Activity 9.1: Recognizing Qualitative Designs

Practice Thinking Like a Researcher Activity 9.2: Collecting Data for a Qualitative Study

Practice Thinking Like a Researcher Activity 9.3: Planning and Conducting Interviews

As an example of a qualitative research study, we present excerpts from Robin Smith’s doctoral dissertation conducted at Syracuse University (Smith, 1999). The study was a multiple case study that also incorporated elements of grounded theory research and content analysis.

The study focused on five high school students who had significant intellectual disabilities. In particular, it examined the nature of the students’ involvement and participation in high school classrooms. It also looked at teachers’ perceptions and interpretations of the students’ disabilities and academic performance.

The dissertation’s “Method” chapter begins with an overview of the research strategies used and a rationale for selecting the individuals to be studied. It then presents more specific information about each of the five students: Gerald, Trish, Nick, Tyrone, and Abe (all pseudonyms). We pick up the chapter at the point where it begins a discussion of data collection. As we have done in preceding chapters of this book, we present excerpts on the left and a running commentary on the right.

Data Collection Processes

Data gained in the varied academic settings of the five students assisted in understanding the patterns of academic participation and the meanings and relationships of the five students regarding their academic participation in high school. I gathered data from the following sources:

Observations

Over three school semesters, I conducted observations of five high school students who were attending high school and enrolled in at least one academic subject in the general high school curriculum. These observations totaled 52 visits ranging in length . . . the shortest was 15 minutes . . . the longest, 6 hours. . . . [The author continues with a detailed discussion of the kinds of observations made and the circumstances in which she made them.]

Conversations and Interviews

I had conversations and interviews with adults involved and concerned with the students, such as general and special education teachers, assistants, and parents. I recorded and described these conversations in field notes and transcriptions. . . .

The semi-structured interviews with the parents of each student included the following kinds of questions:

1. Tell me about the history of your child’s schooling.

2. What are the child’s strengths? That is, what is he or she good at?

3. Where does it get hard for the student?

4. How does he or she like high school? How can you tell?

5. What do you see your child learning?

6. What are your goals and dreams for your child?

7. What else should I know about your child to better understand what is happening for him or her in school?

I conducted similar interviews with the special education teachers, which included discussion of their educational goals for the student. I conducted one formal interview with each special education teacher, with further interviews as necessary to enhance my understanding of my data. These other interviews were often in the form of brief conversations during or in between class, interviews by appointment, and phone conversations.

I also conducted interviews with the general education teachers in the form of formal, informal, or brief conversations that fit into the teachers’ schedules. . . . I also had some conversations with the general education teachers by staying a few minutes after class and asking them questions about what I had observed that day or how they thought the student was doing. . . .

I taped and had transcribed in-depth interviews, and I embedded observer comments in the transcribed text as I reviewed it. I wrote down informal conversations as soon as possible, and when possible, wrote during the conversations according to the comfort level of the participants with note taking. I used a Hewlett Packard 200LX-palmtop computer, which enabled me to take legible and detailed notes and add more detail soon after an observation.

Official Records and Documents

Official records and documents were another source of information. At the very end of my study I went to the district office of special education, which kept the official records of all five of the students. I looked in each file to learn what I could about the students’ grades and progress reports, along with the professional assessments and recommendations regarding the students’ schooling. I took notes on my hand-held computer and read long quotes into my tape recorder for later reference and transcription. I took notes on students’ work in class and from some student work I found in the files, and collected samples of their work where possible.

Finally, I relied heavily on very detailed field notes. At first I wrote everything I saw. As I narrowed my focus I consistently included the students’ interactions with adults and peers, their reactions to what was going on, and what other students were doing at the same time. Describing interactions of the nonspeaking students was challenging; due to the crowded conditions of several of the general education classes and my being in a wheelchair, I was not always able to be close enough to the student to observe facial expressions. Fortunately, each student was accessible to me most of the time, especially when I was well into the study and a couple of sympathetic teachers invited the student to sit where I could be close by. Thus many of my observations were able to include whispered dialogue between the student and support person helping with an assignment.

Coding and Analysis

. . . As I collected and analyzed data from preliminary observations, I found issues to explore . . . questions arose that created a need for further observing or interviewing. Using the constant comparative method of analysis (Glaser & Strauss, 1967), I collected data, looked for emerging themes and recurrent events, categorized them, and reevaluated my themes and categories. As I collected more data, I wrote analytic memos about my data, and reevaluated my previous theories as I compared old data with new (Bogdan & Biklen, 1992, pp. 72-75). The themes of academic engagement, generated by my pilot study, continued to expand in depth and breadth, and they generated more themes that guided the development of my study.

For example, Nick, one of the students I observed, sat with his assistant in the last row by the door, separated by another row of desks from the class; he seemed an observer in class lectures and discussions. When his assistant supported him to participate in hands-on activities, the assistant did the task for the student. The educators in the room said to me, “He doesn’t understand much of what’s going on,” and they did not expect him to benefit from the actual curriculum content (“He’s not getting much out of it.”). In contrast, Trish, a student with even less physical coordination and verbal expression, followed a full academic schedule, and many of her teachers considered her to be involved, interested, and learning. This led me to look for signs of expectations of the student and how people evaluated the students. Thus, early data codes such as “expectation,” “perception,” and “assessment” led to a chapter regarding expectations and another regarding types of assessments.

I used Q.S.R. Nudist (QSR-NUD*IST, 1995) to code my data. This program enabled me to identify text segments in various ways, including participants’ names and roles, as well as assigned categories such as “engaged,” “disengaged,” and “academics,” that resulted in 98 data codes. A few of these original codes survived my ongoing revisions and collapsing of categories to my final analysis. I printed categories out in groups and coded them again by hand, testing new coding categories by merging several categories and reexamining the data. For example, many of the text segments that I had labeled “expectation” evolved into “assessment.” Once I had determined that assessment was an important category, I subdivided it into “formal,” “informal,” and “professional,” each with its own set of categories which are explained in my data chapter, “Patterns of Assessment”. . . . Further hand coding yielded the categories I finally used in the chapter on “participation”. . . .

[The author continues the discussion of data coding and other issues and then turns to the subject of values.]

Exposing Researcher Values

During this research I have continuously inspected my expectations and values as a continuing reminder of the role that values have in inquiry. . . . Ongoing self-reflection in memos and discussions with mentors throughout the course of the study helped me identify and account for the interference of my assumptions in my study. . . . For example, sometimes I was tempted to express findings about expectations in cause and effect terms. . . .

I expose my values in my narrative as playing a significant role in my inquiry. In sharing my values in the introduction, and further here, I have attempted to take them into account as I share my data and analysis. For example, as a disability rights advocate, I have hoped that my research regarding students with disabilities would be a contribution toward achieving equality and full integration of people with disabilities. I remained aware of my bias against the self-contained setting, where four of the students in the study were based, in order to see what might actually benefit the students in that setting. I am aware that my bias is related to my advocacy stance against segregation and to the negative accounts of friends who have experienced segregated special education. I also had a prejudice against professional assessments along with the likelihood . . . that I might be influenced by the contents if I read them early in the study. To counter inappropriate influence of this prejudice, I read the assessments at the end of my study and took a class in how to administer psychoeducational assessments. . . .

As I listened to my informants, I was aware of my own assumption that students benefit from academic inclusion and that all students have the right to attain knowledge . . . for my observations and interviews, I kept an open mind to the notion that special education settings do not preclude learning, may even enhance it, and that observing the special education academic experiences could also inform me about student engagement and how they [students] participated in the academic activities.

Leaving the Field

The process of leaving the field was gradual. I was learning less and less from observations by the end of spring. Completing ceasing the first school year observation was precipitated by the beginning of the university summer session and my assignment to spend all day in a suburban school as a student teacher. I was assigned to Trish’s summer school class the second summer session and took notes on that experience. I visited her twice in the fall but was excluded from her general education classes due to overcrowding. Also in the fall, I spent two days with Tyrone. . . . By then I had been analyzing data and felt the main thing lacking was the assessment of material from official records. Waiting until the following summer to look into the records proved wise, as I was able to find them a rich source of data. I actually eased my way out of the field (Bogdan & Biklen, 1992, pp. 104–105) rather than leaving, keeping contacts with many of my informants and calling to find out what is going on with a student or to clarify a question.

Here the author provides information about the amount of time she spent in the field. Her observations varied considerably in duration depending on the situation; we are more apt to see such flexibility in a qualitative study than in a quantitative one.

The interviews were presumably structured in this manner so that similar kinds of information would be obtained about each child.

The author used follow-up unstructured interviews to gather additional information as needed. This strategy is consistent with a grounded theory approach, in which the researcher moves back and forth between data collection and data analysis.

The author used audiotapes and transcriptions to capture the details of in-depth interviews; she also wrote notes about shorter, more informal conversations as soon after they took place as possible. The phrase “according to the comfort level of the participants with note taking” might have been better worded as “to the extent that participants felt comfortable with my note taking.” However, the phrase reflects an appropriate sensitivity about taking notes only when it did not make a participant feel uneasy. And notice the author’s use of a small computer to facilitate data collection!

Why did the author wait until the end to look at school records and documents? Later in this excerpt you will see her reason: She was worried that early knowledge of these records would bias her interpretations of what she observed in the classroom.

This narrowing of focus as the study proceeds is frequently found in qualitative research.

Notice how the author is looking for nonverbal as well as verbal information. Notice, too, how cooperative participants (in this case, some “sympathetic teachers”) can facilitate data collection.

The author conducted an earlier pilot study—something we urge any beginning researcher to do, particularly when planning a qualitative study.

Notice the author’s attention to Nick’s physical distance from other students—a clear, nonverbal indicator that Nick is essentially a nonparticipant, an outsider, in this classroom.

The ability to contrast one situation with another is a key advantage of a multiple case study.

Here we see open coding, the first step in data analysis in a grounded theory study (see Chapter 11).

NUD*IST was an early, groundbreaking computer database program especially suited for data collection in qualitative research.

Here data analysis has moved on to axial coding, where the author is refining her categories and their interconnections (see Chapter 11).

In this section, the author reveals her biases and the strategies she used to counteract those biases. Regular conversations with her university advisor and others helped her identify assumptions she didn’t initially realize she was making.

Here the author describes her bias in favor of inclusion, where students with disabilities learn in general education classrooms alongside their nondisabled peers, rather than in self-contained classrooms, in which students with disabilities are segregated from nondisabled students.

Here we discover why the author waited until the end of her study to look at school records.

Here the author is looking for disconfirming evidence, one effective strategy for minimizing the influence of a researcher’s biases on data interpretation.

In grounded theory terminology, the author has probably saturated her categories at this point: Any additional information is shedding little or no new light on the subject matter.

Notice that the author didn’t just disappear from the scene. Instead, she continued to maintain contact with her participants after her research was completed.

Excerpt is from Academic Engagement of High School Students With Significant Disabilities: A Competence-Oriented Interpretation (pp. 18–30) by R. M. Smith, 1999, unpublished doctoral dissertation, Syracuse University, Syracuse, New York. Reprinted with permission.

Agar, M. H. (1996). The professional stranger: An informal introduction to ethnography (2nd ed.). San Diego, CA: Academic Press.

Barlow, D. H., & Nock, M. K. (2009). Why can’t we be more idiographic in our research? Perspectives on Psychological Science, 4, 19–21.

Birks, M., & Mills, J. (2011). Grounded theory: A practical guide. Thousand Oaks, CA: Sage.

Bloomberg, L. D., & Volpe, M. F. (Eds.) (2008). Completing your qualitative dissertation: A roadmap from beginning to end. Thousand Oaks, CA: Sage.

Brinkmann, S., & Kvale, S. (2014). InterViews: Learning the craft of qualitative research interviewing (3rd ed.). Thousand Oaks, CA: Sage.

Butler-Kisber, L. (2010). Qualitative inquiry: Thematic, narrative and arts-informed perspectives. London: Sage.

Chandler, S. (Ed.). (1992). Qualitative issues in educational research. Theory Into Practice, 31, 87–186.

Charmaz, K. (2014). Constructing grounded theory: A practical guide through qualitative analysis (2nd ed.). Thousand Oaks, CA: Sage.

Corbin, J., & Strauss, A. C. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage.

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Upper Saddle River, NJ: Pearson/Allyn & Bacon.

Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Los Angeles: Sage.

Denzin, N. K., & Lincoln, Y. S. (2013). Strategies of qualitative inquiry (4th ed.). Thousand Oaks, CA: Sage.

Eisner, E. W. (1998). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. Upper Saddle River, NJ: Prentice Hall.

Fetterman, D. M. (2010). Ethnography: Step by step (3rd ed.). Thousand Oaks, CA: Sage.

Gast, D. L., & Ledford, J. R. (2014). Single case research methodology: Applications in special education and behavioral sciences (2nd ed.). New York: Routledge.

Gibson, B., & Hartman, J. (2014). Rediscovering grounded theory. London: Sage.

Glaser, B. (1992). Basics of grounded theory analysis. Mill Valley, CA: Sociology Press.

Graneheim, U. H., & Lundman, B. (2003). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112.

Guest, G., Namey, E. E., & Mitchell, M. L. (2013). Collecting qualitative data: A field manual for applied research. Thousand Oaks, CA: Sage.

Hammersley, M. (2008). Questioning qualitative inquiry: Critical essays. Thousand Oaks, CA: Sage.

Hammersley, M., & Atkinson, P. (2007). Ethnography: Principles in practice (3rd ed.). New York: Routledge.

Hatch, A. J. (2006). Early childhood qualitative research. New York: Routledge.

Hays, D. G., & Singh, A. A. (2012). Qualitative inquiry in clinical and educational settings. New York: Guilford Press.

Heidegger, M. (2005). Introduction to phenomenological research (D. O. Dahlstrom, Trans.). Bloomington: Indiana University Press.

Hesse-Biber, S. N., & Leavy, P. (Eds.) (2011). Handbook of emergent methods. New York: Guilford Press.

Jaccard, J., & Jacoby, J. (2010). Theory construction and model-building skills. New York: Guilford Press. (See    Chapter 10, “Grounded and Emergent Theory.”)

James, N., & Busher, H. (2009). Online interviewing. London: Sage.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.

Jorgensen, D. L. (2007). Participant observation: A methodology for human studies. Thousand Oaks, CA: Sage.

Josselson, R. (2013). Interviewing for qualitative inquiry: A relational approach. New York: Guilford Press.

Kamberelis, G., & Dimitriadis, G. (2013). Focus groups: From structured interviews to collective conversations. New York: Routledge.

King, N. (2010). Interviews in qualitative research. Thousand Oaks, CA: Sage.

Krueger, R. A., & Casey, M. A. (2009). Focus groups: A practical guide for applied research (4th ed.). Thousand Oaks, CA: Sage.

Latimer, J. (Ed.) (2003). Advanced qualitative research for nursing. New York: Wiley.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

Locke, L. F. (1989). Qualitative research as a form of scientific inquiry in sport and physical education. Research Quarterly for Exercise and Sport, 60, 1–20.

Morgan, D. L. (1998). The focus group guidebook. Thousand Oaks, CA: Sage.

Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.

Neuman, W. L. (2011). Social research methods: Qualitative and quantitative approaches (7th ed.). Boston: Allyn & Bacon. (See Chapter 13, “Field Research and Focus Group Research.”)

Peshkin, A. (1988). Understanding complexity: A gift of qualitative research. Anthropology and Education Quarterly, 19, 416–424.

Richards, L., & Morse, J. M. (2013). Read me first for a user’s guide to qualitative methods (3rd ed.). Thousand Oaks, CA.

Richardson, J. T. E. (1999). The concepts and methods of phenomenological research. Review of Educational Research, 69, 53–82.

Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage.

Salmons, J. S. (2014). Qualitative online interviews (2nd ed.). Thousand Oaks, CA: Sage.

Sandelowski, M. (2000). Whatever happened to qualitative description? Research in Nursing and Health, 23, 334–340.

Savin-Baden, M., & Major, C. H. (2013). Qualitative research: The essential guide to theory and practice. New York: Routledge.

Schram, T. H. (2006). Conceptualizing and proposing qualitative research (2nd ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.

Shank, G. D. (2006). Qualitative research: A personal skills approach (2nd ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.

Smith, J. A., Flowers, P., & Larkin, M. (2009). Interpretative phenomenological analysis: Theory, method and research. London: Sage.

Smith, M. L. (1987). Publishing qualitative research. American Educational Research Journal, 24, 173–183.

Stake, R. E. (2006). Multiple case study analysis. New York: Guilford Press.

Stake, R. E. (2010). Qualitative research: Studying how things work. New York: Guilford Press.

Strauss, A., & Corbin, J. (1994). Grounded theory methodology: An overview. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (pp. 273–285). Thousand Oaks, CA: Sage.

Tashakkori, A., & Teddlie, C. (Eds.) (2010). Mixed methods in social & behavioral research. Thousand Oaks, CA: Sage.

Yin, R. K. (2013). Case study research: Design and methods (5th ed.). Thousand Oaks, CA: Sage.

Yin, R. K. (2011). Qualitative research from start to finish. New York: Guilford Press.

(Leedy 276-277)

Leedy, Paul D., Jeanne Ormrod. Practical Research: Planning and Design, 11th Edition. Pearson Learning Solutions, 12/2014. VitalBook file.

The citation provided is a guideline. Please check each citation for accuracy before use.