Instructions - Case Study Outline 1. Read the following case study and interview questions; complete the interview with a couple. Week 8 Case Study: Couple Interviews Couple Interviews Students are to

Mixed methods research is quickly gaining in popularity in the social sciences because of its ability to explore complex and multifaceted phenomena from a variety of perspectives (Tashakkori & Teddle, 2003). Researchers no longer need to choose between understanding depth of human experience and generalizability, for mixed methods has the advantage of being able to account for both narratives and standardized data. As such, mixed methods have much to offer the marriage and family therapy (MFT) field because it excels in areas that therapists want to understand: processes of change, evaluations of interventions, and therapeutic relation- ships. Researchers and professionals in MFT have called for an increase in the use of mixed methods by marriage and family therapists (Sprenkle & Piercy, 2005), yet little is known about the scope and application of these methods. Only through understanding the current state of mixed methods research in our field can we seek to improve upon it. A comprehensive content analysis of the extant published literature can inform the profession about ways to improve the uses of mixed methods research in MFT. Sprenkle and Piercy (2005) argue that a central trend in MFT research is pluralism, espe- cially the embracing of the benefits of both quantitative and qualitative methods. This pluralism could improve the study of change in therapy (Sprenkle & Piercy, 2005), therefore increasing the rigor of research within the field. By having research methods that are applicable to many aspects of human experience and are at the same time accessible by diverse professionals, mixed methods may help close the gap between clinicians and researchers that has grown in MFT (Sprenkle, 2003). The possible benefits of mixed methods research are clear, but the details of its use in family therapy research are unknown. To promote sound research in this new methodology, we conducted this study as a first step in understanding the current state of mixed methods and multimethod research in the MFT field. Additionally, we hope to illuminate mixed methods methodology to make it more Laura Gambrel MA, is a Doctoral candidate, and John L. Butler VI, MS, is a Doctoral student, Department of Human Development, Marriage and Family Therapy Program, Virginia Tech. This research began in a graduate class and we are grateful for the contributions and feedback from our professor, Dr. Elizabeth Creamer, and our classmates. We also thank Dr. Megan Dolbin-MacNab for her feedback on an earlier draft of this manuscript. Aspects of this research were displayed in a poster presentation at the 26th Annual Graduate Student Assembly Research Symposium, at Virginia Tech in Blacksburg, Virginia in March 2010. Address correspondence to Laura Gambrel, Virginia Tech Family Therapy Center, 840 University City Boulevard Suite 1 (0515) Blacksburg, VA 24061; E-mail: [email protected] Journal of Marital and Family Therapy doi: 10.1111/j.1752-0606.2011.00260.xApril 2013, Vol. 39, No. 2, 163–181 April 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 163 accessible to MFT researchers. Multimethod research is an umbrella term for any study including a variety of methodologies, which could be qualitative and quantitative. Hence, all mixed methods studies are multimethod, but not all multimethod research is defined as mixed methods. Because mixed methods research is a new methodology, this terminology is rarely used explicitly in describing the design of a research study (Bryman, 2006; Creswell & Plano Clark, 2007). Therefore, our content analysis included all research studies with a combination of quantitative and qualitative methods to categorize them and discuss the methodology used.

This content analysis provides a foundation for promoting further rigor in MFT research, an essential need for the future of our profession (Sprenkle, 2003). LITERATURE REVIEW Mixed methods research—‘‘the third methodological movement’’ (Tashakkori & Teddle, 2003, p. ix)—has risen amidst the paradigm wars of proponents of either quantitative or quali- tative research. Mixed methods is:

A research design with philosophical assumptions as well as methods of inquiry. As a methodology, it involves philosophical assumptions that guide the direction of the col- lection and analysis of data and the mixture of qualitative and quantitative approaches in many phases in the research process. As a method, it focuses on collecting, analyz- ing, and mixing both quantitative and qualitative data in a single study or series of studies. (Creswell & Plano Clark, 2007, p. 5) Mixed methods approaches have been growing over the last decade (Bryman, 2006; Tashakkori & Teddle, 2003). Leaders in the field of mixed methods are naming it as its own methodology, not just a combination of other methods, because it requires its own theoretical premises, languaging, design, data collection, and data analysis techniques (Creswell & Plano Clark, 2007).

Mixed methods studies are distinguished from multimethod studies in that there is a point in mixed methods research when qualitative and quantitative data are mixed, merged, connected, or embedded (Creswell & Plano Clark, 2007). The combining of qualitative and quantitative in this way leads to new information that would not be gained if these results were to remain distinct (Creswell & Plano Clark, 2007). Multimethod studies may include a variety of procedures, but do not mix the qualitative and quantitative data (Creswell & Plano Clark, 2007). For example, researchers may want to discover how families feel about their therapists. Researchers could first ask participants open-ended questions during focus groups and then follow with a quantitative survey of a larger sample. In order for this study to be considered mixed methods, it would require collecting and analyzing both qualitative and quantitative data, and also mixing during one of four phases: design (e.g., authors specifically stating that they are using a mixed methods design and structure it that way); data collection (e.g., selecting questions for the survey based on focus group results); data analysis and results (e.g., transforming focus group themes into quantitative data, or a table showing the interaction between quantitative means and exemplary quotes from the focus groups); or the interpretation stage (e.g., developing theory related to the convergence or divergence of the survey and focus group findings). If the themes from the focus groups and the data from the surveys were collected, analyzed, and interpreted separately, the study would be consid- ered multimethod.

According to Creswell and Plano Clark (2007), there are four main designs of mixed methods research: triangulation, exploratory, explanatory, and embedded. Triangulation designs, which can be represented with the notation (QUAN + QUAL), are the most common and usually give equal priority to qualitative and quantitative methods, which are used concurrently. An example of a triangulation study would be a survey with both closed- and open-ended questions that were analyzed deductively and inductively, and then these results were merged to corroborate them.

Exploratory designs (QUALfiquan) are sequential, emphasize qualitative methods, and con- nect qualitative and quantitative data. They could be used for instrument development. Explana- tory designs (QUANfiqual) are also sequential and connect quantitative and qualitative data, 164JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 but give priority to quantitative methods. A common example of this type of design would be to connect quantitative data with findings from a focus group to more fully understand the quantita- tive results. Embedded designs, which can be represented as QUAN (qual) or QUAL (quan), are most frequently used in the medical field, specifically for experimental research, although they are noted as being hard to distinguish from the other mixed methods designs (Creswell & Plano Clark, 2007). Often they are multiphase studies and give greater weight to quantitative methods. One example would be a clinical trial of a new medicine to treat depression, accompanied by focus groups and semi-structured interviews throughout the trial and at follow-up to understand participants’ experiences of the drug treatment in addition to empirical outcomes.

There may be many good reasons to use a mixed methods design for research, including triangulation, answering different questions, gaining a more complete understanding of phe- nomena, or validating results (Creswell & Plano Clark, 2007). Mixed methods may also be used when a purpose of the study cannot be met or a problem cannot be answered by qualitative or quantitative methods alone (Creswell & Plano Clark, 2007; Johnson & Onwuegbuzie, 2004).

While there are many benefits to this method, it can also be costly, time consuming, and it requires expertise in both inductive and deductive methods, all of which are obstacles to imple- menting quality mixed methods studies (Hanson, Creswell, Piano Clark, Petska, & Creswell, 2005; Plano Clark, Huddleston-Casas, Churchill, Green, & Garrett, 2008). Marriage and family therapists in university settings report that it is important that research designs be practical and not overly time-consuming or complex (McWey et al., 2002), which could contribute to researchers’ hesitancy to use mixed methods. However, well-planned mixed methods designs can diminish the impact of some of the problems with the complexities of mixed methods research (Plano Clark et al., 2008).

Beyond practical and logistical barriers, theoretical and epistemological views can also prevent researchers from combining qualitative and quantitative methodologies (Johnson & Onwuegbuzie, 2004). Methodologists may object to the idea that true mixed methods research is possible because of underlying conflicts between inductive and deductive researcher world- views. This objection has been a barrier to the development of mixed methods, as there are people in all fields who do not see how these methods can be combined without undermining their epistemology (Creswell & Plano Clark, 2007).

One answer to this dilemma is pragmatism, in which mixed methods research is rooted.

Greene and Caracelli (2003) argue that qualitative and quantitative methods have been falsely dichotomized. To see a new way of knowing, one must step out of an either⁄or mentality, which traps people into seeing false categories of data, design, and research. Instead, the pragmatic approach is to use research methods as appropriate to answer research questions. This approach allows the mixing of quantitative and qualitative methodologies without contradicting a philo- sophical presupposition because it rejects the premise that qualitative and quxantitative are inher- ently incompatible (Greene & Caracelli, 2003; Johnson & Onwuegbuzie, 2004). The pragmatic approach may be especially appealing to MFT researchers because of the desire to conduct research with clinical and real-world applicability (Tashakkori & Teddle, 2003). Also, family ther- apists are becoming increasingly accepting of diverse research methodologies (Piercy & Benson, 2005; Sprenkle & Piercy, 2005); hence, MFT may have less resistance to mixed methods than other fields.

However, in the past, there has been little application of multimethod research in MFT. In a content analysis of 199 articles from three popular MFT journals from 1994 to 1999, researchers found that only 8% (n= 16) used both qualitative and quantitative methodologies (Hawley, Bailey, & Pennick, 2000). It is unclear which of these articles would be considered mixed methods, thus further investigation is warranted. Additionally, authors in a mixed meth- ods content analysis of family science research in four well-known journals from 1996 to 2005 found that out of 2,142 articles, only 19 (<1%) met the criteria for mixed methods (Plano Clark et al., 2008). In a related field, researchers examined four journals in school psychology research from 2001 to 2004 and found that 13.7% of studies were mixed methods (Powell, Mihalas, Onwuegbuzie, Suldo, & Daley, 2008). Mixed methods research is able to contribute to a deeper understanding of a phenomenon than mulitmethod research alone, yet little is known about the extent of mixed methods versus multimethod research in MFT. From these results, it April 2013JOURNAL OF MARITAL AND FAMILY THERAPY165 is likely that mixed methods research is being underutilized in our field; thus, a comprehensive study into existent published research is necessary to determine whether this assumption is correct and to provide a basis for recommendations and future research. STUDY SCOPE AND PURPOSE The purpose of this study is to conduct a content analysis investigating the combined use of qualitative and quantitative methods in MFT empirical literature, including published mixed methods and multimethod studies. We used a triangulation mixed methods design (Creswell & Plano Clark, 2007) for this content analysis, with pragmatism as our guiding theoretical frame- work. A triangulation mixed methods design is used to ‘‘validate or expand quantitative results with qualitative data’’ (Creswell & Plano Clark, 2007, p. 62). We collected both quantitative and qualitative data concurrently to bring together the strengths of both forms of research by converging results and findings. 1 As previously stated, a mixed methods research study cannot only utilize both qualitative and quantitative methods, but must also combine them in meaningful ways. In this study, mix- ing of quantitative and qualitative data occurred at all stages of research, including the design, data collection, data analysis and results, and data interpretation phases (see Appendix A). In the analysis and results phase, a mixing table demonstrates the interrelationship between quali- tative themes and research design type. Because of the emphasis on quantitative methods and the concurrent design, this study has the following notation: QUAN + qual (Creswell & Plano Clark, 2007). To analyze the nature of MFT research using both qualitative and quantitative methodologies, we pose the following questions:

Quantitative:What are the characteristics of mixed methods and multimethod studies in MFT literature?

Qualitative:What are the topics of research and rationales for using both quantitative and qualitative methods in mixed methods and multimethod studies in MFT literature?

Mixed:How do the characteristics of mixed methods and multimethod studies in MFT litera- ture relate with their topics and rationales? METHODS Search Criteria This study parallels content analyses about the use of mixed methods in other social sci- ence fields (Hanson et al., 2005; Plano Clark et al., 2008). We located and identified mixed methods and multimethod empirical articles by searching eight popular and respected journals in the MFT field. Those journals includedAmerican Journal of Family Therapy(AJFT),Con- temporary Family Therapy(CFT),Family Process(FP),Journal of Couple and Relationship Therapy(JCRT),Journal of Family Psychotherapy(JFP),Journal of Family Therapy(JFT), Journal of Marital and Family Therapy(JMFT), andJournal of Systemic Therapies(JST). As is common practice, we limited our search to include articles that were published during the past 10 years (January 1999 to November 2009). Per the search criteria suggestion of Creswell and Plano Clark (2007) and similar to Plano Clark et al. (2008), we searched titles and abstracts for the following logic operators and search terms: Mixed method* OR multimeth- od OR [(quantitative OR survey OR experimental OR questionnaire) AND (qualitative OR ethnography OR focus group OR interview)]. The latter search term expanded upon Plano Clark et al. (2008) search parameters ‘‘[(quantitative OR survey) AND (qualitative OR inter- view)]’’ (p. 1547) to capture articles with other common quantitative and qualitative research designs. We conducted our search using the psychology and sociology compilation of EB- SCOhost databases, which includes, among others, the Family and Society Studies World- wide, PsycINFO, Psychology and Behavioral Sciences Collection, and Academic Search Complete databases.

To categorize studies as mixed methods, we used the following criteria:

1. must collect both quantitative and qualitative data; 2. must analyze both quantitative and qualitative data; and 166JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 3. data ‘‘need to be ‘mixed’ in some way so that together they form a more complete picture of the problem than they do when standing alone’’ (Creswell and Plano Clark’s (2007), p. 7).

The use of both deductive quantitative methods and inductive qualitative approaches was key to inclusion in this study as ‘‘the basic premise of the definition is that the combination of quantitative and qualitative approaches provides a better understanding of research problems than either approach alone’’ (Creswell & Plano Clark, 2007, pp. 8–9). The mixing component of the categorizing criteria was operationalized as ‘‘merging or converging the two datasets by actually bringing them together; connecting the two datasets by having one build on the other; or embedding one dataset within the other so that one type of data provides a supportive role for the other’’ (p. 7). Mixing can occur at any of the following four phases of research: design, data collection, data analysis (often in results section), and data interpretation (often in discus- sion section). We determined that articles would need to mix data in half of the research phases (at least two of four) to be considered mixed methods studies. If the article did not meet this level of mixing, but still had both quantitative and qualitative data analysis and collection, then it was categorized as multimethod.

Article Selection Our initial search of keywords produced the following hits in these journals: AJFT (8), CFT (8), FP (15), JCRT (7), JFP (12), JFT (3), JMFT (18), and JST (0), for a total of 71 arti- cles that satisfied the search criteria. However, as is often the case with keyword inquiries in search engines, there were a number of article hits that were false positives. For instance, 36 articles were excluded outright as they mentioned the words quantitative, survey, experimental, questionnaire, qualitative, ethnography, focus group, or interview, but did not actually incorpo- rate quantitative and qualitative data collection and analysis. From this pool, three articles were excluded as they did not meet the inclusion criteria, including articles that were purely the- oretical in nature or part of a series of studies. As our unit of analysis was a single article, we excluded two studies that utilized both qualitative and quantitative methods in a single research study, but then discussed this research in two separate articles. Although having a series of arti- cles does meet Creswell and Plano Clark’s (2007) definition of mixed methods research, the excluded articles lacked sufficient information about the mixing of both qualitative and quanti- tative aspects of their research and therefore could not be used in our analyses. Finally, no arti- cles were located from JST, and no articles met inclusion criteria from JFT, which left 32 empirical articles in six journals for the final analysis (see Appendix B).

Quantitative, Qualitative, and Mixed Analyses For our quantitative analysis, we developed a coding scheme informed by previous mixed methods literature (Hanson et al., 2005; Plano Clark et al., 2008). We used quantitative meth- ods to count and calculate descriptive characteristics of the articles. Concurrent with this data collection, our qualitative analysis explored emergent themes of the topics being researched in MFT and the stated rationales for using both qualitative and quantitative methods. Our mixing of quantitative and qualitative methods and data is evident throughout our research process, from design to discussion.

To begin this process, we used a coin toss to randomly assign articles to each researcher to analyze. This process produced an unequal distribution of articles, so we used a random num- ber generator to facilitate equal distribution for analysis. We are trained at the doctoral-level in quantitative and qualitative methodologies, in mixed methods, and in coding data. We then took an article at the beginning, coded it independently, and compared and contrasted discrep- ancies until consensus on all codes was established; this served as a content analysis reliability check. Our quantitative coding included determining the priority (which method was used more heavily quantitative, qualitative, or equal), timing (order of quantitative and qualitative data collection, either sequential or concurrent), mixed methods design type, theoretical framework, sample size and procedure, timing of mixing quantitative and qualitative data, the qualitative and quantitative research procedures, and the population studied in each article. Our qualita- tive coding examined the topic of the article and explicit rationale stated by the authors for April 2013JOURNAL OF MARITAL AND FAMILY THERAPY167 using both qualitative and quantitative methods for their research. Our qualitative analysis fol- lowed the work of Rossman and Rallis (2003) where text was inductively coded on a molar level, and then categorical themes were created to organize these codes.

Our study included quantitative, qualitative, and mixed analyses. We used Microsoft Excel to tabulate and analyze our quantitative outputs, allowing the coding rubric to emerge along with the data. Qualitative analysis included coding of themes in the topics of study as well as the authors’ rationale for using both quantitative and qualitative methods. Mixed analyses included merging the quantitative and qualitative data during the analysis and interpretation. RESULTS AND FINDINGS As part of our quantitative analysis, we found that 16 of the studies met the criteria for mixed methods research that we established, and 16 studies were multimethod. Figure 1 shows the breakdown of mixed and multimethod research by journal and number of articles. We also found that multimethod studies have a larger sample size for quantitative data (Mean= 96, median = 38) compared with mixed methods studies (M= 35, median= 36). The same holds true for qualitative data comparing multimethod (M= 83, median = 22) to mixed methods (M= 23, median = 15) studies. In terms of research methods, all mixed methods articles used surveys for quantitative data collection (n= 16), and a variety of qualitative procedures, with the most popular approaches being semi-structured interviews (n= 11) and the rest using open-ended survey items (n= 5). Multimethod studies were more varied in their qualitative methods including interviews (n= 9), focus groups (n= 4), and opened survey items (n= 3).

For quantitative data collection, multimethod studies used quasi-experimental, program evalua- tion, and survey design, with survey again being the most popular (n= 13).

Discussion of the theoretical or philosophical underpinnings of mixed methods research methodology was largely absent from the articles. No authors made explicit references to the philosophical roots of mixed methods research methodology in pragmatism, nor did any authors explicitly discuss the theoretical conflicts in combining qualitative and quantitative methods. Timing for mixing of quantitative and qualitative was split with a total of 15 concur- rent designs and 17 sequential designs, with no real trends for mixed versus multimethod stud- ies. No mixed methods studies had equal weighting, with an even split between quantitative (n= 8) and qualitative (n= 8) being given priority. Multimethod studies favored quantitative methods (n= 9), then qualitative (n= 4) and finally equal (n= 3). Triangulation was the favored type of mixed methods design (n= 7), followed by explanatory (n= 6), and explor- atory (n= 3), with no embedded designs being used. Over the years, there appears to be a slight trend of increased multimethod and mixed methods studies (e.g., two articles published in 1999 and six in 2007).

TheJournal of Marital and Family Therapy(JMFT) held the highest number of mixed methods articles (n= 6) in our sample, comprising 37.5% of mixed methods articles. It also held the highest number of multimethod articles (n= 4) and total number of articles having the combined usage of quantitative and qualitative methods (n= 10).Family Process(FP) fol- lowed with the next highest number of total articles (n= 7), divided into multimethod (n=5) 0 1 2 3 4 5 6 7 AJFT CFT FP JCRT JFP JMFT Journal Number of Articles Multi Mixed Figure 1.Mixed and multimethod studies by journal.

168JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 and mixed methods (n= 2). This result is not surprising as JMFT and FP have been recognized as top-tier journals in the field, and consequently, would be expected to utilize cutting edge designs and methodologies. Additionally, these journals emphasize empirical research, which would make them more likely to include studies combining quantitative and qualitative methodologies.

The unit of analysis studied in the articles included a total of eighteen studying individuals, eleven studying couples, and three studying families. The unit of analysis by article for mixed methods studies was individual (n= 7), couple (n= 8), and family (n= 1), whereas multi- method studies had individual (n= 11), couple (n= 3), and family (n= 2).

Qualitative analyses were completed regarding the topics of studies and the stated rationale of utilizing both qualitative and quantitative methods. Topics of studies were organized into eight categories:

1. Family therapy education and training; 2. Model or program evaluation; 3. Couples relationships; 4. Process of therapy–client perspectives; 5. Practice of family therapy–therapist perspectives; 6. Family functioning; 7. Women and gender issues; 8. International issues.

Topic categories were related to mixed methods and multimethod studies (see Table 1), with some topics being emphasized more in certain types of studies. Family therapy education and training included topics of MFT graduate school choice, technology in MFT training, and how family therapists are being trained to work with children. The category of model or pro- gram evaluation included efficacy research into solution-focused brief therapy and evaluation of an empathy training program. Couples relationship topics investigated married and unmarried partnerships, not the process of couples therapy. This included how religion and ethnicity affect intimacy in marriage and how couples balance work and family life. The next category, process of therapy from a client’s perspective, included client feedback about change processes and the experience of therapy, such as how clients in couple therapy viewed their own change. The practice of family therapy from therapist perspectives included studies about how therapists use creativity in their own practice and whether or not family was involved in therapy. Overall, the topics in this theme examined how therapists in clinical settings are actually doing therapy.

The category of family functioning was related to the investigation of family dynamics out- side of therapy, including: intergenerational relationships, and family processes that contribute to Schizophrenia. The women and gender issues theme included topics about feminist theory and women’s beliefs about female circumcision. International issues focused on the impact on individuals, families, and couples of larger, national issues including: the impact on a family of a member being kidnapped in Colombia, and marital stress under the threat of forced reloca- Table 1 Relationship of Topic Categories to Type of Study Mixed methods Multimethod Both Topic categoryProcess of therapy—client perspectivesInternational issues Women and gender issues Model and program evaluationPractice of family therapy—therapist perspectives Family functioning Couples relationships Family therapy education and training Note.Topics could be coded in more than one category. April 2013JOURNAL OF MARITAL AND FAMILY THERAPY169 tion in Golan Heights, between Israel and Syria. The most popular theme was related to MFT education (n= 8) followed by couples relationships at (n= 7). These results can be seen in Figure 2. No patterns seemed to emerge based on topics or categories and type of mixed meth- ods design. Also, there were no trends for various topics over time. See Appendix C for a com- plete list of articles’ codes of study topics, arranged by theme.

Rationale statements for why authors used a combination of quantitative and qualitative methods were organized into five categories: greater understanding and more information (n= 7), qualitative methods to expand or support quantitative methods (n= 5), to triangulate and confirm findings (n= 3), none given (n= 16), and other (n= 1). Greater understanding and more information included purpose statements such as: to gain ‘‘richer understanding of the family variables’’ (Klever, 2003, p. 434). Qualitative findings that expand or support quanti- tative data included: ‘‘provide a more in-depth understanding of the results of the quantitative analyses’’ (Heller & Wood, p. 245). The category, triangulate and confirm findings, included rationales about complementing results and having more trustworthy findings. No rationale given—none—emerged as the most popular category and was used to code articles that did not explicitly state the reasons why the research included both qualitative and quantitative methods.

To clarify, many of these articles had purpose statements, but these statements did not include a rationale for using a mixed or multimethod design. The popularity of rationale statement cat- egories is shown in Figure 3. See Appendix D for a complete list of articles’ codes for ratio- nales, arranged by category.

Of mixed methods articles, 31% (n= 5) had no stated rationale and 69% (n= 11) of multimethod articles had no stated rationale for using both quantitative and qualitative meth- ods. Hence, the majority of mixed methods articles have an explicit rationale for using both methods. There is no trend for use of explicit rationale overtime. Rationales for using both qualitative and quantitative data, when compared to study design, are shown in Table 2. 0 1 2 3 4 5 6 7 8 9 Education & TrainingCouples Process of therapy- ClientPractice of therapy- TherapistFamily FunctioningWomen & Gender IssuesInt'l Issues Model/Prog Evaluation Study Topic Number of Codes Figure 2.Number of study topic codes in each category. 0 2 4 6 8 10 12 14 16 18 Understanding Qual Supports QuanTriangulate None Other Rationale Number of Articles Figure 3.Number of articles per category of rationale statement.

170JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 DISCUSSION The purpose of this study was to understand how mixed methods and multimethod research was utilized in published MFT research studies in the last 10 years. Our main finding is that there is a lack of mixed methods research in MFT overall. Of an estimated 2,400 published articles (excluding media reviews) in eight journals in the last 10 years, 32 (approxi- mately 1.3%) studies were published that used a combination of quantitative and qualitative methods, with only 16 (approximately 0.6%) that met the definition of mixed methods research supplied by Creswell and Plano Clark (2007). Hence, researchers may be missing an opportu- nity to implement an innovative methodology that is consistent with the complex nature of MFT research. When mixed methods research is performed well, quantitative and qualitative methods can reinforce each other, giving a fuller picture of a phenomenon. For example, Olson and Russell (2004) gained insight into how quantitative measures of change intersect with cli- ents’ own experiences of therapy by asking clients about their scores on measures of client out- comes. Klever (2009), drawing on Bowen theory, investigated family functioning with interviews and survey data in a longitudinal study and presented material with clear mixing tables in his results section. In an innovative design, Wynne et al. (2006) presented case vign- ettes along with quantitative measures of family functioning to enhance the understanding of genetic and environmental interactions in regards to Schizophrenia.

Furthermore, we discovered that no authors, in mixed or multimethod studies, discussed explicit theories about how to combine qualitative and quantitative methods. Hence, no one discussed the possible objection that some researchers make of the theoretical conflicts with using deductive and inductive methods of inquiry. The exclusion of theory in social sciences has been noted as problematic (Shoemaker, Tankard, & Lasorsa, 2004), which is particularly the case for mixed methods as a new methodology (Plano Clark et al., 2008). Clinical theoreti- cal frameworks were also limited as has been found to be true of family therapy research in general (Hawley & Geske, 2000), not just multimethod research.

Related to this finding, many articles also lacked overt rationales for using both qual- itative and quantitative methods. More mixed methods articles had rationale statements, and over half of multimethod studies did not have a rationale. For those who did include rationales, the most common reasons for using both qualitative and quantitative methods were for increased understanding of phenomena and to use qualitative methods to support and expand quantitative results. This means that most family therapy researchers were using both methods to gain a more complete and in-depth sense of the results, which implies seek- ing complementary findings. However, divergent results can often contribute to understanding complex phenomena, just as Al-Krenawi and Wiesel-Lev (1999) found women were more Table 2 Rationale Category Organized by Study Design Rationale categoryNumber of articles per study design Total Explanatory Exploratory Triangulation Multimethod Greater understanding and more information212 3 8 Qual will expand⁄support quant201 1 4 Triangulate, confirm findings002 1 3 None 2 1 2 11 16 Other-greater flexibility 0 1 0 0 1 Total 6 3 7 16 32 April 2013JOURNAL OF MARITAL AND FAMILY THERAPY171 likely to discuss their negative views toward female circumcision in quantitative measures than in interviews.

We also discovered that the mixed methods articles in our sample all used survey measures for quantitative data and interviews and surveys for qualitative investigation. This lack of vari- ety shows limitations in the research that is currently published in MFT journals. Because much may be gained by the ability to show cause and effect in experimental research and the in-depth understanding of ethnography or additional contributions of focus groups, we recommend more diversity of methods. Another weakness was that some researchers had limited or poorly executed qualitative data collection and analyses, an area that should be improved upon for quality mixed methods research to be possible. For example, Carson, Becker, Vance, and Forth (2003) and Hertlein and Lambert-Shute (2007) collected qualitative data through open-ended questions in on-line surveys. This type of data collection does not allow for the in-depth under- standing of personal experience that is characteristic of qualitative inquiry. None of our articles used embedded designs, which was not surprising because of the potential challenges with such a design (Creswell & Plano Clark, 2007). However, as the use of mixed methods continues to grow, MFT researchers could benefit from incorporating this design in future research because of its ability to add participant voices to experimental research. Research on therapy model effectiveness is lacking in the MFT field as a whole (Hawley et al., 2000); it is therefore encour- aging to see many authors of multimethod studies focusing on this area. Mixed methods research was shown to be used for incorporating client perspectives into process of change research, which is also a beneficial addition to MFT research.

All the studies in our sample that focused on the process of therapy from the client per- spective were mixed methods, and all but one of both topics of couples relationships and family functioning was also mixed methods. Mixed methods articles had almost twice as many rela- tional units of analysis—couples and families—than did multimethod studies. Hence, a strength of MFT mixed methods research is that it is being used to study systems. Finally, we found that the most researched topics in multimethod and mixed methods research in MFT are mar- riage and family therapy education and couples research. Hawley and collegues (2000) found that couples, and training and supervision are the first and third most prevalent categories in MFT research, respectively. Thus, this finding might be more a product of prevalence of research in the entire field, rather than just multimethod and mixed methods studies. CONCLUSION Limitations Despite the valuable information about MFT research gained from this study, there are several shortcoming associated with this research. For instance, this search of multimethod and mixed methods studies in the MFT literature cannot be considered exhaustive. Our choice of first- and second-tier journal articles focused mostly on U.S.-based journals (with the exception ofJournal of Family Therapy, based in the United Kingdom). There are several international journals which could have been considered for the study. Additionally, we did not examine MFT research published in journals in related fields, such as psychology or social work. Future research may also examine how other fields are using mixed methods research to study family systems to further broaden insights into weaknesses and strengths of this methodology.

Additionally, as mixed methods research is a new methodology, there are many areas that are not yet standardized, making our analyses difficult. For example, as there were not explicit guidelines set forth in mixed methods literature as to the timing of mixing (i.e., mixing in each of the four phases of research), this was left up to our interpretation. It may be that other scholars would have different definitions of mixed versus multimethod research, thus marking another reason for the necessity of standardization.

Finally, although we had an extensive search, our relatively small sample size limited our ability to carry out quantitative analyses. We are confident that our search criteria was appro- priate for this study, but we acknowledge that studies that did not discuss both their qualitative and quantitative methods in their abstracts or use words typically associated with them were 172JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 not included in our analyses (e.g., Hertlein & Piercy, 2008). 2If we expanded our search criteria to a longer time period, we may have encountered more articles, but then the results would have been less relevant to modern researchers. Also, it is possible that our decision to analyze only published research may give a skewed impression of the limited use of mixed methods research, as many studies may exist that used this methodology but were not accepted for publication. However, a brief look at the JMFT editorial annual report for 2007–2008 shows that of 110 submitted research manuscripts, only five—less than 5%—were classified as mixed methods (Chenail & Cronin, 2008), which parallels our findings for published mixed methods research.

Research Implications This study highlights a dearth of published MFT multimethod and mixed methods research. To increase rigorous use of mixed methods research, we have several recommenda- tions for the MFT researcher. Currently, there is little consensus for a common language for terminology and methods in the mixed methods field, which can lead to misunderstandings among researchers. Therefore, we suggest MFT researchers adopt the standardized language of mixed methods design types and purpose statements as discussed by Creswell and Plano Clark (2007) to minimize confusion. We also encourage researchers to specifically use the termsmixed methodsandmultimethodin titles, abstracts, and methods sections for further clarity. Addition- ally, we suggest that authors in the field of MFT focus on a variety of quantitative and qualita- tive methods, instead of almost exclusively on survey data. Diversifying methods, especially when mixing deductive and inductive approaches, will broaden and strengthen the conclusions that are reached.

Moreover, we recommend that researchers make explicit decisions about how they will mix the data before they start their study. We argue that mixing purposefully by establishing and fol- lowing through with a mixing plan is an essential component of mixed methods studies, differen- tiating them from multimethod studies. Stating how and when mixing will occur is also important as these mixing statements will inform researchers that mixing was intentional from the design stage rather than as an afterthought, as well as make it easier to replicate the study.

Researchers should clearly state their rationale for using and mixing quantitative and qualitative methods, as this is one of the evaluation criteria proposed by Creswell and Plano Clark (2007).

Finally, we suggest that future studies have a model or theory to guide their research, as detailed in Shoemaker et al. (2004). A theoretical perspective would inform both the process and the outcome of research, providing valuable insight into the nature of the results. Having a theoretical background might provide a practical framework for ideas and make linkages between quantitative results and qualitative findings.

In this study, we have collected and presented quantitative, qualitative, and mixed analyses of published articles that have the combined use of quantitative and qualitative approaches in MFT empirical literature. In an effort to be transparent and pragmatic, we provide the follow- ing summary of this mixed methods content analysis, which could serve as a template for clari- fying future mixed methods studies:

1.Citation: Gambrel, L. E., & Butler, J. L., VI (2011). Mixed methods research in mar- riage and family therapy: A content analysis.

2.Unit of Analysis: Artifacts (individual journal articles).

3.Sample Size (Quan and Qual): 32.

4.Sampling Method: Purposive.

5.Explicit Theory: Pragmatism.

6.Timing: Concurrent.

7.Weighting: Quantitative emphasis.

8.Mixing: Merge the data.

9.Timing of Mixing: All stages.

10.Quan Research Design: Content analysis.

11.Qual Research Design: Content analysis.

12.Rationale for Quan⁄Qual Collection: Triangulate and confirm findings. April 2013JOURNAL OF MARITAL AND FAMILY THERAPY173 13.Mixed Methods Research Design: Triangulation.

14.Mixed Methods Design Notation: QUAN + qual 3.

Overall, mixed methods research has the potential to contribute much to the MFT field, but it is currently being underutilized. Marriage and family therapists need solid research to remain viable in today’s world of evidence-based practice. Integrated research designs and methods like mixed methods can make the MFT field particularly competitive. We encourage MFT researchers to consider using this new methodology intentionally, explicitly, and properly to investigate the complex questions that characterize our field. REFERENCES Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done?Qualitative Research,6(1), 97–113.

Chenail, R. J., & Cronin, K. (2008). Editor’s annual report.Journal of Marital & Family Therapy,34(4), 515.

doi:10.1111/j.1752-0606.2008.00100.x.

Creswell, J. W., & Plano Clark, V. L. (2007).Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Greene, J. C., & Caracelli, V. J. (2003). Making paradigmatic sense of mixed methods practice. In A. Tashakkori & C. Teddue (Eds.),Handbook of mixed methods in social and behavioural research(pp. 91–110). Thousand Oaks: Sage Publications.

Hanson, W. B., Creswell, J. W., Plano Clark, Y. L., Petska, K. S., & Creswell, D. (2005). Mixed methods research designs in counseling psychology.Journal of Counseling Psychology,52, 224–235.

Hawley, D. R., & Geske, S. (2000). The use of theory in family therapy research: A content analysis of family therapy journals.Journal of Marital and Family Therapy,26(1), 17–22.

Hawley, D. R., Bailey, C. E., & Pennick, K. A. (2000). A content analysis of research in family therapy journals.

Journal of Marital and Family Therapy,26(1), 9–16.

Hertlein, K. M., & Piercy, F. P. (2008). Therapists’ assessment and treatment of internet infidelity cases.Journal of Marital & Family Therapy,34(4), 481. doi:10.1111/j.1752-0606.2008.00090.x.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come.Educational Researcher,33(7), 14–26.

McWey, L. M., West, S. H., Ruble, N. M., Handy, A. K., Handy, D. G., Koshy, M., et al. (2002). The practice of clinical research in accredited marriage and family therapy programs.Journal of Marital and Family Ther- apy,28, 85–92.

Piercy, F. P., & Benson, K. (2005). Aesthetic forms of data representation in qualitative family therapy research.

Journal of Marital and Family Therapy,31, 107–119.

Plano Clark, V. L., Huddleston-Casas, C. A., Churchill, S. L., Green, D. O. N., & Garrett, A. L. (2008). Mixed methods approaches in family science research.Journal of Family Issues,29, 1543–1566.

Powell, H., Mihalas, S., Onwuegbuzie, A. J., Suldo, S., & Daley, C. E. (2008). Mixed methods research in school psychology: A mixed methods investigation of trends in the literature.Psychology in the Schools,45(4), 291– 309.

Rossman, G. B., & Rallis, S. F. (2003).Learning in the field: An introduction to qualitative research. Thousand Oaks, CA: Sage.

Shoemaker, P. J., Tankard, J. W., & Lasorsa, D. L. (2004).How to build social science theories. Thousand Oaks, CA: Sage.

Sprenkle, D. H. (2003). Effectiveness research in marriage and family therapy: Introduction.Journal of Marital and Family Therapy,29, 85–96.

Sprenkle, D. H., & Piercy, F. P. (2005). Pluralism, diversity, and sophistication in family therapy research. In D. H. Sprenkle & F. P. Piercy (Eds.),Research methods in family therapy(2nd ed.) (pp. 3–18). New York:

Guilford.

Tashakkori, A., & Teddle, C. (2003).Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage. CONTENT ANALYZED REFERENCES American Journal of Family Therapy (2) Yarhouse, M. A., Pawlowski, L. M., & Tan, E. S. N. (2003). Intact marriages in which one partner dis-identifies with experiences of same-sex attraction.American Journal of Family Therapy,31(5), 375–394.

Zimmerman, T. S., Haddock, S. A., Current, L. R., & Ziemba, S. (2003). Intimate partnership: Foundation to the successful balance of family and work.American Journal of Family Therapy,31(2), 107–124.

174JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 Contemporary Family Therapy (6) Carson, D. K., Becker, K. W., Vance, K. E., & Forth, N. L. (2003). The role of creativity in marriage and family therapy practice: A national online study.Contemporary Family Therapy: An International Journal,25, 89– 109.

Hendrix, C. C., Fournier, D. G., & Briggs, K. (2001). Impact of co-therapy teams on client outcomes and thera- pist training in marriage and family therapy.Contemporary Family Therapy: An International Journal,23(1), 63–82.

Klever, P. (2003). Intergenerational fusion and nuclear family functioning.Contemporary Family Therapy: An International Journal,25(4), 431–451.

Klever, P. (2009). The primary triangle and variation in nuclear family functioning.Contemporary Family Ther- apy: An International Journal,31(2), 140–159.

Lim, S.-L., & Hernandez, P. (2007). The WebQuest: An illustration of instructional technology implementation in MFT training.Contemporary Family Therapy: An International Journal,29(3), 163–175.

Olson, M. M., & Russell, C. S. (2004). Understanding change in conjoint psychotherapy: Inviting clients to com- ment upon the validity of standardized change scores.Contemporary Family Therapy: An International Jour- nal,26(3), 261–278.

Journal of Marital and Family Therapy (10) Blow, A. J., Morrison, N. C., Tamaren, K., Wright, K., Schaafsma, M., & Nadaud, A. (2009). Change processes in couple therapy: An intensive case analysis of one couple using a common factors lens.Journal of Marital and Family Therapy,35(3), 350–368.

Heller, P. E., & Wood, B. (2000). The influence of religious and ethnic differences on marital intimacy: Intermar- riage versus intramarriage.Journal of Marital and Family Therapy,26(2), 241–252.

Hertlein, K. M., & Lambert-Shute, J. (2007). Factors influencing student selection of marriage and family therapy graduate programs.Journal of Marital and Family Therapy,33(1), 18–34.

Klever, P. (2009). Goal direction and effectiveness, emotional maturity, and nuclear family functioning.Journal of Marital and Family Therapy,35(3), 308–324.

Miller, J. K., & Slive, A. (2004). Breaking down the barriers to clinical service delivery: Walk-in family therapy.

Journal of Marital and Family Therapy,30(1), 95–103.

Miller, J. K., Linville, D., Todahl, J., & Metcalfe, J. (2009). Using mock trials to teach students forensic core competencies in marriage and family therapy.Journal of Marital and Family Therapy,35(4), 456– 465.

Russell, C. S., DuPree, W. J., Beggs, M. A., Peterson, C. M., & Anderson, M. P. (2007). Responding to remediation and gatekeeping challenges in supervision.Journal of Marital and Family Therapy,33(2), 227– 244.

Schacht, R. L., Dimidjian, S., George, W. H., & Berns, S. B. (2009). Domestic violence assessment procedures among couple therapists.Journal of Marital and Family Therapy,35(1), 47–59.

Shamai, M., & Lev, R. (1999). Marital quality among couples living under the threat of forced relocation: The case of families in the Golan Heights.Journal of Marital and Family Therapy,25(2), 237–252.

Sori, C. F., & Sprenkle, D. H. (2004). Training family therapists to work with children and families: A modified Delphi study.Journal of Marital and Family Therapy,30(4), 479–495.

Family Process (7) Al-Krenawi, A., & Wiesel-Lev, R. (1999). Attitudes toward and perceived psychosocial impact of female circum- cision as practiced among the Bedouin-Arabs of the Negev.Family Process,38, 431–443.

Clay, C. M., Ellis, M. A., Griffin, M. L., Amodeo, M., & Fassler, I. R. (2007). Black women and white women:

Do perceptions of childhood family environment differ?Family Process,46, 243–356.

Goodrich, T. J., & Silverstein, L. B. (2005). Now you see it, now you don’t: Feminist training in family therapy.

Family Process,44, 267–281.

Marshall, T., & Solomon, P. (2004). Provider contact with families of adults with severe mental illness: Taking a closer look.Family Process,43, 209–216.

Matos, M., Torres, R., Santiago, R., Jurado, M., & Rodrı´ guez, I. (2006). Adaptation of parent-child interaction therapy for Puerto Rican families: A preliminary study.Family Process,45, 205–222.

Shields, C. G., & Rousseau, S. J. (2004). A pilot study of an intervention for breast cancer survivors and their spouses.Family Process,43, 95–107.

Wynne, L. C., Tienari, P., Sorri, A., Lahti, I., Moring, J., & Wahlberg, K. (2006). Genotype-environment interac- tion in the Schizophrenia spectrum: Qualitative observations.Family Process,45, 435–447.

April 2013JOURNAL OF MARITAL AND FAMILY THERAPY175 Journal of Couple & Relationship Therapy (3) Angera, J., & Long, E. (2006). Qualitative and quantitative evaluations of an empathy training program for cou- ples in marriage and romantic relationships.Journal of Couple & Relationship Therapy,5(1), 1–26.

Cheung, M. (2008). Resilience of older immigrant couples long-term marital satisfaction as a protective factor.

Journal of Couple and Relationship Therapy,7(1), 19–38.

Ward, M. R. (2007). Clients’ perceptions of the therapeutic process: A common factors approach.Journal of Couple & Relationship Therapy,6(3), 25–43.

Journal of Family Psychotherapy (4) Cunanan, E. D., & McCollum, E. E. (2006). What works when learning Solution-Focused Brief Therapy: A qual- itative study of trainees’ experiences.Journal of Family Psychotherapy,17(1), 49–65.

Hertlein, K. M., Lambert-Shute, J., & Piercy, F. P. (2009). The journal-reading habits of practicing MFTs.Jour- nal of Family Psychotherapy,20, 28–45.

Molina, B., Agudelo, M. E., De los Rı´ os, A., Builes, M. V., Ospina, A., Arroyave, R., et al. (2005). Kidnapping:

Its effects on the beliefs and the structure of relationships in a group of families in Antioquia.Journal of Family Psychotherapy,16(3), 39–55.

Quick, E. K., & Gizzo, D. P. (2007). The ‘‘doing what works’’ group: A quantitative and qualitative analysis of Solution-Focused Group Therapy.Journal of Family Psychotherapy,18(3), 65–84. NOTES 1Purpose statement adapted from Creswell and Plano Clark (2007), p. 99.2We are appreciative of an anonymous reviewer for this contribution.3Adapted from Creswell and Plano Clark (2007), p. 85. APPENDIX A Triangulation Mixed Methods Design 4 Collection Analysis Results Convergence Interpretation Structured content analysis MS Excel© Percentages and count occurrences QUAN emphasis qual included in mixing tablesThemes and narratives-content analysis MS Excel© Note memos and emergent themes Unequal weighting- supplementary qual Compare and contrast QUAN with qual qual qual qual QUAN QUAN QUAN + qualConverge data Expand QUAN results with qual findings Combining data sets in discussion and implications QUAN 4Appreciation expressed to Sharon J. Butler for help with figure illustration. 176JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 APPENDIX B Multimethod and Mixed Methods Articles in MFT Study Code Title Design Notation Al-Krenawi and Wiesel-Lev (1999) FP1 Attitudes toward and perceived psychosocial impact of female circumcision as practiced among the Bedouin-Arabs of the NegevMultimethod — Angera and Long (2006) JCRT1 Qualitative and quantitative evaluations of an empa- thy training program for couples in marriage and romantic relationshipsMultimethod — Blow et al. (2009) JMFT1 Change processes in couple therapy: An intensive case analysis of one couple using a common factors lensTriangulation QUAL + quan Carson et al. (2003) CFT1 The role of creativity in marriage and family therapy practice: A national online studyMultimethod — Cheung (2008) JCRT2 Resilience of older immigrant couples long-term marital satisfaction as a protective factorTriangulation quan + QUAL Clay et al. (2007) FP2 Black women and White women: Do perceptions of childhood family environment differ?Multimethod — Cunanan and McCollum (2006) JFP1 What works when learning Solution-Focused Brief Therapy: A qualitative study of trainees’ experiencesMultimethod — Goodrich and Silverstein (2005) FP3 Now you see it, now you do not: Feminist training in family therapyMultimethod — Heller and Wood (2000) JMFT2 The influence of religious and ethnic differences on marital intimacy: Intermarriage versus intramarriageExplanatory QUANfiqual Hendrix et al. (2001) CFT2 Impact of co-therapy teams on client outcomes and therapist training in marriage and family therapyMultimethod — Hertlein and Lambert-Shute (2007) JMFT3 Factors influencing student selection of marriage and family therapy graduate programsTriangulation QUAN + qual April 2013JOURNAL OF MARITAL AND FAMILY THERAPY177 Appendix B (Continued) Study Code Title Design Notation Hertlein et al.

(2009)JFP2 The journal-reading habits of practicing MFTs Exploratory QUALfiquan Klever (2003) CFT3 Intergenerational fusion and nuclear family func- tioningExplanatory QUANfiqual Klever (2009) CFT4 The primary triangle and variation in nuclear family functioningExplanatory QUANfiqual Klever (2009) JMFT4 Goal direction and effectiveness, emotional matu- rity, and nuclear family functioningExplanatory QUANfiqual Lim and Hernan- dez (2007)CFT5 The WebQuest: An illustration of instructional technology implementation in MFT trainingMultimethod — Marshall and Solomon (2004)FP4 Provider contact with families of adults with severe mental illness: Taking a closer lookExplanatory QUANfiqual Matos et al.

(2006)FP5 Adaptation of parent–child interaction therapy for Puerto Rican families: A preliminary studyMultimethod — Miller and Slive (2004)JMFT5 Breaking down the barriers to clinical service delivery: Walk-in family therapyMultimethod — Miller et al.

(2009)JMFT6 Using mock trials to teach students forensic core competencies in marriage and family therapyMultimethod — Molina et al.

(2005)JFP3 Kidnapping: Its effects on the beliefs and the structure of relationships in a group of families in AntioquiaMultimethod — Olson and Rus- sell (2004)CFT6 Understanding change in conjoint psychotherapy:

Inviting clients to comment upon the validity of standardized change scoresExploratory quanfiQUAL Quick and Gizzo (2007)JFP4 The ‘‘doing what works’’ group: A quantitative and qualitative analysis of Solution-Focused Group TherapyMultimethod — 178JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 Appendix B (Continued) Study Code Title Design Notation Russell et al.

(2007)JMFT7 Responding to remediation and gatekeeping chal- lenges in supervisionExplanatory QUANfiqual Schacht et al.

(2009)JMFT8 Domestic violence assessment procedures among couple therapistsMultimethod — Shamai and Lev (1999)JMFT9 Marital quality among couples living under the threat of forced relocation: The case of families in the Golan HeightsMultimethod — Shields and Rousseau (2004)FP6 A pilot study of an intervention for breast cancer survivors and their spousesMultimethod — Sori and Sprenkle (2004)JMFT10 Training family therapists to work with children and families: A modified Delphi studyExploratory QUALfiquan Ward (2007) JCRT3 Clients’ perceptions of the therapeutic process: A common factors approachTriangulation QUAN + qual Wynne et al.

(2006)FP7 Genotype–environment interaction in the Schizo- phrenia spectrum: Qualitative observationsTriangulation QUAL + quan Yarhouse et al.

(2003)AJFT1 Intact marriages in which one partner dis-identi- fies with experiences of same-sex attractionTriangulation QUAL + quan Zimmerman et al. (2003)AJFT2 Intimate partnership: Foundation to the successful balance of family and workTriangulation QUAL + quan Note.Code derived from journal title and frequency of articles within journal; AJFT = American Journal of Family Therapy; CFT = Contemporary Family Therapy; FP = Family Process; JCRT = Journal of Couple & Relationship Therapy; JFP = Journal of Family Psychotherapy; JMFT = Journal of Marital and Family Therapy. April 2013JOURNAL OF MARITAL AND FAMILY THERAPY179 APPENDIX C Codes of Articles’ Study Topics, Arranged by Topic Theme 1. Family therapy education and training JMFT3: graduate MFT school choice (MM) CFT2: co-therapy; MFT training (Multi) JMFT10: clinical training—children and families; therapist skill⁄comfort level; play therapy (MM) CFT5: WebQuest technique in MFT training (Multi) FP3: feminist training in COAMFT programs (Multi) JFP1: trainee perspective; family therapy training; SFBT (Multi) JMFT6: forensic competency training (Multi) JMFT7: supervision challenges; remediation and gatekeeping of students (MM) 2. Model and program evaluation JFP4: efficacy of SFBT (Multi) FP5: cultural, PCIT efficacy (Multi) JCRT1: empathy training; program evaluation (Multi) JMFT5: clinical availability (Multi) FP6: treatment techniques; breast cancer; spouses (Multi) 3. Couples relationships AJFT1: coping, same-sex attraction, and marital satisfaction (MM) AJFT2: family⁄work balance, dual earners, marital relationship (MM) JMFT4: marriage relationships, differentiation (MM) JCRT2: immigration; marital satisfaction; acculturation (MM) JMFT2: religion; ethnicity; intimacy in marriage (MM) JMFT9: marital stress, international turmoil (Multi) CFT4: adult child family⁄marriage relationships; triangles; nuclear family functioning (MM) 4. Process of therapy—client perspectives CFT6: change in couples therapy (MM) JMFT1: change in couples therapy (MM) JCRT3: common factors; client perspective; therapeutic experience (MM) 5. Practice of family therapy—therapist perspectives CFT1: creativity in MFT practice (multi) FP4: family involvement in agency, individuals with mental illness (MM) JMFT8: domestic violence screening (Multi) JFP2: clinical journal reading; bridge gap between clinicians and researchers (MM) 6. Family functioning CFT4: adult child family⁄marriage relationships; triangles; nuclear family functioning (MM) FP7: nature versus nurture, Schizophrenia and family (MM) JFP3: family impact of kidnapping, international⁄cultural issue (Multi) CFT3: nuclear family functioning; intergenerational fusion (MM) 7. Women and gender issues FP1: female circumcision, international⁄cultural issue (Multi) FP2: childhood family; ethnicity; women studies (Multi) FP3: feminist training in COAMFT programs (Multi) 8. International issues JFP3: family impact of kidnapping, international⁄cultural issue (Multi) FP1: female circumcision, international⁄cultural issue (Multi) JM1. FT9: marital stress, international turmoil (Multi) Note.MM = mixed method; Multi = multimethod.

180JOURNAL OF MARITAL AND FAMILY THERAPYApril 2013 APPENDIX D Codes of Article’s Rationales for Using Qualitative and Quantitative Methods, Arranged by Rationale Category 1. Greater understanding and more information JMFT9: Greater understanding of dynamics (Multi) FP2: Explanation⁄understanding (Multi) JMFT7: Expand and clarify (MM) CFT2: Ground findings, and strengthen understanding of complexity (Multi) CFT3: Deeper understanding of the family (MM) JMFT1: To gather sufficient evidence (MM) JCRT2: Richer understanding of couples (MM) 2. Qualitative will expand⁄support quantitative JMFT2: Richer understanding of the quantitative results (MM) JMFT3: Further understanding of quantitative data (MM) CFT4: Elaborate on quantitative, thick description (MM) JFP3: Qualitative to support quantitative findings and lead to new ideas (Multi) CFT6: Corroborate and confirm quantitative, give context and description to quantitative (MM) 3. Triangulate, confirm findings JCRT1: Minimize weaknesses in both quantitative and qualitative; more trustworthy findings (Multi) AJFT2: Triangulation, add depth and complement findings (MM) FP7: Confirm and expand, leading to new ideas (MM) 4. None provided JMFT5: None (Multi) JMFT6: None (Multi) JMFT8: None (Multi) JFP4: None (Multi) JFP1: None (Multi) FP3: None (Multi) FP5: None (Multi) FP6: None (Multi) FP1: None (Multi) CFT1: None (Multi) JCRT3: None (MM) FP4: None (MM) JFP2: None (MM) AJFT1: None (MM) JMFT4: None (MM) CFT5: None (Multi) 5. Other JMFT10: Exploratory nature of the study, flexibility (MM) Note.MM = mixed method; Multi = multimethod.

April 2013JOURNAL OF MARITAL AND FAMILY THERAPY181