When Should Mixed Methods Be Used?

profileEveteg
Mixed_Methods_Research_Merging_Theory_with_Practic..._----_Pg_18--41.pdf

Introduction to Mixed Methods Research 3

within the city. He wrote in 1926: “In so far as social structure can be defined in terms of position, social changes may be defined in terms of movement: and society exhibits in one of its aspects that can be measured and described in mathematical formulas.” He took an interest in land values, or street-car transfers, or the volume of traffic at intersections, as indexes for underlying social processes. (p. 153)

What are Mixed Methods?

In general, researchers who use mixed methods employ a research design that uses both quantitative and qualitative data to answer a particular question or set of questions. This combination of methods “involve[s] the collection, analysis, and integration of quantitative and qualitative data in a single or multiphase study” (Hanson, Creswell, Plano Clark, Petska, & Creswell, 2005, p. 224). The term “multimethods” refers to the mixing of methods by combining two or more qualitative methods in a single research study (such as in-depth interviewing and partici- pant observation) or by using two or more quantitative methods (such as a survey and experiment) in a single research study.

Mixed methods is a rich field for the combination of data because with this design “words, pictures, and narrative can be used to add meaning to numbers” ( Johnson & Onwuegbuzie, 2004, p. 21). In other words, what we generally consider qualitative data—“words, pictures, and narrative”—can be combined with quantitative, numerical data from a larger-scale study on the same issue, allowing our research results to be generalized for future studies and examinations.

Why Use Mixed Methods?

Greene, Caracelli, and Graham (1989) list five specific reasons that researchers should consider using mixed methods. The first, triangula- tion, seems to be the most commonly cited reason that mixed methods are incorporated into research. Triangulation—or, more specifically, methods triangulation, in the context of methods alone— refers to the use of more than one method while studying the same research ques- tion in order to “examine the same dimension of a research problem” ( Jick, 1979, p. 602). The researcher is looking for a convergence of the data collected by all methods in a study to enhance the credibility of the research findings. Triangulation ultimately fortifies and enriches a

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

4 M I X E D M E T H O D S R E S E A R C H

study’s conclusions, making them more acceptable to advocates of both qualitative and quantitative methods.

The second reason to consider incorporating a mixed methods design is complementarity. Complementarity allows the researcher to gain a fuller understanding of the research problem and/or to clarify a given research result. This is accomplished by utilizing both quantita- tive and qualitative data and not just the numerical or narrative expla- nation alone to understand the social story in its entirety. Both com- plementarity and triangulation are useful “for cross- validation when multiple methods produce comparable data” (Yauch & Steudel, 2003, p. 466). Complementarity has proven useful in several research studies, and a strong example is Yauch and Steudel’s (2003) examination of the organizational cultures of two small manufacturers. In their work, Yauch and Steudel utilized both qualitative and quantitative research methods. Not only did the triangulation of the qualitative and quanti- tative data secure the validity of their study, but the complementarity of the two datasets produced a more thorough comprehension of the organizational cultures in question.

The researchers first used employee interviews to gather a wealth of narrative information and then used their qualitative findings to cre- ate a survey to collect numerical data. Yauch and Steudel’s hope was to use mixed methods to “identify key cultural factors that aided or hindered a company’s ability to successfully implement manufacturing cells [autonomous labor teams]” (2003, p. 467). Combining and cross- ing these data yielded rich results:

Despite the long delay between beginning the qualitative assessment and administering the survey, the OCI [survey] was an important means of triangulation for two of the cultural factors identified and had the potential to reveal additional cultural dimensions that the qualitative analysis might have missed. (Yauch & Steudel, 2003, p. 476)

This study illustrates the power and possibilities inherent in mixed methods research; the researchers unearthed the convergence of the data from interviews and surveys through triangulation as well as the complementarity of the qualitative and quantitative data, which sup- plied them with a greater understanding of the organizational cultures of small manufacturers.

Mixed methods assist the researcher’s total understanding of the research problem; this understanding represents the third reason

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

Introduction to Mixed Methods Research 5

for using mixed methods: development. Mixed methods often aid in the development of a research project by creating a synergistic effect, whereby the “results from one method . . . help develop or inform the other method” (Greene et al., 1989, p. 259). For example, statistical data collected from a quantitative method can often shape interview ques- tions for the qualitative portion of one’s study. Jenkins’s (2001) research on rural adolescents and substance abuse illustrates the potential of the development factor in a mixed methods study. Jenkins administered a structured questionnaire to quantitatively measure students’ drug use. Her study also included a follow-up set of focus-group interviews and open-ended questionnaires intended to capture students’ perceptions of “drug resistance difficulties” to a variety drugs ranging from alco- hol to LSD to various types of narcotics ( Jenkins, 2001, p. 215). The results from conducting both these studies sequentially contributed to Jenkins’s overall understanding of drug abuse among this population. Her initial use of a structured questionnaire provided her with a statisti- cal understanding of student drug use. A follow-up focus-group study provided her the opportunity to triangulate her data (asking whether the findings from both studies agreed), and, in doing so, she found that the results from the focus group were “consistent with the open- ended questionnaire findings . . . [and] provided further clarification and, in some instances, additional information” ( Jenkins, 2001, p. 219). In addition, by implementing a sequential design with the quantita- tive component first and the qualitative second, she was able to attain a “value added” understanding of the results from both studies. Her focus-group data allowed her to clarify and follow up on definitions and uses of terms, such as “peer pressure,” that were used in survey questions and to ground their meaning from the perspective of her respondents.

A fourth reason cited for using mixed methods is initiation; a study’s findings may raise questions or contradictions that will require clarifi- cation, thus initiating a new study. The desired effect of the new study would be to add new insights to existing theories on the phenomenon under examination (Greene et al., 1989). In fact, findings from this study might uncover a completely new social research topic and launch a new investigation, leading us to a fifth reason for doing mixed methods research: expansion. Expansion is intended to “extend the breadth and range of the inquiry” (Greene et al., 1989, p. 259). Producing detailed findings helps enable future research endeavors and allows researchers to continuously employ different and mixed methods in their pursuit of new or modified research questions.

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

6 M I X E D M E T H O D S R E S E A R C H

Greene and her colleagues (1989) provide a useful organizing framework for characterizing the ways researchers have used mixed methods. We can clearly see the positive power and synergy of using these methods to complement one’s research findings. Quantitative information delivered in a “hard data” format is amenable to statistical analyses and standardized tests of reliability and validity. Qualitative data add an in-depth understanding of research results and allow the researcher to explore anomalies or subgroups within the data. Work- ing with both methods gives many researchers a cross-check on their research results. Qualitative data illuminate the meaning of statistical results by adding a narrative understanding to quantitative research findings. Qualitative methods can also assist researchers who want to test the validity of their research questionnaires by sequentially utiliz- ing mixed methods. For instance, an initial qualitative study allows the development of research instruments, such as a questionnaire, that can be used in a large-scale quantitative research study. Similarly, quantita- tive data can assist qualitative researchers by providing them with a broader context within which to place their qualitative data, as well as providing, through survey samples, ways to identify representative cases for their in-depth research. It is in this sense that quantitative data can be useful for establishing generalizability of qualitative results.

All of these reasons provide strong arguments for a researcher to consider a mixed methods approach. The following in-depth example shows the promise of mixed methods research in tackling a complex social problem: obesity in children.

The Fight against Childhood Obesity: An Illustration of the Need for and Importance of Mixed Methods

America is facing a new range of problems in the 21st century. Poverty, war, and health care are among these challenges, but a deeper problem threatens families and children across the country. Childhood obesity has become a nationwide epidemic, and parents and physicians alike are raising concerns. Consider the following article that appeared in The New York Times:

Six-year-old Karlind Dunbar barely touched her dinner, but not for time- honored 6-year-old reasons. The pasta was not the wrong shape. She did not have an urgent date with her dolls.

The problem was the letter Karlind discovered, tucked inside her report card, saying that she had a body mass index in the 80th

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

Introduction to Mixed Methods Research 7

percentile. The first grader did not know what ‘‘index’’ or ‘‘percen- tile’’ meant, or that children scoring in the 5th through 85th percen- tiles are considered normal, while those scoring higher are at risk of being or already overweight.

Yet she became convinced that her teachers were chastising her for overeating.

Since the letter arrived, “my 2-year-old eats more than she does,” said Georgeanna Dunbar, Karlind’s mother, who complained to the school and is trying to help her confused child. “She’s afraid she’s going to get in trouble,” Ms. Dunbar said. (Kantor, 2007, pp. A1, A14)2

Many school districts across the country have adopted the body mass index (BMI) screening test as a weapon for fighting childhood obesity. On the surface, conducting BMI screening tests on students from kindergarten through eighth grade appears to achieve an impor- tant health goal by identifying the percentage of children who are over- weight, a crucial step in fighting childhood obesity. Yet there does not seem to be a research design in place that can fully address this objec- tive if the goal of these data is to assist in stemming the tide of obesity. In fact, it appears as though little thought was ever given as to how these data might empower parents and their children to be part of the prob- lem’s solution. Gathering descriptive quantitative data is not enough to complete the goals of this study. One needs a broader understanding of the social context within which this information is disseminated among families to ensure that both students and parents are prepared to pre- vent childhood obesity and to develop the best strategy for fighting the obesity epidemic now.

There are many questions we might ask to gain a clearer under- standing of the factors nurturing America’s obesity problem. Explor- ing the lived experiences of students and their families and the day-to- day experiences of children’s eating behaviors at school would provide beneficial information. What is the school environment like for stu- dents with respect to eating? What types of relationships do they have with food at home? To what extent, if any, would parents welcome the school’s input into their children’s issues with weight? These are only a few of the many social- context questions that should be addressed so that schools might utilize the best approach for conveying weight- related “bad news” to both students and their families.

Designing a stronger research strategy would play a significant role in this process. For example, one might use a qualitative focus-

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

8 M I X E D M E T H O D S R E S E A R C H

group-study design following the collection of BMI scores. The quan- titative data collected could be presented to a representative group of parents; the researchers, perhaps in conjunction with school and health officials, could assess the parents’ reactions to this aggregated data for their school and also receive input on how to disseminate, if at all, these results to families and students. A mixed methods study consisting of a quantitative component (the BMI score collection) fol- lowed by a qualitative component (focus group) might give research- ers a deeper understanding of parents’ feelings and attitudes toward childhood obesity. The focus group could also collect parents’ sugges- tions as to how these results could be utilized to combat the epidemic. As one mother quoted in The New York Times article commented, “The school provides us with this information with no education about how to use it or what it means” (Kantor, 2007, p. A14). A fictitious causal link exists in many researchers’ minds that letters to parents will trig- ger the logical response of parents taking personal action to stem the tide of weight gain in their children. This strategy has not succeeded, and, in some cases, the opposite reaction seems to have occurred. One report observed the reaction of some parents: “the letters sent home with report cards . . . [were] a shock. Many parents threw them out, outraged to be told how much their children should weigh or uncon- vinced that children who look just fine by local standards are too large by official ones” (Kantor, 2007, p. A14).

In the end, rather than fighting the effects of obesity, the letter made many families feel helpless and victimized by a flawed system. There appears to be little understanding of the physical and mental effects of bad BMI news on parents and children. The assumption that the problem stems from imprinted eating habits learned at home leads to the conclusion that BMI screenings conducted in school would be an effective deterrent to childhood obesity. Yet research has revealed that the problem runs deeper than family eating habits and screenings. In some schools, the food available in the cafeterias was not nutritious and, in fact, exacerbated the problem. In these specific schools, cafete- ria cuisine was the central root of the problem, as schools were “con- tinuing to feed them atrocious quality meals and snacks, with limited if any opportunities for phys-ed in school” (Kantor, 2007, p. A14).

This example illustrates the need for mixed methods research across a range of disciplines from the physical sciences to the social and behavioral sciences. The problems that followed the BMI screening tests and information letters could have been assuaged with a unique

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

Introduction to Mixed Methods Research 9

research method that would probe the numerical and statistical data and unite it with qualitative information collected through interviews with parents and students so that an effective means of childhood obe- sity prevention and reversal might be discovered. Scholars and research- ers are still debating the benefits of mixing methods, but there seems to be significant promise for this revolutionary design.

objectives of this Book

One objective of this book is to reconceptualize the approach to mixed methods research by offering a comprehensive approach that stresses the tight link between theory and research and that centers the research prob- lem in the design and analysis of mixed methods projects, whether they are derived from a quantitative or a qualitative approach.

A second aim of this book is to center qualitative approaches to mixed methods research. Typically, in mixed methods research discourse, quantitative approaches have primacy over qualitative ones. Qualitative approaches stem from a different research logic, one that privileges sub- jective experience and that is open to a multilayered view of the social world. In this book, we focus on the idea that centering a qualitative approach in mixed methods research can be illuminating, useful, and advantageous, especially as a means to get at subjugated knowledge— knowledge that has not been a part of mainstream research inquiry. A mixed methods approach also allows the researcher to get at “subjective experiences” of those researched while providing the means to test out theories generated from in-depth research samples.

A third objective is to provide researchers with a more detailed understanding of qualitative mixed methods perspectives and prac- tices. In addition, it is my hope that those researchers currently practicing mixed methods from a quantitative approach may dialogue with qualitative approaches by reflecting on the ways in which they might integrate a qualitative perspective into their mixed methods practice.

A final goal for this book is visionary and elusive. It is my wish that, in uncovering and centering qualitative approaches to mixed methods, we can open up a dialogue across current mixed methods research approaches and practices that may serve to fuel synergy and innovation in methods practice with the goal of providing a more complex view and understanding of the social world.

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

10 M I X E D M E T H O D S R E S E A R C H

a comprehensive Perspective on Mixed Methods

I argue that the current practice of mixed methods research is exem- plified by a “cart before the horse” approach. Mixed methods designs are driven by research techniques to the detriment of theory-based research. Figure 1.1 presents a “methods- centric” approach to mixed methods research that places methodology (theory) last in the process of choosing a particular mixed methods design. In this mixed meth- ods design (Figure 1.1), methodology (theory) is isolated from the rest of the research model and appears as the last in the design sequence. It is not clear how theory (whether it is implied or explicit) is linked to a specific methods design, but it appears as if the mixed methods design model is methods- centric in that the type of mixed method model selected drives the type of theory chosen.

In their review of trends and issues in mixed methods evalua- tion, Miller and Fredericks (2006) suggested that this misplacement of theory in a mixed methods project is a “problem of logic” that often plagues the choice of mixed methods approaches, especially in evalua- tion research. They noted:

FiGURe 1.1. Methods-centric approach to mixed methods research.

Methods (methods drive the research)

Data

Research Question

Methodology (theory)

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

Introduction to Mixed Methods Research 11

One can, of course, a priori, simply declare that any MM strategy is appropriate for the issue being studied. However, this thinking intro- duces an unwanted arbitrariness to the whole process, which poten- tially undermines the very purpose of advocating for the uniqueness of MM. The situation is further exacerbated by the use of vague rationales for the selection of a particular mixed method, suggesting that the use of MM will result in richer data or stronger inferences. (p. 569)

As I have stated, I believe that current practice of mixed methods research takes a “cart before the horse” approach. I propose, instead, a practice of mixed methods that is firmly rooted within a research con- text with the intention that the method or methods used foster a richer understanding of the research problem under investigation.

a comprehensive approach to Mixed Methods

Figure 1.2 provides a diagram of a comprehensive approach to research. The basic premise of the comprehensive approach is that methodol- ogy provides the theoretical perspective that links a research problem with a particular method or methods. Methodologies are derived from a researcher’s assumptions about the nature of existence (ontology). These assumptions, in turn, lead to their perspective philosophy or set of philosophies on the nature of knowledge building (epistemology) regarding such foundational questions as: Who can know? What can be known? We can think of methodology as a theoretical bridge that con- nects the research problem with the research method. Kushner (2002) underscored the importance of methodology in methods practice:

We cannot talk of “method” alone. To talk about the “interview” apart from its purpose is merely to picture two people engaged in ver- bal exchange. Only when we shift to the level of methodology where we talk about purpose and value does the instrument become suf- ficiently complex to sustain discussion. Method is like a glove which needs the human hand to give it shape and meaning. (p. 252)

Methodology leads the researcher to ask certain research ques- tions and prioritize what questions and issues are most important to study. Researchers within and across disciplines can hold a range of different methodologies that frame their methods practice. They might

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

12 M I X E D M E T H O D S R E S E A R C H

use methodologies that hold up the importance of studying the “lived experiences” of individuals (interpretative methodologies), those that privilege hypothesis testing and causality as the most important goals of social inquiry (positivist and postpositivist methodologies), or meth- odologies that stress issues of power, control, and social justice (trans- formative and critical methodologies).

A methodological perspective is not inherently quantitative or qual- itative in terms of its use of method. For example, those who practice more positivistic methodology— traditionally seen as quantitative—can use qualitative as well as quantitative methods. In fact, qualitative and quantitative methods are carried out within a range of methodologies (theoretical perspectives). Those espousing feminist methodologies— traditionally viewed as qualitative researchers—can use quantitative methods, including surveys and experiments, in their research. The

Ontology

Epistemology

Methodology

Research Problem

Mixed Methods Design

Paradigmatic Viewpoint

Economic factors

Serendipity

Stakeholder interests Review of literature

FiGURe 1.2. Comprehensive approach to mixed methods research.

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

.

Introduction to Mixed Methods Research 13

comprehensive perspective on mixed methods troubles the dualism between qualitative and quantitative methods when we remember that methods lie in the service of methodologies.

Jennifer Greene (2002) captured the important role that a meth- odological framework takes on in a research project:

Most . . . methodologies have preferences for particular methods, but methods gain meaning only from the methodologies that shape and guide their use. . . . An interview does not inherently respect the agency of individual human life; it only does so if guided by and implemented within a methodological framework that advances this stance. So, any discussions of mixing methods . . . must be discus- sions of mixing methodologies, and thus of the complex epistemo- logical and value-based issues that such an idea invokes. (p. 260)

Greene stated that methods are tools and that, in the hands of researchers with certain methodological persuasions, they can be used to promote social justice, maintain the status quo, or promote social transformation. There is a need, then, for researchers to be conscious of the methodological perspective(s) they employ within their research projects. Greene, Benjamin, and Goodyear (2001) referred to this as “thoughtful mixed method planning,” in which the researcher is cog- nizant of his or her particular methodological standpoint. They noted that each researcher should “figure out one’s stance on the ‘paradigm issues’ in mixed method enquiry” (pp. 29–30). Good mixed methods work requires “consciousness of this organizing framework and adher- ence to its guidance for enquiry practice” (p. 30).

It is also important to note, however, …