Challenges of Mixed-Method Research

Jo Weaver

Jo Weaver

Reposted from Anthropology News April 2015 column.

Mixed-method research involves inherent challenges that make it at once more gratifying and more difficult than traditional single-method approaches. By “mixed-method,” I am referring to studies that employ a mixture of qualitative and quantitative methods. This approach is a hallmark of most biocultural research, and those of us committed to this approach believe that the triangulation of multiple methods is a more effective way of capturing human experience than an approach that attempts to represent only quantitative trends or only qualitative individual experience. Mixed methods also have the potential to make our work more intelligible to those outside of anthropology who transact primarily in the quantitative—those, for instance, in public health, psychiatry, or sociology.

Mixed-method studies are fundamentally challenging because they often take twice the work and require methodological expertise in multiple areas: Instead of just conducting an epidemiological survey to learn about the spread and correlates of a disease in a given sample or only conducting illness narrative interviews to learn about individuals’ experiences with a disease, a biocultural researcher is likely to be doing both of these. This requires a fair amount of time, money, training, and logistic agility.

I am certainly not the first to point out the complications involved in mixed-method research (see Bill Dressler’s 5 Things You Need to Know About Statistics: Quantification in Ethnographic Research from Left Coast Press). But the complication continues after the research is done, and these days, I’m finding the post-fieldwork integration of quantitative and qualitative data more difficult than the execution of research itself. How do we combine all those mixed-method data together into a coherent form that accurately represents human experience?

Let me give an example. At the recent March 2015 Society for Applied Anthropology meeting in Pittsburgh, I organized and participated in a session called “Food insecurity and mental health in global perspective.” The purpose was to bring together scholars who are studying the relationships between food insecurity and mental health and to move toward a unified research agenda that might help us identify some of the social pathways that link these two states in widely different parts of the world. This kind of comparative enterprise obviously requires that there be some standardization in the methods used to measure important outcome variables like food insecurity and mental health across locations. Accordingly, most of us assessed mental health through a standard scale like the Hopkins Symptoms Checklist-25 (HSCL-25) or the Center for Epidemiological Studies-Depression (CES-D) scale. While these have been validated for use in many cultural contexts (including those in which we work), they nevertheless reduce a profound experience of human suffering—depression—to a number.

Dr. Steven Schensul, an applied anthropologist with many years of experience in mixed-method research who attended the session, pointed out the relative lack of attention that each presenter gave to mental health. And he was right—most of us did little more in our 15-minute presentations than name the depression assessment scale we were using before moving on. As he reminded us, there is a whole branch of anthropology, psychological anthropology, dedicated to questioning, problematizing, and pluralizing psychiatric diagnostic categories. And indeed, many of those presenting at the session have an arm of our own research dedicated to just this (for instance, see my and Bonnie Kaiser’s recent article in Field Methods, where we suggest an approach to measuring mental health that employs standard scales to appeal to those who need numbers but also develops locally-derived and ethnography-based ways of measuring mental health in a context specific fashion). My response at the time was to say that we as a group are indeed aware of this limitation and to point to some of the more nuanced mental health work we have done in other contexts.

Making a mixed-method study happen is inherently challenging because it often takes twice the work and requires methodological expertise in multiple areas.

Afterward, I kept wondering, if we are all in fact sensitive to the potentially problematic nature of some of the measures we use, then why didn’t we find time to address that in our presentations? And I kept coming back up against the idea that one can only do so much. I don’t mean that as a defense of my research’s shortcomings, but rather to say that it’s a resounding theme in my own experiences of working and writing at the intersection of qualitative and quantitative social sciences. One can only do so much: in 15 minutes, in a single paper, in a single book, in a single study, with that amount of money, in that time frame, with that word count. In a session devoted to the relationships between food insecurity and mental health, then, perhaps it’s not surprising that none of us dwelled on the methods we were using to measure either one—unsurprising, but not necessarily best practices, either.

Now, to get back to my original point, I think these realistic limitations of academic presenting and publishing are part of the reason why I find it so challenging to assemble the qualitative and quantitative data I’m collecting. Human experience is hard to chunk into measurable quantities, single conversations, a 15-minute presentation, or even an article-length manuscript. This is something that all anthropologists struggle with, and it brings up some of the fundamental issues of social science—things like, how do we make our work “speak” to as wide an audience as possible? How do we know that we’re measuring what we think we’re measuring? How do we represent the people we study with fidelity and ethics? How do we even know what their reality is? How do we claim some authority to knowledge about the people we are studying without overstating the case?

In other words, the challenges of biocultural anthropology are the challenges of anthropology in general. We can’t capture it all. But that doesn’t mean we shouldn’t try.

What’s Biological about Biocultural Research? (Part 1)

Reposted from Anthropology News February 2015 column.

Jason DeCaro

Jason DeCaro

Our January column from Bill Dressler harkened to 2005 when, concerned about the absence of an explicit theory of culture in much biocultural research, Bill had written a piece in Ethos entitled “What’s Cultural about Biocultural Research?” While not all of us follow Bill’s approach to the letter, his perspective has been influential in our Biocultural Medical Anthropology PhD program.

One could ask a parallel question: What’s biological about biocultural research? I attend theHuman Biology Association (HBA) meetings on a near-annual basis and encounter more researchers there who consider themselves biocultural than at the American Anthropological Association (as a percentage and a raw number, despite the fact that HBA is much smaller). Nearly all human biologists consider their work biocultural whether or not it hews to Bill’s definition, because human evolution is irreducibly biocultural, and human biologists are interested in that. And, being human biologists, they rarely feel the need to prove that they’re biological enough.

Yet the Biocultural Medical Anthropology program at UA historically has followed a different pathway. If one were to put my doctoral students to date into a box, for instance, it would be labeled psychocultural in caps, with biological in lowercase. Transacting across such boundaries creates a wonderful environment in which someone like me, whose scholarly roots are in biocultural human biology, can advise great students whose topics include HIV-related cultural models in adolescents, Our Lady of Guadalupe and the stress of migration, and how food security is reinforced or mitigated through social networks. This has been a positive experience for all of us, frequently sans biomarkers or explicitly evolutionary hypotheses. (Our program’s very strong tradition of group mentorship and co-advising has a lot to do with this too.)
The inclusion of biomarkers does not by itself make a study biocultural any more than the inclusion of a question about someone’s ancestry or “traditional foods and dress” makes it biocultural.

In short, as I see our students develop, a question that I continually turn over in my mind is: what does it mean to refer to biology here? First, let’s do some debunking.

  • Biocultural research is not necessarily about biomarkers. I have seen biomarkers thrown into studies for their own sake with no significant development of biocultural theory. I have seen them thrown into studies because it was fashionable. I have seem them thrown into studies because they are falsely understood as a shortcut that circumvents all the challenges of assessing stress or some other nebulous concept, by observing and talking to people. The inclusion of biomarkers does not by itself make a study biocultural any more than the inclusion of a question about someone’s ancestry or traditional foods and dress makes it biocultural.

    The inclusion of biomarkers does not by itself make a study biocultural any more than the inclusion of a question about someone’s ancestry or “traditional foods and dress” makes it bioculturalBiomarkers are a method, a tool. They are only as useful as the study design and underlying theory make them. If they don’t make sense in a given study design and are not called for by theory, they should be omitted. They’re neither a sufficient element to constitute a project as biocultural, nor are they necessary. Consider Daniel Lende’s (USF) work in substance use in Colombia or most of the other recent work in neuroanthropology. Not a biomarker to be found in many cases—but careful is paid attention to underlying neurological mechanisms that influence and are shaped by subjective experience.

    Our Lady of Guadalupe can be a focus of biocultural research (Altar in a market of Mexico City by ProtoplasmaKid. CC BY-SA 4.0).

    Our Lady of Guadalupe can be a focus of biocultural research (Altar in a market of Mexico City by ProtoplasmaKid. CC BY-SA 4.0).

  • Biocultural research is not necessarily about genetics (or genomics). This is an extension of an older false equivalence: biology = genes. Biological phenotypes are complexly determined through multi-level interactions among genes, developmental systems, physical and social/cultural environments. The “biology = genes” fallacy is common in discussions with a “nature vs. nurture” tone. Such discussions often take the form of “Is X trait/phenotype biological or cultural,” where “biological” = “genetic” and “cultural” = “anything remotely social or experiential.” In such discussions, the developmental systems that actually produce the phenotype are ignored. Genotypes can be quite helpful in biocultural research, depending on the question, but they also are neither necessary nor sufficient.
  • Biocultural research is not limited to work drawing from evolutionary theory. This one is perhaps more controversial among researchers in my circles, most of whom approach their work from the standpoint of evolutionary theory and hypotheses drawn from it. Much of my work does as well.  Yet it is not necessary to chart the evolutionary history of a physiological system to usefully study it. Again, sometimes developmental theory is more helpful. Sometimes stress theory. Sometimes eco-cultural theory. Denying the importance of the evolution of these systems to their current variation and functioning is a non-starter, but it’s sometimes useful to study the current endpoint of a long process of evolutionary change without explicit reference to how and why we got there. Conversely, there’s danger of evolutionary tokenism. Mentioning Paleolithic diets does not a biocultural study make (even when it’s not blatant misrepresentation or over-interpretation of the Paleo-human data).

So far, I’ve outlined negatives but no positives. If the biological in biocultural research is not per se about biomarkers, genetics, or evolution, then what is the biology?

In our next post, I’ll explore this question in more detail, making the argument that biocultural research is about integrative transactions across theoretical frameworks combined with methodological opportunism. Those methods are chosen from an interdisciplinary toolkit to fit specific hypotheses and research questions rather than a pre-set, unchanging, conventional inventory. I’ll argue that how we operationalize human biology is less important than how we understand it to work. I’ll outline the value of developmental perspectives, without claiming all biocultural research must lean on human development, and the importance of measurable outcomes with biological implications, even when biology is not directly measured to achieve this.