Analyzing the application of mixed method methodology in medical education: a qualitative study | BMC Medical Education

Our analysis identified three themes from the data which were distinct and unique, explaining the construct of the perceptions of using mixed methods in research related to medical education. These themes included:: understanding and perceptions of mixed methods in medical education research, strategies and integration in mixed methods implementation, and mastery in mixed methods: prerequisites and expert consultation in research. No modifications were provided by the participants’ feedback on the findings.

Theme 1: understanding and perceptions of mixed methods in medical education research

The participants offered a range of definitions, associating them with what they believed was comprised of mixed methods research. A few participants believed that the presence of quantitative and qualitative methods in a single study comprised of a mixed methods design. Most believed that it was a design of research that merges a quantitative part with a qualitative part. Others believed that since it is comprised of two components of quantitative and qualitative research, the qualitative research is an inductive part of the research while quantitative research is more of a deductive mode of inquiry; whereas the data that originates from both types of research is called mixed methods design. Some participants believed that the mixing of data during the analytic part consisted of mixed methods research. Most had a clear understanding of the composition of mixed methods research.

Mixed methods is the presence of quantitative and qualitative studies in a single study. 5-Female.

Usually, it involves mixing quantitative and qualitative methodologies in one research project. 12-Female.

It is a combination of exploratory and explanatory methods. It is transferring one piece of data to another or merging data together. 9-Male.

It’s when we analyze quantitative data, and we want to explore more data qualitatively.

7-Male.

Participants understood that they required a strong justification and a need to implement a mixed methods approach to answer their research questions. Some believed that when quantitative data were not sufficient to explain a certain phenomenon or construct, qualitative data can add to it. They believed that adding a qualitative component offered a better understanding of the answers to their research questions particularly inquiring about participants’ perceptions, feelings, and reflections about a particular topic. Others argued that using a mixed methods approach gives a strong backbone to the findings from the studies, especially when investigating a new concept. Others simply believed that conducting mixed methods research would increase their chances of publishing their research in higher ranked journals. Moreover, the participants were also aware that they could not use a mixed methods approach for just any type of research question, as this carries its own limitations in conducting research, including the factors of time, resources, target population, and settings.

Because in medical education, we cannot quantify everything in numbers since medical education has many aspects that have perceptions, emotions, feelings, and reflections to generalize it. 14-Female.

I also found that the mixed methods methodology is frequently used in medical education research and publications especially when you are studying a phenomenon and you want to see the impact of that phenomena. 12-Female.

It depends on research question, we shouldn’t use mixed methods methodology just because of the availability of both quantitative and qualitative data. 15-Male.

Theme 2: strategies and integration in mixed methods implementation

When inquired about if the participants clearly stated the objectives or the research questions being addressed by the mixed methods research, they were divided in their responses, where some clearly identified and stated their objectives while others did not.

I had three research questions to address the aim of the study, a quantitative, a qualitative and a mixed method methodology question. 12-Female.

One question is sufficient in mixed methods methodology to be answered by the quantitative and qualitative study. 14-Female.

Participants were sometimes advised to explore their research questions in more detail, at times more complex than what could be answered through a simplistic design, leading them to consider a mixed-methods research design. Some explained how they searched for literature on their topics, and from there they were inspired to employ a mixed methods design. A participant explained that a mixed methods approach was necessary in her case because she believed it added value to the reliability, validity, and generalizability of the data. Other participants used mixed methods research because they needed to investigate patterns in their samples that quantitative analysis alone could not answer, such as perceptions about a specific phenomenon.

It depends on research question, we shouldn’t use mixed methods methodology just because of the availability of both quantitative and qualitative data. 15-Male.

(I added) a qualitative study to get more in-depth data and which will explain what we are going to find in the quantitative data. I also started reading similar studies to see which method is best for such studies, and I found out that mixed methods methodology was the most suitable one. 12-Female.

I usually use mixed methods methodology studies when I need to explore perceptions, experiences, and challenges because first I must identify these challenges and experiences and then further explore them in-depth. 5-Female.

For most participants, the mixed methods approach was driven by the research questions that they formulated at the outset of planning their research. For some, it was part of the theoretical frameworks they wanted to use in their research, requiring a comprehensive approach. Others just found at a later stage of their research, to employ a mixed methods approach, whereas for others, the available data required them to use this approach.

So, my research questions in the qualitative part were guided by these components, like assignments, clinical evaluations, and exams. 12-Female.

So, it depends on the cognitive or theoretical framework that serves your purpose. 10-Male.

Most participants in our study acknowledged the need for clarity about using an appropriate study design at the outset. A participant said that if the design was wrong at the beginning, it would lead to poor data and half-answered research questions. Participants talked about considering how the data sets would be analyzed in a mixed methods design. They were particularly aware of the need for integration of the findings in the discussion part of their report.

If you have good quantitative and qualitative parts in your study, that doesn’t mean you will be able to mix the results together for a mixed methods methodology study. 10-Male.

The merging of results will also occur when I discuss the results in the discussion part of the study. 15-Male.

When asked about how they prioritized a component in their mixed methods design, participants talked about the way they selected the components. Most believed that both the components supported each other, but they had to consider the factors that could affect the decision about which part of the study could be started first, including the factors of feasibility and resource availability, starting with the quantitative to address the research questions. However, for studies for which participants were required to implement an exploratory design, they mostly started with a qualitative design followed by a quantitative one, which was supported by items from the preliminary analysis of qualitative data. However, a participant said that explanatory designs start with quantitative rather than qualitative designs. Some participants who did not employ a sequential procedure used both components simultaneously. Nevertheless, the participants were aware of the logical sequence of the components of the mixed methods research.

It is very important to prioritize data sets to select the appropriate design. 13- Male.

We still need to determine the time of data collection of one data set over the other even if the data sets have equal priority. 10-Male.

Despite of weather the design is sequential or concurrent, you must decide which data set you will start with, quantitative or qualitative and must give the rationale behind it. 6-Male.

We can prioritize the data in triangulation based on the phenomena that we have and what the relevant data and resources are that we will explore. 5-Female.

If they are not depending on each other then I will see other factors that could affect which one can be started first, such as feasibility and available resources to start with the quantitative to answer the first or second objective then I will do this. 1-Female.

Integration and making sense of the data originating from two different sources of data was another major challenge for the study participants. They talked about situations where they attempted to reduce the data, but simultaneously, they were afraid of losing the richness in their data, which they believed was essential to capture the nuances in the data. A participant talked about the importance of merging by saying that it must be considered at the outset when the data is being collected and later when analysis begins. Nevertheless, most participants, except some, valued that merging was one of the most important aspects of mixed-method studies, as it added more weight and credibility to the study. They were able to give examples of how they merged their data in the discussion sections of their reports. However, many participants believed that their understanding of the methods of merging in such studies was limited; this is the reason, for instance, that a participant explained that he drafted the findings from two sets of data separately within a single report without performing merging of results.

Then I will report it in the discussion part, the relation between both results. 12-Female.

The quantitative and qualitative studies should be linked together and merged to get the mixed methods methodology results. 10-Male.

We didn’t merge the quantitative and qualitative results. I kept them separately. 7-Male.

I stated the quantitative and qualitative results independently. 4-Male.

Data will be integrated in the results and in the discussion parts.

The study should have quantitative results, qualitative results and mixed methods methodology results. 12-Female.

I integrated the data in the results part and these merged results were discussed in the discussion part. 3-Female.

I had no specific technique to present the MMM results. 2-Male.

Yes. I did a table that shows my results for qualitative and quantitative studies and the mixed methods methodology results. 3-Female.

I displayed my mixed methods methodology results using tables. I had to interpret some words into numbers.

4-Male.

Technique to merge data depends on the design used and depends on the researcher. 5-Female.

Theme 3: mastery in mixed methods: prerequisites and expert consultation in research

The participants talked about several issues pertaining to the prerequisites for conducting a mixed methods study. Experience in qualitative research techniques was valued as crucial prerequisites for a researcher’s readiness to use a mixed methods research approach. For the participants, resources, and feasibility, apart from a strong rationale, some experience, and knowledge, were one of the main prerequisites for conducting the qualitative component of the mixed methods research.

The qualitative section specifies the type of qualitative method you will employ focus groups, interviews, formal or informal interviews, and whether you have sufficient manpower resources to conduct this interview. 1-Female.

You must have experience of conducting both quantitative and qualitative studies and good exposure to have a well-structured mixed methods methodology study. 5-Female.

Participants also talked about how using mixed methods research was a challenge for them and how they upgraded their skills to accomplish their complex projects. It required some to do extra reading before they initiated their projects. For many, it meant getting upskilled in qualitative research methods, as they already had adequate training in quantitative skills. In doing so, they believed that access to resources and resourceful people who could impart qualitative skills in their institutions was essential to their projects. Some participants, therefore, indicated the need for formal training in mixed methods research and qualitative techniques. This is because they believed that these types of studies needed to be designed perfectly, since if the design is wrong at the beginning, the data may not be rich enough to answer the research questions.

You must have enough knowledge about the mixed methods methodology studies before doing one. 5-Female.

I personally took workshops in conducting qualitative studies because I only had a background in quantitative studies. 3-Female.

You must have training in mixed methods methodology and exposure to conduct mixed studies. 5-Female.

It is very important to have an expert in mixed methods methodology even if we have experienced researchers in quantitative and qualitative studies. 13-Male.

Participants discussed how important it was to consult an expert with a mixed research methods background, especially at the start. They believed it would enhance the quality of the study because, even if the researcher has a good background in both quantitative and qualitative studies, they still required guidance throughout the study in analysis, writing results, and discussion because it is different than quantitative and qualitative studies. The participants insisted that even if a researcher has a background in both quantitative and qualitative research, it is always better to consult an expert with experience in mixed methods research.

It is very important to have an expert in the field of mixed methods methodology because he will be extremely helpful in integrating the data together. 15-Male.

I think we need experts in mixed methods methodology, even if the researcher has a good background in both quantitative and qualitative studies because you will need them to guide you through the study in analysis part, writing results and discussion parts because it is different than quantitative and qualitative studies. 12-Female.

They did, however, believe that it was difficult to find qualitative researchers who were knowledgeable about advising on mixed methods research. They argued that the supervisors should be aware of the mixed methods design so that they are able to advise on the appropriateness of conducting studies with advanced study designs. Some believed that, ideally, the researcher should have a supervisor with a sound background in mixed methods research because it is not enough that the researcher has either a background in quantitative or qualitative research alone. Whereas others thought that specialist mixed methods researchers may not be needed in situations where a person who is expert equally in quantitative and qualitative approaches.

It is better to have an expert in mixed methods methodology look at the themes and see how you will analyze your data. 3-Female.

We don’t need to involve a mixed methods methodology expert while doing the research if we have researchers with good experience in both quantitative and qualitative studies. 14-Female.