Qualitative methods in PhD theses from general practice in Scandinavia.

Qualitative methodology is gaining increasing attention and esteem in medical research, with general practice research taking a lead. With these methods, human and social interaction and meaning can be explored and shared by systematic interpretation of text from talk, observation or video. Qualitative studies are often included in Ph.D. theses from general practice in Scandinavia. Still, the Ph.D. programs across nations and institutions offer only limited training in qualitative methods. In this opinion article, we draw upon our observations and experiences, unpacking and reflecting upon values and challenges at stake when qualitative studies are included in Ph.D. theses. Hypotheses to explain these observations are presented, followed by suggestions for standards of evaluation and improvement of Ph.D. programs. The authors conclude that multimethod Ph.D. theses should be encouraged in general practice research, in order to offer future researchers an appropriate toolbox.

Qualitative methods in general practice research

Qualitative research methods for the interpretation and analysis of texts already existing or transcribed from talk, observation or video can be used to explore meanings of social and bodily phenomena as how and why human beings act as they do, within their natural context [1]. Qualitative studies explain why promising clinical interventions do not always work in the real world, how patients experience care, how practitioners think or how the complex relations between the healthcare system and the outside world are working [2]. The general practitioner (GP) meets people over time in their social environments, often with undifferentiated symptoms, chronic disease or multimorbidity [3]. Clinical knowledge beyond measures and numbers is necessary to understand not only the diseases, but also the patients with their suffering, strengths and coping [4]. It is no surprise that general practice researchers have played important roles to lead the way for qualitative methods in medicine.

The authors of the present article have participated in the development of qualitative methods within Scandinavian general practice research, achieving experiences and presenting arguments about adequacy and proficiency of methods [1,5-10]. Sharing an interest for education, implementation and scientific standards for qualitative methods in general practice research, we have extensive experience as supervisors as well as evaluators of Ph.D. theses with qualitative studies.

Over years, we have noticed different customs for writing and evaluating Ph.D.-theses within our academic field. Ph.D. practices signify disciplinary norms, which establish and consolidate scholarly standards, often subtly and implicitly. Embedded in official evaluation systems, attitudes institutionalised by such practices have a strong impact on how medical knowledge is constructed. Empirical data about these issues are not easily available. In this opinion article, we draw upon our observations and experiences, unpacking and reflecting upon values and challenges at stake when qualitative studies are included in PhD theses. We restrict our exploration to general practice research in Scandinavia.

Institutional framework and evaluation procedures

A Ph.D. is the final documentation from a training program intended to qualify academics for postdoctoral research, academic supervision and permanent tenure. For PhDs from general practice research, similarities across the three Scandinavian countries (Norway, Sweden and Denmark) are more prominent than differences. The estimated time is three to four years of full-time work, including an educational program and a thesis, which finally is to be evaluated and defended. One or more supervisors guide the candidate throughout the Ph.D. period.

Candidates apply for and are admitted to the Ph.D. program. Educational programs and requirements, differing between institutions and across nations, consist of a minor mandatory curriculum and elective elements. All Ph.D. students attend courses about responsible conduct of research. Modules of research ethics, philosophy of science and biostatistics methods are often components of the program. When qualitative methodology is offered, training is often voluntary and basic, with advanced courses as electives. A Ph.D. thesis is currently typically based on three articles (quantitative studies, qualitative studies or both), published in or submitted to international peer-reviewed journals, and a synopsis with overall presentations and discussions.

The candidate’s progress is evaluated during the program. Intermediate assessment is conducted in all three countries, but formal procedures and content vary across borders and institutions. In Norway and Sweden, a mandatory mid-seminar is conducted, while in Denmark the main supervisor is responsible for a formal assessment, which can include a report with presentation and discussion of the process.

When the thesis is submitted to the university, procedures for assessment differ across countries. In Norway and Denmark, the evaluation committee presents a written statement concluding whether the thesis deserves a public defence. A negative conclusion is usually followed by an invitation to resubmit. In Sweden, some universities conduct pre-assessments, while others leave the whole evaluation to the defence.

After the formal, public defence, the evaluation committee takes the formal decision of acceptance. In Sweden, an opponent who is not a member of the committee exposes the thesis and the candidate, after which the committee makes its final verdict. Evaluation committees and opponents for Ph.D. theses with qualitative studies often include scholars from sociology, anthropology, psychology, linguistics, nursing or philosophy, in addition to general practice researchers.

Acceptable scientific quality of Ph.D. theses – a matter of variation?

Detailed procedures for assessment of the scientific quality of Ph.D. theses differ across Scandinavian medical schools. We have previously described how the role of the opponents and committee is different in Sweden compared to Norway and Denmark. Furthermore, procedures when a thesis is not accepted differ for example even between the universities in Oslo as compared to Bergen. Summarizing our own experiences as members of Ph.D. committees, we suggest that a thesis of good scientific quality, independent of research method, demonstrates consistency between well-defined overall aim and study objectives, as related to design, material, method and results. Distinct presentation of relevant findings is appreciated. Finally, we regard a critical and focused presentation, interpretation and discussion of strengths and limitations of methods and findings, as essential indications of quality,

We have noticed substantial variations in pre-assessments and evaluation from committees for theses where qualitative studies are involved. A comparable level of uncertainty does, to our knowledge, not appear in evaluation of theses based on quantitative studies alone. This is probably because quantitative methods traditionally represent the state of the art within medical research, leading to a more standardized approach to conduct as well as assessment.

Inconsistent practice leads to unpredictability for candidates and supervisors, especially when a thesis is not accepted for defence. According to the rules, reasons for rejection shall be explicated and seem to be very diverse, as are also the type and level of objections required for rejection. Admittedly, some of these theses may be only just sufficiently passable. We have, however, also noticed rejection of theses holding acceptable methodological quality, and even theses with brave and challenging qualitative analysis and theoretical discussions have been rejected. At the same time, qualitative (and quantitative) theses of a much more traditional academic format are being accepted.

Challenges for qualitative research in the Ph.D. programs

As GP researchers, we have long been concerned by the biases of the Ph.D. programs as training for future research, tending to encourage a confined quantitative focus of inquiry. Although multimethod Ph.D. theses are quite frequent within general practice research compared to other fields, Ph.D. programs do not encourage candidates toward multimethod competence. Furthermore, some scholars and institutions argue that only mono-method theses – qualitative studies only, or quantitative studies only – allow candidates the necessary depth to learn the tools of the trade.

However, general practice needs researchers with skills and awareness regarding qualitative as well as quantitative methods, to choose the most appropriate design and tool. We therefore propose a shift, encouraging multimethod competence in Ph.D. programs and theses. The shift implies substantial discussions about methodological skills required for a Ph.D., as well as the assessment of PhD theses with qualitative studies from general practice. A minimum requirement for a thesis including a qualitative study should be a supervisor holding qualitative skills and experience. This is a better alternative than leaving qualitative studies to social scientists, which some medical academic environments do.

A hypothesis to explain the shifting evaluation standards described above is that the interdisciplinarity of evaluation committees imposes methodological standards from different disciplines beyond medicine. General practice calls for knowledge intended for implementation with patients and GPs in a foreseeable future – a goal that may divert from the analytic and theoretical pursuits of a researcher from the humanities or social sciences. Furthermore, academic pursuits are intended to increase the understanding of the discipline itself by developing theories and challenging knowledge that is taken for granted [11]. Hence, persistent attention to and respect for the connections between ontology and epistemology within the distinctive domain where research is supposed to contribute, is necessary for adequate theoretical endeavours [10]. Personally, we are involved in several interdisciplinary collaborations, appreciating the impact of such for the development of health care systems and sustainable knowledge [12]. Yet, we recommend caution when choosing evaluators from other disciplines. Has the potential evaluator sufficient knowledge and understanding of the general practice context to conduct an assessment that recognizes the nature and standards of this academic field?

Sometimes, we suggest, negative assessments or rejections indicate that committee members have misunderstood their assignment. Some committees seem to overwork their evaluation, highlighting their own ideas and views instead of trying to understand and evaluate what the candidate has achieved. The evaluation of a Ph.D. thesis should not be the arena for promoting personal idiosyncrasies such as positions in ongoing methodological debates or omission of certain ‘mandatory’ references. Sometimes, the evaluation committee returns the thesis to the candidate, suggesting minor or major revisions. Such responses, comparable to the dialogue format expected by peer reviewers of a journal, might improve the quality, but neglect the question of whether the thesis merits the standard of deserving a public defence; in the version, it was submitted. Clear evaluation instructions contribute to prevention of such pitfalls.

Developing qualitative research through Ph.D. assessment

We suggest that particular criteria must be added in evaluation of Ph.D. theses incorporating qualitative studies. Basic criteria include demonstration of reflexivity [13], detailed presentation of the analysis process, a qualified discussion of validity as well as information power of the data. Appropriate application of theory for analysis and a well-written account of relevant findings are also important. Such aspects can better be elaborated in the synopsis than in the brief format of the articles. Consequently, the synopsis will be the most important issue for assessment.

In our opinion, the synopsis of a first-class thesis includes a focused discussion of relevant methodical challenges as well as a theory-supported synthesis of findings from all the included articles. A decent thesis may, however, be assessed as acceptable without being outstanding. After all, the Ph.D. represents the endpoint of systematic research training, and not the level of a Nobel Prize. A weak thesis is often characterized by too broadly defined aims, leading to excessive amounts of empirical data. The subsequent superficial analysis leads to descriptive presentations of trivial phenomena well known in advance, sometimes to be confused with the candidate’s preconceptions. Discussion of data quality or validity is often neglected, thereby undermining the trustworthiness of the results. Other recurrent flaws are lacking understanding of philosophy of science underlying the inductive logic of the interpretative paradigm, indicated for example by apologies for a sample perceived as small by quantitative standards, detailed though general explanations for the limited generalizability of qualitative studies, missing discussions about external validity beyond the population level, or subjectivity perceived as bias to be avoided. Theses demonstrating lack of reflexivity by omitting considerations about the role of the researcher are also seen.

Too often, theses with qualitative studies are overloaded with undigested grandiloquent theory and philosophy, lacking a clear connection to the aims and findings. On the other hand, superficial or textbook-like general discussions of strengths and weaknesses without specific connection to the actual project indicate an inadequate academic level. Studies presented as mixed methods seem to be especially vulnerable, with candidates not always distinguishing sufficiently clear between the methods they have used and their epistemological foundations.

Implications for academic practice

The last decades, qualitative methodology has earned increasing recognition in medical research, but is still often ignored in Ph.D. programs and by academic supervisors. Based on reflections upon observations and experiences, we have in this opinion paper suggested that substantial and unpredictable variation regarding evaluation of theses with qualitative studies indicates a lack of consensus regarding scientific quality within this field. There is no reason to endorse acceptance of Ph.D. theses of inferior quality based on the excuse that qualitative methods are still young and underdeveloped. Such practices will lead to later substandard research, supervision and teaching and jeopardize the reputation of qualitative research methods. A minimum requirement for a Ph.D. project incorporating qualitative studies is a supervisor with adequate methodological skills. Future GP researchers need more than one tool in their toolbox to design a study with sufficient validity for the broad range of study questions in the field of general practice.