Reviewing Relevant Phd Dissertations

The following reviews were selected based on their relevance to evaluation, youth, evaluative / critical thinking and potential for application to my future research design.

Using Empowerment Evaluation with Youth by Sarah Heath (Heath, 2021)from the University of Ottawa examines youth participation and engagement in an empowerment evaluation setting. This approach is unique as it aims to both conduct an evaluation while also involving program stakeholders and beneficiaries in the process, which results in enhanced skills to conduct their own evaluations.

The dissertation is structured to reflect three papers that each provide detail on sample, instruments, data collection and analysis procedures, findings and summary. The author explains a conceptual framework, which draws on a review of literature, including theories of program evaluation, collaborative and participatory evaluation literature that focuses on youth and research literature on empowerment evaluation.

The research involved a two-phase, mixed methods case study approach; the first involved a survey of targeted interest groups from the American Evaluation Association to learn more about evaluator experiences with empowerment evaluation and their experiences with youth, which informed interview guides that would be implemented through interviews with the evaluators; the second involved engagement with students participating in the University of Ottawa’s Faculty of Engineering science, technology, engineering and math program and their participation in an empowerment evaluation exercise and interviews with them therafter.

The findings indicated that most evaluators who conducted evaluations of programs involving youth do not use empowerment evaluation, but instead preferred outcome evaluations or theory driven approaches. This was due to a lack of stakeholder interest or training and instruction on empowerment evaluation. Evaluators, therefore had low confidence in their abilities to carry out an empowerment evaluation targeting youth. The author also observed that when empowerment evaluation is used, stakeholders generally seem to be deeply involved. Interestingly, evaluator perceptions of their own skills were seen to either facilitate or hinder the use of empowerment evaluation, depending their experiences, knowledge and professional training. Students felt that empowerment evaluation was unique through its challenging of their perceptions of program evaluation and that their voice mattered. The time it takes to lead and participate in an evaluation were seen as a limitation by both evaluators and students alike, which may also influence the degree of empowerment felt throughout the process.

Future research could gather input from evaluators outside of North America or under a more general guise to broaden data collection. Longitudinal studies of youth involvement in empowerment evaluation is a worthy endeavour and one that students expressed interest in particiating. Evidence suggested that regular interactions with an evaluator could encourage youth participation, but questionable if it could be sustained. On a longer term basis, the author suggested that a multiple event case study of an empowerment evaluation would be interesting and likewise, an assessment of longer term outcomes for youth that are involved in empowerment evaluation.

__________________

Improving Use through Evaluation Participant Engagement by Jennifer L Hughes (Hughes, 2024) from Queen’s University examines the beliefs, strategies and practices of evaluation scholars, practitioners and students affiliated with an evaluation network concerning Evaluation Participant Engagement (EPE).

This study is an exploratory multiple method qualitative bounded case study across three phases, consisting of 1) focus groups with scholars, practitioners and students, 2) nested case study of an in-progress evaluation based on the all in for Youth (AIFY) program in Alberta and 3) a data party with participants from 1 and 2 (a data party, the author explains, is a limited-time collaboration with individuals to jointly engage with each other and the data from a study to respond to emerging findings) involving deductive analysis. Although the dissertation contains three phases of the study, it is structured in a narrative format.

The findings reflected positive influences on EPE by purposeful acts of relationship building, which was also connected to the themes of contextual understanding and enhanced capacity to participate in evaluations, and think evaluatively. Relationship building included references to purposeful engagement, transparency and trust. The analysis showed that the positive interactions between relationship building, EPE and use could contribute to a connection between relationships and the extent of use.

The author suggests future research could exploring evaluation practice partnerships (EPP) to create capacity over time, based on stronger, more trusting relationships that lead to improved engagement and use. Likewise, future research could build on these findings that show a connection between culturally responsive and equity focused evaluations to EPE.

Limitations of this research should focus on its very small sample sizes, a single case study design and navigating the Covid-19 pandemic.

______________

Exploring Students’ Perspectives on Critical Thinking Integration in College Pre-Health Science Education by Bernard Ho (Ho, 2020) from the Ontario Institute for Studies in Education at the University of Toronto, is an interesting dissertation as he writes as a practitioner, teaching health sciences to students and is motivated to conduct the research based on his own observations and desire to improve education for his students.

This research uses a case study of a single cohort to address three research questions focusing critical thinking, its role in pre-health science courses and the challenges in integrating critical thinking in to the course design. Using a constructivist/interpretivist framework and methodological approach, the dissertation is structured according to the three research questions, which are bookended by a literature, methodology and implications section. The Methods used in this study included surveys, classroom observations and interviews and the data generated from pre-health science students (n=26) was analysed by conducting an inductive-deductive content analysis.

Findings showed that an elusive definition of critical thinking persists with some common themes. As well, student data revealed that their perceptions of importance was stronger in regards to the content of their health science education than the notion of applying critical thinking within it. Observationally, the author discusses the challenges of applying critical thinking within a healthcare setting and a culture which focuses on scientific language and process. The author suggests that critical thinking integration health science education can be done with some creativity and alignment with what is already in operation and within the scope of scientific culture, such as applying critical thinking concepts during labs.

The author discusses at length the implications of the research on health care education and subsequent practice and deviates from other scholars and expresses his ideas for future research to a lesser extent in advancing scholarly literature and more in the context of his own experimentation and motivation to encourage critical thinking application in health science education and advance his own practical ideas, such as modifications to curricula.