Transformative Evaluation case study: Evaluation of Student Colleagues scheme

Achtaridou, Elpida and Winter, Jennie (2023) Transformative Evaluation case study: Evaluation of Student Colleagues scheme. [Report]

Preview

Text Transformative Evaluation case study Evaluation of Student Colleagues scheme.pdf - Published Version Available under License Creative Commons Attribution Non-commercial. Download (278kB) | Preview

Abstract

Programme Context A small institution, Plymouth Marjon University (PMU) was granted degree-awarding powers in 2013. The Student Colleague (SC) scheme was developed in 2019 in the context of the university’s Access and Participation Plan (APP), which identified gaps in student employability outcomes for students in POLAR4 Q1 and 2 compared to Q3-5 and students with and without a declared disability. Acknowledging that a student’s access to quality employability experiences can be limited, particularly in cases of disadvantage and/or disability, the scheme offers a range of university-based, paid graduate-level employment opportunities open to student applications. Evaluation Context At the time of the study, the SC scheme’s implementation was in flux; efforts to improve the scheme meant that revisions, changes and additional activities were either in progress or planned. Further, not all of the stakeholders contributed to the development of the initial Theory of Change. The project team chose to use Transformative Evaluation (TE), a fork of the Most Significant Change (MSC) methodology, to evaluate the impact of the programme. TE is a qualitative evaluation method that examines practice in its natural settings to make sense of the outcomes and processes in terms of the meanings people bring to it (Cooper, 2017). TE has specific qualities that the project team felt made it appropriate to scrutinise, question, and reinforce the existing Theory of Change and to explore the programme outcomes: • TE uses story generation to expose the experiences recipients have of a practice. These stories become the subject of a series of critically reflective activities that include relevant stakeholders who focus on the learning that takes place for those stakeholders and the organisations they are part of. This enables evaluators to identify important processes, relationships, and events that influence the success of the scheme, but are hard to quantify and are poorly captured using traditional evaluation techniques. • TE offers a transparent, inclusive, and democratic way to generate, analyse, and respond to stories, and promotes collaborative accountability by exploring the issues surfaced by stories within a shared and structured format. These qualities suited this evaluation as logic chains between the intervention and target metrics were yet to be fully developed. TE supported this articulation and helped identify which aspects of the scheme led to intended outcomes of interest. • TE enables practitioners to become evaluators and support their professional development through hands-on experience and reflective practice. Engaging more staff in research and evaluation activities is one of PMU’s strategic aims. • TE brings to the fore voices of marginalized and underrepresented groups. This was particularly relevant to the target population of this evaluation, which included students who are underrepresented in HE. Although TE is related to Most Significant Change methodology, the process and series of steps vary. MSC is often used in large-scale interventions, such as international development. As used here, TE is well adapted to smaller-scale projects. For a useful comparison between TE and MSC see the PMU local evaluation report, p.19.

Item Type:	Report
Depositing User:	Ms Raisa Burton
Date Deposited:	13 Mar 2024 09:51
Last Modified:	13 Mar 2024 09:51
URI:	https://marjon.repository.guildhe.ac.uk/id/eprint/17805
Related URLs:	https://cdn.taso.org.uk/ (Publisher URL)

Actions (login required)

Edit Item