The past months have seen increasing talks about efficiency in higher education (HE). This is an uncomfortable topic for universities, not something academics are used to considering on a daily basis. And still, we may have to. As a political scientist turned educationalist, it is quite painful to observe how political decisions negatively impact the higher education landscape, often attacking its foundations and core principles. And this is happening across the world, at a different scale and in different ways. There is one question that I keep returning to, and it surprises me how difficult it is to answer it, even after decades of working in HE. How do we measure the quality of education? What is good teaching? And more broadly, what is good quality education? How can we measure it in a proper way?
A lot of people in academia, especially outside STEM (Science, technology, engineering, and mathematics), are reluctant to use any kind of metrics. After all, education is not (only) about numbers, there are so many nuances that are very hard, if not impossible, to measure. This may well be one of the reasons universities have a difficult time drawing up a strong stance on the efficiency discussion. If we cannot really prove with hard evidence (including numbers!) that the methods we use are effective and well suited, we are making ourselves vulnerable to the internal and external forces that are pushing for “education on a tight budget”.
Some ways of measuring quality
There are many formal ways of measuring - and indeed, ensuring - the quality of education. Accreditation criteria, for instance, are aimed at evaluating whether the education provided by a HE institution lives up to the formal quality standards established at national or international level. Practically speaking, what this comes down to is a set of often very abstract and rigid criteria and checkboxes that universities have to fill in on a regular basis. What is labelled as a quality assurance mechanism all too often ends up being a mere bureaucratic exercise (and a costly one, at that), very far removed from actually measuring and safeguarding the quality of education.
Then we have the university rankings. Highly visible and flashy, their purpose is to drive up enrolments, in a highly competitive HE landscape. Based on sometimes doubtful metrics, they paint a superficial picture of quality assessment, often biased and highly relative. Again, very little genuine quality evaluation at the core, mostly replaced by marketing efforts to find the best packaging. Are these accurate ways of measuring? What are we missing?
What is quality education?
For me, as educator and faculty developer, quality education means, first and foremost, a coherent, meaningful student experience. This does not only translate in high quality courses, but also in aligned, consolidated curricula, personalised, student-centred learning and assessment spaces (like ePortfolios, Problem-Based Learning, Project-Based Learning) and collaborative learning opportunities. Quality also entails an inclusive learning space-seriously thinking about who our students are, what they bring to the classroom and what they need. Quality education needs a supportive learning environment- making the appropriate resources and infrastructure available to students and educators. I am talking here about various services such as career counselling, student support, Centres for Teaching and Learning (CTLs) and IT systems that support the student journey. Often underrated, they are crucial to student success. Another prerequisite for quality education is having educators that feel empowered and supported to enhance their teaching, and more importantly, an environment where educators regularly talk about teaching, ideally as much and as often as they talk about their research. And, very importantly, quality education means education that reaches and impacts different target audiences. This means looking beyond our traditional students, descending from the “Ivory Tower”, and trying to engage more broadly and thus respond to different societal demands.
The current dilemma
As universities want to thrive in a highly competitive environment, they strive to provide excellence in education. The calls for efficiency force us to think where we can compromise and what is non-negotiable, where compromising would actually mean we are not standing by our educational principles and quality standards. It is precisely to answer this question that we need a good understanding of, and possible ways of measuring, what is excellent education, what is high quality education and what is good enough education (the minimum we can accept). And, of course, what resources go into each of them. Only when we have some of these answers can we draw the line and make some informed decisions. This is a process that takes time and strategic thinking, although recently university leaders have been under pressure to take quick decisions, foregoing at least parts of this process. The key question is how far can we go in order to preserve the core principles while being mindful of the costs? Using educational technologies can, for instance, contribute to a more flexible and inclusive learning environment, but we need to seriously consider to what extent and in which ways technology can support learning and not see it simply as a cost-cutting exercise. When implemented well, educational technology requires, at least in the beginning, quite a high investment, both in terms of infrastructure and in terms of support, something that educational leaders often choose to turn a blind eye to.
What should we measure?
In order to measure the quality of education, we have to take into account three major aspects: the student experience, the teaching staff experience and the connections to the society, at a broader level.
Student evaluations are the most common tool for measuring student experience. But do the evaluations we currently use capture the entire student experience? They often focus on the evaluation of teaching, and even then, the questions and the metrics used are debatable. What I consider most relevant to explore and assess are: the various types of interaction that take place in the physical or virtual classroom and beyond; the forms of assessment and their impact on student learning; the alignment of various courses within a programme; the level of interdisciplinarity and transdisciplinarity, including the involvement of stakeholders from outside academia, and, finally, what skills and competences our students acquire.
Teaching staff experience is a perspective that is often missed, unfortunately. This could be captured through self-reflection, discussions with peers, engagement with professional development activities, to name just a few ways in which we can get an insight into what educators think and do. In terms of links to society as a whole, it’s important to explore what students end up doing after graduation and to what extent their education helped them perform well on the labour market. Evaluating this requires longitudinal studies, which are time and resource intensive, but can offer rich insights that enable universities to refine their educational offer and continuously ensure quality and relevance.
Each of us, as educators, students, leaders or administrators in HE, needs to seriously reflect on what good education means for them. To have a real impact, this individual reflection should be complemented by discussions at department and at institutional level. It is only through this process, which does not happen overnight, that we explicitly acknowledge our values and our boundaries and can start developing a vision that prioritises quality while not neglecting the increasing constraints faced by the HE system.
Further reading
Measuring Quality as if Quality Mattered by Alex Usher;
Measuring the Quality of University Education: Beyond the Nonsense of University Rankings by Paul Ashwin;
Measuring teaching quality in higher education: assessing selection bias in course evaluations by Maarten Goos & Anna Salomons;
Measuring Quality in Higher Education: A competency approach by James Warn & Tranter Paul.
