Engagement and wellbeing analytics: the whole is greater than the sum of the parts

Engagement and wellbeing analytics: the whole is greater than the sum of the parts

By Rachel Maxwell, Principal Advisor at Kortext.

Data at the heart of student support

A successful and integrated framework for academic support that is built around students comprises three core elements: data, theory and people. The university ‘ethos’ around student support frames the collection and use of data that in turn are both interpreted and used by staff to collaboratively design meaningful interventions with students to support engagement, wellbeing and academic development. The data proxies used to support academic engagement are proven and well-established (see Foster and Siddle, 2019; Rimmington, 2024; University of Essex, 2023). Data proxies for wellbeing are more embryonic in nature and it is less clear how to effectively use both data sets effectively to maximise the overall impact on student success.

Mental health matters

That the sector, if not the country, is facing an unprecedented crisis in the mental health of young people is well established. Underreporting or non-disclosure of issues masks the true scale of the picture, and the increasing severity of those issues imposes an additional layer of complexity and resource for higher education providers to address.

Ways to address the crisis, using student data, are therefore logical and essential, but also unclear. The Jisc Core Specification for Student Engagement Analytics identifies five wellbeing data points that indicate risks to retention and continuation alongside six more traditional student engagement data points. The inclusion of wellbeing analytics is an essential part of a whole provider approach to supporting student success alongside access and participation activity or the embedding of the University Mental Health Charter from Student Minds. Successful initiatives can now be shared via TASO’s Student Mental Health Evidence Hub.

The evaluation of an Office for Students mental health and analytics project at Northumbria University concluded that student wellbeing can be accurately predicted and can provide operational value to intervention models within student support in addition to students requiring academic support identified through engagement or learning/learner analytics. And while poor mental health is likely to evidence itself in non-engagement, not all non-engagement is indicative of a wellbeing risk.

… but it’s complex

Universities grappling with the thorny issue of accurately identifying students who are struggling and need support with their mental health will naturally be considering whether the Northumbria approach can be successfully transferred and scaled up within their own settings. Answering this question is particularly important in the case of initial non-disclosure or subsequent development of mental health issues, particularly given the fairly significant caveats associated with the project:

  • Data cleanliness, accuracy and availability is essential – but it was only possible following a decade-long data and digital transformation project at the university
  • Over 800 data variables were reviewed alongside dynamic data from relevant systems and associated student support facilities
  • Human decision-making by mental health and wellbeing experts remains central, to ‘see’ the person behind the risk rating, avoid potential ‘blind spots’, false positives and ‘misses’, and, crucially, to understand how an individual’s mental health is actually impacting their university experience
  • Although deemed successful, the Northumbria project has not (yet) resulted in a deliverable service.

The whole is greater than the sum of the parts

The Kortext student engagement analytics product, StREAM, provides an effective comparison point with early work to turn wellbeing indicators into effective data proxies suitable for risk determination.

One critical difference is that StREAM can effectively identify risk with an average of 90% accuracy based on data drawn from just 2 core systems – the VLE and the student record. However, identification of the causes of disengagement comes only through meaningful conversations with students, based both on their data and on contextual information about personal and demographic circumstances. It is important that the significance of those circumstances is explored collaboratively with the student at a relevant time to determine subjective impact, rather than presuming risk in advance.

In light of the mental health crisis, effective, holistic student support requires the use of analytics based on both engagement and wellbeing to provide frontline staff with a richer picture of their students. This approach will also enable universities to demonstrate that they have discharged their legal responsibilities to their students as fully as possible. Waiting until a possible mental health situation is starting to manifest in a student’s engagement data may be seen as too late and potentially too risky, being reliant upon all staff members to identify and act upon risk at the precise moment the student starts to disengage with their learning. While the need to provide ongoing information, advice and guidance to all students has long been identified as good practice, tailoring that messaging based on predictive and unsubstantiated subjective risk requires handling with care.

What next for health and wellbeing analytics?

Deploying engagement and wellbeing analytics together across an institution is complex. One size will not fit all in terms of using one approach to achieve dual objectives (retention/continuation and wellbeing), nor will the approach be the same across all institutions. More research is required to explore a range of questions, including:

  1. How many of the students identified as being ‘at risk’ by an engagement analytics system require mental health support?
  2. How many of those who don’t (at least initially) disclose a mental health condition, were subsequently identified as having low or no engagement by an engagement analytics system?
  3. Would the use of the wellbeing analytics proxies identify the same group of students as having mental health concerns as those picked up by an engagement analytics system and, following a conversation, be appropriately categorised as having a mental health concern?
  4. What level of confidence can be placed in each data set in terms of identifying the right students and, critically, doing so at the right time?
  5. Can the wellbeing data points inform the development of a mental health algorithm, when such data points are not easily reduced to a 1 or 0?
  6. What are the policy implications of a combined approach – both across the sector and within institutions – to demonstrate that a university has actively and meaningfully met their legal responsibilities for all students?
  7. How can ‘prior knowledge of a possible risk’ be combined with near real-time data in a student analytics platform to pinpoint an acute mental health situation and support early intervention?

Here at Kortext, we are interested in undertaking in-depth research with universities and others to explore these questions and find ways to use both data sets to support successful academic outcomes and a healthy student population. If you’re interested, please let us know here: www.kortext.com/stream/contact

Source link