by Rachel Brooks
‘Intractable’ datafication?
Over recent years, both policymakers and university leaders have extolled the virtues of moving to a more metricised higher education sector: statistics about student satisfaction with their degree programme are held to improve the decision-making processes of prospective students, while data analytics are purported to help the shift to more personalised learning, for example. Moreover, academic studies have contended that datafication has become an ‘intractable’ part of higher education institutions (HEIs) across the world.
Nevertheless, our research (conducted in ten English HEIs, funded by TASO) – of data use with respect to widening participation to undergraduate ‘sandwich’ courses (where students spend a year on a work placement, typically during the third year of a four-year degree programme) – indicates that, despite the strong claims about the advantages of making more and better use of data, in this particular area of activity at least, significant constraints operate, limiting the advantages that can accrue through datafication.
Little evidence of widespread data use
Our interviewees were those responsible for sandwich course provision in their HEI. While most thought that data could offer useful insights into the effectiveness of their area of activity, there was little evidence of ‘intractable’ data use. This was for three main reasons. First, in some cases, interviewees explained that no relevant data were collected – in relation to access to sandwich courses and/or the outcomes of such courses. Second, in some HEIs, relevant data were collected but not analysed. Such evidence tends to support the contention that ‘data lakes’ are emerging, as HEIs collect more and more data that often remain untapped. Third, in other cases, appropriate data were collected and analysed, but in a very limited manner. For example, one interviewee explained how data were collected and analysed in relation to the participation of students from under-represented ethnic groups, but not with respect to any other widening participation categories. This limited form of datafication, in which only some social characteristics were datafied, was not, therefore, able to inform any action with respect to the participation of widening participation students generally. Indeed, across all ten HEIs, there was only one example of where data were used in a systematic fashion to help analyse who was accessing sandwich courses within the institution, and the extent to which they were representative of the wider student population.
Constraints on data use
Lack of institutional capacity
In explaining this absence of data use, the most commonly identified constraint was the lack of institutional capacity to collect and/or analyse appropriate data. For example, one interviewee commented that they did not have a very good data system for placements – ‘we are still quite Excel- based’. Excel spreadsheets were viewed as limited as they could not be easily shared or updated, and data were relatively hard to manipulate. This, according to the interviewee, made collection of appropriate data laborious, and systematic analysis of the data difficult. Interviewees also pointed to the limited time staff had available to analyse data that the institution had collected.
Prioritisation of ‘externally-facing’ data
Several interviewees described how ‘externally-facing data’ – i.e. that required by regulatory bodies and/or that fed into national and international league tables – was commonly prioritised, leaving little time for information officers to devote to generating and/or analysing data for internal purposes. One interviewee, for example, was unclear about what data, if any, were collected about equity gaps but believed that they were generally only pulled together for high-level reports ‘such as for the TEF’.
Institutional cultures
A further barrier to using data to analyse access to and outcomes of sandwich courses was perceived to be the wider culture of the institution, including its attitude to risk. An interviewee explained that the data collected in their institution was limited to two main variables – subject of study and fee status (home or international) – because of ‘ongoing cautiousness at the university about how some of that data is used and how it’s shared with different teams’.
In addition, many participants outlined the struggles they had faced in gaining access to relevant data, and in influencing decisions about what should be collected and what analyses should be run. Several spoke of having to ‘request’ particular analyses to be run (which could be turned down), leading to a fairly ad hoc and inefficient way of proceeding, and illustrating the relative lack of agency accorded to staff – typically occupying mid-level organisational roles – in accessing and manipulating data.
Reflections
Examining a discrete set of activities within the UK higher education sector – those relating to sandwich courses – provides a useful lens to examine quotidian practices with respect to the availability and use of data. Despite the strong emphasis on data by government bodies and HEI senior management teams, as well as the claims made about the ‘intractability’ of HEI data use in the academic literature, our research suggests that datafication is perhaps not as widespread as some have claimed. Indeed, it indicates that some areas of activity – even those linked to high profile political and institutional priorities (in this case, employability and widening participation) – have remained largely untouched by ‘intractable’ datafication, with relevant data either not being collected or, where it is collected, not being made available to staff working in pertinent areas.
As a consequence, the extent to which students from widening participation backgrounds were accessing sandwich courses – and then succeeding on them – relative to their peers typically remained invisible. While the majority of our interviewees were able to speculate on the extent of any under-representation and/or poor experience, this was typically on the basis of anecdotal evidence and their own ‘sense’ of how inequalities were played out in this area. Although reflecting on professional experience is obviously important, many inequalities may not be visible to staff (for example, if a student chooses not to talk about their neurodiversity or first-in-family status), even if they have regular contact with those eligible to take a sandwich course. Moreover, given the status often accorded to quantitative data within the senior management teams of universities, the lack of any statistical reporting about inequalities by social characteristic, as they pertain to sandwich courses, makes it highly likely that such issues will struggle to gain the attention of senior leaders. The barriers to the effective use of metrics highlighted above may thus have a direct impact on HEIs’ capacity to recognise and address inequalities.
The research on which this blog is based was carried out with Jill Timms (University of Surrey) and is discussed in more detail in this article Institutional constraints to higher education datafication: an English case study | Higher Education
Rachel Brooks is Professor of Higher Education at the University of Oxford and current President of the British Sociological Association. She has conducted a wide range of research on the sociology of higher education; her most recent book is Constructing the Higher Education Student: perspectives from across Europe, published (open access) with Policy Press.
An international learned society, concerned with supporting research and researchers into Higher Education
