Tag: REF

  • The Karateka vs the Sumo Wrestler: what REF 2029 means for research leadership in UK universities

    The Karateka vs the Sumo Wrestler: what REF 2029 means for research leadership in UK universities

    This blog was kindly authored by Dr Antonios Kelarakis, Reader in Polymers and Nanomaterials, University of Lancashire

    UK universities increasingly reward size, visibility and institutional influence. Yet many of the discoveries that underpin scientific progress come from researchers whose work is slow, specialist and largely invisible – the academic karateka, whose precision contrasts sharply with the highly visible, institution-shaping sumo wrestler. With reforms to the Research Excellence Framework (REF) for 2029 confirmed in December 2025, there is now an opportunity to rebalance what we value in research leadership and to better align institutional incentives with how knowledge is actually produced.

    In today’s academic world, two very different research styles are stepping onto the mat.

    The karateka is defined by focus and precision. They dedicate themselves to mastering a single research field, refining a theory, improving a method or laying the foundations for a new diagnostic or experimental technique. Every publication is carefully considered, every contribution is incremental but cumulative. Their ambition is depth rather than scale, and they aim to reach previously inaccessible insights. These researchers often form the invisible engine of scientific progress. Their work may attract little attention beyond specialist communities, yet its influence is long-lasting and foundational.

    The sumo wrestler, by contrast, plays a broader game. Their strength lies in size, coordination and visibility. They lead large research groups, oversee multiple interdisciplinary projects and accumulate titles, affiliations and advisory roles. Their calendars are filled with conferences, policy briefings and media engagement. They shape research agendas as much as individual ideas and act as the public face of modern academia. While the karateka advances knowledge through precision, the sumo wrestler moves institutions through mass and momentum.

    A shifting balance of power

    For much of scientific history, the karateka was the primary driver of discovery. The laws of physics, advances in chemistry and the development of new materials and analytical techniques have typically emerged from decades of focused work by scholars deeply embedded in a single domain.

    In recent years, however, the balance in UK academia has tilted. Universities increasingly reward visibility, scale, collaboration and institutional contribution – metrics that naturally favour the sumo wrestler. Funding requirements emphasise partnerships, pathways to impact and the management of large consortia. Universities respond rationally by supporting researchers who can deliver coordination, profile and strategic alignment.

    The karateka, meanwhile, often struggles to justify slow, methodical work in systems dominated by short-term indicators. Their contributions are essential, but they are not always easily captured by institutional performance metrics or institutional narratives.

    Why REF matters now

    The REF has always been a powerful signal of what universities should value. Decisions taken as part of the REF 2029 reforms strengthen the emphasis on research culture, long-term contribution and the environments that sustain excellence, alongside continued recognition of impact.

    Under the revised framework, assessment is weighted across three elements: Contribution to Knowledge and Understanding (55%), Engagement and Impact (25%) and Strategy, People and Research Environment (20%), assessed at both disciplinary and institutional levels. This represents a clear shift from REF 2021, where the role of environment was more limited.

    This change matters. By strengthening the role of research environment and contribution, REF 2029 creates an opening for universities to recognise how excellence is actually sustained; through deep expertise, stable methods, supportive cultures and long-term institutional investment. Research outputs remain central, but they no longer crowd out other forms of contribution to the same extent.

    Karateka-style scholarship has often struggled to fit neatly into REF narratives. Breakthroughs take time, develop incrementally and may not translate into demonstrable impact within a single cycle. Yet many celebrated impact case studies ultimately rest on foundational research generated by specialist researchers whose work is less visible and harder to narrate.

    From critique to policy

    The reforms give universities greater scope and responsibility to act differently. REF 2029 does not dictate outcomes, but it reshapes the conditions under which institutions define excellence.

    In practical terms, universities can now use the framework to reaffirm the value of:

    • deep, specialist expertise, even when audiences are narrow
    • long-term, foundational inquiry that underpins later impact
    • precision scholarship that strengthens methods and disciplines
    • small, focused teams that are often more intellectually productive than large consortia

    REF 2029 offers a chance to rebalance the contest without lowering the bar for excellence. Protecting space for karateka-style research is not a retreat from impact; it is a precondition for it. When depth is preserved, leadership has something genuinely worth amplifying: impact that endures rather than merely dazzles.

    Source link

  • REF 2029 talks about people again but early career labour is still hard to see

    REF 2029 talks about people again but early career labour is still hard to see

    REF 2029 guidance now confirms that the previously proposed people, culture and environment (PCE) element has been renamed strategy, people and research environment (SPRE). Its weighting has been set at 20 per cent, while the main contributions to knowledge and understanding element will make up 55 per cent of the overall profile. Compared with REF 2021, outputs no longer carry the 60 per cent weighting they once did, and the environment component has increased from 15 to 20 per cent.

    Supporters of the change, including Wellcome’s John-Arne Røttingen, have been clear that this is not intended as a downgrading of research culture, instead describing the move as a rebrand designed to prevent “culture” becoming politicised, and as a way of preserving the momentum of efforts to improve research environments.

    For early-career academics at the most insecure end of the system, however, research labour still sits outside what is easiest to count. What resists straightforward counting is also what is least likely to be protected.

    Hidden research expectations

    I am one year out of my PhD, in which I explored the “care-full” and “careless” dimensions of academic work. I graduated expecting that the next few years would involve short-term teaching, fractional contracts or, if things went well, fixed-term research roles. I also entered this stage of my working life knowing that, whatever job I took, I would need to keep publishing to stand any real chance of staying in higher education.

    I write this with short-term teaching arrangements in mind. Within these roles, there is an unspoken contradiction. Many teaching contracts formally exclude research. At the same time, research remains a condition of future employability. It appears in shortlisting criteria, promotion thresholds and hiring decisions. The result is that research becomes an informal obligation. It is returned to between classes and tutorials, and carried into evenings, weekends and term breaks.

    This is where the reframing of “culture” now matters.

    Sustainability without supported labour

    In REF 2021, the environment element required institutions to demonstrate the “vitality and sustainability” of their research environments. Guidance defined this in terms of research strategy, doctoral pipelines, research income, mentoring structures for early-career researchers and the capacity to continue producing high-scoring outputs. In arts, humanities and social sciences units in particular, panels praised institutions that could demonstrate early-career development pathways, including reduced teaching loads, research leave and internal funding.

    SPRE retains the same two criteria of vitality and sustainability. In REF 2029, these will now be assessed through both an institution-level statement, weighted at 60 per cent of the SPRE score, and a unit-level statement at 40 per cent. The institution-level statement places explicit emphasis on strategy as the main way in which research environments and cultures are now explained.

    This version of sustainability rests on the assumption that research labour is formally recognised and resourced. It does not capture the volume of research produced under contracts where research does not appear in workload models or time allocation at all. In practice, sustainability comes to mean whether outputs keep appearing, rather than whether the people producing them can realistically go on working like this when their next job may depend on it.

    The limits of research expectation

    It is true that REF 2029 introduces a substantive-link rule and allows outputs from staff on part-time or non-standard contracts, so long as they meet the 0.2 FTE, 12-month employment and research-expectation threshold. This complicates any straightforward claim that REF excludes precarious researchers. It also places the power of recognition firmly at institutional level.

    REF 2029 requires that a contract include a “research expectation,” while the guidance does not require institutions to prove that time, funding or workload adjustment were provided to support the research. The term “research expectation” itself remains vague, and in practice it may amount to little more than a nominal clause. That ambiguity allows outputs to be counted even when the labour behind them was carried out under precarious, unsustainable conditions.

    Culture was never going to be a perfect remedy. As Lizzie Gadd has already argued in her “my culture is better than yours” critique of competitive approaches to research culture, the sector’s engagement with culture has been uneven and often reflects the priorities of research-intensive, or more accurately funding-intensive, institutions and STEM disciplines. Even so, culture was the one part of the framework with the reach to ask how research expectations attach themselves to people, workloads and contracts. Political? Maybe. But what about precarity isn’t political.

    What still counts

    All of this is unfolding in the context of a wider financial crisis across higher education. Falling international recruitment, rising costs and long-term funding pressure have placed many providers under severe strain, with arts, humanities and social science provision often among the most exposed. In this environment, universities trade on the career aspirations of early-career academics to manage costs, relying on their, our, my hopes of progression to sustain teaching at lower pay and with fewer protections.

    We now have a sector full of strategies, including ever more detailed strategies for people and research environments, and very little shared vision of what a sustainable early-career academic life should look like. With REF 2029 restoring the dominance of outputs and re-casting culture as a subsidiary part of institutional strategy, a clear message is taking shape. Outputs still count. The conditions under which those outputs are produced count for far less.

    Source link

  • Podcast: REF 2029, franchising crackdown, year in review

    Podcast: REF 2029, franchising crackdown, year in review

    This week on the podcast we examine what the rebooted 2029 Research Excellence Framework will mean for universities’ research strategies, research culture, and future funding – including the new “strategy, people and research environment” element and the renewed focus on contribution to knowledge and understanding through research outputs.

    Plus we discuss the government’s crackdown on franchised higher education provision and student loan eligibility, and we look back at the defining moments of 2025 in higher education policy – from regulation and finance to admissions, academic freedom and research – and consider what they might signal for universities in 2026 and beyond.

    With Steph Harris, Director of Policy at Universities UK, Andy Westwood, Professor of Public Policy, Government and Business at the University of Manchester, Michael Salmon, News Editor at Wonkhe and presented by Jim Dickinson, Associate Editor at Wonkhe.

     

    On the site:

    Re-thinking research support for English universities: Research England’s programme of work during the REF 2029 pause

    Everything you need to know about REF 2029

    Study a Bachelors DEGREE without paying a single penny? You’re on

    Weekend courses can’t get student loans

    Sub-contractual providers need to register with OfS

    You can subscribe to the podcast on Apple Podcasts, YouTube Music, Spotify, Acast, Amazon Music, Deezer, RadioPublic, Podchaser, Castbox, Player FM, Stitcher,

    Transcript (auto generated)

     

    Source link

  • Everything you need to know about REF 2029

    Everything you need to know about REF 2029

    REF 2029 has been unpaused and with it will undoubtedly come a whole new wave of disagreement and debate. Much like the research ecosystem itself it is an unpredictable beast forever buffeted by its participants, leaders, and funders.

    To get immediately to the headlines. People, Culture, and Environment (PCE) has been relabelled as Strategy, People and Research Environment (SRPE). The weighting for the new element is 20 per cent of the total dropped from the 25 per cent weighting originally allocated to PCE. Contribution to Knowledge and Understanding (CKU) (the output one), has been boosted to 55 per cent, and Engagement and Impact (E&I) (the impact case study one) has remained at 25 per cent.

    There is a significant attempt to reduce the burden of the exercise through reverting to some of the narrative practices of REF 2021 in research environments while reducing the need for new data to be collected. E&I has remained pretty much the same and there are some concessions on portability that will only partially assuage the concerns of people concerned by this sort of thing.

    So: a bit more for the things that researchers produce and a bit less for how they produce them.

    Strategy people research and environment

    The big frame for REF 2029 has been that research is a team sport. This is why Research England and its devolved counterparts have sought to remove the relationship between researchers and research outputs. However, this led to a forever debate on who actually produces research between institutions or researches and whose work should be measured. In effect, should an average researcher be boosted by an exceptional research environment or should an exceptional researcher be held back by an average research environment.

    SPRE asks institutions and units to demonstrate how their strategies contribute toward the development of people and good research environments. This will be done primarily through narrative with metrics to support. There is flexibility in how providers may demonstrate their work in this area but the core idea is that the work should be accompanied by a clear strategic intent.

    The actual basket of work that can fit under SPRE is varied and might include evidence of improving research cultures, new partnerships, collaborations, the development of new policies, and a range of evidence of improving culture metrics. The major change from what had been proposed is the underpinning focus on strategy and by extension the broader range of activity providers are likely to submit. Culture is still very much there but it is part of a range of activity.

    SPRE will be assessed at both an institution and unit level. The assessment will be through a statement similar to the unit level statements from the environment element in the 2021 exercise. However, the institution level score will make up 60 per cent of the overall score for each Unit of Assessment (UoA) and documentation linked to the UoA itself will make up the remaining 40 per cent of the overall score. In effect, this means that the research infrastructure of the institution will have a greater impact than the research infrastructure of the unit where research is actually produced.

    The changes to SPRE have partially emerged from the PCE pilots. Their conclusion was that it would be possible to assess PCE, but that the approach would need some adaptations for a full scale exercise. Some of the challenges included: the phenomenon of larger institutions scoring better purely because they had access to more evidence, the need for simple and timely data collection, and a need for clearer guidance and simpler processes. In short, it is technically possible to measure PCE in a robust way but it is hard to implement – which was a view shared by many at the start of the exercise.

    Measures

    The argument in favour of the 60:40 split is that it incentivises providers to improve their research environments across the whole institution. In what will be partially good news to the minister there is also a renewed focus on rewarding providers that are focussed on aligning their activities with their strategic intent in people, research, and environments.

    While we do not yet have all of the criteria, the submission burden seems to be lower than many feared. As well as the statements at a unit and institution level there will be a data requirement at an institutional level which it is anticipated may include: which units are submitted, volume, research doctoral degrees awards, and annual research income by source.

    At a unit level there are similar set of measures with some nuance. In the initial set of decisions it was proposed then PCE now SPRE could include

    […]EDI data (that are already collected via the HESA staff record), quantitative or qualitative information on the career progression and paths of current and former research staff, outcomes of staff surveys, data around open research practices, and qualitative information on approaches to improve research robustness and reproducibility.

    There are criteria yet to be published but it is suggested that issues of equality will be looked at primarily through the statements, and through calibration with the People and Diversity Advisory Panel and the Research Diversity Advisory Panel during the panel assessment stage. The data burden will be less and ideally not newly collected.

    CKU OK

    SPRE will also now be the place where institutions submit context, structure, and strategy, about their units. Disciplinary statements have been removed entirely from Contribution to Knowledge and Understanding (CKU) and Engagement and Impact (E&I). This might look like a rearranging of the same information but it also impacts the overall weightings.

    In the previous model CKU accounted for 50 per cent overall including outputs and the statement. In effect, CKU now accounts for 55 per cent of the weighting while focusing only on outputs.

    REF is now an exercise which is still majority related to the perceived quality of research outputs. There is now an upper limit on individual submissions of five per unit unless there is an explanation why it exceeds this. However, there is no requirement that every researcher submits (the decoupling process). Providers will have to produce a statement on how their submissions are representative through and each unit will be expected to provide an overview of their work and a statement of representation.

    On the other big debate the portability rules have remained broadly the same. To recap, in REF 2014 the whole output was captured by whichever institution a researcher was at, at the REF census date. In REF 2021 if a researcher moved between institutions the output was captured by both. In REF 2029 the initial proposal was that the output will be captured by the institution where there is a “substantive link.” Research England has made a slight concession and will allow long-form submissions to be portable for a five-year period with sufficient justification.

    What remains unresolved

    There is a political element to all of this, of course. In the post-16 white paper it was made explicit that

    We anticipate that institutions will be recognised and rewarded, including through the Research Excellence Framework (REF) and Quality Related funding, for demonstrating clarity of purpose, demonstrating alignment with government priorities, and for measurable impact, where appropriate. While government will continue to invest across the full spectrum of research, we expect universities to be explicit about their contributions and to use this framework to guide strategic decisions.

    REF 2029, as currently set out, does not do this. Unless there are further announcements on the relationship between REF and funding, REF will do a different version of what it has always done. It assesses the research that is put in front of it. There is no additional weighting for alignment to government priorities, there are no changes to impact measurements, and while there is a focus on alignment between activity and strategic intent it is up for institutions to define what that strategic intent is. There have been efforts to reduce the burden from the initial decisions but this does not seem to be a significantly less burdensome activity than REF 2021.

    The minister might be pleased that the word “strategy” has replaced “culture”, and with some fiddling with weighting, but the direction of travel across the whole exercise has remained broadly intact. It is not quite the cultural revolution that was promised nor is it an output focussed exercise that some wanted. It’s a bit of a compromise but only a little bit.

    What now

    The response of the sector will largely determine whether these changes are viewed as a success. Ultimately, REF is a political project. It is not simply an input into the dispassionate allocation of public money but requires decisions on what is valued. There is a version of the REF which is only about research outputs. There is a possible version which is only about research environments and there are hundreds of weighting, criteria, frameworks, rules, and regulations in between.

    The reasonable criticism of Research England is that it made radical changes to REF 2021 and could not bring the sector with it for REF 2029. At times, it felt like the public explanation was about how a series of technical changes to the exercise achieved a set of good outcomes for the sector without vigorously explaining what good was, who would lose out, and why the trade offs were worth it.

    These new decisions are either a messy middle ground or a genius compromise. They cede ground to those concerned about outputs by changing weightings and moving criteria but it maintains culture as a key focus. They provide room to include more culture focussed statements without complex metrics. And they are politically astute enough to talk about strategy, even if the strategy isn’t the same as the government’s in every institution.

    The worst possible result would be the ongoing argument between providers, between providers and funders, and between funders and government. The unedifying spectacle of a noisy debate on why elements of the sector’s own research exercise is not fit for purpose distracts from both the enormous administrative burden of the exercise and the political case for why the sector should command significant research funding.

    Source link

  • Re-thinking research support for English universities: Research England’s programme of work during the REF 2029 pause

    Re-thinking research support for English universities: Research England’s programme of work during the REF 2029 pause

    In September, Science Minister Lord Vallance announced a pause to developing REF 2029 to allow REF and the funding bodies to take stock. Today, REF 2029 work resumes with a refreshed focus to support a UK research system that delivers knowledge and innovation with impact, improving lives and creating growth across the country.

    Research England has undertaken a parallel programme of work during the pause, intended to deliver outcomes that align with Government’s priorities and vision for higher education as outlined in the recently published Post-16 Education and Skills white paper. Calling this a pause doesn’t reflect the complexity, pace and challenge faced in delivering the programme over the last three months.

    Since September, we have:

    • explored the option of baseline performance in research culture being a condition of funding
    • considered how our funding allocation mechanisms in England could be modified to better reward quality, as part of our ongoing review of Strategic Institutional Research Funding (SIRF)
    • fast-tracked existing activity related to the allocation of mainstream quality-related research funding (QR).
    • developed our plans to consider the future of research assessment.

    Over the last three months to progress this work, we’ve engaged thoughtfully with groups across the English higher education and research sector, as well as with the devolved funding bodies, to help us understand the wider context and refine our approaches. Let me outline where we’ve got to – and where we’re going next – with the work we’ve been doing.

    Setting a baseline for research cultures

    Each university, department and team are unique. They have their own values, priorities and ways of working. I therefore like to think of ‘research cultures and environments’, using the term in plural, to reflect this diversity. The report of the REF People, Culture and Environment pilot, also published today, confirms that there is excellent practice in this area across the higher education sector. REF 2029 offers an opportunity to recognise and reward those institutions and units that are creating the open, inclusive and collaborative environments that enable excellent research and researchers to thrive.

    At the same time, we think there are some minimum standards that should be expected of all providers in receipt of public funding. To promote these standards, we will be strengthening the terms and conditions of Research England funding related to research culture. In the first instance, this will mean a shift from expecting certain standards to be met, to requiring institutions to meet them.

    We are very conscious not to increase burden on the sector or create unnecessary bureaucracy. This will only succeed by engaging closely with the sector to understand how this can work effectively in practice. To this end, we will be engaging with groups in early 2026 to establish rigorous standards that are relevant across the diversity of English institutions. As far as possible, we will use existing reporting mechanisms such as the annual assurance report provided by signatories to the Research Integrity Concordat. While meeting the conditions will not be optional, we will support institutions that don’t yet meet all the requirements, working together and utilising additional reporting to help with and monitor improvements. And because research cultures aren’t static, we will evolve our conditions over time to reflect changes in the sector.

    This will lead to sector-wide improvements that we can all get behind:

    • support for everyone who contributes to excellent and impactful research: researchers, technicians and others in vital research-enabling roles, across all career stages
    • ensuring research in England continues to be done with integrity
    • ensuring that is also done openly
    • strengthening responsible research assessment.

    Our next steps are to engage with the sector and relevant groups as part of the process of making changes to our terms and conditions of funding, and to establish low-burden assurance mechanisms. For example, working as part of the Researcher Development Concordat Strategy Group, we will collectively streamline and strengthen the concordat, making it easier for institutions to implement this important cross-sectoral agreement.

    These changes will complement the assessment of excellent research environments in the REF and the inspiring practice we see across the sector. Championing vibrant research cultures and environments is a mission that transcends the REF — it’s the foundation for maintaining and enhancing the UK’s world-leading research, and we will continue to work with the devolved funding bodies to fulfil the mission.

    Modelling funding mechanisms

    The formula-based, flexible research funding Research England distributes to English universities is crucial to underpinning the HE research landscape, and supporting the

    financial sustainability of the sector. We are aware that that this funding is increasingly being spread more thinly.

    As part of the review of strategic institutional research funding (SIRF), we are working to understand the wider effectiveness of our funding approaches and consider alternative allocation mechanisms. Work on this review is continuing at speed. We will provide an update to the sector next year on progress, as well as the publication of the independent evaluation of SIRF, anticipated in early 2026.

    Building on this, we have been considering how our existing mechanisms in England could be modified to better reward quality of research. This work looks at how different strands of SIRF – from mainstream QR to specialist provider funding – overlap, and how that affects university finances across English regions and across institution types. We are continuing to explore options for refining our mainstream QR formula and considering the consequences of those different options. This is a complex piece of work, requiring greater time and attention, and we expect next year to be a key period of engagement with the sector.

    The journey ahead

    While it may seem early to start thinking about assessment after REF 2029, approaches to research assessment are evolving rapidly and it is important that we are able to embrace the opportunities offered by new technologies and data sources when the moment comes. We have heard loud and clear that early clarity on guidance reduces burden for institutions and we want to be ready to offer that clarity. A programme of work that maximises the opportunity offered by REF 2029 to shape the foundation for future frameworks will be commencing in spring 2026.

    Another priority will be to consider how Research England as the funding body for England, and as part of UKRI, can support the government’s aim to encourage a greater focus on areas of strength in the English higher education sector, drawing on the excellence within all our institutions. As I said at the ARMA conference earlier in the year, there is a real opportunity for universities to identify and focus on the unique contributions they make in research.

    The end of the year will provide the sector (and my colleagues in Research England and the REF teams) with some much-needed rest. January 2026 will see us pick back up a reinvigorated SIRF review, informed by the REF pause activity. We will continue to refine our research funding and policy to – as UKRI’s new mission so deftly puts it – advance knowledge, improve lives and drive growth.

    Source link

  • Before we automate REF there are three issues we need to talk about

    Before we automate REF there are three issues we need to talk about

    The long-awaited REF-AI report prompts the sector to imagine an increasingly automated REF, but also leaves several important questions unanswered about what such a future might mean for the people and practices that underpin research assessment. Before we embed AI more deeply into REF2029, we need to pause and reflect on three issues that deserve much greater attention, starting with the long-term risks to disciplinary expertise.

    Long-term impacts: Efficiency gains and the risk of skills erosion

    Recommendation 15 in the report proposes that: “REF assessments should include a human verification step… confirming that final judgements rest on human academic expertise.”

    This feels sensible on the surface. But the longer-term implications warrant more attention. Across many sectors, evidence shows that when automation takes on tasks requiring expert judgement, human expertise can slowly erode as roles shift from analysis to oversight. The report itself recognises this trend when discussing labour substitution and task reallocation.

    REF processes already rely heavily on signals, heuristics and proxies, particularly under time pressure. Introducing AI may further reduce opportunities for deep disciplinary reading in panel work. If this happens, then by the 2030s or 2040s, the experts needed to meaningfully verify AI-generated assessments may become harder to sustain.

    This is not an argument against using AI, but rather a suggestion that we need to consider the long-term stewardship of disciplinary expertise, and ensure that any AI integration strengthens, rather than displaces, human judgement. We don’t yet have expertise in how to collaborate effectively with AI systems and their outputs. This needs to be developed as a conscious endeavour to ensure that AI supports research assessment responsibly.

    Learning from Responsible Research Assessment (RRA)

    Over more than a decade, frameworks such as DORA, CoARA, the Hong Kong Principles and the Leiden Manifesto have laid out clear principles for responsible use of quantitative indicators, transparency, equity, and disciplinary diversity. The REF-AI report notes that in the interviews conducted: “Seldom was mention made of responsible research assessment initiatives such as DORA and CoARA… There is no clear view that the deployment of GenAI tools in the REF is antithetical to the ambitions of such initiatives.” But the absence of discussion in the focus groups does not necessarily mean a positive alignment, it may simply indicate that RRA principles were not a prominent reference point in the design or facilitation of the project.

    A fuller analysis could explore how AI intersects with core RRA questions, including: i) How do we assess what we value, not just what is machine-readable? ii) How do we prevent AI from amplifying systemic inequities? iii) How do we ensure transparency in systems underpinned by proprietary models? and iv) How do we avoid metrics-by-stealth re-entering the REF through automated tools? These considerations are essential, not peripheral, to thinking about AI in research assessment.

    Representation: A report on bias that overlooks some of its own challenges

    Finally, representation. As the authors have acknowledged themselves, it is hard to ignore that the authorship team comprises four men, three of which are senior and white. This matters, not as a criticism of the individuals involved, but because who examines AI uptake shapes how issues of bias, fairness and inclusion are framed. Generative AI systems are widely acknowledged as being trained on text that contains gendered, racialised and geographical biases; the report also notes that: “Concerns of bias and inaccuracy related to GenAI tools are widely acknowledged…” What is less evident, however, is a deeper engagement with how these biases might play out within a national research assessment exercise that already shows uneven outcomes for different groups.

    A similar issue arises in the dataset. Half of the interviewees were from Russell Group institutions, despite the Russell Group representing around 15 per cent of REF-submitting HEIs. The report itself notes that experimentation with AI is concentrated in well-resourced institutions: “Variation in experimentation with GenAI tools is mainly influenced… by institutional resource capacity.”

    Given this, the weighting of the sample will skew the perspectives represented. This does not necessarily invalidate the findings, but it does raise questions about whether further, broader consultation would strengthen confidence in the conclusions drawn.

    Doing it better?

    The report does an excellent job of surfacing current institutional anxieties. Larger, well-resourced universities appear more open to integrating AI into REF processes; others are more cautious. Survey findings suggest notable scepticism among academics, particularly in Arts, Humanities and Social Sciences. Despite this, the report signals a direction of travel in which REF “inevitably” becomes AI-enabled and eventually “fully automated.” Whether this future is desirable, or indeed equitable, remains an open question.

    The REF-AI report is therefore best read as an important starting point. For the next phase, it will be vital that Research England broadens the conversation to include a wider diversity of voices, including experts in equality and inclusion, disciplinary communities concerned about long-term skills, those with deep experience in RRA, smaller institutions, and early career researchers who will inherit whatever system emerges.

    This more diverse team must be given licence to make bold decisions about not just what’s inevitable but what’s desirable for the research ecosystem the REF ultimately seeks to monitor and shape. We cannot simply pay lip service to principles of responsible research assessment, equity, diversity and inclusion, and ignore the resulting outcomes of the decision-making processes shaped by those principles.

    AI will undoubtedly shape aspects of future research governance and assessments. The challenge, now, is to ensure that its integration reflects sector values, not just technological possibility.

    Source link

  • Generative AI and the REF: closing the gap between policy and practice

    Generative AI and the REF: closing the gap between policy and practice

    This blog was kindly authored by Liam Earney, Managing Director, HE and Research, Jisc.

    The REF-AI report, which received funding from Research England and co-authored by Jisc and Centre for Higher Education Transformations (CHET), was designed to provide evidence to help the sector prepare for the next REF. Its findings show that Generative AI is already shaping the approaches that universities adopt. Some approaches are cautious and exploratory, some are inventive and innovative, and most of it is happening quietly in the background. GenAI in research practice is no longer theoretical; it is part of the day-to-day reality of research, and research assessment.

    For Jisc, some of the findings in the report are unsurprising. We see every day how digital capability is uneven across the sector, and how new tools arrive before governance has had a chance to catch up. The report highlights an important gap between emerging practice and policy – a gap that the sector can now work collaboratively to close. UKRI has already issued guidance on generative AI use in funding applications and assessment: emphasising honesty, rigour, transparency, and confidentiality. Yet the REF context still lacks equivalent clarity, leaving institutions to interpret best practice alone. This work was funded by Research England to inform future guidance and support, ensuring that the sector has the evidence it needs to navigate GenAI responsibly.

    The REF-AI report rightly places integrity at the heart of its recommendations. Recommendation 1 is critical to support transparency and avoid misunderstandings: every university should publish a clear policy on using Generative AI in research, and specifically in REF work. That policy should outline what is acceptable and require staff to disclose when AI has helped shape a submission.

    This is about trust and about laying the groundwork for a fair assessment system. At present, too much GenAI use is happening under the radar, without shared language or common expectations. Clarity and consistency will help maintain trust in an exercise that underpins the distribution of public research funding.

    Unpicking a patchwork of inconsistencies

    We now have insight into real practice across UK universities. Some are already using GenAI to trawl for impact evidence, to help shape narratives, and even to review or score outputs. Others are experimenting with bespoke tools or home-grown systems designed to streamline their internal processes.

    This kind of activity is usually driven by good intentions. Teams are trying to cope with rising workloads and the increased complexity that comes with each REF cycle. But when different institutions use different tools in different ways, the result is not greater clarity. It is a patchwork of inconsistent practices and a risk that those involved do not clearly understand the role GenAI has played.

    The report notes that most universities still lack formal guidance and that internal policy discussions are only just beginning. In fact, practice has moved so far ahead of governance that many colleagues are unaware of how much GenAI is already embedded in their own institution’s REF preparation, or for professional services, how much GenAI is already being used by their researchers.

    The sector digital divide

    This is where the sector can work together, with support from Jisc and others, to help narrow the divide that exists. The survey results tell us that many academics are deeply sceptical of GenAI in almost every part of the REF. Strong disagreement is common and, in some areas, reaches seventy per cent or more. Only a small minority sees value in GenAI for developing impact case studies.

    In contrast, interviews with senior leaders reveal a growing sense that institutions cannot afford to ignore this technology. Several Pro Vice Chancellors told us that GenAI is here to stay and that the sector has a responsibility to work out how to use it safely and responsibly.

    This tension is familiar to Jisc. GenAI literacy is uneven, as is confidence, and even general digital capability. Our role is to help universities navigate that unevenness. In learning and teaching, this need is well understood, with our AI literacy programme for teaching staff well established. The REF AI findings make clear that similar support will be needed for research staff.

    Why national action matters

    If we leave GenAI use entirely to local experimentation, we will widen the digital divide between those who can invest in bespoke tools and those who cannot. The extent to which institutions can benefit from GenAI is tightly bound to their resources and existing expertise. A national research assessment exercise cannot afford to leave that unaddressed.

    We also need to address research integrity, and that should be the foundation for anything we do next. If the sector wants a safe and fair path forward, then transparency must come first. That is why Recommendation 1 matters. The report suggests universities should consider steps such as:

    • define where GenAI can and cannot be used
    • require disclosure of GenAI involvement in REF related work
    • embed these decisions into their broader research integrity and ethics frameworks

    As the report notes that current thinking about GenAI rarely connects with responsible research assessment initiatives such as DORA or CoARA, that gap has to close.

    Creating the conditions for innovation

    These steps do not limit innovation; they make innovation possible in a responsible way. At Jisc we already hear from institutions looking for advice on secure, trustworthy GenAI environments. They want support that will enable experimentation without compromising data protection, confidentiality or research ethics. They want clarity on how to balance efficiency gains with academic oversight. And they want to avoid replicating the mistakes of early digital adoption, where local solutions grew faster than shared standards.

    The REF AI report gives the sector the evidence it needs to move from informal practice to a clear, managed approach.

    The next REF will arrive at a time of major financial strain and major technological change. GenAI can help reduce burden and improve consistency, but only if it is used transparently and with a shared commitment to integrity. With the right safeguards, GenAI could support fairness in the assessment of UK research.

    From Jisc’s perspective, this is the moment to work together. Universities need policies. Panels need guidance. And the sector will need shared infrastructure that levels the field rather than widening existing gaps.

    Source link

  • REF should be about technical professionals too

    REF should be about technical professionals too

    Every great discovery begins long before a headline or journal article.

    Behind every experiment, dataset, and lecture lies a community of highly skilled technical professionals, technologists, facility managers, and infrastructure specialists. They design and maintain the systems that make research work, train others to use complex equipment, and ensure data integrity and reproducibility. Yet their contribution has too often been invisible in how we assess and reward research excellence.

    The pause in the Research Excellence Framework (REF) is more than a scheduling adjustment, it’s a moment to reflect on what we value within the UK research and innovation sector.

    If we are serious about supporting excellence, we must recognise all those who make it possible, not just those whose names appear on papers or grants, but the whole team, including technical professionals whose expertise enables every discovery.

    Making people visible in research culture

    Over the past decade, there has been growing recognition that research culture, including visibility, recognition, and support for technical professionals is central to delivering world-class outcomes. Initiatives such as the Technician Commitment, now backed by more than 140 universities and research institutes, have led the way in embedding good practice around technical professional careers, progression, and recognition.

    Alongside this, the UK Institute for Technical Skills and Strategy (UK ITSS) continues to advocate for technical professionals nationally to ensure they are visible and their inputs are recognised within the UK’s research and innovation system. These developments have helped reshape how universities think about people, culture, and environment, creating the conditions where all contributors to research and innovation can thrive.

    A national capability – not a hidden workforce

    This shift is not just about fairness or inclusion, it’s about the UK’s ability to deliver on its strategic ambitions. Technical professionals are critical to achieving the goals set out in the UK Government’s Modern Industrial Strategy and to the success of frontier technologies such as artificial intelligence, quantum, engineering biology, advanced connectivity, and semiconductors. These frontier sectors rely on technical specialists to design, operate, and maintain the underpinning infrastructure on which research and innovation depend.

    Without a stable, well-supported technical professional workforce, the UK risks losing the very capacity it needs to remain globally competitive. Attracting, training, and retaining this talent necessitates that technical roles are visible and recognised – not treated as peripheral to research, but as essential to it.

    Why REF matters

    This is where the People, Culture and Environment (PCE) element of the REF becomes critical. REF has always shaped behaviour across the sector. Its weighting signals what the UK values in research and innovation. Some have argued that PCE should be reduced (or indeed removed) to simplify the REF process, ease administrative burden, or avoid what they see as subjectivity in the assessment of research culture. Others have suggested a greater emphasis on environment would shift focus away from research excellence, or that culture work is too challenging to consistently assess across institutions. But these arguments overlook something fundamental, that the quality of our research, the excellence we deliver as a sector, is intrinsically tied to the conditions in which it is produced. As such, reducing the weighting of PCE would send a contradictory message, that culture, collaboration, and support for people are secondary to outputs rather than two sides of the same coin.

    The Stern Review and the Future Research Assessment Programme both recognised the need for a greater focus on research and innovation environments. PCE is not an optional extra, it is fundamental to research integrity, innovation, and excellence. A justifiably robust weighting reflects this reality and gives institutions the incentive to continue investing in healthy, supportive, and inclusive environments.

    Universities have already made significant progress on this by developing new data systems, engaging staff, and benchmarking culture change. There is clear evidence that the proposed PCE focus has driven positive shifts in institutional behaviour. To step away from this now would risk undoing that progress and undermine the growing recognition of technical professionals as central to research and innovation success.

    Including technical professionals explicitly within REF delivers real benefits for both technical professionals and their institutions, and ultimately strengthens research excellence. For technicians, recognition within the PCE element encourages universities to create the kind of environments in which they can thrive – cultures that value their expertise, provide clearer career pathways, invest in skills, and ensure they have the support and infrastructure to contribute fully to research. Crucially, REF 2029 also enables institutions to submit outputs led by technical colleagues, recognising their role in developing methods, tools, data, and innovations that directly advance knowledge.

    For universities, embedding this broader community within PCE strengthens the systems REF is designed to assess. It drives safer, more efficient and sustainable facilities, improves data quality and integrity, and fosters collaborative, well-supported research environments. By incentivising investment in skilled, stable, and, empowered technical teams, the inclusion of technicians enhances the reliability, reproducibility, and innovation potential of research – ultimately raising the standard of research excellence across the institution.

    From hidden to central

    REF has the power not only to measure excellence, but to shape it. By maintaining a strong focus on people and culture, it can encourage institutions to build the frameworks, leadership roles, and recognition mechanisms that enable all contributors, whether technical, academic, or professional, to contribute and excel.

    In doing so, REF can help normalise good practice, embed openness and transparency, and ensure that the environments underpinning discovery are as innovative and excellence driven as the research itself.

    Technical professionals have always been at the heart of UK research. Their skill, creativity, and dedication underpin every discovery, innovation, and breakthrough. What’s changing now is visibility. Their contribution is increasingly recognised and celebrated as foundational to research excellence and national capability.

    As REF evolves, it must continue to reward the environments that nurture, develop, and sustain technical expertise. In doing so, it can help ensure that technical professionals are not just acknowledged but firmly established at the centre of the UK’s research and innovation system – visible, recognised, and vital (as ever) to its future success.

    Source link

  • The REF helps make research open, transparent, and credible- let’s not lose that

    The REF helps make research open, transparent, and credible- let’s not lose that

    The pause to reflect on REF 2029 has reignited debate about what the exercise should encompass – and in particular whether and how research culture should be assessed.

    Open research is a core component of a strong research culture. Now is the time to take stock of what has been achieved, and to consider how REF can promote the next stage of culture change around open research.

    Open research can mean many things in different fields, as the UNESCO Recommendation on Open Science makes clear. Wherever it is practiced, open research shifts focus away from outputs and onto processes, with the understanding that if we make the processes around research excellent, then excellent outcomes will follow

    Trust

    Being open allows quality assurance processes to work, and therefore research to be trustworthy. Although not all aspects of research can be open (sensitive personal data, for example), an approach to learning about the world that is as open as possible differentiates academic research from almost all other routes to knowledge. Open research is not just a set of practices – it’s part of the culture we build around integrity, collaboration and accountability.

    But doing research openly takes time, expertise, support and resources. As a result, researchers can feel vulnerable. They can worry that taking the time to focus on high-quality research processes might delay publication and risk them being scooped, or that including costs for open research in funding bids might make them less likely to be funded; they worry about jeopardising their careers. Unless all actors in the research ecosystem engage, then some researchers and some institutions will feel that they put themselves at a disadvantage.

    Open research is, therefore, a collective action problem, requiring not only policy levers but a culture shift in how research is conducted and disseminated, which is where the REF comes in.

    REF 2021

    Of all the things that influence how research is done and managed in the UK HE sector, the REF is the one that perhaps attracts most attention, despite far fewer funds being guided by its outcome than are distributed to HEIs in other ways.

    One of the reasons for this attention is that REF is one of the few mechanisms to address collective action problems and drive cultural change in the sector. It does this in two ways, by setting minimum standards for a submission, and by setting some defined assessment criteria beyond those minimum standards. Both mechanisms provide incentives for submitting institutions to behave in particular ways. It is not enough for institutions to simply say that they behave in this way – by making submissions open, the REF makes institutions accountable for their claims, in the same way as researchers are made accountable when they share their data, code and materials.

    So, then, how has this worked in practice?

    A review of the main panel reports from REF 2021 shows that evidence of open research was visible across all four main panels, but unevenly distributed. Panel A highlighted internationally significant leadership in Public Health, Health Services and Primary Care (UoA 2) and Psychology, Psychiatry and Neuroscience (UoA 4), while Panel B noted embedded practices in Chemistry (UoA 8) and urged Computer Science and Informatics (UoA 11) to make a wider shift towards open science through sharing data, software, and protocols. Panel C pointed to strong examples in Geography and Environment Studies (UoA 14), and in Archaeology (UoA 15), where collaboration, transparency, and reproducibility were particularly evident. By contrast, Panel D – and parts of Panel C – showed how definitions of open research can be more complex, because what constitutes ‘open research’ is perhaps much more nuanced and varied in these disciplines, and these disciplines did not always demonstrate how they were engaging with institutional priorities on open research and supporting a culture of research integrity. Overall, then, open research did not feature in the reports on most UoAs.

    It is clear that in 2021 there was progress, in part guided by the inclusion in the REF guidance of a clear indicator. However, there is still a long way to go and it is clear open research was understood and evidenced in ways that could exclude some research fields, epistemologies and transparent research practices.

    REF 2029

    With REF 2029, the new People, Culture and Environment element has created a stronger incentive to drive culture change across the sector. Institutions are embracing the move beyond compliance, making openness and transparency a core part of everyday research practice. However, alignment between this sector move, REF policy and funder action remains essential to address this collective action problem and therefore ensure that this progress is maintained.

    To step back now would not only risk slowing, or even undoing, progress, but would send confused signals that openness and transparency may be optional extras rather than essentials for a trusted research system. Embedding this move is not optional: a culture of openness is essential for the sustainability of UK research and development, for the quality of research processes, and for ensuring that outputs are not just excellent, but also trustworthy in a time of mass misinformation.

    Openness, transparency and accountability are key attributes of research, and hallmarks of the culture that we want to see in the sector now and in the future. Critically, coordinated sector-wide, institutional and individual actions are all needed to embed more openness into everyday research practices. This is not just about compliance – it is about a genuine culture shift in how research is conducted, shared and preserved. It is about doing the right thing in the right way. If that is accepted, then we would challenge those advocating for reducing the importance of those practices in the REF: what is your alternative, and will it command public trust?

     

    This article was supported by contributions from:

    Michel Belyk (Edge Hill University), Nik Bessis (Edge Hill University), Cyclia Bolibaugh (University of York), Will Cawthorn (University of Edinburgh), Joe Corneli (Oxford Brookes University), Thomas Evans (University of Greenwich), Eleanora Gandolfi (University of Surrey), Jim Grange (Keele University), Corinne Jola (Abertay University), Hamid Khan (Imperial College, London), Gemma Learmonth (University of Stirling), Natasha Mauthner (Newcastle University), Charlotte Pennington (Aston University), Etienne Roesch (University of Reading), Daniela Schmidt (University of Bristol), Suzanne Stewart (University of Chester), Richard Thomas (University of Leicester), Steven Vidovic (University of Southampton), Eric White (Oxford Brookes University).

    Source link

  • The disagreements on REF cannot go on forever – it may be time for a compromise

    The disagreements on REF cannot go on forever – it may be time for a compromise

    The submission deadline for REF is autumn 2028. It is not very far away and there are still live debates on significant parts of the exercise without an obvious way forward in sight.

    As the Contributions to Knowledge and Understanding guidance makes clear there are still significant areas where guidance is being awaited. The People, Culture and Environment (PCE) criteria and definitions will be published in autumn this year. Undoubtedly, this will kick off rounds of further debate on REF and its purposes. It feels like there is a lot left to do with not much time left to do it in.

    Compromise

    The four UK higher education funding bodies could take a view that the levels of disquiet in the sector about REF, and what I am hearing at the events I go to and from the people I speak to it does seem significant, will eventually dissipate as the business of REF gets underway.

    This now seems unlikely. It is clear that there are increasingly entrenched views on the workability or not of the new portability measures, and there is still the ongoing debate on the extent to which research culture can be measured. Research England has sought to take the sector toward ends which have broad support, improving the diversity and conditions of research, but there is much less consensus on how to get there.

    The consequences for continuing as is are unpredictable but they are potentially significant. At the most practical level the people working on REF only have so much resource and bandwidth. The debate about the future of REF will not go away as more guidance is released, in fact the debate is likely to intensify, and getting to submission where there is still significant disagreement will drain resources and time.

    The debate also crowds out the other work that is going on in research. All the while that the future of REF is being debated it is time taken away from all of the funding which is not allocated through REF, all of the problems with research that do not stem from this quinquennial exercise, and the myriad of other research issues that sit beyond the sector’s big research audit. The REF looms large in the imagination of the sector but the current impasse is eclipsing much else.

    If the government believes that REF does not have broad support from the sector it could intervene. It is faulty to assume that the REF is an inevitable part of the research landscape. As Chancellor, Gordon Brown attempted to axe its predecessor on the basis that it had become too burdensome. Former advisor to the Prime Minister Dominic Cummings also wished to bin the REF. UCU opposed REF 2014. Think Tank UK Day One also published a well shared paper on the argument for scrapping the current REF.

    The REF has survived because of lack of better alternatives, its skilful management, and its broad if not sometimes qualified support. The moment the political pain of REF outweighs its perceived research benefits it will be ripe for scrapping by a government committed to reducing costs and reducing the research burden.

    The future

    The premise of the new REF is that research is a team sport and the efforts of the team that create the research should be measured and therefore rewarded. The corollary of identifying research as a product of a unit rather than an individual is that the players, in this case researchers and university staff, have had their skills unduly diminished, hidden, or otherwise not accounted for because of pervasive biases in the research landscape.

    It is impossible to argue that by any reasonable measure there aren’t significant issues with equality in research. This impacts the lives and career prospects of researchers and the UK economy as whole. It would be an issue for any serious research funder to back away from work that seeks to improve the diversity of research.

    It is in this light where perhaps the biggest risk of all lies for Research England. If it pushes on with the metrics and measures it currently has and the result of REF is seen as unfair or structurally unsound it will do irreversible harm to the wider culture agenda. The idea of measuring people, culture, and environment will be put into the “too hard to do” box.

    This work is too important to be done quickly but the urgency of the challenge cannot be dropped. It is an unenviable position to be in.

    REF 2030?

    If a conclusion is reached that it is not feasible to carry the sector toward a new REF in time for 2029 there only seems to be one route forward which is to return to a system more like 2021. This is not because the system was perfect (albeit it was generally seen as a good exercise) but because it would be unfeasible to carry out further system changes at this stage. Pushing the exercise back to 2030 would mean allocating funding from an exercise completed almost a decade prior. It seems untenable to do so because of how much institutions will have changed in this period.

    The work going on to measure PCE is not only helpful in the context of REF but alongside work coming out of the Metascience Unit and UKRI centrally, among others, part of the way in which the sector can be supported to measure and build a better research culture and environment. This work within the pilots is of such importance that it would make sense to stand these groups up over a long time period with a view to building to the next exercise, while improving practice within universities more generally on an ongoing basis.

    As I wrote back in 2023 complexity in REF is worthwhile where it enhances university research. The complexity has now become the crux of the debate. If Research England reaches the conclusion that the cost and complexity of the desired future outstrips the capacity and knowledge of the present, the opportunity is to pause, pilot, learn, improve, and go again.

    Tactical compromise for now – with the explicit intention of taking time to agree a strategic direction on research as more of a shared and less of an individual endeavour – is possible. To do so it will require making the political and practical case for a different future (as well as the moral one) ever more explicit, explaining the trade-offs it will involve, and crucially building a consensus on how that future will be funded and measured. Next year is a decade on from the Stern Review; perhaps it is time for another independent review of REF.

    A better future for research is possible but only where the government, funders, institutions, and researchers are aligned.

    Source link