Tag: reflection

  • Pause for REFlection: Time to review the role of generative AI in REF2029

    Pause for REFlection: Time to review the role of generative AI in REF2029

    Author:
    Nick Hillman

    Published:

    • This blog has been kindly written for HEPI by Richard Watermeyer (Professor of Higher Education and Co-Director of the Centre for Higher Education at the University of Bristol), Tom Crick (Professor of Digital Policy at Swansea University) and Lawrie Phipps (Professor of Digital Leadership at the University of Chester and Senior Research Lead at Jisc).
    • On Tuesday, HEPI and Cambridge University Press & Assessment will be hosting the UK launch of the OECD’s Education at a Glance. On Wednesday, we will be hosting a webinar on students’ cost of living with TechnologyOne – for more information on booking a free place, see here.

    For as long as there has been national research assessment exercises (REF, RAE or otherwise), there have been efforts to improve the way with which research is evaluated and Quality Related (QR) research funding consequently distributed. Where REF2014 stands out for its introduction of impact as a measure of what counts as research excellence, for REF2029, it has been all about research culture. Though where impact has become an integral dimension of the REF, the installation of research culture (into a far weightier environment or as has been proposed People, Culture and Environment (PCE) statement) as a criterion of excellence appears far less assured, especially when set against a three-month extension to REF2029 plans. 

    A temporary pause on proceedings has been announced by Sir Patrick Vallance, the UK Government’s Minister for Science, as a means to ensure that the REF provides ‘a credible assessment of quality’. The corollary of such is that the hitherto proposed formula (many parts of which remain formally undeclared – much to the frustration of universities’ REF personnel and indeed researchers) is not quite fit for purpose, and certainly not so if the REF is to ‘support the government’s economic and social missions’. Thus, it may transpire that research culture is ultimately downplayed or omitted from the REF. For some, this volte face, if it materialises, may be greeted with relief; a pragmatic step-back from the jaws of an accountability regime that has become excessively complex, costly and inefficient (if not even estranged from the core business of evaluating and then funding so-called ‘excellent’ research) and despite proclamations at the conclusion of its every instalment, that next time it will be less burdensome.   

    While the potential backtrack on research culture and potential abandonment of PCE statements will be focused on to explain the REF’s most recent hiatus, these may be only cameos to discussion of its wider credibility and utility; a discussion which appears to be reaching apotheosis, not least given the financial difficulties endemic to the UK sector, which the REF, with its substantial cost, is counted as further exacerbating. Moreover, as we are finding in our current research, the REF may have entered a period not limited to incremental reform and tinkering at the edges but wholesale revision; and this as a consequence of higher education’s seemingly unstoppable colonisation by artificial intelligence. 

    With recent funding from Research England, we have undertaken to consult with research leaders and specialist REF personnel embedded across 17 UK HEIs – including large, research-intensive institutions and those historically with a more modest REF footprint, to gain an understanding of existing views of and practices in the adoption of generative AI tools for REF purposes. While our study has thrown up multiple views as to the utility and efficacy of using generative AI tools for REF purposes, it has nonetheless revealed broad consensus that the REF will inevitably become more AI-infused and enabled, if not ultimately, if it is to survive, entirely automated. The use of generative AI for purposes of narrative generation, evidence reconnaissance, and scoring of core REF components (research outputs and impact case studies) have all been mooted as potential applications with significant cost and labour-saving affordances and applications which might also get closer to ongoing, real-time assessments of research quality, unrestricted to seven-year assessment cycles. Yet the use of generative AI has also been (often strongly) cautioned against for the myriad ways with which it is implicated and engendered with bias and inaccuracy (as a ‘black box’ tool) and can itself be gamed in multiple ways, for instance in ‘adversarial white text’. This is coupled with wider ongoing scientific and technical considerations regarding transparency, provenance and reproducibility. Some even interpret its use as antithetical to the terms of responsible research evaluation set out by collectives like CoARA and COPE.

    Notwithstanding, such various objections, we are witnessing these tools being used extensively (if in many settings tacitly and tentatively) by academics and professional services staff involved in REF preparations. We are also being presented with a view that the use of GenAI tools by REF panels in four years’ time is a fait accompli, especially given the speed by which the tools are being innovated. It may even be that GenAI tools could be purposed in ways that circumvent the challenges of human judgement, the current pause intimates, in the evaluation of research culture. Moreover, if the credibility and integrity of the REF ultimately rests in its capacity to demonstrate excellence via alignment with Government missions (particularly ‘R&D for growth’), then we are already seeing evidence of how AI technologies can achieve this.

    While arguments have been previously made that the REF offers good value for (public) money, the immediate joint contexts of severe financial hardship for the sector; ambivalence as to the organisational credibility of the REF as currently proposed; and the attractiveness of AI solutions may produce a new calculation. This is a calculation, however, which the sector must own, and transparently and honestly. It should not be wholly outsourced, and especially not to one of a small number of dominant technology vendors. A period of review must attend not only to the constituent parts of the REF but how these are actioned and responded to. A guidebook for GenAI use in the REF is exigent and this must place consistent practice at its heart. The current and likely escalating impact of Generative AI on the REF cannot be overlooked if it is to be claimed as a credible assessment of quality. The question then remains: is three months enough? 

    Notes

    • The REF-AI study is due to report in January 2026. It is a research collaboration between the universities of Bristol and Swansea and Jisc.
    • With generous thanks to Professor Huw Morris (UCL IoE) for his input into earlier drafts of this article.

    Source link

  • Getting it ‘right’ – a reflection on integrating Service Learning at scale into a large Faculty of Science and Engineering

    Getting it ‘right’ – a reflection on integrating Service Learning at scale into a large Faculty of Science and Engineering

    This blog was kindly authored by Professor Lynne Bianchi, Vice Dean for Social Responsibility & Equality, Diversity, Inclusion and Accessibility, at the University of Manchester

    I recently had the fortune to be part of a panel discussing the place of Service Learning in higher education, chaired by HEPI. My reflections before and since may inspire you to take time to think about your perspective on the nature and role of Service Learning in fast-changing university and civic landscapes. In its simplest sense, Service Learning is an educational approach that combines academic study with community service.

    In my role within a large science and engineering faculty, I have rallied our staff and students to think seriously about the features, advantages and benefits of Service Learning in science and engineering contexts. For our university, this teaching and learning approach isn’t new, with expertise in the biomedical sciences and humanities teaching us much about the way in which undergraduate students can create benefit for our local communities whilst enriching their own academic experiences.

    In this blog, I build on my own background as a teacher and higher education academic and draw on my experience in curriculum design when focusing on how we can provide authentic and impactful Service Learning experiences for our undergraduates.

    What do we mean by the ‘right’ learning experiences?

    It doesn’t take long working in this area to unearth a wide range of terms that are used interchangeably – from place-based learning, real-world learning, community-engaged learning, practice-based learning, critical urban pedagogy, industry-inspired learning and more. A gelling feature is that to get Service Learning working well there must be an authentic benefit to each party involved. The students should develop skills and understanding directly required within their degree, and the partner should have a problem explored, solved, or informed. In essence, the experience must lead to a ‘win-win’ outcome(s) to be genuine.

    In our context in science and engineering, we have envisioned Service Learning working well, and considered this to include when:

    For students:

    • Learning has relevance: work on a project, individually or in groups, is contextualised by a problem, issue or challenge that is authentic (as opposed to hypothetical).
    • Learning has resonance: developing and applying skills and knowledge to inform the problem, issue or project that dovetails with existing course specifications and requirements.

    For partners:

    • They are engaged: partners are involved in the design and delivery of the project to some extent. This may vary in the depth or level of engagement and requires both sides to appreciate the needs of each other.
    • They are enriching: partners identify real issues that matter and expose elements of the work environment that enrich students’ awareness of the workplace and career pathways.

    When is the right time for students to engage in service learning?

    I am still pondering this question as there are so many variables and options that influence the choice. Which year group should service learning drop into? Or, does a developmental over time approach suit better? Is Service Learning more impactful in the later undergraduate years, or should it be an integral part of each year of their experience with us? Realistically, there won’t be a one-size-fits-all all model, and there are benefits and challenges to each. What will need to underpin whichever approach we take, will be the focused need to elicit the starting points of our students, our staff and our partners in whichever context.

    Going from ‘zero to hero’ in Service Learning will require training and support for all parties. My experience working across the STEM sector for nearly three decades has taught me that no one partner is the same as another – what is a big deal to one can mean nothing to another. My thinking is that we need to see each person involved in the Service Learning experience as a core ‘partner’ and each has learning starting points, aspirations and apprehensions. Our role as programme leaders is to identify a progression model that appreciates that this is ‘learning’ and that scaffolds and key training will be required at different times – even within the process itself.

    What support will be required to mobilise this model at scale?

    In my early career at this university, I spent time within the Teaching & Learning Student Experience Professional Support teams, where I saw firsthand the integral way that any university programme relies on expertise in taking theoretical ideas into practice. The interplay between project management, planning, timetabling, eLearning, marketing and communications and student experience support teams, to name some, will have play such critical roles in achieving excellence in Service Learning. Working at scale in our faculty across 10 different discipline areas, will require integrated work with other faculties to harness the power of interdisciplinary projects and digital support for course delivery and assessment that can embrace an internal-external interface.

    Support for scaling up will also require a culture of risk-taking to be valued and championed. Over the introductory years, we need to provide a sense of supported exploration, a culture of learning and reflection, and an ethos where failure is rarely a negative, but an opportunity. Of course, science and engineering disciplines bring with them our obligations to accrediting bodies, and a close dialogue with them about ambition, relevance and need for this enriching approach needs to be clearly articulated and agreed so that any course alteration becomes a course invigoration rather than a compromise.

    Faculty culture and the way the university and the sector views and reviews SL will have a significant implication on practice and people feeling safe to innovate. As the university forges and launches its 2035 strategy the spaces for innovation and development are increasingly championed, and the months and years ahead will be ones to watch in terms of establishing a refreshed version of teaching and learning for our students.

    In closing this short exploration of Service Learning, I can feel a positive tension in the air – the excitement to work together to further invigorate our student experience whilst supporting our staff and partners to embrace varied new opportunities. The ‘getting it right’ story will have many chapters, many endings as the genres, characters and plots are there for us all to create – or more pertinently ‘co-create’! What drives me most to remain in this space of uncertainty for a while longer is the anticipation of creating experiences that truly make a difference for good. As our universities transform themselves over the coming years, I invite you to join us in the dialogue and development as we have so much to learn through collaboration.

    Source link

  • A 20-Year Reflection on Transparency and the Illusion of Access (Glen McGhee)

    A 20-Year Reflection on Transparency and the Illusion of Access (Glen McGhee)

    The cancellation of the latest NACIQI (National Advisory Committee on Institutional Quality and Integrity) meeting brought back bitter memories that refuse to fade. 

    It’s been twenty years since I traveled to Washington, DC—dressed in my best lobbying attire and carrying a meticulous roster of Department of Education staff—to visit the Office of Postsecondary Education (OPE) on K Street. My goal was simple, even noble: to seek answers about the opaque workings of accreditation in American higher education. What I encountered instead was a wall of silence, surveillance, and authoritarianism.

    I stepped off the elevator on the seventh floor of the Department building and signed in. Under “Purpose of Visit,” I wrote: Reform. I was calm, professional, and respectful. I asked to see the NACIQI Chair, Bonnie, hoping that she would be willing to speak with me about a system that, even then, was falling into disrepair. But what happened next still infuriates me.

    Within seconds, two armed, uniformed guards approached me. They didn’t ask questions. They gave an ultimatum: leave or be arrested.

    I eventually complied, descending into the lobby, still stunned. From there I began dialing—one by one—through the directory of names I had so carefully assembled. I called staffers, analysts, assistants, anyone who might answer. Not a single person picked up. I could feel the eyes of the guards watching me, one of them posted on the mezzanine like a sniper keeping watch over a public enemy. I was not dangerous. I was not disruptive. I was, however, unwanted.

    The next day, I turned to my Congressman, Allen Boyd, whose LA generously tried to intervene. His office contacted OPE, attempting to broker a meeting on my behalf. The Department didn’t even return his call. Apparently, a sitting member of Congress—who didn’t sit on a high-ranking committee—carried no weight at the fortress of federal education oversight.

    This most recent overstepping by US ED—unilaterally postponing NACIQI’s Summer 2025 meeting—reminds observers of how limited the oversight provided by NACIQI really is. It is, apparently, nothing more than a performative shell that fulfills ceremonial functions, and not much more.

    I would argue that this latest episode reveals that NACIQI is less an independent watchdog and more a ceremonial body with limited real power, and so my view differs somewhat from David Halperin, because he sees more substantive activity than I do.

    The history of ACICS (Accrediting Council for Independent Colleges and Schools) and SACS (Southern Association of Colleges) appearing before NACIQI illustrates how regulatory capture can manifest not only through industry influence, but also through bureaucratic design and process control. The OPE’s central role, combined with NACIQI’s limited enforcement power, has allowed failing accreditors to retain recognition for years, even in the face of overwhelming evidence of noncompliance and harm to students.

    The illusion of accountability has long been a feature of the accreditation system, not a flaw. NACIQI meetings, when they occur, are tightly scripted, with carefully managed testimony and limited public engagement. The real decisions are made elsewhere, behind closed doors, often under the influence of powerful lobbying groups and entrenched bureaucracies that resist transparency and reform at every turn.

    Despite the increasing scrutiny on higher education and growing public awareness of student debt, poor educational outcomes, and sham institutions, the federal recognition of accreditors remains an elite-controlled process. It is a closed loop. Institutions, accreditors, and government officials all play their roles in a carefully choreographed performance that rarely leads to systemic change. The result is a system that protects institutions at the expense of students, particularly the most vulnerable—low-income, first-generation, and minority students who are often targeted by predatory schools hiding behind federal accreditation.

    This is the reality of the U.S. Department of Education’s accreditation apparatus: inaccessible, unaccountable, and increasingly symbolic. NACIQI, far from being an independent advisory body, has always functioned as a ceremonial front for political appointees and entrenched interests. It is, as I see it, just another arm of Vishnu—multiplicitous, all-seeing, but ultimately indifferent to critique or reform. Whether it’s chaired by a bureaucrat or a former wrestling executive like Linda McMahon, the outcome is the same: the process is rigged to exclude dissent and suppress scrutiny.

    And yet, pundits today still fail to grasp the implications. They speak of accreditation as if it were a technocratic process guided by evidence and integrity. They act as if NACIQI were a neutral arbiter. But I know otherwise, because I was there—thrown out, silenced, and treated like a trespasser in the very institution that claims to protect educational quality and student interest.

    This is more than personal bitterness. It’s about structural rot. When critics are expelled, when staff are muzzled, and when public servants ignore elected representatives, we are not dealing with oversight—we are witnessing capture. Accreditation in this country serves the accreditors and the institutions, not students, not taxpayers, and certainly not reformers.

    Two decades later, the anger remains. So does the silence.


    Sources:
    Department of Education building directory and procedures (2005)
    Congressional Office of Rep. Allen Boyd (archival record, 2005)
    Public notices regarding NACIQI meeting cancellations (2024–2025)
    David Halperin, Republic Report

    Source link

  • Lifting as We Climb: A Reflection on Mentorship, Growth, and Leadership in Nursing Education – Faculty Focus

    Lifting as We Climb: A Reflection on Mentorship, Growth, and Leadership in Nursing Education – Faculty Focus

    Source link

  • How You Can Habituate the Circular Model of Reflection: Before-Action, During-Action, After-Action, and Beyond-Action – Faculty Focus

    How You Can Habituate the Circular Model of Reflection: Before-Action, During-Action, After-Action, and Beyond-Action – Faculty Focus

    Source link

  • How You Can Habituate the Circular Model of Reflection: Before-Action, During-Action, After-Action, and Beyond-Action – Faculty Focus

    How You Can Habituate the Circular Model of Reflection: Before-Action, During-Action, After-Action, and Beyond-Action – Faculty Focus

    Source link

  • Empower Learners for the Age of AI: a reflection – Sijen

    Empower Learners for the Age of AI: a reflection – Sijen

    During the Empower Learners for the Age of AI (ELAI) conference earlier in December 2022, it became apparent to me personally that not only does Artificial intelligence (AI) have the potential to revolutionize the field of education, but that it already is. But beyond the hype and enthusiasm there are enormous strategic policy decisions to be made, by governments, institutions, faculty and individual students. Some of the ‘end is nigh’ messages circulating on Social Media in the light of the recent release of ChatGPT are fanciful click-bait, some however, fire a warning shot across the bow of complacent educators.

    It is certainly true to say that if your teaching approach is to deliver content knowledge and assess the retention and regurgitation of that same content knowledge then, yes, AI is another nail in that particular coffin. If you are still delivering learning experiences the same way that you did in the 1990s, despite Google Search (b.1998) and Wikipedia (b.2001), I am amazed you are still functioning. What the emerging fascination about AI is delivering an accelerated pace to the self-reflective processes that all university leadership should be undertaking continuously.

    AI advocates argue that by leveraging the power of AI, educators can personalize learning for each student, provide real-time feedback and support, and automate administrative tasks. Critics argue that AI dehumanises the learning process, is incapable of modelling the very human behaviours we want our students to emulate, and that AI can be used to cheat. Like any technology, AI also has its disadvantages and limitations. I want to unpack these from three different perspectives, the individual student, faculty, and institutions.


    Get in touch with me if your institution is looking to develop its strategic approach to AI.


    Individual Learner

    For learners whose experience is often orientated around learning management systems, or virtual learning environments, existing learning analytics are being augmented with AI capabilities. Where in the past students might be offered branching scenarios that were preset by learning designers, the addition of AI functionality offers the prospect of algorithms that more deeply analyze a student’s performance and learning approaches, and provide customized content and feedback that is tailored to their individual needs. This is often touted as especially beneficial for students who may have learning disabilities or those who are struggling to keep up with the pace of a traditional classroom, but surely the benefit is universal when realised. We are not quite there yet. Identifying ‘actionable insights’ is possible, the recommended actions harder to define.

    The downside for the individual learner will come from poorly conceived and implemented AI opportunities within institutions. Being told to complete a task by a system, rather than by a tutor, will be received very differently depending on the epistemological framework that you, as a student, operate within. There is a danger that companies presenting solutions that may work for continuing professional development will fail to recognise that a 10 year old has a different relationship with knowledge. As an assistant to faculty, AI is potentially invaluable, as a replacement for tutor direction it will not work for the majority of younger learners within formal learning programmes.

    Digital equity becomes important too. There will undoubtedly be students today, from K-12 through to University, who will be submitting written work generated by ChatGPT. Currently free, for ‘research’ purposes (them researching us), ChatGPT is being raved about across social media platforms for anyone who needs to author content. But for every student that is digitally literate enough to have found their way to the OpenAI platform and can use the tool, there will be others who do not have access to a machine at home, or the bandwidth to make use of the internet, or even to have the internet at all. Merely accessing the tools can be a challenge.

    The third aspect of AI implementation for individuals is around personal digital identity. Everyone, regardless of their age or context, needs to recognise that ‘nothing in life is free’. Whenever you use a free web service you are inevitably being mined for data, which in turn allows the provider of that service to sell your presence on their platform to advertisers. Teaching young people about the two fundamental economic models that operate online, subscription services and surveillance capitalism, MUST be part of ever curriculum. I would argue this needs to be introduced in primary schools and built on in secondary. We know that AI data models require huge datasets to be meaningful, so our data is what fuels these AI processes.

    Faculty

    Undoubtedly faculty will gain through AI algorithms ability to provide real-time feedback and support, to continuously monitor a student’s progress and provide immediate feedback and suggestions for improvement. On a cohort basis this is proving invaluable already, allowing faculty to adjust the pace or focus of content and learning approaches. A skilled faculty member can also, within the time allowed to them, to differentiate their instruction helping students to stay engaged and motivated. Monitoring students’ progress through well structured learning analytics is already available through online platforms.

    What of the in-classroom teaching spaces. One of the sessions at ELAI showcased AI operating in a classroom, interpreting students body language, interactions and even eye tracking. Teachers will tell you that class sizes are a prime determinant of student success. Smaller classes mean that teachers can ‘read the room’ and adjust their approaches accordingly. AI could allow class sizes beyond any claim to be manageable by individual faculty.

    One could imagine a school built with extensive surveillance capability, with every classroom with total audio and visual detection, with physical behaviour algorithms, eye tracking and audio analysis. In that future, the advocates would suggest that the role of the faculty becomes more of a stage manager rather than a subject authority. Critics would argue a classroom without a meaningful human presence is a factory.

    Institutions

    The attraction for institutions of AI is the promise to automate administrative tasks, such as grading assignments and providing progress reports, currently provided by teaching faculty. This in theory frees up those educators to focus on other important tasks, such as providing personalized instruction and support.

    However, one concern touched on at ELAI was the danger of AI reinforcing existing biases and inequalities in education. An AI algorithm is only as good as the data it has been trained on. If that data is biased, its decisions will also be biased. This could lead to unfair treatment of certain students, and could further exacerbate existing disparities in education. AI will work well with homogenous cohorts where the perpetuation of accepted knowledge and approaches is what is expected, less well with diverse cohorts in the context of challenging assumptions.

    This is a problem. In a world in which we need students to be digitally literate and AI literate, to challenge assumptions but also recognise that some sources are verified and others are not, institutions that implement AI based on existing cohorts is likely to restrict the intellectual growth of those that follow.

    Institutions rightly express concerns about the cost of both implementing AI in education and the costs associated with monitoring its use. While the initial investment in AI technologies may be significant, the long-term cost savings and potential benefits may make it worthwhile. No one can be certain how the market will unfurl. It’s possible that many AI applications become incredibly cheap under some model of surveillance capitalism so as to be negligible, even free. However, many of the AI applications, such as ChatGPT, use enormous computing power, little is cacheable and retained for reuse, and these are likely to become costly.

    Institutions wanting to explore the use of AI are likely to find they are being presented with additional, or ‘upgraded’ modules to their existing Enterprise Management Systems or Learning Platforms.

    Conclusion

    It is true that AI has the potential to revolutionize the field of education by providing personalized instruction and support, real-time feedback, and automated administrative tasks. However, institutions need to be wary of the potential for bias, aware of privacy issues and very attentive to the nature of the learning experiences they enable.


    Get in touch with me if your institution is looking to develop its strategic approach to AI.


    Image created using DALL-E

    Source link