Tag: Wonkhe

  • How postdocs get on | Wonkhe

    How postdocs get on | Wonkhe

    A new paper in The Proceedings of the National Academy of Sciences suggests that the postdoc period, the period of flexible, temporary and often insecure work after a PhD but before a permanent role, is “no less critical than the Ph.D. in determining future academic careers.”

    The paper by Duan et al shows that in America the PhD studying period is not in and of itself the only determinant of whether a student will secure an academic role. They demonstrate that

    Those whose productivity went down during the postdoc, and those without a hit paper during this period are significantly more likely to drop out of academia than others

    The paper also argues that the research active and geographically mobile do better in securing a faculty job. The authors argue this is because their “diverse academic experience gives an advantage.”

    Waiting around

    The paper suggests that waiting for a faculty job is usually the wrong path. This is to say that doing a PhD, working as a postdoc, and then securing a coveted faculty role, sounds more straightforward on paper than it is in reality.

    This is interesting in itself but it’s also a question of how a market for talent functions. And this is important because the whole economy relies on universities selecting academics that are the best in the field, not just those more likely to get picked by dint of good fortune or demographics.

    The most obvious way to look at how this market works is to look at how universities themselves describe it. Let’s move away from the US examples and look to the UK.

    There is lots of advice for PhDs seeking to break into the academic job market. This slideshow from LSE presents the kind of information that is typically shared with aspiring academics. The slides suggest the UK academic job-market is more multivariate than the US with greater flexibility in choosing between teaching and research routes without the possibility of tenure in either. The slides, albeit now a bit dated, show that promotion depends on a mixture of teaching, research, service, and public engagement. So far so REF.

    The University of Salford’s advice from 2023 includes a good chunk of information on the administrative responsibilities of academics and its broader emphasis for the aspirant academic is a practical one. Their guidance looks at the kinds of skills an academic needs including: passion, communication skills, team-working, and networking skills. And advice on getting that first academic job foremost of which is publishing research.

    The University of Oxford has an extensive and nuanced set of guidance which brings the dynamics of the labour market into sharper relief. This particular section captures the sentiment that undercuts much of the academic career guidance “Only a tiny percentage of PhD graduates become professors; the vast majority take their research and teaching training to make significant contributions in other fields.”

    These are only three examples amongst dozens of guides on getting an academic career split across hundreds and hundreds of pages. The underlying themes are that getting a first academic role is hard, it is largely based on research record (and luck) and the extent to which a PhD student has been published, and building a broad skills base with flexibility over job role and location is helpful.

    Jobs

    Of course as well as being educators, institutions are also employers. An analysis of academic job postings in Europe demonstrates that research is the primary job criteria for early career academics with emphasis on teaching and other skills becoming more important as academics progress up the career ladder. Albeit, as pointed out on Wonkhe, within the UK there is a significant growth in teaching only academic contracts. Even more specifically within the UK there is a literature over many decades which emphasises the importance of teaching, writing, and networking, the porosity between programmes and other institutions in careers, and the global precarity of junior academics

    The challenge that emerges for the PhD and early post-doc breaking into and through the job market are therefore twofold. The first is that the skills and experience required to secure a first permanent academic role are effectively the same as someone already carrying out a full-time academic role. This is a big hurdle to clear. The second is that the conditions of postdoc students, particularly the lack of stability, makes acquiring those skills difficult.

    In line with the study emerging from the US, if a UK post-doc wants to get on academic literature suggests they are best-placed to do so by being fortunate enough to have high-quality instruction and they may benefit from structured support through programmes like Prosper.

    There is potentially an endless list of the ways in which PhD and postdoc study shape future academic careers. The analysis here does not even touch on the various ways in which social and economic inequalities shut down or otherwise open up career paths.

    Nevertheless, the UK’s industrial strategy relies on a pipeline of academics progressing in both established fields and emergent ones. The lack of institutionalised knowledge on not just who gets on but the conditions through which students get on presents not only an institutional risk in losing talent in the academic pipeline but an economic one in allowing future academics to slip out of the system.

    Source link

  • Ventriloquising the student interest | Wonkhe

    Ventriloquising the student interest | Wonkhe

    Following the devastating review offered by the 2023 report of the Industry and Regulators Committee of the House of Lords, the Office for Students’ (OfS) proposed strategy makes a great play of being centred around “the student interest”.

    But while it recognises that students have diverse and changeable views about their interests, it is still significant that it characterises these as “the student interest” rather than “students’ interests”.

    The reason for doing this is that it makes it much more rhetorically powerful to claim you are doing something in relation to an interest that is definitive, rather than interests which are multivarious and shifting.

    And be clear, the OfS proposed strategy shows a huge appetite to intervene in higher education in the name of “the student interest”.

    Much talk, no sources

    In the draft, OfS boasts that it has done a great deal of work to renew its understanding of the student interest – polling students, holding focus groups, hosting engagement sessions and talking to their own student panel.

    But two things are particularly noticeable about this work. First, whilst a lot of other sources are referenced in their strategy consultation, this is one area where no evidence is provided.

    This means the OfS interpretation of the outcomes of this consultation cannot be interrogated in any way. Clearly OfS knows best how to interpret this interest and isn’t interested in collective conversations to explore its ambiguities and complexities.

    Second, none of this work involves open ended engagement with students and their representative organisations (who appear to have been excluded completely, or at least their involvement is not detailed). They are all forms of consultation in which OfS would have framed the terms and agenda of the discussions (non-decision-making power, as Steven Lukes would have it). It’s consultation – but within tightly defined limits of what can legitimately be said.

    This seems to explain the remarkable number of priorities in the strategy (freedom of speech, mental health, sexual harassment) that are said to be in the student interest but previously appeared in ministerial letters outlining the strategic priorities of the OfS.

    Get a job

    Perhaps most concerning is that the government/treasury logic that the only real reason for going to university is to get a well-paid job is now central to the student interest. Sometimes this is done more subtly by positioning it in the (never-)popular student language of “a return on investment”:

    …in return for their investment of time, money and hard work they [students] expect that education to continue to provide value into the longer-term, including in ways that they may not be able to anticipate while they study (p.12).

    At other times, we are left in no doubt that the primary function of higher education is to serve the economy:

    Our proposals…will support a higher education system equipped to cultivate the skills the country needs and increase employer confidence in the value of English higher education qualifications. High quality higher education will be accessible to more people, and students from all backgrounds will be better able to engage with and benefit from high quality higher education, supporting a more equal society which makes better use of untapped talent and latent potential. The supply of skilled graduates will support local and national economies alike, while the ‘public goods’ associated with high quality higher education will accrue to a wide range of individuals and communities. Public goods include economic growth, a more equal society and greater knowledge understanding (OfS 2024 p.30-31).

    So what we are left with is a proposed strategy that makes powerful claims to be grounded in the student interest – but which could have easily formed part of the last government’s response to the Augar review.

    Whose priorities?

    Through its consultation on its proposed strategy, OfS has presented the priorities of the previous government as if they are drawn straight from its engagement with students.

    We don’t yet know the higher education priorities of the current government, but given the proposed strategy was published under their watch it looks like we are moving in a depressingly familiar direction.

    It is worth reflecting on the profound injustice of this. Students are expected to pay back the cost of their higher education and now have the previous government’s priorities presented as their interest so that OfS can intervene in higher education.

    Yes, you have to pay – but the government and its friendly neighbourhood regulator are here to tell you why you want to pay! It seems that despite the excoriating criticism of the House of Lords Committee, OfS have not really learned how to engage with students or to reflect and reconcile their interests.

    Source link

  • Higher education postcard: Dartington | Wonkhe

    Higher education postcard: Dartington | Wonkhe

    Dartington Hall is a splendid old country house.

    Its great hall dates from 1388, and has a wonderful hammerbeam roof, an a porch where the arms of Richard II can still be seen. The trouble with keeping old buildings going – when you’re no longer a medieval feudal lord, and when wages have risen – is that upkeep is pretty tough. So it was fortunate that Dartington was bought, in 1925, by Leonard and Dorothy Elmhirst.

    The Elmhirsts were interesting. Leonard was from minor English gentry, poor but clearly clever (he completed his degree in agriculture from Cornell in two years). Dorothy inherited at 17 a fortune (about half a billion dollars in todays money). She met Leonard when he was seeking donations to support his club for international students at Cornell. Romance blossomed: they married in 1925.

    But before then, Leonard had accepted a job as secretary to Nobel-prize-winning polymath, poet and painter Rabindranath Tagore. This took him to India, where he supported Tagore’s work on rural reconstruction. This influenced him and Dorothy to attempt something similar in the UK, and in 1925 they purchased Dartington Hall. This became a home for all sorts of experimental work – on agriculture and rural economics and society, and also arts and creativity. About 1500 people worked on the estate; it gave concerts (the BBC broadcast the English Singers Quartet on 24 November 1934, and on Saturday 1 December 1934 a concert by Claud Biggs on the piano, accompanying contralto Astra Desmond); and the Western Morning News and Daily Gazette was entranced, on 28 May 1934, with Uday Shan Kar’s Hindu dancing.

    In 1935 a charitable trust was established to run the estate, and a a wide variety of activities continued. These included summer schools (such as the Fabian Society school which is pictured on the card), concerts, classes of all sorts. Post war, these activities became more significant: an adult education centre was established in 1955, and in 1961 Dartington College of the Arts was founded. Initially this focused on training teachers in the arts: music, dance and drama, and visual arts.

    In 1973 the college gained “assisted” status and received funding from Devon County Council to extend its offer to provide undergraduate degrees. Degrees were awarded by the Council for National Academic Awards (CNAA), the body which awarded degrees across the non-university higher education sector. This was prompted by a shift in government policy to require teacher education to be to degree-level (the first of many such shifts in professional education over the years).

    The changes in funding arrangements in the late 1980s – the removal of polytechnics and colleges from local authority control, and the creation of the Polytechnics and Colleges Funding Council – created financial challenges for Dartington, as it lost its special funding. These were addressed by closing some programmes and rationalising others. And again, in 2006, the college faced financial difficulties. Scale appears to have been the problem, exacerbated by the college not owning its buildings and therefore being unable easily to expand student numbers.

    This time the problems were insurmountable, and the college merged what was then Falmouth University College, and is now Falmouth University. The provision was moved away from the Dartington site.

    But that isn’t the end of the story for higher education at Dartington: Dartington Hall Trust is registered with the Office for Students and established a provider with two faculties: Dartington School of Arts and Schumacher College (each of which continued the work of former colleges associated with the Dartington site).

    Back to the postcard: we’ve encountered the Fabians before, in the establishment of the London School of Economics. They seem to have run summer schools at Dartington for several summers in the 1940s at least, but I haven’t been able to track down which one this card depicts. Can anyone recognise any of the earnest (Bevin or otherwise) socialists?

    And here’s the jigsaw for you to have a go at. Have fun!

    Source link

  • Making SEISA official | Wonkhe

    Making SEISA official | Wonkhe

    Developing a new official statistic is a process that can span several years.

    Work on SEISA began in 2020 and this blog outlines the journey to official statistics designation and some key findings that have emerged along the way. Let’s firstly recap why HESA needed a new deprivation index.

    The rationale behind pursuing this project stemmed from an Office for Statistics Regulation (OSR) report which noted that post-16 education statistics lacked a UK-wide deprivation metric. Under the Code of Practice for Statistics, HESA are required to innovate and fill identified statistical gaps that align with our area of specialism.

    Fast forward almost six years and the UK Statistics Authority have reiterated the importance of UK-wide comparable statistics in their response to the 2024 Lievesley Review.

    Breaking down barriers

    While higher education policy may be devolved, all nations have ambitions to ensure there is equal opportunity for all. Policymakers and the higher education sector agree that universities have a pivotal role in breaking down barriers to opportunity and that relevant data is needed to meet this mission. Having UK-wide comparable statistics relating to deprivation based on SEISA can provide the empirical evidence required to understand where progress is being made and for this to be used across the four nations to share best practice.

    In developing SEISA, we referred to OSR guidance to produce research that examines the full value of a new statistic before it is classed as an ‘official statistic in development’. We published a series of working papers in 2021 and 2022, with the latter including comparisons to the Indices of Deprivation (the main area-based measure utilised among policymakers at present). We also illustrated why area-based measures remain useful in activities designed to promote equal opportunity.

    Our research indicated that the final indexes derived from the Indices of Deprivation in each nation were effective at catching deprived localities in large urban areas, such as London and Glasgow, but that SEISA added value by picking up deprivation in towns and cities outside of these major conurbations. This included places located within former mining, manufacturing and industrial communities across the UK, like Doncaster or the Black Country in the West Midlands, as well as Rhondda and Caerphilly in Wales. The examples below come from our interactive maps for SEISA using Census 2011 data.

    An area of Doncaster that lies within decile 4 of the English Index of Multiple Deprivation (2019)

    An area of Caerphilly that lies within decile 5 of the Welsh Index of Multiple Deprivation (2019)

    We also observed that SEISA tended to capture a greater proportion of rural areas in the bottom quintile when compared with the equivalent quintile of the Index of Multiple Deprivation in each nation.

    Furthermore, in Scotland, the bottom quintile of the Scottish Index of Multiple Deprivation does not contain any locations in the Scottish islands, whereas the lowest quintile of SEISA covers all council areas in the country. These points are highlighted by the examples below from rural Shropshire and the Shetland Islands, which also show the benefit that SEISA offers by being based on smaller areas (in terms of population size) than those used to form the Indices of Deprivation. That is, drawing upon a smaller geographic domain enables pockets of deprivation to be identified that are otherwise surrounded by less deprived neighbourhoods.

    A rural area of Shropshire that is placed in decile 5 of the English Index of Multiple Deprivation (2019)

    An area of the Shetland Islands that is within decile 7 of the Scottish Index of Multiple Deprivation (2020)

    Becoming an official statistic

    Alongside illustrating value, our initial research had to consider data quality and whether our measure correlated with deprivation as expected. Previous literature has highlighted how the likelihood of experiencing deprivation increases if you are a household that is;

    • On a low income
    • Lives in social housing
    • A lone parent family
    • In poor health

    Examining how SEISA was associated with these variables gave us the assurance that it was ready to become an ‘official statistic in development’. As we noted when we announced our intention for the measure to be assigned this badge for up to two years, a key factor we needed to establish during this time period was the consistency in the findings (and hence methodological approach) when Census 2021-22 data became available in Autumn 2024.

    Recreating SEISA using the latest Census records across all nations, we found there was a high level of stability in the results between the 2011 and 2021-22 Census collections. For instance, our summary page shows the steadiness in the associations between SEISA and income, housing, family composition and health, with an example of this provided below.

    The association between SEISA and family composition in Census 2011 and 2021-22

    Over the past twelve months, we’ve been gratified to see applications of SEISA in the higher education sector and beyond. We’ve had feedback on how practitioners are using SEISA to support their widening participation activities in higher education and interest from councils working on equality of opportunity in early years education. The measure is now available via the Local Insight database used by local government and charities to source data for their work.

    It’s evident therefore that SEISA has the potential to help break down barriers to opportunity across the UK and is already being deployed by data users to support their activities. The demonstrable value of SEISA and its consistency following the update to Census 2021-22 data mean that we can now remove the ‘in development’ badge and label SEISA as an official statistic.

    View the data for SEISA based on the Census 2021-22 collection, alongside a more detailed insight into why SEISA is now an official statistic, on the HESA website.

    Please feel free to submit any feedback you have on SEISA to [email protected].

    Read HESA’s latest research releases and if you would like to be kept updated on future publications, you can sign-up to our mailing list.

    Source link

  • Keep talking about data | Wonkhe

    Keep talking about data | Wonkhe

    How’s your student data this morning?

    Depending on how close you sit to your institutional student data systems, your answer may range from a bemused shrug to an anguished yelp.

    In the most part, we remain blissfully unaware of how much work it currently takes to derive useful and actionable insights from the various data traces our students leave behind them. We’ve all seen the advertisements promising seamless systems integration and a tangible improvement in the student experience, but in most cases the reality is far different.

    James Gray’s aim is to start a meaningful conversation about how we get there and what systems need to be in place to make it happen – at a sector as well as a provider level. As he says:

    There is a genuine predictive value in using data to design future solutions to engage students and drive improved outcomes. We now have the technical capability to bring content, data, and context together in a way that simply has not been possible before now.”

    All well and good, but just because we have the technology doesn’t mean we have the data in the right place or the right format – the problem is, as Helen O’Sullivan has already pointed out on Wonkhe, silos.

    Think again about your student data.

    Some of it is in your student information system (assessment performance, module choices), which may or may not link to the application tracking systems that got students on to courses in the first place. You’ll also have information about how students engage with your virtual learning environment, what books they are reading in the library, how they interact with support services, whether and how often they attend in person, and their (disclosed) underlying health conditions and specific support needs.

    The value of this stuff is clear – but without a whole-institution strategic approach to data it remains just a possibility. James notes that:

    We have learned that a focus on the student digital journey and institutional digital transformation means that we need to bring data silos together, both in terms of use and collection. There needs to be a coherent strategy to drive deployment and data use.

    But how do we get there? From what James has seen overseas, in the big online US providers like Georgia Tech and Arizona State data is something that is managed strategically at the highest levels of university leadership. It’s perhaps a truism to suggest that if you really care about something it needs ownership at a senior level, but having that level of buy-in unlocks the resource and momentum that a big project like this needs.

    We also talked about the finer-grained aspects of implementation – James felt that the way to bring students and staff on board is to clearly demonstrate the benefits, and listen (and respond) to concerns. That latter is essential because “you will annoy folks”.

    Is it worth this annoyance to unlock gains in productivity and effectiveness? Ideally, we’d all be focused on getting the greatest benefit from our resources – but often processes and common practices are arranged in sub-optimal ways for historical reasons, and rewiring large parts of someone’s role is a big ask. The hope is that the new way will prove simpler and less arduous, so it absolutely makes sense to focus on potential benefits and their realisation – and bringing in staff voices at the design stage can make for gains in autonomy and job satisfaction.

    The other end of the problem concerns procurement. Many providers have updated their student records systems in recent years in response to the demands of the Data Futures programme. The trend has been away from bespoke and customised solutions and towards commercial off-the-shelf (COTS) procurement: the thinking here being that updates and modifications are easier to apply consistently with a standard install.

    As James outlines, providers are looking at a “buy, build, or partner” decision – and institutions with different goals (and at different stages of data maturity) may choose different options. There is though enormous value in senior leaders talking across institutions about decisions such as these. “We had to go through the same process” James outlined. “In the end we decided to focus on our existing partnership with Microsoft to build a cutting edge data warehouse, and data ingestion, hierarchy and management process leveraging Azure and MS Fabric with direct connectivity to Gen AI capabilities to support our university customers with their data, and digital transformation journey.” – there is certainly both knowledge and hard-won experience out there about the different trade-offs, but what university leader wants to tell a competitor about the time they spent thousands of pounds on a platform that didn’t communicate with the rest of their data ecosystem?

    As Claire Taylor recently noted on Wonkhe there is a power in relationships and networks among senior leaders that exist to share learning for the benefit of many. It is becoming increasingly clear that higher education is a data-intensive sector – so every provider should feel empowered to make one of the most important decisions they will make in the light of a collective understanding of the landscape.

    This article is published in association with Kortext. Join us at an upcoming Kortext LIVE event in London, Manchester and Edinburgh in January and February 2025 to find out more about Wonkhe and Kortext’s work on leading digital capability for learning, teaching and student success.

    Source link

  • Podcast: Visegrad special | Wonkhe

    Podcast: Visegrad special | Wonkhe

    This week on the podcast Jim, Mack and team are on a bus around the Visegrad countries where they’ve been exploring student experience, representation and rights, discounted dorms and a set of countries where students have been leading change.

    Plus Disabled Students UK has its access insights survey out, and we discuss changes to the Renter’s Rights Bill.

    With Katie Jackson, Faculty of Humanities Officer at the University of Manchester SU, Seán Keaney, Academic Officer at University of Limerick Student Life, Gary Hughes, CEO at Durham SU, Mack Marshall, Community and Policy Officer at Wonkhe and presented by Jim Dickinson, Associate Editor at Wonkhe.

    Read more

    On Day -1 of this year’s magical mystery tour around Europe and students, the team come across plenty of protests for democracy, on Day 0 of the tour we find students in the centre of both the past and the future for Hungary, on Day 1 the team put down some roots and build some belonging at camp, on the second evening the team try to work out if they have enough points for a dorm in Slovakia, and on Day 2 the team get community building and pot roasting.

    Source link

  • Podcast special: Writing for Wonkhe

    Podcast special: Writing for Wonkhe

    In this special seasonal edition of the Wonkhe Show, we discuss how you can contribute to the higher education debate by writing for the site.

    Plus we discuss the importance of communicating academic and professional insights to wider audiences, and we take you inside our editorial process – which is all about clear arguments and diverse perspectives.

    With Adam Matthews, Senior Research Fellow at the School of Education at the University of Birmingham, Michael Salmon, News Editor at Wonkhe, David Kernohan, Deputy Editor at Wonkhe and presented by Debbie McVitty, Editor at Wonkhe.

    Higher Education Policy into Practice (Online) PGCert

    Writing for Wonkhe

    Source link

  • The data dark ages | Wonkhe

    The data dark ages | Wonkhe

    Is there something going wrong with large surveys?

    We asked a bunch of people but they didn’t answer. That’s been the story of the Labour Force Survey (LFS) and the Annual Population Survey (APS) – two venerable fixtures in the Office for National Statistics (ONS) arsenal of data collections.

    Both have just lost their accreditation as official statistics. A statement from the Office for Statistical Regulation highlights just how much of the data we use to understand the world around us is at risk as a result: statistics about employment are affected by the LFS concerns, whereas APS covers everything from regional labour markets, to household income, to basic stuff about the population of the UK by nationality. These are huge, fundamental, sources of information on the way people work and live.

    The LFS response rate has historically been around 50 per cent, but it had fallen to 40 per cent by 2020 and is now below 20 per cent. The APS is an additional sample using the LFS approach – current advice suggests that response rates have deteriorated to the extent that it is no longer safe to use APS data at local authority level (the resolution it was designed to be used at).

    What’s going on?

    With so much of our understanding of social policy issues coming through survey data, problems like these feel almost existential in scope. Online survey tools have made it easier to design and conduct surveys – and often design in the kind of good survey development practices that used to be the domain of specialists. Theoretically, it should be easier to run good quality surveys than ever before – certainly we see more of them (we even run them ourselves).

    Is it simply a matter of survey fatigue? Or are people less likely to (less willing to?) give information to researchers for reasons of trust?

    In our world of higher education, we have recently seen the Graduate Outcomes response rate drop below 50 per cent for the first time, casting doubt as to its suitability as a regulatory measure. The survey still has accredited official statistics status, and there has been important work done on understanding the impact of non-response bias – but it is a concerning trend. The national student survey (NSS) is an outlier here – it has a 72 per cent response rate last time round (so you can be fairly confident in validity right down to course level), but it does enjoy an unusually good level of survey population awareness even despite the removal of a requirement for providers to promote the survey to students. And of course, many of the more egregious issues with HESA Student have been founded on student characteristics – the kind of thing gathered during enrollment or entry surveys.

    A survey of the literature

    There is a literature on survey response rates in published research. A meta-analysis by Wu et al (Computers in Human Behavior, 2022) found that, at this point, the average online survey result was 44.1 per cent – finding benefits for using (as NSS does) a clearly defined and refined population, pre-contacting participants, and using reminders. A smaller study by Diaker et al (Journal of Survey Statistics and Methodology, 2020) found that, in general, online surveys yield lower response rates (on average, 12 percentage point lower) than other approaches.

    Interestingly, Holton et al (Human Relations, 2022) show an increase in response rates over time in a sample of 1014 journals, and do not find a statistically significant difference linked to survey modes.

    ONS itself works with the ESRC-funded Survey Futures project, which:

    aims to deliver a step change in survey research to ensure that it will remain possible in the UK to carry out high quality social surveys of the kinds required by the public and academic sectors to monitor and understand society, and to provide an evidence base for policy

    It feels like timely stuff. Nine strands of work in the first phase included work on mode effects, and on addressing non-response.

    Fixing surveys

    ONS have been taking steps to repair LFS – implementing some of the recontacting/reminder approaches that have been successfully implemented and documented in the academic literature. There’s a renewed focus on households that include young people, and a return to the larger sample sizes we saw during the pandemic (when the whole survey had to be conducted remotely). Reweighting has led to a bunch of tweaks to the way samples are chosen, and non-responses accounted for.

    Longer term, the Transformed Labour Force Survey (TLFS) is already being trialed, though the initial March 2024 plans for full introduction has been revised to allow for further testing – important given a bias towards older age group responses, and an increased level of partial responses. Yes, there’s a lessons learned review. The old LFS and the new, online first, TLFS will be running together at least until early 2025 – with a knock on impact on APS.

    But it is worth bearing in mind that, even given the changes made to drive up responses, trial TLFS response rates have been hovering around just below 40 per cent. This is a return to 2020 levels, addressing some of the recent damage, but a long way from the historic norm.

    Survey fatigue

    More usually the term “survey fatigue” is used to describe the impact of additional questions on completion rate – respondents tire during long surveys (as Jeong et al observe in the Journal of Development Economics) and deliberately choose not to answer questions to hasten the end of the survey.

    But it is possible to consider the idea of a civilisational survey fatigue. Arguably, large parts of the online economy are propped up on the collection and reuse of personal data, which can then be used to target advertisements and reminders. Increasingly, you now have to pay to opt out of targeted ads on websites – assuming you can view the website at all without paying. After a period of abeyance, concerns around data privacy are beginning to reemerge. Forms of social media that rely on a constant drive to share personal information are unexpectedly beginning to struggle – for younger generations participatory social media is more likely to be a group chat or discord server, while formerly participatory services like YouTube and TikTok have become platforms for media consumption.

    In the world of public opinion research the struggle with response rates has partially been met via a switch from randomised phone or in-person to the use of pre-vetted online panels. This (as with the rise of focus groups) has generated a new cadre of “professional respondents” – with huge implications for the validity of polling even when weighting is applied.

    Governments and industry are moving towards administrative data – the most recognisable example in higher education being the LEO dataset of graduate salaries. But this brings problems in itself – LEO lets us know how much income graduates pay tax on from their main job, but deals poorly with the portfolio careers that are the expectation of many graduates. LEO never cut it as a policymaking tool precisely because of how broadbrush it is.

    In a world where everything is data driven, what happens when the quality of data drops? If we were ever making good, data-driven decisions, a problem with the raw material suggests a problem with the end product. There are methodological and statistical workarounds, but the trend appears to be shifting away from people being happy to give out personal information without compensation. User interaction data – the traces we create as we interact with everything from ecommerce to online learning – are for now unaffected, but are necessarily limited in scope and explanatory value.

    We’ve lived through a generation where data seemed unlimited. What tools do we need to survive a data dark age?

    Source link