Inserting brief quiz questions into an online lecture can boost learning and may reduce racial achievement gaps, even when students are tuning in remotely in a distracting environment.
That’s a main finding of our recent research published in Communications Psychology. With co-authors Dahwi Ahn, Hymnjyot Gill and Karl Szpunar, we present evidence that adding mini-quizzes into an online lecture in science, technology, engineering or mathematics – collectively known as STEM – can boost learning, especially for Black students.
In our study, we included over 700 students from two large public universities and five two-year community colleges across the U.S. and Canada. All the students watched a 20-minute video lecture on a STEM topic. Each lecture was divided into four 5-minute segments, and following each segment, the students either answered four brief quiz questions or viewed four slides reviewing the content they’d just seen.
This procedure was designed to mimic two kinds of instructions: those in which students must answer in-lecture questions and those in which the instructor regularly goes over recently covered content in class.
All students were tested on the lecture content both at the end of the lecture and a day later.
When Black students in our study watched a lecture without intermittent quizzes, they underperformed Asian, white and Latino students by about 17%. This achievement gap was reduced to a statistically nonsignificant 3% when students answered intermittent quiz questions. We believe this is because the intermittent quizzes help students stay engaged with the lecture.
To simulate the real-world environments that students face during online classes, we manipulated distractions by having some participants watch just the lecture; the rest watched the lecture with either distracting memes on the side or with TikTok videos playing next to it.
Surprisingly, the TikTok videos enhanced learning for students who received review slides. They performed about 8% better on the end-of-day tests than those who were not shown any memes or videos, and similar to the students who answered intermittent quiz questions. Our data further showed that this unexpected finding occurred because the TikTok videos encouraged participants to keep watching the lecture.
For educators interested in using these tactics, it is important to know that the intermittent quizzing intervention only works if students must answer the questions. This is different from asking questions in a class and waiting for a volunteer to answer. As many teachers know, most students never answer questions in class. If students’ minds are wandering, the requirement of answering questions at regular intervals brings students’ attention back to the lecture.
This intervention is also different from just giving students breaks during which they engage in other activities, such as doodling, answering brain teaser questions or playing a video game.
Why it matters
Online education has grown dramatically since the pandemic. Between 2004 and 2016, the percentage of college students enrolling in fully online degrees rose from 5% to 10%. But by 2022, that number nearly tripled to 27%.
Our study therefore offers a scalable, cost-effective way for schools to increase the effectiveness of online education for all students.
What’s next?
We are now exploring how to further refine this intervention through experimental work among both university and community college students.
As opposed to observational studies, in which researchers track student behaviors and are subject to confounding and extraneous influences, our randomized-controlled study allows us to ascertain the effectiveness of the in-class intervention.
Our ongoing research examines the optimal timing and frequency of in-lecture quizzes. We want to ensure that very frequent quizzes will not hinder student engagement or learning.
The results of this study may help provide guidance to educators for optimal implementation of in-lecture quizzes.
The Research Brief is a short take on interesting academic work.
A recent report from the Campaign for College Opportunity notes strategies and programs that support graduation outcomes for students from historically underrepresented backgrounds.
Tashi-Delek/E+/Getty Images
Higher education can be an agent for positive change in students’ lives, providing personal, intellectual and socioeconomic growth opportunities. But not all of these outcomes are realized by every student.
An April report by the Campaign for College Opportunity outlines some of the challenges Black students face in pursuit of higher education, as well as measures that colleges can take to address disparities in completion and persistence rates.
What’s the need: Since fall 2019, Black enrollment in higher education has declined more rapidly than that of other races. Black students currently make up about 10 percent of all undergraduates enrolled in the U.S., but roughly 14 percent of the total U.S. population.
Among U.S. adults, about 32 percent of Black Americans have completed some college but have yet to earn a bachelor’s degree—four percentage points higher than the average American (28 percent) but roughly the same as people belonging to two or more races (32 percent), Native Hawaiians and Pacific Islanders (32 percent), and American Indian and Alaska Native populations (34 percent).
Despite the challenges students of color face while pursuing a degree, most learners say college is worth it in the long run for their careers. Still, balancing academics and other obligations, strains on mental health and feelings of isolation can be unexpected costs associated with college, according to a 2024 report from the Pell Institute.
DEI Under Attack
Since Trump retook office in January, his administration has sought to eliminate diversity, equity and inclusion practices. A Feb. 14 Dear Colleague letter from the Department of Education to colleges and universities sought to issue guidance on which race-based practices besides those used in admissions—which the Supreme Court struck down in 2023—would no longer be permitted. The letter cited scholarships and programs that were exclusively available to students based on their race. An FAQ page from the department notes that race or cultural heritage education or celebrations are not prohibited, so long as they are open to everyone on campus.
Recommendations: Based on existing research, the report authors outlined six strategies to improve Black student outcomes.
Demystify the college experience. High school partnerships and pathway programs, including summer programs and dual-enrollment opportunities, can positively impact Black students’ college trajectories.
Improve transfer. Invest in two-year colleges as access points and transfer launchpads for Black students who may want to earn a bachelor’s degree at four-year institutions. Additionally, strong partnerships between two- and four-year colleges can address culture gaps and ensure the four-year institution is equipped to help Black and other transfer students thrive.
Address college affordability. Institutions should invest in avenues and resources to ensure Black students, and others, can pay for tuition, fees, technology, supplies, living experiences and other costs associated with college. “Having a robust portfolio of grants, scholarships and other financial support for Black and low-income students is essential,” according to the report. Students of color are also more likely to report basic needs insecurity, so creating holistic financial resources that ensure students have suitable food, housing and transportation is critical.
Invest in representation. Establishing “Black-affirming” spaces, including resource centers, honors colleges, studies programs and media and art collections can improve students’ sense of belonging on campus, as well as counter negative stereotypes regarding Black students. Similarly, ensuring Black students have a seat at the table for decision-making processes allows them opportunities to advocate for their needs.
Prioritize faculty development. Centers for teaching and learning can provide educators with resources and guidance on how to best serve underrepresented minority groups, including Black students.
Create co-curricular learning opportunities. Faculty-led research, pre-apprenticeship programs and workforce development programs can engage Black students on campus and give them the necessary skills to launch their careers.
Examples of success: In addition to highlighting initiatives that can promote student success, the report also names five institutions that have developed effective programs to improve Black student outcomes.
Compton College provides no-cost food to students through a variety of ways, including an on-campus food pantry, a partnership with the Los Angeles Regional Food Bank and free meals at the Everytable Cafeteria on campus. The college also broke ground on its first student housing facility earlier this year, creating more opportunities to minimize the risk of housing insecurity or homelessness for vulnerable students.
Last year Sacramento State University established the Black Honors College, which provides wraparound support for students interested in learning about Black history and culture. The program, which is open to all students, celebrates Black excellence through mentorship by hand-selected faculty and staff, designated housing and personalized support for participants.
The City University of New York created the Black Male Initiative in 2005, an inclusive 15-project initiative focused on improving enrollment and graduation rates of students from underrepresented populations. Most recently, the program has evolved to include wellness and career development.
Spelman College invested millions of dollars in promoting holistic student wellness, in part by creating a new fitness center and introducing fitness classes, cooking demonstrations and mental health workshops. The initiative is designed to address health concerns that disproportionately impact Black women, including high blood pressure, Type 2 diabetes, heart disease, breast cancer and strokes.
The University of California, San Diego, is home to the Black Academic Excellence Initiative, which strives to improve the experiences of Black students, faculty and staff members on campus. The initiative provides scholarship funds for students and has established a hub for historically Black fraternities and sororities, called the Divine Nine.
On Wednesday, April 30, the University at Albany’s campus buzzed with energy as students, faculty and staff bounced between poster fairs, musical performances, student presentations and other exhibitions. Showcase Day, a newer campus tradition, reserves one day in the spring term to celebrate various student achievements from the year, including dance performances, internship experiences and scientific research.
In addition to boosting campus engagement, the initiative highlights the important work of the University at Albany and invites outside groups to partner with the institution, Provost Carol Kim said.
The background: Kim was inspired to create Showcase Day after kick-starting a similar initiative at her previous institution, the University of Maine. After a few delays due to COVID-19, the University at Albany launched Showcase Day in 2023.
“Post-COVID, our campus felt an ennui,” Kim said. National research shows decreased levels of student participation in campus activities, including faculty-led research, since 2020. “How do we energize or develop more engagement on campus, get people excited again? This event has made a huge difference.”
Many colleges and universities host research symposia in the spring to honor and demonstrate student achievement throughout the academic year, typically in STEM courses or faculty-led research.
UAlbany’s event, however, engages undergraduate and graduate students across colleges, exposing students to opportunities within their discipline and beyond, as well as in graduate studies. Around 37 percent of UAlbany students are first generation, and they may be unaware of the various avenues of experiential learning or research at the institution, Kim said.
The initiative also breaks disciplinary silos, exposing individuals to different kinds of academic work in ways that build campus culture, Kim said. “It’s natural for many faculty, staff and students to stay in their college, in their departments, and many times they don’t know what their colleagues and peers are doing.”
How it works: Showcase Day is a one-day event that unites various student presentations, including posters, artistic performances and demonstrations, under one umbrella.
The day is integrated into the calendar as a no class academic day, which means that while classes are not canceled, professors typically assign students work related to Showcase Day. That could include a review of a theatrical demonstration or a summary of a poster presentation.
One of the most important elements of establishing a campuswide symposium was getting buy-in from campus leadership and the University Senate, Kim said. The event requires the support of hundreds of volunteers, making outside support through sponsorships and community partners another essential element.
Photos by Brian Busher/University at Albany
This year, 2,200 students participated in the event, with over 1,300 unique presentations delivered to an audience of faculty, staff, prospective students, donors, legislators and industry leaders, as well as middle and high school students.
Student projects ranged from research presentations on air quality and native plants to an orchestra performance and robotics demonstration. Many colleges assign an end-of-term project within courses or majors that lend themselves to a Showcase presentation, Kim said; others are student-prompted creations such as internship work experience reflections.
Showcasing Student Success
Other innovative approaches colleges and universities have taken to highlight student achievement include:
At the University of Dayton, an interdisciplinary partnership between graphic design and biology students produces high-quality research posters.
A centralized hub breaks barriers for students interested in experiential learning and research opportunities to identify open positions and engage at San Francisco State University.
A statewide research journal for undergraduate students in Florida provides greater opportunities for learners to share their research beyond institutional journals.
A research festival at Tennessee Tech University celebrates student work in English composition courses.
What’s next: Since Showcase Day launched in 2023, hundreds of students have participated in the event. Student and staff feedback shows that the event has been a positive influence on campus culture, inspiring pride in the participants and the work being done at the institution, Kim said.
“From facilities to student affairs and academic affairs, they’re very proud of their part in contributing to this showcase,” Kim said.
In the future, university leaders would like to see more engagement with potential employers, embedding career development or engagement as a piece of the event. Kim also sees potential in extending the event to multiple days, allowing campus members to participate in a greater number of activities.
Do you have an academic intervention that might help others improve student success? Tell us about it.
The PIE Live Europe, held between March 11-12 in central London, brought together leading figures in the international education sector. Delegates at the two-day conference heard key immigration updates, debated the future of the ELT sector and highlighted the value that international students bring to the UK.
Famed for its university rankings, QS has expanded under Quacquarelli’s leadership to employ more than 900 people from over 30 countries.
Quacqarelli said it was a “great feeling” to win the award, having been a supporter of The PIE since it was a “fledgeling business”.
On what was next for QS, he added: “We’re really committed to providing trusted data and insights to the higher education sector and we really believe in the need for universities to transform, to adopt AI – so we’ve launched a responsible AI consortium with Imperial College.
“And we really believe the need to deliver the emerging skills of the fifth industrial revolution, so we are developing huge amounts of insight on skills and occupations… to identify whatever skills are going to be demanded by employers of the future to guide curricular reform and university transformation.”
You can watch his full interview in the video below.
CHRISTCHURCH, New Zealand — Many students in New Zealand have a story to tell about “streaming” — being grouped into separate math classes based on their perceived ability to master the subject.
Manaaki Waretini-Beaumont, now 18 and an environmental science major at the University of Canterbury, learned about the downside of streaming when she enrolled in Avonside Girls’, a 1,000-student high school in Christchurch.
Avonside starts at Year 9, equivalent to eighth grade in the United States, and ends at Year 13, equivalent to 12th grade. Before the start of her Year 9 term, Waretini-Beaumont and her fellow students were divided up into groups to take tests in “maths,” reading comprehension, and patterns and shapes.
Afterward, the students were separated into lettered groups that spelled out the word B-I-N-O-C-U-L-A-R-S. Waretini-Beaumont was a “9-N” student in mathematics — as she describes it, “the top of the middle block.”
But she said she didn’t feel comfortable as one of the few Māori students in the class.
“I felt like I wasn’t good enough to be in that space,” said Waretini-Beaumont, whose iwi, or tribal affiliations, are Te Āti Haunui-A-Pāpārangi, Ngāti Rangi, Ngāti Apa, Ngāti Paoa. “If there was something I wasn’t understanding, I felt like I wasn’t able to say that, because I’m supposed to be in the smart class with all these smart people.”
So she shifted to another mathematics class with her Māori friends, who were in the “S” classes.
“Being in two different spaces, I could really see the change,” Waretini-Beaumont said. “At the top classes, the teachers’ language towards the students was always positive and it was always encouraging. And they really wanted students to learn and were trying to help them.”
Manaaki Waretini-Beaumont experienced the effects of “streaming,” or academic ability tracking, during her time as a high school student at Avonside Girls’ School in Christchurch, New Zealand. Credit: Image provided by Richie Mills/Ngāi Tahu
In the classroom where her friends were assigned, in contrast, the mathematics work mostly amounted to simple worksheets — “coloring pages and word find,” Waretini-Beaumont said.
For years, much like in the United States, New Zealand has worried about sliding student proficiency in mathematics, as captured by both national and international test scores. Later this month — the beginning of the New Zealand school year — the country is launching an overhaul of mathematics instruction that education leaders hope will reverse the trend.
But other groups in the country have been trying to approach the problem of academic achievement from a different angle. They believe that streaming is driving achievement gaps in the country, including in mathematics. Tokona te Raki/Māori Futures Collective, a think tank focused on youth, has been working since 2019 to persuade schools to voluntarily end the practice by 2030. The initiative is called “Kōkirihia”— Māori for “take action.”
Streaming is just one of many ways that schools group students by academic ability. Ability grouping can include separating students into vocational or university tracks at different schools as early as age 10, as is common in Germany and other Western European countries. But it could also include teachers creating informal and non-permanent groupings within their own classrooms to provide enrichment or extra support to students who need it.
In New Zealand, critics say streaming pushes two groups into so-called “cabbage,” or lower-level mathematics, at a disproportionate rate: Māori students, who are indigenous to New Zealand, and students who are Pasifika, the New Zealand term for people from Samoa, Tonga and other nations in the Pacific Islands.
In the 14th century, the Polynesian ancestors of today’s Māori migrated thousands of miles by canoe to what they called Aotearoa, the land of the long white cloud. Hundreds of years later, English settlers came to engage in trade and now represent the majority ethnic group in New Zealand. In 1840, the two groups signed the Treaty of Waitangi that established New Zealand’s bicultural identity.
Many youth with Pacific Island backgrounds are descended from people who were encouraged to move to New Zealand after World War II to address a labor shortage.
Both Māori and Pasifika are a fast-growing, and young, population. By the 2040s, more than a third of children in the country are expected to identify as Māori, according to Stats NZ, the country’s official data agency.
The New Zealand Ministry of Education’s official stance discourages streaming, but the country’s more than 2,500 schools operate with a great deal of independence: Principals have similar powers and responsibilities as school superintendents in the United States, and each school has an elected board that sets policy and manages budgets.
New Zealand does not track streaming or ability grouping by race or ethnicity, but surveys show it is common: Eighty percent of students are in schools that group students by ability level in mathematics, according to a 2022 survey conducted by the Program for International Student Assessment.
Other data shows a wide academic gap among students of different ethnicities in New Zealand.
Students at May Road School in Auckland, New Zealand, work through a lesson on fractions. Credit: Becki Moss for The Hechinger Report
In the Auckland region, the country’s most densely populated of 16 regions in all, 76 percent of Asian students left secondary school with the highest of three levels on the country’s National Certificate of Educational Achievement in 2022. Like a high school diploma, the NCEA Level 3 is a minimum qualification to enter college in New Zealand.
About 66 percent of Pākehā, or white, students left school with that credential. About 46 percent of Pasifika students and 40 percent of Māori students did the same.
In comparison, the high school graduation rate by race and ethnicity in the United States in the 2021-22 school year was 94 percent for Asian American/Pacific Islander students, 90 percent for white students, 83 percent for Hispanic students, 81 percent for Black students and 74 percent for American Indian/Alaskan Native students.
Misbah Sadat, the newly appointed principal at Kuranui College, a high school 50 miles northeast of the capital of Wellington, began actively working to “destream” mathematics courses soon after emigrating to New Zealand in 2009 and becoming a teacher there.
As head of mathematics at a high school called Horowhenua College, she started by identifying promising Māori students on her own, moving them to higher level classes, and mentoring them, as described in a Ministry of Education newsletter.
Eventually she convinced her colleagues at Horowhenua to create mixed-ability classes rather than dividing the students. She continued the same work as deputy principal at Onslow College in suburban Wellington, where she worked before her new appointment.
The streaming practice comes from a patronizing mindset, said Sadat, who was also a math teacher in Montgomery County, Maryland.
Schools are telling parents that their children might be lost and overwhelmed in a more rigorous class. In actuality, “We have demoted some students to learn crap,” she said. “And then we are saying that at age 16, ‘You are dumb at maths.’ How dare we decide what a young person is capable of or not capable of?”
Students at Kaiapoi North School in suburban Christchurch, New Zealand, work through a multiplication problem in chalk on the playground blacktop. Credit: Becki Moss for The Hechinger Report
Both of New Zealand’s unions for elementary and secondary teachers signed onto the pledge to end streaming by 2030. In a newsletter to members, the elementary teachers union noted that its members have noticed “a sense of ingrained hopelessness that comes with being in the ‘cabbage’ classes.”
But in the same newsletter, another teacher said educators struggle with the mix of abilities in one classroom, along with managing behavior challenges.
David Pomeroy, a senior lecturer in education at the University of Canterbury in Christchurch, is studying schools that have committed to reducing their reliance on streaming.
It’s a difficult task, he said. So many teachers are accustomed to the practice, since they went through it in school themselves. Parents of students in high-level classes are worried their children will be shortchanged. Teachers also say that it is easier to work with students who are all roughly on the same skill level.
And then there is an emotional connection to the practice, Pomeroy said. Unlike in the United States, lower-level mathematics classes are often taught by teachers who have a lot of classroom experience and who express real fondness for their students, he said. Pushing students too hard is seen as setting them up for repeated failure, which teachers were reluctant to do.
Abby Zonneveld’s bulletin board at St. Clair School in Dunedin, New Zealand, asked students to describe their “tūrangawaewae,” or place where they feel a special connection. Credit: Becki Moss for The Hechinger Report
“Even if they accepted streaming wasn’t the right next step, they wanted to protect them from anything that could damage their confidence,” Pomeroy said.
For schools that have made a commitment to reducing or ending streaming, he said, one useful tool has been to bring mathematics teachers in different schools together so they can work through challenges, such as lesson planning, and share successes.
The research into the benefits or harms of academic tracking or streaming show mixed results. In 2016, a group of researchers compiled all the best U.S-based research on ability grouping and acceleration at that point, going back for a century. They found certain kinds of ability grouping, such as placing highly gifted students together, was a benefit to those students. But grouping students in high- or low-performing classes did not show any benefit or detriment for students.
The New Zealand Initiative, a right-of-center think tank, said that the country should conduct its own research on the effects of streaming in the country, rather than relying primarily on research done elsewhere and on qualitative reports that primarily capture feelings about the practice. “Research suggests that lowerstream students are often taught less engaging content by less experienced teachers. So, it may not be streaming itself that increases gaps in achievement but streaming done poorly,” the initiative said in a report.
But the efforts to reduce streaming voluntarily seem to be catching on.
When looking at all academic subjects, not just mathematics, principals on a 2022 PISA survey said 67 percent of students in New Zealand are grouped by ability into different classes for at least some subjects. That’s a drop from 2015, when 90 percent of principals reported that students were grouped into different classes in their schools.
The change is welcome, said Waretini-Beaumont, who works on social media for Tokona te Raki. Streaming “has more impact than just cutting off some opportunities and stopping someone from doing calculus,” she said. “Our grandparents have been streamed and they don’t know it was even a thing. They just thought they were dumb.”
The Hechinger Report provides in-depth, fact-based, unbiased reporting on education that is free to all readers. But that doesn’t mean it’s free to produce. Our work keeps educators and the public informed about pressing issues at schools and on campuses throughout the country. We tell the whole story, even when the details are inconvenient. Help us keep doing that.
The New Mexico Public Education Department has updated its student achievement data reporting website — NM Vistas — with a renovated layout and school performance data from the 2023-2024 academic year, with expectations for additional information to be released in January 2025.
NM Vistas is crucial to informing New Mexicans about school performance and progress at the school, district and state levels through yearly report cards. The site displays student reading, math and science proficiency rates taken from state assessments, as required by the federal Every Student Succeeds Act. Districts and schools receive scores between 0 and 100 based on performance, and schools also receive designations indicating the level of support the school requires to improve.
Other information on the site includes graduation rates, attendance and student achievement growth. Data also shows rates among specific student demographics, including race, gender, disability, economic indicators and more.
PED Deputy Secretary of Teaching, Learning and Innovation, Amanda DeBell told NM Education in an interview that this year’s recreation of the NM Vistas site came from a desire to go beyond the state’s requirements for school performance data.
“We knew that New Mexico VISTAs had a ton of potential to be a tool that our communities could use,” DeBell said.
One new data point added to NM Vistas this year is early literacy rates, which measures the percentage of students in grades K-2 who are reading proficiently at their grade level. Currently, federal law only requires proficiency rates for grades 3-8 to be published, and New Mexico also publishes 11th grade SAT scores. In the 2023-2024 school year, 34.6% of students grades K-2 were proficient in reading, the data says.
DeBell said several advisory groups encouraged the PED to report early literacy data through NM Vistas.
“We were missing some key data-telling opportunities by not publishing the early literacy [rates] on our website, so we made a real effort to get those early literacy teachers the kudos that they deserve by demonstrating the scores,” DeBell said.
The PED also added data on individual schools through badges indicating specific programs and resources the school offers. For example, Ace Leadership High School in Albuquerque has two badges: one for being a community school offering wraparound services to students and families, and another for qualifying for the career and technical education-focused Innovation Zone program.
“What we are really trying to do is provide a sort of one-stop shopping for families and community members to highlight all of the work that schools are doing,” DeBell said.
The updated NM Vistas website has removed a few things as well, most notably the entire 2021-2022 NM Vistas data set. DeBell said this was because the PED changed the way it measured student growth data, which resulted in the 2021-2022 school year’s data being incomparable to the most recent two years.
“You could not say that the schools in 2021-2022 were doing the same as 2022-2023 or 2023-2024, because the mechanism for calculating their scores was different,” DeBell said.
However, this does leave NM Vistas with less data overall, only allowing viewers to compare scores from the latest data set to last year’s.
In January 2025, several new indicators are expected to be uploaded to the site, including:
Student performance levels: Reports the percentage of students who are novices, nearing proficiency, proficient and advanced in reading, math and science at each school, rather than only separating between proficient and not proficient.
Results for The Nation’s Report Card (also known as NAEP): Compares student proficiencies between US states.
Educator qualifications: DeBell said this would include information on individual schools’ numbers of newer teachers, substitute teachers covering vacancies and more.
College enrollment rates: only to be statewide numbers indicating the percentage of New Mexico students attending college after graduating, but DeBell said she later hopes the PED can narrow down by each K-12 school.
Per-pupil spending: How much money each school, district and the state spends per-student on average.
School climate: Links the viewer to results of school climate surveys asking students, parents and teachers how they feel about their school experience.
Alternate assessment participation: Percentage of students who take a different assessment in place of the NM-MSSA or SAT.
“We want VISTAs to be super, super responsive, and we want families to be able to use this and get good information,” DeBell said. “We will continue to evolve this until it’s at its 100th iteration, if it takes that much.”
This year, the PED released statewide assessment results for the 2023-2024 school year to NM Vistas on Nov. 15. Results show 39% of New Mexico students are proficient in reading, 23% are proficient in math and 38% are proficient in science. Compared to last year’s scores, reading proficiency increased by 1%, math proficiency decreased by 1% and science proficiency increased by 4%.
