The College Fix

Survey incentivized the poorest students to respond

It’s no secret among academic researchers that journalists frequently mischaracterize their research, likely because the journalists don’t understand it. Perhaps the most misreported statistic in higher ed, the 1-in-5 campus-rape figure, has been explicitly disavowed by the researcher who led the study.

But sometimes reporters and their editors just want a better headline than the research can deliver.

Here’s how The New York Times characterized a new survey on “food insecurity” among college students by Temple University researchers:

Tuition or Dinner? Nearly Half of College Students Surveyed in a New Report Are Going Hungry

What the Times left out: everything about the survey methodology. Why? Probably because it would ruin the entire premise of the story it wanted.

Robert VerBruggen of National Review, an expert in statistical analysis, highlights everything the Times left out of its report in a new column.

First, the definition of “food insecurity.” The #RealCollege survey by Temple’s Hope Center for College, Community and Justice asked students from 100 institutions if they had been food insecure in the past 30 days. How the Times defines the term: “the state of having limited or uncertain access to food.”

Recent data show that “the problem is more serious and widespread” than indicated by anecdotal reports, “affecting almost half of the student population at community and public colleges” – 45 percent, according to reporter Kaya Laterman.

Here’s how the survey actually defines food insecurity, according to VerBruggen:

[A] student could be classified as “food insecure” if he said he worried whether his food would run out, “couldn’t afford to eat balanced meals,” and ate a bit less “because there wasn’t enough money for food.”

Food insecurity is not synonymous with “physiological sensations of hunger.” That is simply one “extreme” variation of the phenomenon.

MORE: Students claim to stop eating, university grants their liberal wish list

Also left out of the Times report: the response rate. It was an “estimated” 5.8 percent. The researchers did not draw a subsample “due to legal and financial restrictions,” and it was emailed to students, meaning that those who filled it out 1) had email access and 2) actually used email, a notoriously unpopular communications method for today’s students.

As VerBruggen explains, “[t]he people who answered it are almost certainly not representative of the general population. The NYT simply ignores this fact.”

His final complaint is with the survey’s own appraisal of its soundness. rather than the Times hiding its caveats.

Led by Prof. Sara Goldrick-Rab, who left the University of Wisconsin to found the Hope Center after comparing former Gov. Scott Walker to Adolf Hitler, the survey team claimed that it provided only “negligible” incentives to students to participate.

VerBruggen tracked down the email ad for the survey, and concludes it “carries a significant risk of skewing the sample toward poorer, struggling students”:

Subject: #RealCollege: Speak out – chance to win $100!

Making it in college these days can be tough. We want to help.

Colleges and universities need to know about the lives of real students like you so that they can offer more support. After you complete the survey, you can enter a drawing to receive a $100 award.

The NYT uncritically wrote up a survey on college-student "food insecurity" that had a 5.8% response rate and got participants through an email offering the chance to win $100 and saying "Making it in college these days can be tough. We want to help." https://t.co/DMwMVSmdgk

— Robert VerBruggen (@RAVerBruggen) May 3, 2019

You don’t have to consult a conservative think tank for research that challenges Goldrick-Rab’s findings. The left-of-center Urban Institute previously noted the low response rates and “nonrepresentative samples” in previous surveys conducted by Goldrick-Rab and others.

As VerBruggen notes, the Urban Institute used 15 years of Current Population Survey data – “freely available, nationally representative data” – to conclude in 2017 that food insecurity rates are about a quarter of the new Hope Center survey’s findings.

The worst insecurity, as you might guess, was suffered by students who attended two-year colleges after the 2008 recession, yet their rates fell back to only nominally higher than four-year colleges by 2015.

It took VerBruggen about an hour to find the major flaws in the reporting and the survey itself: “I looked into it because the idea that 45 percent of college students are struggling to find enough to eat struck me as patently absurd.”

Read the column and New York Times report.

MORE: UW professor who compared Scott Walker to Hitler lands at Temple

IMAGE: Osugi/Shutterstock

Like The College Fix on Facebook / Follow us on Twitter