The results of Gallup’s recent poll of college graduates generated attention-grabbing headlines such as “Student Loan Debt: Major Barrier to Entrepreneurship” and “One Way To Make College Worth It: Help Students Feel Like Someone Cares About Them.” Unfortunately, the analysis that produced those conclusions would not pass muster in an undergraduate research methods course.

The Gallup-Purdue index, released last month, asked college graduates a series of questions about their experiences during and after college. Nationally representative polls like this are an excellent method for gauging public opinion in a systematic way, rather than relying on anecdotes or biased samples. Yet, while these surveys can describe the public’s stated views, they cannot usually isolate cause and effect. The Gallup-Purdue Index fails miserably because it makes inappropriate causal inferences from survey data rather than simply describing what respondents said.

The Gallup analysis makes two types of fundamental errors: confusing opinions and facts and mistaking correlation for causation. Two examples from the 2015 report illustrate how Gallup leapt from its survey results to misleading conclusions.

First, the Gallup-Purdue report claims to have measured the “consequences of student loan debt,” stating, for example, that “[a] third or more [of recent graduates] have delayed purchasing a house or a car because of their [student] debt, and nearly one in five have put off starting their own business.” These findings are based on graduates’ responses to questions about whether they have delayed certain activities due to the student loans.

But people’s opinions about the consequences of student loans are not the same thing as scientific evidence of loan effects. This fact is illustrated by another recent poll, in which 30 percent of respondents said they would sell one of their organs in order to get rid of their student loans. In response, economist Susan Dynarski wryly noted, “We do not see millions of debt-free, single-kidneyed college grads.”

Polls can tell us people’s opinions, such as what share of Americans think climate change is real, but not scientific facts, such as whether climate change is real. Likewise, a Gallup poll can tell us how many college graduates feel that student debt has prevented them from starting a business. But understanding the possible causal relationship between student debt and entrepreneurship must overcome significant conceptual and empirical challenges, as my colleague Sandy Baum has discussed.

Second, Gallup claims to have identified aspects of the college experience that positively affect graduates later in life: “[the] results reaffirm the importance to undergraduates of supportive relationships with professors and mentors. If employed graduates strongly agree that they had professors who cared about them as people, they had at least one professor who made them excited about learning and they had a mentor who encouraged them to pursue their goals and dreams, their odds of being engaged at work nearly double.”

This type of conclusion violates the mantra that every undergraduate learns in Statistics 101: “correlation is not causation.” Gallup makes these claims based on survey respondents’ answers to questions about their college experiences and their current jobs. On average, those who responded more positively to the questions about college also responded more positively to the questions about their current work (taking into account a personality measure based on other survey questions).

Survey results describing how college graduates look back on their time in college and rate their current employment are potentially useful, but they do not come close to establishing a causal relationship between the two. In addition to the fact that respondents may not accurately remember their college experiences, it is highly likely that students who chose to participate in certain activities differ systematically from those who did not participate. For example, students who sought out faculty mentoring may have been more motivated than those who did not (in ways not captured by the personality measure), and thus more likely to also find greater success later in life.

Survey research has the potential to contribute to the study of higher education in the United States. For example, a survey of college graduates could document how well they understand how much they have borrowed and their repayment options. But the Gallup-Purdue Index flunks the most basic principles of social science by drawing the kinds of conclusions that survey data can never support.