Introductory statistics is a required course for students in many college programs, but many students end up withdrawing or failing. There are also substantial gaps in course achievement by student income, first generation status, and history of developmental education.
A new Urban Institute report finds potential for technology-enhanced instruction to improve student academic outcomes and satisfaction in introductory statistics at four-year colleges. But the results do not hold in community colleges, and the technology interventions we studied did not close gaps for less-advantaged students.
Technology-enhanced college instruction clearly has promise, but more research is needed to determine how to translate the advantages of technology to nontraditional and community college students. This project also raised important issues with the adoption and use of technology in higher education.
The Adaptive Learning in Statistics (ALiS) project, led by Ithaka S+R and partners, brought together nine participating colleges to see if a new technology intervention would increase student achievement in the course and close gaps between students from different groups.
ALiS instructors used an online learning tool in their introductory statistics classrooms as a replacement for a traditional textbook. The tool offered adaptive activities based on each student’s knowledge and gave instructors real-time, detailed information about student progress. Students engaged with the online tool to learn new material in statistics, while instructors were asked to reinforce the content through hands-on activities in a “flipped classroom” approach.
The study results highlighted avenues for new research. Across all institutions, students in ALiS sections were not better off than those in traditional sections, and the gaps between student subgroups remained. But there were promising results at four-year colleges, where ALiS students scored higher on grades, pass rates, statistics competency, and course satisfaction.
Students at two-year colleges did no better on academic outcomes, and they were less satisfied in the course. Past studies (PDF) on technology-enhanced classroom instruction have also found limited positive impacts and lower student satisfaction.
Four-year college students had more success in the course and were more satisfied than their two-year peers
Though there was also variation across individual colleges, the different results seem to be driven by important distinctions between two-year and four-year college contexts.
Community college students have more obligations outside the classroom, such as employment and dependent children (PDF). They are also more likely to be English-language learners and therefore have lower English-language reading comprehension skills. These factors may make it harder for students to self-teach new concepts outside of class meeting times.
In addition, the flipped classroom approach may have been a welcome change for four-year college students accustomed to large lecture halls, but it may not have been as welcomed by community college students who would otherwise have been in a smaller setting and therefore would have preferred targeted instruction during class time.
Finally, the hiring of instructors varied across colleges. Two-year colleges in the study were more likely to employ adjunct instructors, who had little time to familiarize themselves with new technology and with the flipped classroom approach before beginning the semester.
Directions for future research
To further understand how technology can improve student outcomes, close gaps, and enhance instruction, we need more information on the skills and tools that students and instructors need.
The Ithaka S+R team provided a portfolio of resources and an active learning community for ALiS instructors. Adopting both a technology tool and the flipped classroom approach was challenging, particularly for adjunct faculty. But students still struggled with reading literacy, technological literacy, self-teaching, and finding sufficient time to complete activities. Targeted research on the conditions necessary for success with this type of intervention may inform future efforts.
It would be valuable to know if fully online learners or students in hybrid courses, who have fewer in-class meetings and more time to focus on completing activities in the online tool, would fare better because they come into the semester expecting to engage with technology-mediated learning. The ALiS evaluation focused only on one model of course delivery, where technology-introduced content outside of class and was combined with in-person classroom meeting time to replace what otherwise might have been a traditional live lecture course. We did not gain insight into how this compared with exclusively online courses.
The study raises questions that may guide future research. For which institutions and students does this technology-mediated approach make sense? What supports could better lead nontraditional students to success? And could better preparation for instructors or shared learning communities across institutions improve their grasp and application of technology?
Investigation into these questions can help us understand the factors underlying variations in student achievement—technological literacy, pedagogy, and instructor and student characteristics—which can help us work toward more equitable outcomes for students.
Tune in and subscribe today.
The Urban Institute podcast, Evidence in Action, inspires changemakers to lead with evidence and act with equity. Co-hosted by Urban President Sarah Rosen Wartell and Executive Vice President Kimberlyn Leary, every episode features in-depth discussions with experts and leaders on topics ranging from how to advance equity, to designing innovative solutions that achieve community impact, to what it means to practice evidence-based leadership.