Does AI truly aid student learning? A recent study conducted in Turkey offers a cautionary perspective on the role of AI in education.
Researchers from the University of Pennsylvania found that Turkish high school students who used ChatGPT while practicing math problems performed worse on subsequent math tests than those who didn’t use the chatbot.
While students with ChatGPT access solved 48 per cent more practice problems correctly, they scored 17 per cent lower on tests covering the same material. A third group of students, who had access to a modified version of ChatGPT acting as a tutor, performed even better on practice problems, solving 127 per cent more accurately compared to their peers. However, these AI-assisted students saw no improvement in their test scores. Those who completed the practice problems independently performed just as well on the tests.
The researchers, in a paper titled “Generative AI Can Harm Learning,” cautioned educators and parents that current AI tools like ChatGPT may significantly hinder students’ ability to learn. Even a refined version of ChatGPT, designed to offer hints rather than direct answers, didn’t lead to improved test performance.
Read More: Weight-loss drugs could transform healthcare by slowing aging, scientists say
The core issue, according to the researchers, lies in students using the chatbot as a “crutch.” Instead of developing problem-solving skills, many students simply asked ChatGPT for the correct answers. Additionally, ChatGPT’s accuracy was a concern; while its arithmetic calculations were wrong 8 per cent of the time, the bigger problem was that it provided incorrect step-by-step solutions 42 per cent of the time. The tutoring version of ChatGPT, however, was programmed with correct solutions, reducing these errors.
The draft paper, released on SSRN in July 2024, has yet to undergo peer review and may still be revised. However, the study involved nearly 1,000 students across grades nine to 11, offering a broad perspective on AI’s impact on learning. Students were divided into three groups: one using ChatGPT, one using an AI tutor, and one without any technological aid. After practicing math problems, all students took a test to measure their learning.
The findings suggested that AI can foster overconfidence. Despite their lower test scores, students who used ChatGPT didn’t perceive that they had learned less. Those with access to the AI tutor believed they had performed significantly better, though their test results showed otherwise. The study highlighted a common issue: our perceptions of learning often don’t match reality.
The researchers compared reliance on ChatGPT to autopilot technology in aviation. Just as pilots must still be able to fly without autopilot, students need to develop their own problem-solving abilities without over-relying on AI tools. This tradeoff isn’t new in education—just as calculators have reduced the need for manual arithmetic, tools like ChatGPT may help students get answers but could undermine deeper learning.
