Can ChatGPT Help Students Learn?
OpenAI’s new artificial intelligence chatbot ChatGPT has received intense press coverage since its launch in November 2022. And with the release of its newest version known as GPT-4 only five months later, its developers are claiming some spectacular educational accomplishments. These jaw-dropping but so far unverified claims include 90th percentile scores on standardized tests like the Scholastic Aptitude Test, the Graduate Record Examination, the Law School Admissions Test, the Uniform Bar Examination, and 26 of the Advanced Placement exams.
But curiously absent from press reports are questions about whether such a chatbot can actually help students learn. And yes, as we’ll see below, these AI platforms can aid learning in a variety of ways.
But before we explore some of the ways these programs can help students earn better grades, we need to talk about a substantial limitation of these programs. This drawback could very well be the main reason why the press hasn’t devoted much attention to how these platforms can help learners.
The AI Accuracy Problem
The likely reason for this press hesitancy is that large language models (LLMs) like ChatGPT, Microsoft’s Bing AI, and Google’s even newer Bard frequently return unexpected (and at times) nonsensical results when responding to user prompts that ask about facts and details. Why?
When the model is not aware of relevant details, it frequently makes them up. This effect is called “AI hallucination,” and this tendency can frequently make such software unusable when students are trying to use these models to help them learn detailed or technical subjects.
This is a known weakness of ChatGPT. “Asking it for facts is usually a mistake,” says Dr. Ethan Mollick, a management professor who teaches MBA students and undergraduates at the University of Pennsylvania’s Wharton School who’s emerged as one of the world’s preeminent authorities on AI in higher education.
“It is often wrong, or out-of-date, or will refuse to answer the question. ChatGPT is not Google,” says Dr. Mollick.
These problems aren’t limited to ChatGPT. Although the latest 4.0 subscription version may perform better, the current free version of ChatGPT isn’t connected to the live internet and wasn’t trained on any resources after 2021.
By contrast, Bing AI can “surf” the web through a live web connection and cites URLs to support its results ostensibly. But not only does Bing AI still have severe problems with hallucination, the web URLs that it cites as references are frequently irrelevant, incorrect, or misleading.
To make matters worse, these platforms can come across as extraordinarily confident about the ways they present unequivocally wrong information. They can present falsehoods with such unwavering conviction that they’ve even fooled expert authorities, including Ivy League professors like Dr. Mollick, with years of teaching and research experience. And their incorrect results even include mathematical calculations, equations, and statistics, along with computer code that initially appears correct, but upon execution, crashes.
In practice, hallucinations like these mean that pretty much any factual information returned from these programs needs to be rigorously fact-checked before students can use the results in their studies. In fact, one clueless YouTuber’s clickbait video about how ChatGPT is poised to revolutionize learning was assailed with hundreds of student comments after the clip barely mentioned hallucinations. Those commentators attacked how ChatGPT had confidently given them incorrect answers to their questions about topics in college courses like chemistry, physics, computer science, psychology, and English literature.
AI Accuracy Workarounds
Nevertheless, just because LLMs require conscientious fact-checking doesn’t mean that they are completely useless at helping students learn. In part, several workarounds have been suggested to verify the accuracy of the results returned by these platforms.
Before asking the AI for study help through one of the techniques listed below, here are some ways to confirm accuracy:
Self-Provide Sources
Provide relevant excerpts from reading assignments like textbook chapters or research by directly pasting that content into the chat window. Because memory is limited on these platforms, lengthy excerpts may need to be broken up into smaller segments for ChatGPT 3.5 and Bing AI, but this is less of a concern for ChatGPT 4.0. If the AI platform hangs while evaluating a long excerpt, open a new conversation in a separate browser tab or enter the prompt “Did you time out?”
Request Source Citations
Add language such as “cite all your sources for this information along with website URLs” at the end of a ChatGPT prompt. Students should then check the sources that the platform provides.
Search on Google
A quick cross-check on Google can rule out inaccuracies. This might be easier when using Google’s Bard AI platform, but most computer and mobile device operating systems already include a “Search with Google” command in their contextual menus.
Ask the AI to Assume an Expert’s Role
Machine learning authority Allie Miller, who managed AI projects at Amazon and IBM for almost seven years and who now charges $1,000 an hour for consulting, describes in this podcast how such a technique can improve accuracy:
A prompt that I love is the “calling upon an expert” sort of prompt: “Act like a public speaking coach,” or “Act like a bestselling novelist,” and then you ask the actual prompt. It’s a very simple edit for people to take advantage of, and there are 150 examples of these online. I love that it’s playing the role of an expert. . .maybe there’s no one right answer, but it does elevate the quality of that output.
Several of the examples of this kind of prompt that appear in this GitHub repository are particularly useful for students. Here are two of the best:
Act as a Math Teacher
I want you to act as a math teacher. I will provide some mathematical equations or concepts, and it will be your job to explain them in easy-to-understand terms. This could include providing step-by-step instructions for solving a problem, demonstrating various techniques with visuals or suggesting online resources for further study. My first request is “I need help understanding how probability works.”
Act as a Motivational Coach
I want you to act as a motivational coach. I will provide you with some information about someone’s goals and challenges, and it will be your job to come up with strategies that can help this person achieve their goals. This could involve providing positive affirmations, giving helpful advice or suggesting activities they can do to reach their end goal. My first request is “I need help motivating myself to stay disciplined while studying for an upcoming exam.”
ChatGPT Strategies for Learning
With the above workarounds in mind, here are a few ways in which these new technologies can help students learn:
Explaining Confusing Concepts
Several of Dr. Mollick’s MBA students told him that they asked ChatGPT to explain concepts to them “like they were ten years old.” This is also known as the Feynman Technique, a study skill invented by the Nobel Prize-winning physicist Richard Feynman.
The technique is based on the idea that to understand a concept truly, you should be able to explain it in simple terms using only the most essential information. By applying this technique, Feynman could break down complex ideas into simpler concepts and understand them at a deeper level. This approach can be used for any subject or topic, but can be especially helpful for students studying science, technology, engineering, math, or business.
Although accuracy can be an issue, Dr. Mollick says that In this case, “the students were using this as a check on their own knowledge, where I think AI can be more effective.” He also adds that “the use of AI as an explainer is one that educators and technologists should be thinking about.”
Correcting Errors
Dr. Mollick also reports that his students told him they fed the problems on examinations or problem sets that they got wrong into ChatGPT, then asked for explanations of their mistakes and how to correct their errors.
Summarizing Passages
Although some computer operating systems like Apple’s have offered summary functions for years, most of these were fairly primitive because they would condense passages by omitting entire sentences, resulting in incomplete or inaccurate results that weren’t usable as timesavers during research.
Although imperfect and always requiring fact-checking, ChatGPT offers a significant advance. The chatbot appears to be very good at summarizing academic papers and articles in ways that preserve much of the emphasis of the ideas expressed by the narrative or argument, a timesaving capability when compiling research needed to write a superior academic paper. ChatGPT can also summarize transcripts from meetings and interviews as well.
Organizing Facts Into Outlines
Students who depend upon memorizing lengthy and complex outlines before examinations—like law students—should find this capability a huge time-saver. ChatGPT appears to perform surprisingly well when it comes to logically organizing disparate facts into outlines with several levels of subordination.
Practicing With Simulated Test Questions
Decades of students at university campuses where the PLATO e-learning computers and software systems were installed had practiced answering examination questions from test banks stocked with thousands of multiple-choice items from old tests. These days, ChatGPT can actually write sample multiple-choice questions that closely simulate the test items on actual exams and even explain why certain choices are correct or incorrect.
Foreign Language Conversation Practice
A chatbot like ChatGPT can be particularly effective at engaging in conversations with students who are learning new languages. Although the chatbot has been reported to make spelling, vocabulary, syntax, and idiomatic errors, the mistakes don’t appear to deter students from practicing new conversation skills in this way. What’s more, the chatbot is always available to talk late at night after language “study buddies” have gone to sleep.
Strategic Study Planning
Students under tight deadlines, such as during finals week, can ask ChatGPT to suggest possible study plans. When students have limited time available, these plans can help students realistically anticipate the time they can afford, as well as set realistic time boundaries when studying critical topics or practicing calculations for exams.
Eternal Vigilance Required?
To sum up, ChatGPT and other large language models like Bing AI and Google’s Bard can definitely help students learn. However, their chronic accuracy problems require students to always perform fact-checking for the time being until the developers of these platforms release new and improved versions.
To effectively use these models for learning, students need to verify the accuracy of their responses through workarounds—such as providing relevant excerpts from reading assignments, requesting source citations, searching on Google, or asking the AI to assume the role of an expert. Nevertheless, despite their limitations, LLMs like these hold tremendous promise for higher education, and should rapidly evolve and improve in future releases.
