Is Education the AI Industry’s First Killer Application?

According to EdSurge’s Editor, Jeffrey Young, tech companies appear to have targeted education as a “killer application” for artificial intelligence platforms, an area that could significantly drive sales and company growth. Young cites several demonstrations during the late spring of 2024 by OpenAI, Google, and other firms that focused on educational applications of their latest chatbots and AI platforms. OpenAI also launched ChatGPT Edu, a new partnership program for colleges similar to the firm’s arrangement with Arizona State University.

Author and higher education consultant José Bowen told Young that “both Google and OpenAI are gunning for education. They see this both as a great use case and also as a tremendous market.”

Although the tech industry’s sudden enthusiasm for education seems to have materialized overnight, one key factor appears to have propelled the AI industry’s interest towards education as the first killer app, and particularly towards tutoring delivered through a large language model like GPT-4o. That key factor was reduced risk to the education sector from inaccurate content brought on by AI hallucinations—a technical challenge that’s hindered the adoption of AI platforms across a broad spectrum of other industries.

In a September 2024 op-ed in the Boston Globe, Babson College management professor Peter Cohan points out that companies have spent $150 billion during 2024 alone on AI-related hardware, software and services. However, he writes that at the same time, “companies are also terrified of being sued if the technology hallucinates.” That fear has rendered them hesitant to deploy those investments, he says.

One widely-reported lawsuit has already been decided. According to Ars Technica, in February 2024 a Canadian court forced Air Canada to honor a refund policy that the airline’s customer service chatbot had made up out of thin air during one such nonsensical hallucination. The chatbot told a customer that he would be owed a refund for a flight he never took, and actually cut a deal to pay the customer. After Air Canada refused payment, the customer sued, and the court forced the airline to pay a partial refund on grounds of negligent misrepresentation because it “did not take reasonable care to ensure its chatbot was accurate.”

By contrast, education as an industry probably faces lower risks from litigation than most other sectors over a chatbot’s providing incorrect information because of hallucinations. Although lawsuits can always occur, it’s difficult to conceive of a situation where, for example, a college student could prove some sort of harm with measurable damages by relying on incorrect information from a hallucinating chatbot that resulted in a poor grade on a term paper. After all, undergraduates are always responsible for citing authoritative sources and checking their validity even if they use chatbots for research, like so many of the students in this new Harvard University study.

Nevertheless, despite reduced risks from hallucinations in the education industry, the promise of AI-based tutoring has sparked controversy. Experts have recently questioned how effective human tutoring could be at boosting student performance. Some have also raised doubts about whether artificial intelligence-based tutoring can ever deliver gains in achievement even remotely equivalent to the gains that human tutors can deliver.

In 1984, University of Chicago educational psychologist Dr. Benjamin Bloom asserted that tutoring was actually able to boost student performance by two “sigmas” in statistics parlance. That expression means two standard deviations above the mean, roughly elevating a student’s achievement from the 50th to the 98th percentile.

Known as Dr. Bloom’s Two Sigma Problem, that effect is massive, and far greater than the results from any educational modality available then or since. It’s the equivalent of consistently transforming a “C” student into a potential Rhodes Scholar.

Dr. Bloom argued that most students possess the capacity for much greater learning than they could achieve inside classrooms, but educators couldn’t possibly begin to tap that potential because in-person tutoring for every student would be prohibitively impractical and expensive. Dr. Bloom also argued that inventing interventions with lower costs that could approach tutoring’s benefits posed the major challenge facing education at the time.

In the 40 years since this theory was published, Dr. Bloom’s assertions have been accepted uncritically by many educators along with the press. Researchers have cited his work in scholarly journals more than 2,000 times during the past 10 years, out of more than 5,000 of his citations overall. He’s also partially responsible for the widespread mythology around the world that tutors can work miracles.

Now, what does Dr. Bloom’s work have to do with education as the AI industry’s first potential killer application? That connection becomes more clear once one understands how experts define the term “killer app.” Most of these definitions focus on how such an app offers benefits that drive adoption of a new technology platform on a massive scale. For example, Business Insider’s tech editor Alistair Barr offers this definition:

A killer app is an application that is so useful and so easy to use that it convinces everyday people to adopt a whole new technology en masse. Spreadsheets and word-processing software made many individuals buy personal computers for the first time. The internet, possibly the biggest killer app of all, made us all buy smartphones, tablets, and a host of other connected devices.

A killer application provides a solution for a widespread use case, and the use case creates a market for the killer app’s solution that drives adoption of the platform on which the app runs, such as devices or operating systems. As Barr points out, we’ve seen applications like these repeatedly in the software industry, starting with spreadsheets on standalone PCs in the 1980s, followed by the internet in the 1990s, and then followed by Apple’s iTunes platform which drove sales of the iPod, iPhone and many other mobile devices. Recent killer apps that have also driven sales of smartphones, tablets and computers include TikTok, Snapchat and WhatsApp.

Investopedia points out that such killer apps are typically associated with huge numbers of users who use the apps frequently or intensely, and development firms with massive valuations. These apps can also provide substantial competitive advantages, brand loyalty, and profitability for their firms.

Today, tutoring delivered through an inexpensive AI app instead of by an expensive human tutor might qualify as just such a widespread use case with a potentially massive, lucrative market. Many tech industry experts and company spokespersons have recently argued as much during conferences, in scholarly papers, and in the press. That market for economical tutoring could potentially drive broad adoption of AI platforms, along with the upgraded hardware and software that users would need to purchase that those platforms require.

Now, here’s how Dr. Bloom’s purported results relate to education as the AI industry’s first killer app: In short, several tech firms and executives have attempted to boost the public’s demand for economical tutoring via AI by hyping the results claimed for tutoring by Dr. Bloom.

But Glenda Morgan, an industry analyst with the Phoenix-based educational technology consulting firm Phil Hill and Associates, argues in a June 2024 article that the AI industry’s emphasis on tutoring stems from a misinterpretation and oversimplification of Bloom’s Two Sigma Problem. “Too often people look at Bloom’s work and say, ‘hey, tutoring produces this big effect and has been unreachable to most people because of cost, and AI changes that,’” Morgan writes.

In fact, Dr. Paul von Hippel at the University of Texas at Austin recently cited several research studies showing that an excessive focus on tutoring hasn’t even come close to replicating Dr. Bloom’s purported two sigma gains:

More recently, a 2020 meta-analysis of randomized studies by Andre Nickow, Philip Oreopoulos, and Vincent Quan found that the average effect of tutoring was 0.37 standard deviations, or 14 percentile points—“impressive,” as the authors wrote, but far from two sigmas. Among 96 tutoring studies the authors reviewed, none produced a two-sigma effect. . .

The idea that tutoring consistently raises achievement by two standard deviations is exaggerated and oversimplified. . .In short, Bloom’s two-sigma claim had some basis in fact, but it also contained elements of fiction.

Morgan’s criticisms of the tech industry’s hyped misinterpretation of the Two Sigma Problem don’t end there. She then argues that the types of AI-based tutoring demonstrated by OpenAI and Google do not actually approximate the kinds of analysis of a student’s problem-solving approach that an experienced human tutor in practice actually performs:

Thus far I am unconvinced that the kinds of tutoring currently offered via AI matches the concept of watching a student’s thought processes and identifying the core issues they aren’t understanding.

Instead, AI tutoring today seems to consist of breaking down problems into component parts and explaining the components. This is no doubt helpful, but it is not tutoring in the true sense of the word.

In other words, Morgan appears to suggest that the tech industry relies on exaggerated and oversimplified research in attempts to pump up demand for the latest AI platforms, along with the new hardware and software required to run them. And the industry persists in pursuing those tactics even though the product they’re selling as “tutoring” probably isn’t advanced enough to approximate the analytical skills that an experienced human tutor actually provides to a student.

That doesn’t mean tutoring via AI can’t play a helpful educational role in certain circumstances, or that it won’t provide adequate value to students and parents relative to its cost. But it certainly remains unclear at this stage as to whether the initial generation of tutoring via AI will deliver enough value to students, and if that value will drive sufficient adoption of artificial intelligence platforms to qualify as AI’s first true killer application.