My Homework Helper Was Wrong: the full story
Part One
It was already late on Friday night. Zoe had a book report due on Monday, and because she was tired, she opened her study helper, an AI program on her laptop, and typed a question. "Can you recommend a good novel about a girl who lives on a boat?" The answer appeared within seconds, smooth and friendly, as if a very clever friend had been waiting patiently for her question.
"Of course!" said the helper. "You might enjoy the novel The Silver Orchard Sail by Marian Holloway. It is about a twelve-year-old named Wren who sails around an island with her grandfather." Zoe felt reassured, because the title sounded perfect and the helper had sounded so certain. She wrote it at the top of her page and started organizing her report. She could hardly wait to tell her friends about it.
On Saturday she went to the library with her older brother Dev, and they walked past one shelf after another before she finally asked at the desk. The librarian typed in the title and shook her head. "I do not see that book anywhere," she said. "Are you sure you have the right title?" Zoe's stomach sank, and she read the author's name out loud. "Marian Holloway," said the librarian. "I do not know her either."
Back home, Zoe told her mother what had happened. "AI chatbots can make up facts," her mother explained, "and they can even make up the sources they name, which is why people should check what an AI tells them before they trust it." Zoe hesitated. "But it sounded so sure," she said. "AI can be wrong even when it sounds sure," said her mother, "so a confident answer is not proof."
"Why would it do that?" asked Zoe. Her brother Dev looked up from his phone. "Think of it like autocomplete," he said. "A language AI works by predicting which word is likely to come next. It learned from enormous collections of writing, so it can copy what a book title usually looks like." He sketched a few words on a napkin. "It guessed words that fit the pattern, but it does not think the way a person does."
Zoe tried a new plan. She asked the librarian for help. The librarian walked her to a real shelf of real books about girls and boats, and Zoe picked one with a faded blue cover and a title she could hold in her hand. "This one exists," she said, and she laughed. Her mother nodded. "A good way to check an AI is to look the facts up on other reliable pages, and to ask a person who knows," she said.
That evening, Zoe carefully wrote the title of the real book at the top of her page. Under it she wrote a note to herself: Check before you trust. She did not stop using her study helper, but she used it differently and more carefully now, treating it like a quick suggestion from a friend instead of the final authority. When it named a book, she searched for the book, and when it gave a fact, she looked for a second source.