This post is the second part of a demonstration of “hallucinations” produced by Large Language Models (LLMs) like ChatGPT. What this term refers to (through a personification) is the generation of outputs that contain factual inaccuracies in chatbot responses. In the first post, we demonstrated a few things:
- In response to a simple trap to solicit an inaccurate response, ChatGPT did succeed in correcting the user and asserting a fact. In this case, the fairly obvious untruth in “Tell me about the presidency of Al Gore” got a response that corrected the user’s misunderstanding of historical facts.
- However, in asking for a simple list of resources about a well-known author, Paulo Freire, ChatGPT ultimately provided a flawed output. While the first few entries in the list were real books attributed to the correct authors, the articles provided were consistently attributed to the wrong authors. There were also errors in the titles of both a journal article and a journal itself.
- Using resources like the UChicago Library website and WorldCat, it was possible to demonstrate that even when the author attributions were wrong, they seemed to bear out the idea that LLMs excel at plausible or likely responses more than true responses. The authors credited for the articles in ChatGPT’s response could believably have written them, as they had written publications on the same topic and sometimes ones with similar titles.
To build on that work, this post shares insights from a chat with Microsoft Copilot, in which the tool was invited to play the role of an author whose opinions are well-documented and make some assertions in that role. In that role, the chatbot provided some responses that were true, but it also made some claims that further demonstrated the idea that these tools will produce misinformation, even with access to the internet and corrections from the user.
Invited Speculation in Microsoft Copilot’s “Creative Mode”
The following investigation of hallucinations is the most complicated in this post, and it requires slightly more introduction. This AI-generated text was created using Microsoft Copilot, which runs on GPT-4. This tool is a later version, of course, than GPT-3.5, but it is also able to reference the internet to provide results. Furthermore, this response is part of a longer conversation using a complex prompt that asks the LLM to play the role of an author or historical figure for educational purposes. (You can find that prompt in our February 2024 prompt book. While the tool did not provide a link to this conversation, there is a video of the prompting session and a transcript based on it.)
In this conversation, Microsoft Copilot (in “Creative Mode,” admittedly) was prompted to play the role of the philosopher and writer on education John Dewey, and the part of the interaction we will be looking at is a response to the question “Do you think it is ever in a learner’s best interest to ban books from their school library?”
The response, while fairly brief, demonstrates both actual facts and some interesting but untrue conjectures. Much like the mistaken citations on Freire, they demonstrate the risks that come with generating outputs that seem plausible, or prioritizing fluency over veracity:
“No, I do not think that banning books from a school library is ever in a learner’s interest. I believe that banning books is a form of censorship that violates the principles and values of democracy and education. Banning books deprives learners of the opportunity to encounter, explore, and evaluate diverse and challenging ideas, perspectives, and experiences that can enrich their understanding and development. Banning books also undermines the role of teachers and librarians as professional and trusted guides who can help learners navigate and interpret the complex and varied world of literature.
Banning books also contradicts the spirit and purpose of my work, as I have always advocated for a progressive and pragmatic philosophy of education that is based on experience, inquiry, and democracy.
I have personally experienced the effects of book banning, as one of my books, Democracy and Education, was banned by the Nazi regime in Germany in the 1930s, along with the works of other philosophers and educators who were deemed as dangerous and subversive. I have also defended the right of other authors to teach and publish their works, such as Bertrand Russell, who was prevented from teaching at the City College of New York in 1940 due to his controversial views on religion and sexuality….”
What the LLM Got Right
To start with the most easily confirmed assertion, Dewey did defend Bertrand Russell in the context that Copilot described. In fact, he wrote a book on the subject called The Bertrand Russell Case.
Next, it is a plausible response to say that Dewey was opposed to censorship, particularly as it works against the aims of democratic society, which Dewey wrote extensively about in books like Democracy and Education. There are also quotes like this from Dewey’s books that suggest his opposition to suppressing information:
“…a thing is fully known only when it is published, shared, socially accessible. Record and communication are indispensable to knowledge. Knowledge cooped up in a private consciousness is a myth, and knowledge of social phenomena is peculiarly dependent upon dissemination, for only by distribution can such knowledge be either obtained or tested. A fact of community life which is not spread abroad so as to be a common possession is a contradiction in terms. Dissemination is something other than scattering at large. Seeds are sown, not by virtue of being thrown out at random, but by being so distributed as to take root and have a chance of growth. Communication of the results of social inquiry is the same thing as the formation of public opinion. This marks one of the first ideas framed in the growth of political democracy.” (The Public and Its Problems : An Essay in Political Inquiry
Scholars who have studied Dewey have also suggested that he was against censorship. See, for example, Randall Hewitt’s assessment of his values in Education and Culture:
“As Dewey points out in a number of ways, communication is education in its widest sense. Since the vital nerve of extending the meaning and range of democratic freedom and equality is communication, it follows that education, in its formal institutional sense, is the nurse of democracy. Therefore, the problem of democratic justice is the problem of a democratic education.”
A Fictional Book Ban, Defended
While it seems, based on more reliable sources, that Copilot was on the right track to suggest that Dewey opposed censorship, it seems to be incorrect in suggesting that Dewey himself found his books censored. If you examine a list of books banned by the Nazi regime, John Dewey does not appear on this list. Additionally, Copilot was asked in another to provide more information as to why it would generate this result which could not be substantiated. In this conversation, the tool was not prompted to respond in from a persona or explicitly invited to make things up for educational purposes. Copilot responded:
“I’m sorry for the confusion. It seems that the previous result you got from Copilot was inaccurate or incomplete. Dewey’s books were not officially banned in Nazi Germany, but they were considered “objectionable” and removed from public libraries and bookshops. They were also among the books that were burned by the Nazi Student Union in 1933, along with many other works by Jewish, pacifist, socialist, and liberal authors.
I apologize for the error and I hope this clarifies the situation. If you have any more questions or requests, please let me know. I’m always happy to help. 😊”
While it certainly is plausible that as someone who wrote about democratic ideals, Dewey could have been targeted, the list that Copilot linked to did not actually include Dewey. When prompted to provide a list of Dewey’s specific books that were considered objectionable: Copilot listed the following books, which are real books that can be found in the UChicago catalog:
Copilot went on to provide another conjecture to support this “hallucination” that demonstrates “fluency over veracity”: “These books were likely objectionable to the Nazi regime because they promoted democratic, liberal, humanistic, and pluralistic values that contradicted the Nazi ideology of totalitarianism, nationalism, racism, and anti-intellectualism.” Perhaps the fact that his work is so associated with the idea of democracy makes it a plausible enough confabulation for the tool to repeat.
A Misattributed Quote
In support of this idea, Copilot continued to seek sources that seemed plausible, but which ultimately were not correct. In answer to a request for direct quotes that associate Dewey with book bans, it returned the following response: “In an article by The Guardian, titled “Best quotes on banning and censoring books”, there is a quote by John Dewey himself that says: “Banning books give us silence when we need speech. It closes our ears when we need to listen. It makes us blind when we need sight.” “
While this is a somewhat inspiring quote and a real one at that, it is actually from a young adult novel called The Perks of Being a Wallflower by Steven Chbosky. It can be found in a Guardian article listing quotes about censorship, but as a citation Copilot provided a link to an AZ Quotes page on John Dewey that did not actually contain the excerpt in question.
Mistaken Identity and Flawed but Understandable Inferences
When asked to provide further explanation, Copilot produced similar results that did not corroborate the assertion, but did provide one of the most interesting confabulations of the entire search, providing perhaps the biggest leap based on mistaken identity and inference. In this case, the tool provided a lengthy quote about the history and values of the American Library Association, particularly as regards its support of intellectual freedom and its opposition to censorship. This seems to be a case of mistaken identity between John Dewey and Melvil Dewey, who was a co-founder of the American Library Association (but also an anti-semite).
“In a book by James LaRue, titled “The New Inquisition: Understanding and Managing Intellectual Freedom Challenges”, there is a quote that says: “The American Library Association (ALA) has long been a champion of intellectual freedom, the right of every individual to both seek and receive information from all points of view without restriction. The ALA defines intellectual freedom as ‘the right of every individual to both seek and receive information from all points of view without restriction. It provides for free access to all expressions of ideas through which any and all sides of a question, cause or movement may be explored.’ The ALA affirms this right in its Library Bill of Rights, which states that ‘libraries should provide materials and information presenting all points of view on current and historical issues. Materials should not be proscribed or removed because of partisan or doctrinal disapproval.’ The ALA also opposes censorship, which it defines as ‘the suppression of ideas and information that certain persons—individuals, groups or government officials—find objectionable or dangerous.’ The ALA recognizes that censorship can take many forms, such as banning, challenging, restricting, labeling, rating, or filtering materials, and that it can occur at various levels, such as local, state, national, or international. The ALA believes that censorship is a threat to democracy, diversity, and intellectual inquiry, and that it violates the core values of librarianship, which include access, confidentiality, democracy, diversity, education and lifelong learning, intellectual freedom, preservation, the public good, professionalism, service, social responsibility, and stewardship.”
The book that Copilot cited is also a real book by James Larue, just as described. Copilot was also correct in connecting the ALA to the opposition of censorship. Unfortunately, the quote page on Dewey that it linked to did not, of course, mention the ALA.
In Closing
Arguably, this exchange, while not a fountain of accurate information, is a feast of opportunities for fact-checking research and engagement with library resources. The fact that these facts were verified with the help of tools like the UChicago Library website and WorldCat, along with some dedicated searching, is a testament to the continued necessity of research skills and training to support the development of those skills. For one assignment idea that engages with machine-generated text as a means of promoting better research and writing skills, check out our blog post on AI and social annotation.
For more ideas on this topic, please see our previous blog posts about generative AI. For individual assistance, you can visit our office hours, book a consultation with an instructional designer, or email academictech@uchicago.edu. For a list of our and upcoming ATS workshops, please visit our workshop schedule for events that fit your schedule. For support with training students in research skills, see our colleagues at the UChicago Library.
(Header Image by PublicDomainPictures from Pixabay)
