As artificial intelligence (AI) becomes increasingly embedded in our daily lives, understanding the nuances of its behavior is necessary. Generative AI hallucinations are when an AI system produces outputs that are factually incorrect or misleading despite appearing credible. 

Generative Language Models are designed to create content based on patterns learned from large datasets. While they have powerful capabilities to generate coherent text, images, and other types of content, they can also generate incorrect or nonsensical responses, understanding this phenomenon is crucial to knowing when to trust your AI system. 

Generative AI hallucinations occur when the AI generates information that is not only factually incorrect but also appears convincing. This can manifest in the following ways:

Fabricating information

The AI might create details or entire narratives that are false. This behavior is particularly problematic in contexts requiring factual accuracy, such as news reporting or scientific research. 

For instance, in the 2020 "GPT-3 and the ‘Fake Scientist’ Incident," OpenAI's GPT-3 generated detailed yet entirely fictional scientific research papers. These papers included invented authors and fabricated data, illustrating how the AI can convincingly create misleading scientific content.

Another example occurred in 2023 when Google’s Bard was found to generate misleading information about historical events and scientific facts. Bard provided inaccurate dates and events in its responses, demonstrating its capacity to fabricate historical narratives that seemed plausible but were ultimately incorrect.

Contextual misunderstanding

If an AI system misunderstands the context, it can result in irrelevant or nonsensical output. For instance, it might generate a believable-sounding answer that has no grounding in actual knowledge.

Extrapolating from inaccurate data

The AI might base its responses on outdated or incorrect training data, which can lead to errors or misconceptions.

Understanding hallucinations is important to mitigate this risk to ensure accurate and reliable output for maintaining user trust and effective decision-making.  In healthcare, for example, a hallucination can lead to incorrect medical advice. Moreover, if hallucinations enter into news or communications, they can lead to misinformation spread or faulty recommendations, causing the public to lose trust in the source of this information.

At AI Sweden, TrustLLM represents an initiative to advance European large language models (LLMs). AI Sweden is tasked with collecting and processing training data as well as developing and training the foundational models for this project. The goal is to create open, trustworthy, and sustainable LLMs with a primary focus on Germanic languages. This effort will lay the groundwork for a next-generation, modular, and extensible ecosystem of European AI, enhancing context-aware human-machine interaction across diverse applications.

Contact us if you are a partner in AI Sweden and want to learn more or engage with AI Sweden in AI security.

→ Sign up here to receive updates from the AI Security Newsletter!

Related projects at AI Sweden

TrustLLM: https://www.ai.se/en/project/trustllm 

FormAI: https://www.ai.se/en/project/formai 

Read more about our Natural Language Understanding initiatives: https://www.ai.se/en/labs/natural-language-understanding

July 22, 2024 by Madeleine Xia