The EU AI Act is the first comprehensive AI regulation. It requires high-risk AI systems to be accurate, robust, and transparent. AI hallucination - confidently generating false information - violates all three requirements. Non-compliance fines reach 35 million EUR or 7% of global revenue, whichever is higher.
Accuracy (Article 15). High-risk AI systems must achieve "appropriate levels of accuracy" for their intended purpose. A system that hallucinates - generating false information and presenting it as true - fails this requirement by definition.
Robustness (Article 15). Systems must be resilient to errors and inconsistencies. Hallucination is the opposite of robustness - it is the system generating errors as a normal part of operation, not as an exceptional failure.
Transparency (Article 13). Users must be able to understand and appropriately use the system's output. When a system generates hallucinated content with the same confidence as accurate content, users cannot distinguish reliable output from fabricated output. Transparency is undermined.
Human oversight (Article 14). The Act requires that humans can effectively oversee the system. But hallucinations are designed to look correct - the model generates them with full confidence and proper formatting. Human oversight of a system that produces indistinguishable true and false output is not meaningful oversight.
The EU AI Act does not mention "hallucination" by name. It does not need to. Every hallucination violates at least one of its core requirements.
The Act classifies AI systems by risk level. High-risk systems face the strictest requirements. These include AI used in:
Healthcare. Clinical decision support, diagnostic AI, medical device software. A hallucinated dosage or fabricated guideline in these contexts is both an accuracy failure and a safety risk.
Legal. AI used in judicial systems, legal research tools, automated legal analysis. Fabricated citations and misrepresented holdings are accuracy failures with professional consequences.
Financial services. Credit scoring, risk assessment, fraud detection, compliance tools. Hallucinated financial data or fabricated regulatory requirements create compliance and market risk.
Employment. Recruitment tools, performance evaluation, workforce management AI. Hallucinated assessments or fabricated candidate information affect hiring decisions.
Critical infrastructure. AI operating on infrastructure systems. Hallucinated commands or fabricated system states can cause outages or security breaches.
Accuracy metrics. Organisations must measure and document their AI system's accuracy, including hallucination rates. "We use GPT-4" is not a compliance statement. Specific, measured hallucination rates for your use case, with your data, in your deployment context are required.
Risk management. Hallucination risk must be identified, assessed, and mitigated as part of the required risk management system. This means acknowledging that the model can generate false output and implementing controls to catch it.
Verification mechanisms. The most direct path to compliance is implementing a verification layer that validates AI output before it reaches users. This addresses accuracy (verified output is accurate), robustness (verification catches errors), and transparency (users can distinguish verified from unverified claims).
Documentation. The technical documentation must describe how the system manages hallucination risk. What mitigation measures are in place, what their effectiveness is, and what residual risk remains.
The Act does not prescribe specific technical solutions. It sets requirements and leaves implementation to providers. But the requirements point clearly toward verification.
A verification layer that checks AI output against reality before delivery directly addresses the Act's accuracy requirement. It provides measurable, documented accuracy improvement. It creates an audit trail showing which outputs were verified and which were not.
Check provides this for infrastructure operations. It reads the actual environment state, injects it into the AI's context, and validates every output before execution. The same verification architecture applies to any high-risk domain - the principle is the same, only the data source changes.
120 verifications a day free. No card, no signup.