You deployed an AI system. It works well in testing. Then it hallucinates in production - on queries you never tested, with context you never anticipated, at 3 AM on a Saturday. Hallucination monitoring is how you catch it before users do.
AI systems hallucinate more in production than in testing. Testing uses curated prompts. Production uses whatever users type. Testing has human review. Production runs at scale. Testing covers the scenarios you thought of. Production encounters the ones you did not.
Hallucination rates also change over time. Model updates, knowledge cutoffs, context drift, and evolving user behaviour all affect how often and where the model fabricates. A system that was accurate last month may hallucinate on topics that changed since its training data was assembled.
You need continuous monitoring, not one-time evaluation. The same way you monitor uptime, latency, and error rates, you need to monitor hallucination rate as a production metric.
Claim verification rate. What percentage of factual claims in the model's output can be verified against your knowledge base or external sources? A drop in this rate signals increasing hallucination.
Confidence-accuracy correlation. Track whether the model's expressed confidence (hedging language, certainty markers) correlates with actual accuracy. If calibration drifts - the model becomes more confident while accuracy drops - hallucination risk is increasing.
Novel entity generation. Monitor for the model generating entities (names, citations, URLs, product names) that do not exist in your knowledge base. New entities are not always hallucinations - but a spike in unverifiable entities is a red flag.
User feedback signals. Track "that's wrong" feedback, correction requests, and follow-up questions that suggest the previous answer was fabricated. User-reported hallucinations are a lagging indicator but validate automated detection.
Domain-specific consistency. For each domain your AI operates in, monitor whether responses are consistent with known facts in that domain. A medical AI that starts suggesting deprecated drug interactions. A legal AI that cites overturned precedents. Domain-specific drift signals hallucination.
Monitoring tells you the model is hallucinating. It does not stop the hallucination from reaching the user. Detection after delivery is an incident. Verification before delivery is prevention.
Monitoring is necessary. It is not sufficient. Monitoring detects hallucination after it happens. It tells you the rate is 3% today, or that a specific topic has spiked to 12%. This is useful for system health. It does not help the user who received the hallucinated response.
Detection catches some hallucinations before delivery - automated classifiers that flag likely fabrications. This helps but has a false negative rate. Some hallucinations pass the detector.
Prevention is the only approach that stops hallucination at the source. Ground the model in reality. Give it the actual data before it responds. Validate its output against that data before delivery. Check does this - it is not a monitor. It is a verification layer. The hallucination is caught before it reaches the user, not after.
The ideal production system has both: verification to prevent hallucination, and monitoring to confirm the verification is working and to catch edge cases where even verification might have gaps.
120 verifications a day free. No card, no signup.