What is Eval (LLM Evaluation)?
A systematic method for measuring the quality of AI-generated answers — often using a second model as a judge — so quality is tracked rather than assumed.
You cannot manage what you do not measure, and AI answer quality is no exception. Evals score generated answers against criteria such as groundedness (does the answer follow from the retrieved sources?), completeness, and correctness.
The LLM-as-judge pattern uses a capable model to score answers at scale — imperfect but consistent, and far cheaper than human review of every conversation. Judged scores trend meaningfully: a docs migration that quietly broke retrieval shows up as a score drop before customers complain.
Mature eval practice combines automated judging with human signals (user feedback, review queues) and drill-down tooling (traces) to diagnose why a score moved.
How BeforeQuery uses this
BeforeQuery runs LLM-judge evals per knowledge base, collects per-message user feedback, and records traces of every retrieval — so you see quality trend over time, catch regressions after source changes, and trace any weak answer to its cause. Simulations extend the same discipline to pre-launch ticket replay.
Related terms
Hallucination
When a language model generates fluent, confident content that is factually wrong or entirely invented.
Citations
Links from an AI answer back to the specific source documents used to generate it, allowing users to verify every claim.
RAG (Retrieval-Augmented Generation)
An AI architecture that retrieves relevant documents from a trusted knowledge source and passes them to a language model, so answers are grounded in real content instead of the model's training data.
From definition to deployment
Connect a knowledge source and see cited AI answers on your own content in minutes — free, no credit card required.
Get Started Free