Production Evals For Agentic AI Systems - Nishant Gupta, Meta Superintelligence Labs

TL;DR

• 1

  Agentic AI shifts evaluation from answers to workflows: Instead of asking 'did the model generate the correct answer?', teams must ask 'did the system behave correctly?' across planning, tool use, recovery, and multi-agent coordination.

• 2

  Failure modes are hierarchical, not just hallucinations: Below reasoning errors lie memory failures, retrieval failures, safety failures, and at the top multi-agent coordination breakdowns. Evaluating only model output misses most production risk.

• 3

  Production telemetry is the highest-value evaluation signal: Every real-user interaction becomes evaluation data. Execution traces, user outcomes, escalations, and feedback signals are far more representative than any offline benchmark.

• 4

  Think like an SRE, not a researcher: The goal is not maximizing benchmark accuracy but maximizing dependable outcomes. Reliability, availability, latency, cost, and recovery become North Star metrics; accuracy is just one input.

• 5

  Agent systems drift silently: Model updates, prompt changes, tool changes, and shifting user behavior cause reliability to degrade slowly. Without continuous monitoring, teams don't discover drift until users complain.

• 6

  Evaluation is becoming part of the control plane, not a separate tool: The industry is moving toward an architecture where evaluation runs as an always-on service, collecting telemetry, running simulations, coordinating human review, and governing behavior in production.