VideoVideo:AI Engineer
Building Production-Grade Agent Evals: Lessons from Lyft
Nick Ung and Akshay Sharma from Lyft present an end-to-end framework for evaluating customer support AI agents. They detail why default offline simulations produce misleading pass rates and show how fine-tuned adversarial user simulators and calibrated LLM judges create trustworthy benchmarks.