Agent Seer: Synthesizing Scenarios from Specification Understanding

Agent Seer is a pipeline that generates realistic multi-turn dialogue scenarios for evaluating AI agents using tool specifications alone, without live tool access or manual curation. It synthesizes scenarios from function names, descriptions, and parameter schemas, achieving strong tool-calling correctness and conversational coherence across diverse domains. The study finds parameter schema complexity impacts quality more than tool-suite size, with argument value accuracy as a key failure mode.