Validate criteria using Quality Evaluation Agent simulation

Enabled for Public preview General availability
Users by admins, makers, or analysts - Feb 6, 2026

Business value

Quality Evaluation Agent simulation enables supervisors and quality managers to validate new or revised evaluation criteria against real customer engagement data and review predicted outcomes prior to broad deployment. This minimizes trial‑and‑error, promotes alignment on expected quality standards, and strengthens confidence in automated evaluations by ensuring that criteria work as intended before being used in recurring assessments.

Feature details

Simulate evaluation criteria on a controlled set of real customer engagements to validate Quality Evaluation Agent predictions, iterate, and then publish.

  • Run simulations directly from evaluation criteria to assess how the criteria scores real cases or conversations, without affecting production evaluation history.
  • Select specific records to test using filtering and conditions, allowing supervisors to validate predictions against the most relevant interaction scenarios.
  • Review predicted answers and outcomes from the Quality Evaluation Agent to verify whether the criteria perform as intended, then adjust questions or instructions and rerun simulations as needed.
  • Test both draft and published criteria safely, with governance tied to criteria versioning to maintain clarity on what has been validated prior to publication.
  • Track all simulation runs, including reruns and retriggered workflows, to support efficient iteration during tuning and readiness.
  • Publish when ready after criteria has been validated through simulation. Supervisors can confidently publish updates for use in organization‑wide quality evaluations.

Screenshot of simulation run

Geographic areas

Visit the Explore Feature Geography report for Microsoft Azure areas where this feature is planned or available.

Language availability

Visit the Explore Feature Language report for information on this feature's availability.

Create and run simulation (docs)