Measure the pause the caller experiences

A voice demo can feel quick without showing how an AI agent behaves during a real business request. For your evaluation, define response time as the interval from the caller finishing a turn to the first audible response. Record the measurement method and keep it consistent across candidates.

This is a proposed evaluation method, not a published Voxistry benchmark. It sets no universal latency target and contains no measured product results.

Use a repeatable call set

Prepare the same set of scenarios for each configuration: a simple information question, a request requiring a system lookup, an interruption, unclear speech and a human handoff. Include different call conditions representative of your intended users.

Record configuration, test date, channel, region, scenario, sample count and failures. State whether a result includes the business-system lookup. Otherwise, two apparently comparable response-time numbers may describe different work.

Look at the distribution and the failure cases

Keep individual observations as well as a summary. Review the median and slower responses, and report sample size with any percentile. Do not remove failed calls from the report: record when no response arrived, when audio overlapped and when the worker spoke before the required information was available.

Listen to a sample of calls alongside the timing data. An early acknowledgement can reduce silence, but measure the time to a useful answer separately. A prompt “I am checking” and a completed answer are different events.

Use a scorecard that includes the work

  • Turn response: how long until audible speech begins?
  • Useful answer: when does the caller receive the information needed?
  • Interruption: does the AI agent stop and respond to the revised request?
  • Action completion: does the business system show the requested outcome?
  • Handoff: did a person accept the conversation with usable context?
  • Recovery: what happens after a lookup, provider or transfer failure?

Set your acceptance thresholds with the operational owner before testing. Record any tradeoff between speed, answer quality and completion instead of declaring a winner from the fastest demonstration.

Ask for the evidence behind the number

Request the test scope, recordings permitted for review, timestamps and unresolved failures. If engineers use distributed tracing, ask how it connects to the caller-visible measurements. OpenTelemetry’s observability primer provides background on examining system behavior; technical traces alone are not a completed-call outcome.

Explore voice operations and the AI operations scorecard to prepare a broader evaluation.