Keep the voice agent snappy

You are serving a real-time voice agent: a caller speaks, the model answers, and anything over ~600 ms of model latency feels like a dropped beat. Peak load is 20 requests/second, and because calls arrive in bursts you need 30% throughput headroom. Quality must stay ≥ 80%. The board weights latency heaviest here — but a config that buys low latency with a rack of idle GPUs will lose to one that gets there leanly. Speculative decoding and batch size are your sharpest levers.

Loading the simulator…

The four axes are recomputed on our servers from the config you submit — the metrics you see are a faithful preview of the same model, so a forged score is impossible. Only players who choose a handle and opt in appear on the public board; we never show your email.