See how well your agents work
AI quality shows each agent's success rate, review score, thumbs up, answers with sources, cost per successful run and how often your team overrides it, over the last 7, 30 or 90 days.
AI quality shows how well the agents in your workspace work. It puts the numbers for every agent on one page, with totals for the whole workspace at the top.
When you’d use this
- You want to know which agents succeed most often and which ones fail.
- You want to see what one good result from an agent costs.
- You want to know how often your team rejects what an agent suggests.
- You turned on Review real runs for an agent and want to follow its review score.
Finding it
Sidebar → Agents → AI quality.
Anyone who can see your agents can open this page.
Agents that need attention
The box at the top of the page lists agents that need attention. Each row names the agent, says what happened and when, and has a link that opens the agent. When nothing needs attention, the box says so.
An agent is listed for one of two reasons:
| Reason | What it means | When it leaves the list |
|---|---|---|
| Missed schedule | A scheduled run was due more than an hour ago and did not start. The row says why, for example the person the agent acts for paused its permission, or its previous run has not finished | When the agent runs again |
| Quality drop | The weekly check found that the agent got worse: your team rejected more of its work, its tools failed more often, or its review score fell | 7 days after it was noticed |
The agent’s owner and the people chosen for Agent notices also get a notice in their inbox. An agent misses at most one notice a day for a missed schedule, and one a week for a quality drop.
You only see agents you are allowed to see. A person who cannot see an agent does not see its row here or its notice.
Pick a period
The tabs at the top choose the period: the last 7 days, 30 days or 90 days. The page opens on 30 days. Every number on the page covers the period you pick.
What each number means
| Number | What it counts |
|---|---|
| Success rate | Runs that completed, out of runs that finished. A failed or timed-out run counts against it. A cancelled run is not counted either way |
| Review score | The share of reviewed runs that passed review. Runs are reviewed only for agents with Review real runs turned on in their Settings |
| Thumbs up | Thumbs up, out of all thumbs up and thumbs down people gave the agent’s results |
| Answers with sources | Of the runs whose answers can cite a source, the share that cite at least one |
| Cost per successful run | Everything the period’s runs cost, failed ones included, divided by the runs that completed |
| Overridden by your team | Of the agent’s suggestions and findings someone decided on, the share they rejected or dismissed |
A run that stopped because something it needed was missing is not counted as a run.
“Not measured”
A number reads Not measured when there is nothing to measure it from. For example:
- Review score for an agent without Review real runs turned on.
- Thumbs up for an agent nobody has rated.
- Answers with sources for an agent whose answers never cite sources.
“Not measured” is not the same as 0%. A 0% means something was measured and none of it passed.
The workspace totals
The cards at the top add up every agent in the list below them. When the review score has more than one week of reviews, its card shows a small trend line, one point per week.
One row per agent
Below the totals, each agent you turned on has a row with the same numbers. Click an agent’s name to open its page on the Overview tab.
An agent that has not run in the period still has a row, with Not measured where there is nothing to show. If you have not turned on any agent, the page shows the steps to turn one on.
Related
AI can make mistakes. Review it before you rely on it.