Falaah Falaah AI
Announcements

Check an Agent Change Before You Save It

The new Evaluate tab tries a change to an agent's settings, model or prompt on work it already did, and lists the answers that got worse first. Nothing is saved until you save it.

FT
Falaah Team
· · 3 min read
Check an Agent Change Before You Save It

Every agent in Muin now has an Evaluate tab. It answers one question before you save a change: does this make the agent’s answers worse?

Build checks from past runs

Open an agent and click Evaluate, then Build from past runs. Pick how many runs (up to 50) and how far back: the last 7, 30 or 90 days. Reviewed runs first is on by default, so runs someone approved or corrected come first.

Each run becomes a check. A reviewed run uses the answer that person approved or corrected. A run nobody reviewed uses what the agent produced at the time, marked Not reviewed.

Fix what is wrong

Open the set of checks. Edit answer sets what a correct answer looks like, and Included leaves a check in or out. A check is scored by matching fields, or judged by AI against a sentence you write.

Compare a change

Under Compare a change, choose what to try:

  • Unsaved settings, including a different model.
  • A new prompt, for an agent you built.
  • A saved prompt version, to see whether an older one did better.

Before it runs, the panel says “This will run N cases twice and use about X credits.” Each check runs twice: once as the agent works today and once with your change. Nothing is saved, sent or changed in your records.

On an agent you built, Compare before publishing in the page header opens this with a new prompt ready to edit.

Read the results

Regressions come first: checks that passed today and fail with your change. Then Improvements, Unchanged and Not graded. Each side shows how many checks passed, and the results show the credits used and the time per case.

When checks fail, Suggested changes proposes edits written from those failures. Nothing is applied for you. Copy the text, open the agent’s settings and save it yourself.

What it costs

A comparison uses AI credits, listed as Quality checks on the Usage Breakdown tab. It runs only when you start it. Saving or publishing an agent does not start one. See AI Credits.