Evaluating with $eval
Send simulated users through the real thing, keep their traces as evidence, and let that decide the next task.
Why simulated users#
Most of what you need to learn early about a build is coarse: the step nobody finishes, the wording nobody understands, the button nobody finds. Those failures show up in a simulated run, they show up in minutes rather than weeks, and they show up before the work is sunk.
Evaluation in Mirasim is an explicit skill, not an always-on review pass. You invoke `$eval` when you want evidence, and the simulation runs locally.
ArtifactsScreenshots, traces, commands and reports all land here, anchored to the host that produced them. A conclusion you cannot trace back to a run is treated as no conclusion. — this control is not on the captured screen.
The buttons in this panel work — click them to look around.
Running an evaluation#
The workflow is deliberate about what counts as signal, and the order matters.
- Fix the version and name the decision
Pin the exact build under test and state what you will do differently depending on the outcome. An evaluation that cannot change a decision is a report nobody reads.
- Create isolated personas and natural scenarios
The way you would pick a test group: the impatient first-timer, the expert who skips onboarding, the skeptic looking for a reason to leave — each with a task they would plausibly have, not a script to follow.
- Give each user the real entry point
A clean context and the actual product: a browser, a phone, a shell, a conversation, or a rendered artifact. Runtime actions and observations are the evidence — reading the source is not treated as user evidence.
- Read the evidence, then decide
Screenshots, commands, traces and artifacts are preserved, with what was observed kept separate from what was inferred. Then keep the change, revert it, or re-verify with held-out cases.
Reading the result#
A run is worth reading in a specific order, most useful first:
- Where did users actually get stuck? — the runtime action or observation where the run broke down, with the trace preserved so you can watch it happen.
- What was observed, versus what was inferred? — kept separate on purpose, so you can trust the first and argue with the second.
- What is the next task? — each observed failure converts directly into a build task for the next turn of the loop.
Past runs stay in the app: ⌘K, then Eval runs, opens the list of recorded simulation runs, so last week's round is one search away rather than a folder you have to remember.
Something wrong or missing on this page? Tell us