Mirasim Documentation
Evaluating with $eval
Send simulated users through the real thing, keep their traces as evidence, and let that decide the next task.
Why simulated users
Most of what you need to learn early about a build is coarse: the step nobody finishes, the wording nobody understands, the button nobody finds. Those failures show up in a simulated run, they show up in minutes rather than weeks, and they show up before the work is sunk.
Evaluation in Mirasim is an explicit skill, not an always-on review pass. You invoke `$eval` when you want evidence, and the simulation runs locally.
ArtifactsScreenshots, traces, commands and reports all land here, anchored to the host that produced them. A conclusion you cannot trace back to a run is treated as no conclusion. — this control is not on the captured screen.
The buttons in this panel work — click them to look around.
Running an evaluation
The workflow is deliberate about what counts as signal, and the order matters.
- Fix the version and name the decision
Pin the exact build under test and state what you will do differently depending on the outcome. An evaluation that cannot change a decision is a report nobody reads.
- Create isolated personas and natural scenarios
The way you would pick a test group: the impatient first-timer, the expert who skips onboarding, the skeptic looking for a reason to leave — each with a task they would plausibly have, not a script to follow.
- Give each user the real entry point
A clean context and the actual product: a browser, a phone, a shell, a conversation, or a rendered artifact. Runtime actions and observations are the evidence — reading the source is not treated as user evidence.
- Read the evidence, then decide
Screenshots, commands, traces and artifacts are preserved, with what was observed kept separate from what was inferred. Then keep the change, revert it, or re-verify with held-out cases.
Reading the result
A run is worth reading in a specific order, most useful first:
- Where did users actually get stuck? — the runtime action or observation where the run broke down, with the trace preserved so you can watch it happen.
- What was observed, versus what was inferred? — kept separate on purpose, so you can trust the first and argue with the second.
- What is the next task? — each observed failure converts directly into a build task for the next turn of the loop.
Past runs stay in the app: ⌘K, then Eval runs, opens the list of recorded simulation runs, so last week's round is one search away rather than a folder you have to remember.
Something wrong or missing on this page? Tell us