Skip to content

Scenario CI

Do not discover a broken scenario in front of a client.

A scenario can look fine in the editor and still leak the answer, give in too easily or score something the learner never had a chance to demonstrate. Run automated checks and simulated learners before you publish.

What gets tested before publishing

  • Whether hidden information stays hidden

    Tests try to extract private facts without earning them and check that they do not leak into the learner brief, API responses or the counterpart behaviour.

  • Whether the scenario can actually be completed

    Simulations look for unreachable success conditions, counterparts that never move and branches that end the conversation before the learner can do the work.

  • Whether the rubric measures what it claims

    A criterion needs evidence in the conversation. If it cannot be scored from the transcript or scenario state, the test flags it before publication.

Simulated learners instead of one perfect happy path

Different behaviour profiles run the scenario: prepared, passive, pushy and rule-bending. You see where the AI counterpart breaks character, not only whether one ideal conversation happens to work.

Published versions stay measurable and repeatable

  • Immutable version

    Once published, a learner practices against a specific scenario version. Later edits do not rewrite the history behind earlier results.

  • Regression after a change

    Change a prompt, persona or criterion and run the same checks again, so fixing one behaviour does not quietly break another.

  • Quality proof before delivery

    A training company can treat scenario validation as quality assurance for its own product instead of experimenting on the first paying cohort.

Product

Test your first scenario before you ship it

Build a draft from an exercise you already teach and see where the simulation breaks before a learner does.