What is Impact Simulation?
An Impact Simulation runs a policy version against a large set of historical execution data, then compares the results against a baseline version. Think of it as a regression test suite — before deploying a new version, you can see exactly how it would have performed on past traffic.Impact Simulation vs Dry Run: A Dry Run tests one input at a time for quick iteration. An Impact Simulation tests hundreds or thousands of inputs at once and produces aggregate metrics (match rate, metric deltas, per-rule statistics). Use Dry Runs during development; use Impact Simulation before deployment.
What You Get
Reading the Change Distribution
One aggregate delta looks the same whether many records moved a little, one record moved a lot, or large moves in both directions cancelled out. Those three carry very different risk, so the distribution reports the shape as well as the total.
The whole object is
null when no baseline version was compared, and on runs that completed
before the field existed.
largestIncrease and largestDecrease are null when the aggregation is COUNT, because the
per-record difference is then only -1, 0 or +1 and no single record moves further than any other.
Running a Simulation
Console
Navigate to a policy group → Simulation tab → select candidate version, baseline version, date range → click Start.CLI
metricConfig.targetVariable points at any key in the response — input facts, mutated facts, or generatedVariables (including the {key}__delta keys produced by MUTATE_FACT). Pick the variable whose aggregate behavior you want to compare across versions.COMPLETED or FAILED:
Dataset Types
Record Limits
options.maxRecords bounds how many records one simulation processes.
A
maxRecords above the hard cap is rejected with 400 rather than clamped down to it.
An explicit maxRecords is honored for every type, MANUAL included.
Every value in a MANUAL or UPLOADED dataset has the same value limits as a value in an execution call. If a value is over a limit, starting the simulation fails with 400 (P-064), and the message gives the value’s path, such as dataset[0].paymentAmount for paymentAmount in the first record.
A MANUAL dataset is part of the request body, so the request that starts the simulation has the same request body limit as an execution call. A larger request fails with 413 (C-011), and one nested deeper than the limit fails with 400 (C-012); the simulation does not start. Put a larger dataset in a file and use UPLOADED (File Upload Dataset).
File Upload Dataset
Upload a CSV or JSON file to use as simulation input. This is useful when you have custom test data that doesn’t come from execution history.Supported Formats
- CSV
- JSON
First row must be a header with fact keys. Each subsequent row is one test record.Type inference: Numbers, booleans (
true/false), and strings are automatically detected. A cell holding a JSON array, such as ["gold","vip"], becomes a list, so a CSV file can carry a list fact; quote the cell, as CSV requires for a value with commas. Empty values become null. Quoted fields with commas are supported.AN-040. In Excel, save as CSV UTF-8, not plain CSV. The byte order mark (BOM) that this format adds at the start of the file is accepted.
Download a Template
Don’t know the expected columns? Download a template pre-filled with example values based on the version’s required facts. Each example fits its fact’s type and value constraints, so the template passes the simulation’s type and value checks as it is:- Console
- CLI
- API
In the New Simulation dialog, select File Upload (CSV / JSON) as dataset type. Click CSV or JSON under “Download template.”
Upload & Run Simulation
- Console
- CLI
- MCP
- Go to Impact Simulations → New Simulation
- Select policy group and target version
- Set Dataset Type to File Upload (CSV / JSON)
- Drag & drop your file or click to select
- Wait for the Uploaded badge
- Click Start Simulation
Managing Simulations
# Metric Distribution section, with the columns
ChangedRecords,IncreasedRecords,DecreasedRecords,LargestIncrease,LargestDecrease,MeasuredRecords.
The section is omitted entirely when there was no baseline, and the two Largest cells are left
empty when the aggregation is COUNT.
Best Practices
- Always set a baseline. Without a baseline version, you only see absolute numbers — no deltas.
- Use a meaningful date range. At least 7 days of data for statistically significant results.
- Check
maxRecords. Start with 5,000–10,000 for quick validation, and read Record Limits above before you read a capped result as a random sample. - Run simulation before every production deployment.
Next Steps
Dry Run
Quick single-input validation during development.
A/B Testing
Compare versions with live production traffic.

