Impact Simulation

What is Impact Simulation?

An Impact Simulation runs a policy version against a large set of historical execution data, then compares the results against a baseline version. Think of it as a regression test suite — before deploying a new version, you can see exactly how it would have performed on past traffic.

Impact Simulation vs Dry Run: A Dry Run tests one input at a time for quick iteration. An Impact Simulation tests hundreds or thousands of inputs at once and produces aggregate metrics (match rate, metric deltas, per-rule statistics). Use Dry Runs during development; use Impact Simulation before deployment.

What You Get

Metric	Description
Match Rate	Percentage of processed records where at least one rule matched. The denominator excludes records that errored, since a rule match could not be judged for those
Per-Rule Stats	How many times each rule matched and its metric contribution
Metric Delta	Difference in aggregate output between baseline and candidate
Policy Impact	Matched count delta, match rate delta, metric value delta
Change Distribution	How many records changed, how many rose and fell, and the largest single move in each direction

Reading the Change Distribution

One aggregate delta looks the same whether many records moved a little, one record moved a lot, or large moves in both directions cancelled out. Those three carry very different risk, so the distribution reports the shape as well as the total.

Field	Meaning
`changedRecords`	Records whose value differs between the two versions
`increasedRecords` · `decreasedRecords`	The same count split by direction
`largestIncrease` · `largestDecrease`	The biggest single-record move in each direction
`measuredRecords`	Records where the target variable is present in both versions. This is what `AVG` divides by

measuredRecords is not the denominator of the change counts. The two are counted over different populations. The change counts run over every processed record and treat an absent variable as zero, while measuredRecords counts only records where the variable is genuinely present in both versions. A record that has it in one version only is counted as changed but not as measured, so changedRecords can exceed measuredRecords.

The whole object is null when no baseline version was compared, and on runs that completed before the field existed — a stored result is returned as-is, so older runs carry no such key. largestIncrease and largestDecrease are null when the aggregation is COUNT, because the per-record difference is then only -1, 0 or +1 and no single record moves further than any other.

Running a Simulation

Console

Navigate to a policy group → Simulation tab → select candidate version, baseline version, date range → click Start.

CLI

lexq analytics simulation start --json '{
  "policyVersionId": "<candidate-version-id>",
  "dataset": {
    "type": "HISTORICAL",
    "source": "EXECUTION_LOGS",
    "from": "2025-01-01",
    "to": "2025-01-31"
  },
  "options": {
    "baselinePolicyVersionId": "<baseline-version-id>",
    "includeRuleStats": true,
    "maxRecords": 10000,
    "metricConfig": {
      "targetVariable": "paymentAmount__delta",
      "aggregationType": "SUM"
    }
  }
}'

metricConfig.targetVariable points at any key in the response — input facts, mutated facts, or generatedVariables (including the {key}__delta keys produced by MUTATE_FACT). Pick the variable whose aggregate behavior you want to compare across versions.

Simulations run asynchronously. Poll until status is COMPLETED or FAILED:

lexq analytics simulation status --id <simulationId>

Dataset Types

Type	Source	Description
`HISTORICAL`	`EXECUTION_LOGS`	Re-run past execution inputs from the specified date range, scoped to the target version’s policy group
`MANUAL`	`REQUEST_BODY`	Provide inputs directly in the request body
`UPLOADED`	`S3_BUCKET`	Use a pre-uploaded CSV or JSON dataset from S3

Record Limits

options.maxRecords bounds how many records one simulation processes.

Setting	Value
Hard cap	100,000 records per simulation
Default sample (`maxRecords` omitted)	1,000 records for `HISTORICAL` and `UPLOADED` · the whole array for `MANUAL`

A maxRecords above the hard cap is rejected with 400. The request is never silently truncated: a truncated run reports 100% progress against the truncated denominator, so nothing on screen would tell you the rest was dropped. The default only applies where the engine cannot know the size in advance. HISTORICAL is a date range and UPLOADED is a file in object storage. MANUAL is an array you just sent, so defaulting it would silently discard data you provided explicitly. An explicit maxRecords is honored for every type, MANUAL included.

The sample is not random. HISTORICAL reads the date range newest first (by createdAt then id, both descending, so the order is the same on every run) and takes the leading maxRecords. UPLOADED takes the leading rows in file order. A 30-day range that hits the cap measures your most recent executions, not a spread across the month — narrow the range rather than trusting the cap to sample it for you.

File Upload Dataset

Upload a CSV or JSON file to use as simulation input. This is useful when you have custom test data that doesn’t come from execution history.

Supported Formats

CSV
JSON

First row must be a header with fact keys. Each subsequent row is one test record.

paymentAmount,customerTier,isFirstPurchase
150000,VIP,true
50000,REGULAR,false
80000,VIP,false

Type inference: Numbers, booleans (true/false), and strings are automatically detected. Empty values become null. Quoted fields with commas are supported.

A JSON array of objects, where each object is a set of facts.

[
    {"paymentAmount": 150000, "customerTier": "VIP", "isFirstPurchase": true},
    {"paymentAmount": 50000, "customerTier": "REGULAR", "isFirstPurchase": false}
]

Single-object format is also accepted (wraps into a 1-element array):

{"facts": {"paymentAmount": 150000, "customerTier": "VIP"}}

Max file size: 10 MB

Download a Template

Don’t know the expected columns? Download a template pre-filled with example values based on the version’s required facts:

Console
CLI
API

In the New Simulation dialog, select File Upload (CSV / JSON) as dataset type. Click CSV or JSON under “Download template.”

lexq analytics dataset template \
--group-id <groupId> \
--version-id <versionId> \
--format csv \
--output template.csv

GET /api/v1/partners/analytics/groups/{groupId}/versions/{versionId}/dataset-template?format=csv

Upload & Run Simulation

Console
CLI
MCP

Go to Impact Simulations → New Simulation
Select policy group and target version
Set Dataset Type to File Upload (CSV / JSON)
Drag & drop your file or click to select
Wait for the Uploaded badge
Click Start Simulation

# Step 1: Upload the file
lexq analytics dataset upload --file ./my-data.csv
# → path: datasets/my-tenant/a1b2c3d4e5f6.csv

# Step 2: Start simulation with the uploaded path
lexq analytics simulation start --json '{
    "policyVersionId": "<versionId>",
    "dataset": {
        "type": "UPLOADED",
        "source": "S3_BUCKET",
        "path": "datasets/my-tenant/a1b2c3d4e5f6.csv"
    },
    "options": {
        "includeRuleStats": true,
        "maxRecords": 10000
    }
}'

lexq_dataset_template → get expected format
lexq_dataset_upload → upload inline CSV/JSON content
lexq_simulation_start → use returned path with type UPLOADED

Managing Simulations

lexq analytics simulation list
lexq analytics simulation list --status COMPLETED
lexq analytics simulation cancel --id <simulationId>
lexq analytics simulation export --id <simulationId> --format json
lexq analytics simulation export --id <simulationId> --format csv

The CSV carries the distribution in its own # Metric Distribution section, with the columns ChangedRecords,IncreasedRecords,DecreasedRecords,LargestIncrease,LargestDecrease,MeasuredRecords. The section is omitted entirely when there was no baseline, and the two Largest cells are left empty when the aggregation is COUNT.

Best Practices

Always set a baseline. Without a baseline version, you only see absolute numbers — no deltas.
Use a meaningful date range. At least 7 days of data for statistically significant results.
Check maxRecords. Start with 5,000–10,000 for quick validation, and read Record Limits above before you read a capped result as a random sample.
Run simulation before every production deployment.

Impact Simulation re-executes rules against historical input facts, and nothing leaves the engine: the 3 action primitives read and write facts, and nothing else. Decision Replay carries the same guarantee.

Getting Started

Core Concepts

Features

What is Impact Simulation?

What You Get

Reading the Change Distribution

Running a Simulation

Console

CLI

Dataset Types

Record Limits

File Upload Dataset

Supported Formats

Download a Template

Upload & Run Simulation

Managing Simulations

Best Practices

Next Steps

Dry Run

A/B Testing

​What is Impact Simulation?

​What You Get

​Reading the Change Distribution

​Running a Simulation

​Console

​CLI

​Dataset Types

​Record Limits

​File Upload Dataset

​Supported Formats

​Download a Template

​Upload & Run Simulation

​Managing Simulations

​Best Practices

​Next Steps

Dry Run

A/B Testing

What is Impact Simulation?

What You Get

Reading the Change Distribution

Running a Simulation

Console

CLI

Dataset Types

Record Limits

File Upload Dataset

Supported Formats

Download a Template

Upload & Run Simulation

Managing Simulations

Best Practices

Next Steps