Overview
Portfolio Evaluation (PE) Backtesting lets you safely test changes to your risk & compliance workflows using your own historical data—so you can see the impact before updating production.
What you can do with PE Backtesting
Validate policy changes: Measure how edits to a workflow (rules, thresholds) would have changed outcomes.
Create an audit trail: Demonstrate pre‑deployment testing for model governance and regulatory reviews.
Keep production clean: All test outputs are marked informational and are automatically excluded from live queues, analytics, and aggregations.
Definitions
Control = the original workflow version used for the historical run.
Test = the new workflow version you want to try.
Results compare Test vs Control.
How to Run a Backtest
Before you begin
Access: You’ll need permission to view Portfolio Evaluations and edit workflows in Alloy. If you don’t see Portfolio Evaluations or can’t save workflow versions, ask your administrator to update your role.
Historical data availability: Backtesting runs on historical entity & event data. If your organization hasn’t loaded historical data, you’ll be prompted to backfill via the Events API before running a test.
Getting started
Logging in & navigation: Log in to your Alloy dashboard. Go to Workflows (left navigation) and select the portfolio workflow you want to test.
Run a backtest (step‑by‑step)
Step 1 — Open the test flow
You can kick off a test from 2 locations:
The workflow versions list page: Click on a workflow → view versions → select “Run Test” on the top right
The workflow editor: Click on the specific workflow version you want to test → click “Test” on the top right. If you have unsaved edits, you’ll be prompted to Save as new version.
Step 2 — Select test type
Select backtest to kick off a backtest, and click “Next” on the top right.
Step 3 — Choose the population (historical dataset)
You have two options:
Compare to a Previous Portfolio Evaluation (recommended): Select an existing historical Portfolio Evaluation run. Backtesting will evaluate the same entities from that run, enabling an apples‑to‑apples comparison.
Set a Custom Date & Time: Pick an exact As‑Of timestamp to evaluate all entities as they existed at that moment (based on your historical data).
Heads up: If no historical data exists, you’ll see a prompt to backfill data before continuing.
Step 4 — Confirm configuration
Review the summary panel:
Control version: the base historical run
Test version: your new saved version
Version changes: what changed between versions
Data As‑Of: the fixed historical moment we reference — we look at how your portfolio and entities looked “as of” this time
Click “Run” test to start.
Step 5 — Review results
When complete, the results page includes:
Header
Control / Test versions and Version changes (what changed between versions)
Entities evaluated (total count)
Link to the original Portfolio Evaluation run (Control) for side‑by‑side comparisons
Summary charts (top)
Side‑by‑side distribution of outcomes (e.g., Suspicious / Dismissed — or your workflow’s outcomes). Use this to gauge the overall impact of the changes made to the Test version.
Entity list (bottom)
A detailed table of Test results at the entity level. You can search by entity name or token to pull up specific entities.
Click “view” on an entity’s test evaluation to open its informational test evaluation. You’ll see the same details you see in a normal production evaluation (entity panel, workflow details, outcome, tags).
Inside the test evaluation, click a tag to open Rule Explainability and understand why that tag fired in the Test version (rule logic, evaluated values, and inputs).
Deep‑dive: Use the header link to the original Portfolio Evaluation run to compare Control vs. Test for a given entity — outcomes and tags side‑by‑side.