Portfolio Evaluation (PE) Backtesting

Overview 

Portfolio Evaluation (PE) Backtesting lets you safely test changes to your risk & compliance workflows using your own historical data—so you can see the impact before updating production.

What you can do with PE Backtesting

• Validate policy changes: Measure how edits to a workflow (rules, thresholds) would have changed outcomes.
• Create an audit trail: Demonstrate pre‑deployment testing for model governance and regulatory reviews.
• Keep production clean: All test outputs are marked informational and are automatically excluded from live queues, analytics, and aggregations.

Definitions

• Control = the original workflow version used for the historical run.
• Test = the new workflow version you want to try.

Results compare Test vs Control.

How to Run a Backtest

Before you begin

• Access: You’ll need permission to view Portfolio Evaluations and edit workflows in Alloy. If you don’t see Portfolio Evaluations or can’t save workflow versions, ask your administrator to update your role.
• Historical data availability: Backtesting runs on historical entity & event data. If your organization hasn’t loaded historical data, you’ll be prompted to backfill via the Events API before running a test.

Getting started

• Logging in & navigation: Log in to your Alloy dashboard. Go to Workflows (left navigation) and select the portfolio workflow you want to test.

Run a backtest (step‑by‑step)

Step 1 — Open the test flow 

You can kick off a test from 2 locations:

• The workflow versions list page: Click on a workflow → view versions → select “Run Test” on the top right
• The workflow editor: Click on the specific workflow version you want to test → click “Test” on the top right. If you have unsaved edits, you’ll be prompted to Save as new version.

Step 2 — Select test type

• Select backtest to kick off a backtest, and click “Next” on the top right.

Step 3 — Choose the population (historical dataset) 

You have two options:

• Compare to a Previous Portfolio Evaluation (recommended): Select an existing historical Portfolio Evaluation run. Backtesting will evaluate the same entities from that run, enabling an apples‑to‑apples comparison.
• Set a Custom Date & Time: Pick an exact As‑Of timestamp to evaluate all entities as they existed at that moment (based on your historical data).

Heads up: If no historical data exists, you’ll see a prompt to backfill data before continuing.

Step 4 — Confirm configuration 

• Review the summary panel:
  • Control version: the base historical run
  • Test version: your new saved version
  • Version changes: what changed between versions
  • Data As‑Of: the fixed historical moment we reference — we look at how your portfolio and entities looked “as of” this time
• Click “Run” test to start.

Step 5 — Review results 

When complete, the results page includes:

Header

• Control / Test versions and Version changes (what changed between versions)
• Entities evaluated (total count)
• Link to the original Portfolio Evaluation run (Control) for side‑by‑side comparisons

Summary charts (top)

Side‑by‑side distribution of outcomes (e.g., Suspicious / Dismissed — or your workflow’s outcomes). Use this to gauge the overall impact of the changes made to the Test version.

Entity list (bottom)

A detailed table of Test results at the entity level. You can search by entity name or token to pull up specific entities.

• Click “view” on an entity’s test evaluation to open its informational test evaluation. You’ll see the same details you see in a normal production evaluation (entity panel, workflow details, outcome, tags).
• Inside the test evaluation, click a tag to open Rule Explainability and understand why that tag fired in the Test version (rule logic, evaluated values, and inputs).
• Deep‑dive: Use the header link to the original Portfolio Evaluation run to compare Control vs. Test for a given entity — outcomes and tags side‑by‑side.