Saykai
Safety Spec

Safety Spec in.
Safety Pack out.

The Safety Spec is how you write down what “safe enough” means. The Safety Pack is the proof of what happened. Saykai enforces the Spec in CI and emits a Pack on every run.

  • > Specs live in your repo next to code.
  • > Packs are generated in CI.
  • > Together they turn every pull request into a safety decision.
.YML
SPEC
CI GATE
.JSON
PACK
Jump to: Overview Examples CI Integration

Two artifacts, one gate.

Saykai is built around a simple idea: you write down what “safe enough” means in a Safety Spec, and Saykai turns that into a repeatable gate on every change.

Safety Spec

A versioned configuration file in your repo that defines:

  • Which scenarios and log replays must run
  • Which metrics matter and how to measure them
  • Thresholds and drift limits the system must respect

Safety Pack

A JSON artifact emitted on every CI run that captures:

  • Which Spec version was used
  • Which scenarios were executed
  • Previous vs current metrics and the final decision

The Safety Spec: what “safe enough” means.

A Safety Spec is the single source of truth for what you consider safe enough for one system. It is written as code, stored in git, and reviewed like any other change.

What goes into a Safety Spec

System Identity

Which system this Spec applies to (for example, warehouse_amr_prod).

Scenario Definitions

The simulations, agent runs, and log replays that must pass on every change.

Metric Definitions

How you measure risk and performance, and which metrics must not regress.

Gating Rules

Hard thresholds and allowed drift, including any metrics that must never regress at all.

/repo/safety/saykai.yml

Example Safety Spec

This is a simplified example for a warehouse AMR fleet.

saykai.yml
spec: system_id: "warehouse_amr_prod" spec_version: "warehouse_spec_v1" scenarios: sim: - id: "tight_aisle_crossing" command: "python sims/tight_aisle.py" replay: - id: "2025-11-21-night-shift" command: "python replays/night.py" metrics: - name: "collisions_per_10k" threshold: { max: 0 } - name: "human_override_pct" threshold: { max: 0.25 } gate: require_no_regressions_on: - "collisions_per_10k" allow_soft_regressions_if: max_relative_regression_pct: 10

> LOGIC EXPLAINED

  • Collisions must always stay at 0 per 10k missions.
  • Any regression in collisions blocks the build.
  • Soft regressions in other metrics are allowed within 10 percent.

Safety Pack Result

What actually happened on this change.

safety_pack.json
{ "system": "warehouse_amr_prod", "change_id": "pr-4829", "decision": "approve", "metrics": { "collisions_per_10k": { "prev": 0, "current": 0 }, "human_override_pct": { "prev": 0.11, "current": 0.18 } }, "scenarios_executed": [ "tight_aisle_crossing", "2025-11-21-night-shift" ], "timestamp": "2025-12-03T10:24:00Z" }
decision approve or block, based on the Spec.
metrics previous vs current values with diffs.
scenarios_executed the exact scenarios and replays that ran.

How Specs and Packs fit into CI.

Once you have a Safety Spec and Saykai wired into CI, every pull request gets the same treatment.

01 // PROPOSE

Engineer opens a change.

The pull request includes code, plus any Spec changes if the safety bar needs to move.

02 // EXECUTE

CI runs Saykai.

Saykai runs scenarios, replays logs, collects metrics, and compares them to the Spec and baseline.

03 // DECIDE

Gate returns pass or block.

If it passes, the Safety Pack is attached to the run. If it fails, the build is blocked with a clear reason based on Spec rules.

Why Specs and Packs beat ad hoc checks.

Repeatable

The same scenarios, metrics, and thresholds run on every change, not just big releases.

Versioned

When incidents or requirements change, you update the Safety Spec through a pull request and the new bar applies to every future change.

Shareable

You can share a Safety Pack with risk or leadership without giving them raw system access.

"Here is the Safety Pack for Release v4.2"

Download JSON

Start defining your first Safety Spec.

We usually start with one system. We list the scenarios and metrics that matter most, and turn that into saykai.yml.

Talk about your Safety Spec