Skip to content

Run

7 emulators 684 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so a downward arrow can be the new tests biting rather than a target regressing.

Suite grew from 655 to 684 tests this run.

That's 29 new tests measured against every target. Movement below compares to the previous run, so a fall here is as likely to be the stricter suite as a real regression.

What changed in the suite this run

Suite on

Grew to 684 tests with a control-plane and table-configuration sweep: the CreateTable/UpdateTable config parameters and the secondary control-plane operations (limits, backups and PITR, exports and imports, Kinesis, resource policies, contributor insights), each characterised against real AWS and probe-skipped where a target doesn't implement it.

The published percentage changed with it. It now measures correctness over implemented operations, Pass / (Pass + Fail), so skips no longer count against the score. A skip is honest scope; a fail is a bug.

  1. live (AWS) · full coverage
    100% ground truth
    Tier 1 100%
    Tier 2 100%
    Tier 3 100%
  2. 0.9.13 · 14 unsupported
    95.5% -0.7pp fell 0.7 percentage points
    Tier 1 94.8%
    Tier 2 94.5%
    Tier 3 97.2%
  3. v0.1.0 · 34 unsupported
    93.7% -1.1pp fell 1.1 percentage points
    Tier 1 93.1%
    Tier 2 93.3%
    Tier 3 94.9%
  4. e981a5afd790 · 9 unsupported
    93.5% -1.3pp fell 1.3 percentage points
    Tier 1 93.9%
    Tier 2 91.3%
    Tier 3 93.9%
  5. 2026.5.0 · 8 unsupported
    88.0% +0.1pp rose 0.1 percentage points
    Tier 1 98.6%
    Tier 2 92.2%
    Tier 3 68.7%
  6. d89f8fcc6b1a · 13 unsupported
    87.0% -0.1pp fell 0.1 percentage points
    Tier 1 97.7%
    Tier 2 88.3%
    Tier 3 69.2%
  7. 4.0.0 · 59 unsupported
    82.9% -1.2pp fell 1.2 percentage points
    Tier 1 95.4%
    Tier 2 16.9%
    Tier 3 82.7%
  8. 9641f520cebd · 10 unsupported
    65.1% -0.4pp fell 0.4 percentage points
    Tier 1 80.9%
    Tier 2 73.7%
    Tier 3 35.0%