Skip to content

LocalStack

2026.5.0 9 recorded runs

88.0%
correct, across 676 of 684 operations · 8 unsupported
Tier 1 98.6%
Tier 2 92.2%
Tier 3 68.7%

Fully supports

Operation areas this target implements completely — every test passes, nothing skipped. Areas it only partly implements, or gets wrong, are in “Where it falls short” below.

  • batchGetItem tier 1
  • batchWriteItem tier 1
  • createTable tier 1
  • deleteItem tier 1
  • deleteTable tier 1
  • describeTable tier 1
  • getItem tier 1
  • listTables tier 1
  • scan tier 1
  • updateTable tier 1
  • kinesis tier 2
  • streams tier 2
  • updateTable tier 2
  • limits tier 3

Conformance over time

80% 90% 100% 93.0% 23 Mar 93.0% 24 Mar 93.5% 24 Apr 92.3% 25 Apr 89.0% 27 Apr 89.0% 23 May 88.3% 24 May 87.9% 25 May 88.0% 26 May

Where it falls short

Operation groups with gaps in the latest run, biggest first. Unsupported means the target doesn't implement that feature at all, often by design. Open one for the exact tests, or see the conformance suite.

  • error-messages Tier 3 41 failing
    View these tests in the suite →
    • BatchGetItem — exact error messages empty RequestItems: full required-parameter error
    • BatchGetItem — exact error messages > 100 keys across all tables: interpolated full error
    • BatchGetItem — exact error messages non-existent table: full ResourceNotFoundException message
    • BatchWriteItem — exact error messages empty RequestItems: full required-parameter error
    • BatchWriteItem — exact error messages > 25 requests: anchored regex on the constraint phrase
    • BatchWriteItem — exact error messages non-existent table: full ResourceNotFoundException message
    • CreateTable — exact error messages missing TableName: full required-parameter error
    • CreateTable — exact error messages short table name (2 chars): minimum length 3 error
    • CreateTable — exact error messages duplicate attribute in KeySchema
    • CreateTable — exact error messages more than 2 KeySchema elements
    • CreateTable — exact error messages invalid KeyType
    • CreateTable — exact error messages invalid AttributeType
    • CreateTable — exact error messages LSI without range key on base table
    • CreateTable — exact error messages duplicate index names
    • DeleteItem — exact error messages non-existent table: full ResourceNotFoundException message
    • DeleteItem — exact error messages malformed Key (missing range key on composite table): full schema-mismatch error
    • GetItem — exact error messages non-existent table: full ResourceNotFoundException message
    • GetItem — exact error messages malformed Key (missing range key on composite table): full schema-mismatch error
    • PutItem — exact error messages missing table name: full validation error string
    • PutItem — exact error messages empty table name: minimum length 1 error
    • PutItem — exact error messages table name too long (256 chars): maximum length 255 error
    • PutItem — exact error messages table name with invalid chars: regex pattern error
    • PutItem — exact error messages duplicate values in SS: full duplicates error
    • Query — exact error messages missing hash key in KeyConditionExpression
    • Query — exact error messages non-key attribute in KeyConditionExpression
    • Query — exact error messages invalid Select value
    • Query — exact error messages Limit of 0
    • Query — exact error messages malformed ExclusiveStartKey: short invalid-starting-key error
    • Scan — exact error messages Segment without TotalSegments: full required-parameter error
    • Scan — exact error messages Segment >= TotalSegments: full out-of-range error
    • Scan — exact error messages TotalSegments without Segment: full required-parameter error
    • Scan — exact error messages Limit of 0: full minimum-value error
    • Scan — exact error messages non-existent table: full ResourceNotFoundException message
    • Scan — exact error messages malformed ExclusiveStartKey: long schema-mismatch error
    • TransactGetItems — exact error messages empty TransactItems: full minimum-length error
    • TransactGetItems — exact error messages > 100 gets: anchored regex on the constraint phrase
    • TransactGetItems — exact error messages non-existent table: full ResourceNotFoundException message
    • TransactGetItems — exact error messages missing key attribute: action-level ValidationError surfaces as TransactionCanceledException
    • TransactWriteItems — exact error messages empty TransactItems: full minimum-length error
    • TransactWriteItems — exact error messages > 100 actions: anchored regex on the constraint phrase
    • TransactWriteItems — exact error messages non-existent table: full ResourceNotFoundException message
  • validation-ordering Tier 3 25 failing
    View these tests in the suite →
    • Batch operations — validation ordering BatchWriteItem rejects empty RequestItems
    • Batch operations — validation ordering BatchGetItem rejects empty RequestItems
    • CreateTable — validation ordering empty TableName reports only tableName constraint
    • CreateTable — validation ordering invalid table name pattern reports only tableName
    • CreateTable — validation ordering reports invalid BillingMode and invalid KeySchema element together
    • CreateTable — validation ordering rejects a duplicate attribute in KeySchema
    • CreateTable — validation ordering rejects more than two KeySchema elements
    • CreateTable — validation ordering rejects an invalid BillingMode on its own
    • DeleteItem — validation ordering empty TableName reports only tableName constraint
    • DeleteItem — validation ordering reports invalid ReturnValues and invalid ReturnConsumedCapacity together
    • DeleteItem — validation ordering invalid table name pattern reports only tableName
    • Non-existent index — error surface Query on a non-existent index reports ValidationException
    • PutItem — validation ordering empty TableName reports tableName constraint (stops early)
    • PutItem — validation ordering empty TableName with invalid ReturnValues reports only tableName
    • PutItem — validation ordering reports invalid ReturnConsumedCapacity, ReturnItemCollectionMetrics, and ReturnValues together
    • PutItem — validation ordering reports invalid table name pattern
    • Query — validation ordering reports invalid TableName pattern
    • Query — validation ordering reports invalid ReturnConsumedCapacity
    • Scan — validation ordering rejects Segment without TotalSegments
    • Scan — validation ordering rejects Segment >= TotalSegments
    • Scan — validation ordering rejects TotalSegments without Segment
    • Scan — validation ordering rejects a negative Segment
    • UpdateItem — validation ordering empty TableName reports only tableName constraint
    • UpdateItem — validation ordering reports invalid ReturnValues and invalid ReturnConsumedCapacity together
    • UpdateItem — validation ordering invalid table name pattern reports only tableName
  • putItem Tier 1 3 failing
    View these tests in the suite →
    • ReturnItemCollectionMetrics PutItem with SIZE returns ItemCollectionMetrics
    • ReturnItemCollectionMetrics DeleteItem with SIZE returns ItemCollectionMetrics
    • ReturnItemCollectionMetrics UpdateItem with SIZE returns ItemCollectionMetrics
  • backups Tier 2 3 skipped
    View these tests in the suite →
    • On-demand backups — lifecycle and restore CreateBackup → DescribeBackup → ListBackups → DeleteBackup
    • On-demand backups — lifecycle and restore RestoreTableFromBackup initiates a restore into a new table
    • On-demand backups — lifecycle and restore DescribeBackup on a deleted backup throws BackupNotFoundException
  • transactions Tier 2 3 failing
    View these tests in the suite →
    • TransactGetItems — ConsumedCapacity INDEXES breakdown includes the table read capacity units
    • TransactGetItems — projection matching nothing omits Item when the projection matches no attribute on a present item
    • TransactWriteItems — ConsumedCapacity charges 2 write capacity units per item
  • export Tier 2 unsupported
    View these tests in the suite →
    • Export and import — S3 ExportTableToPointInTime initiates an export and reports it
    • Export and import — S3 ImportTable ingests S3 data into a new table
  • partiql Tier 2 2 failing
    View these tests in the suite →
    • ExecuteStatement — PartiQL rejects INSERT on an existing item (INSERT is not upsert)
    • ExecuteStatement — PartiQL returns a populated ConsumedCapacity block when requested
  • resourcePolicy Tier 2 unsupported
    View these tests in the suite →
    • Resource policies — Put/Get/Delete GetResourcePolicy on a table with no policy throws PolicyNotFoundException
    • Resource policies — Put/Get/Delete Put then Get round-trips the policy, and Delete removes it
  • tags Tier 2 2 failing
    View these tests in the suite →
    • Tags — validation rejects TagResource with an invalid ARN format
    • Tags — validation rejects ListTagsOfResource with a non-existent ARN
  • query Tier 1 1 failing
    View these tests in the suite →
    • Query — Limit + FilterExpression interaction rejects ExclusiveStartKey that does not match the table schema
  • updateItem Tier 1 1 failing
    View these tests in the suite →
    • UpdateItem — ReturnValues granularity UPDATED_NEW on a nested SET returns only the changed fragment
  • account Tier 2 1 skipped
    View these tests in the suite →
    • Account reads — DescribeLimits, DescribeEndpoints DescribeEndpoints returns at least one endpoint with an address
  • contributorInsights Tier 2 1 failing
    View these tests in the suite →
    • Contributor insights — enable/describe/list enabling transitions the status and lists the table
  • ttl Tier 2 1 failing
    View these tests in the suite →
    • TTL — validation rejects empty attribute name
  • legacy-api Tier 3 1 failing
    View these tests in the suite →
    • Legacy API — AttributeUpdates (legacy UpdateExpression) AttributeUpdates with a Value and no Action defaults to PUT

Run history

Run Total Movement

Suite on

Grew to 684 tests with a control-plane and table-configuration sweep: the CreateTable/UpdateTable config parameters and the secondary control-plane operations (limits, backups and PITR, exports and imports, Kinesis, resource policies, contributor insights), each characterised against real AWS and probe-skipped where a target doesn't implement it.

The published percentage changed with it. It now measures correctness over implemented operations, Pass / (Pass + Fail), so skips no longer count against the score. A skip is honest scope; a fail is a bug.

88.0% +0.1pp rose 0.1 percentage points
87.9% -0.4pp fell 0.4 percentage points

Suite on

Grew to 625 tests, up 24 on the previous run: eleven more in Tier 1 and thirteen more in Tier 3, tightening coverage of core operations and the strict edge cases.

88.3% -0.7pp fell 0.7 percentage points

Suite on

ExtendDB joined the run as a target. The suite itself held steady at 601 tests.

89.0% 0.0pp unchanged

Suite on

Grew to 601 tests, up 29, and every new test landed in Tier 3: a broader strict surface spanning validation ordering, exact error messages, service limits, and the legacy request shapes. Floci and Ministack were added to the run as targets the same day.

89.0% -3.3pp fell 3.3 percentage points
92.3% -1.2pp fell 1.2 percentage points

Suite on

Grew to 572 tests, up 46 on the first run: thirty-six more in Tier 1 and ten more in Tier 2, deepening coverage of the core and complete-feature behaviour.

93.5% +0.6pp rose 0.6 percentage points
93.0% 0.0pp unchanged

Suite on

The suite was established with 526 tests across the three tiers (267 Tier 1, 93 Tier 2, 166 Tier 3), run against Dynalite, DynamoDB Local, Dynoxide, and LocalStack, with live AWS DynamoDB as the baseline.

93.0% first run