Step Functions Express At Least Once FAQ & Answers

Express Workflows may execute more than once for same input event - no exactly-once guarantee (contrast with Standard Workflows' exactly-once). Why: (1) Optimistic concurrency - prioritizes low latency (5-10ms) + high throughput (100K+ starts/sec) over strict guarantees, doesn't persist state before starting, (2) Retry behavior - transient failures trigger automatic retry, may result in duplicate execution if first attempt partially succeeded, (3) No execution history - can't verify if execution already completed before retry.

All tasks must be safe to execute multiple times. Idempotent operations: DynamoDB PutItem (writing same item twice yields same result), SQS SendMessage with deduplication ID, Lambda with unique request ID (function checks if already processed). Avoid non-idempotent: incrementing counters (UPDATE SET count = count + 1 executes twice = wrong count), charging credit cards (duplicate charges), sending notification emails (duplicate sends). Standard Workflows guarantee exactly-once (no idempotency required, but 25x more expensive).

Idempotency patterns (2025): (1) Idempotency tokens - include unique ID: {RequestId: Context.Execution.Id} passed to Lambda, function checks DynamoDB for existing result with that ID, returns cached result if already processed, (2) Conditional writes - DynamoDB ConditionExpression: attribute_not_exists(executionId) fails if item exists, (3) Deduplication - SQS FIFO MessageDeduplicationId: Context.Execution.Id ensures message sent only once even if Express retries, (4) Event sourcing - append-only logs (multiple appends of same event OK, downstream deduplicates by event ID).

Use Express when: (1) High-throughput event processing - millions of events/day (IoT, clickstream), idempotent transformations (filter, enrich, route), (2) API orchestration - fast API Gateway integrations (<200ms), idempotent service calls (GET requests, PUT with same data), (3) Data pipelines - S3 → Lambda → DynamoDB batch processing, idempotent PutItem. Standard required for: (1) Financial transactions (money transfers, payments - duplicate = double charge), (2) Human approvals (duplicate execution confusing), (3) Long-running workflows (>5 minutes - Express times out).

CloudWatch metrics don't distinguish first vs retry execution. Monitoring approach: instrument Lambda functions to log execution IDs, deduplicate in analysis, track duplicate rate (aim <1%). Production best practice: always design Express workflow tasks as idempotent (assume retry), use execution ID in all state-changing operations, validate idempotency in load testing (simulate failures, verify no corruption).

Express Workflows may execute more than once for same input event - no exactly-once guarantee (contrast with Standard Workflows' exactly-once). Why: (1) Optimistic concurrency - prioritizes low latency (5-10ms) + high throughput (100K+ starts/sec) over strict guarantees, doesn't persist state before starting, (2) Retry behavior - transient failures trigger automatic retry, may result in duplicate execution if first attempt partially succeeded, (3) No execution history - can't verify if execution already completed before retry.

All tasks must be safe to execute multiple times. Idempotent operations: DynamoDB PutItem (writing same item twice yields same result), SQS SendMessage with deduplication ID, Lambda with unique request ID (function checks if already processed). Avoid non-idempotent: incrementing counters (UPDATE SET count = count + 1 executes twice = wrong count), charging credit cards (duplicate charges), sending notification emails (duplicate sends). Standard Workflows guarantee exactly-once (no idempotency required, but 25x more expensive).

Idempotency patterns (2025): (1) Idempotency tokens - include unique ID: {RequestId: Context.Execution.Id} passed to Lambda, function checks DynamoDB for existing result with that ID, returns cached result if already processed, (2) Conditional writes - DynamoDB ConditionExpression: attribute_not_exists(executionId) fails if item exists, (3) Deduplication - SQS FIFO MessageDeduplicationId: Context.Execution.Id ensures message sent only once even if Express retries, (4) Event sourcing - append-only logs (multiple appends of same event OK, downstream deduplicates by event ID).

Use Express when: (1) High-throughput event processing - millions of events/day (IoT, clickstream), idempotent transformations (filter, enrich, route), (2) API orchestration - fast API Gateway integrations (<200ms), idempotent service calls (GET requests, PUT with same data), (3) Data pipelines - S3 → Lambda → DynamoDB batch processing, idempotent PutItem. Standard required for: (1) Financial transactions (money transfers, payments - duplicate = double charge), (2) Human approvals (duplicate execution confusing), (3) Long-running workflows (>5 minutes - Express times out).

CloudWatch metrics don't distinguish first vs retry execution. Monitoring approach: instrument Lambda functions to log execution IDs, deduplicate in analysis, track duplicate rate (aim <1%). Production best practice: always design Express workflow tasks as idempotent (assume retry), use execution ID in all state-changing operations, validate idempotency in load testing (simulate failures, verify no corruption).