Orchestrating GenAI on AWS: Step Functions, Lambda Streaming, Retries and Runtime Configuration

Domain 2 scenarios glue services together: a workflow that waits for a person, a response that must stream, an API that must retry, a model that must change without a deploy. These notes list the documented limits and behaviors behind each pattern.

Key points (15)

Streaming · Streaming APIs

Use ConverseStream, InvokeModelWithResponseStream or RetrieveAndGenerateStream to send tokens to the user as they are generated instead of waiting for the full response. Streaming lowers the perceived latency of long answers.

Official source: docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

AWS Lambda · Lambda response streaming

Lambda response streaming improves time to first byte and allows payloads up to 200 MB, versus 6 MB for buffered responses. It works through function URLs, the InvokeWithResponseStream API and the API Gateway proxy integration.

Official source: docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html

AWS Lambda · Response streaming details

The first 6 MB of a streamed response is uncapped; after that the stream is limited to 2 MBps. Streaming is native on Node.js runtimes (other languages need a custom runtime or the Lambda Web Adapter), and function URLs do not support streaming inside a VPC.

Official source: docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html

AWS Step Functions · 256 KiB payload limit

The input or output of a Step Functions task, state or execution is limited to 256 KiB. For larger data, such as long FM outputs, store it in S3 and pass only the reference between states.

Official source: docs.aws.amazon.com/step-functions/latest/dg/service-quotas.html

AWS Step Functions · Standard vs Express

Standard workflows can run up to 1 year and keep up to 25,000 history events; Express workflows run at most 5 minutes. Use Standard for long, auditable orchestrations and Express for short, high-volume ones.

Official source: docs.aws.amazon.com/step-functions/latest/dg/service-quotas.html

AWS Step Functions · Human approval with a callback token

For human review, use a Standard workflow with .waitForTaskToken: the task pauses until the token returns through SendTaskSuccess or SendTaskFailure. Express workflows only support Request Response integrations, so they cannot wait for a callback.

Official source: docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html

AWS Step Functions · Heartbeat on waiting tasks

A task waiting for a task token can wait until the one-year execution limit. Set HeartbeatSeconds so the task fails with States.Timeout if no SendTaskHeartbeat, SendTaskSuccess or SendTaskFailure arrives, and send the token from principals in the same account.

Official source: docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html

AWS SDKs · Standard retry mode

AWS SDK standard retry mode (the default) retries transient and throttling errors with exponential backoff and full jitter, three attempts by default, and a retry quota that makes it fail fast when failures are widespread. It never retries non-retryable errors such as ValidationException.

Official source: docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html

AWS SDKs · Adaptive retry mode

Adaptive mode adds a client-side rate limiter that slows requests when throttling is detected, and it can delay even the initial request. AWS recommends it only for clients that hit a single resource with frequent throttling, not as a general default.

Official source: docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html

Agents and tools · Safeguard agent workflows

Task 2.1 expects safeguarded agent workflows: Step Functions stopping conditions, Lambda timeouts, IAM policies that enforce resource boundaries and circuit breakers. Human review steps can be orchestrated with Step Functions and feedback collected through API Gateway.

Official source: docs.aws.amazon.com/aws-certification/latest/ai-professional-01/ai-professional-01-domain2.html

Agents and tools · MCP servers on AWS

The exam guide places lightweight, stateless MCP servers on AWS Lambda and MCP servers that provide complex tools on Amazon ECS, with MCP client libraries to give consistent access patterns. Strands Agents and AWS Agent Squad are named for multi-agent systems.

Official source: docs.aws.amazon.com/aws-certification/latest/ai-professional-01/ai-professional-01-domain2.html

Converse API · One API for every chat model

The Converse API gives one consistent request format for all Bedrock models that support messages, so you write the code once and switch models. Model-specific parameters go in additionalModelRequestFields, and the previous messages must be sent again on every turn.

Official source: docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

Prompt Management · Reusable prompts, variants and versions

Prompt management stores reusable prompts with variables, lets you compare variants of a prompt and saves versions for deployment. You integrate a prompt by specifying it at inference time or by adding a prompt node to a flow.

Official source: docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html

AWS AppConfig · Change behavior without redeploying

AWS AppConfig changes application behavior in production without redeploying code, using feature flags or free-form configuration. Validators check the data, deployment strategies roll it out gradually and a CloudWatch alarm can trigger an automatic rollback.

Official source: docs.aws.amazon.com/appconfig/latest/userguide/what-is-appconfig.html

AWS AppConfig · AppConfig Agent

Applications read AppConfig data through a local endpoint exposed by the AWS AppConfig Agent, which caches the deployed configuration. That is how a Lambda function can switch models or providers at runtime without a new deployment or a call per request.