Creates an API Gateway WebSocketand Lambda functions that provides a streaming response from the LLMs in Amazon Bedrock.
This sample project demonstrates how to use WebSocket API as a front door to Lambda functions that perform inference on Amazon Bedrock LLMs using the [InvokeModelWithResponseStream API, where the LLM responses are returned in a stream.
With WebSocket API, developers of interactive LLM chatbot interfaces can provide a better user experience by displaying the LLM responses as they are generated (which at times can be long, rather than relying on the synchronous InvokeModel API request.
This pattern deploys one API Gateway WebSocket API, four Lambda functions, and one DynamoDB table.
< Back to all patterns
Visit the GitHub repo for this pattern.
Download
git clone https://github.com/aws-samples/serverless-patterns/ cd serverless-patterns/apigw-websocket-api-bedrock-streamingDeploy
sam deployTesting
See the GitHub repo for detailed testing instructions.
Cleanup
Delete the stack:
sam delete --stack-name STACK_NAME.