[{"data":1,"prerenderedAt":100},["ShallowReactive",2],{"pattern-eventbridge-firehose-opensearch-typescript-cdk":3,"pattern-group-eventbridge-firehose-opensearch-typescript-cdk":6},{"id":4,"title":5,"architectureURL":6,"cleanup":7,"contributors":12,"deploy":14,"description":18,"extension":19,"framework":20,"gitHub":21,"highlight":6,"introBox":25,"language":35,"level":36,"meta":37,"patternArch":38,"patternGroup":6,"patternGroupLabel":6,"resources":74,"stem":91,"testing":92,"videoId":6,"__hash__":99},"patterns\u002Fpatterns\u002Feventbridge-firehose-opensearch-typescript-cdk.json","Amazon EventBridge to Amazon OpenSearch Service via Amazon Data Firehose",null,{"headline":8,"text":9},"Cleanup",[10,11],"\u003Ccode>cd cdk\u003C\u002Fcode>","\u003Ccode>cdk destroy\u003C\u002Fcode>",[13],"content\u002Fcontributors\u002Fantoine-boucherie.json",{"text":15},[10,16,17],"\u003Ccode>npm install\u003C\u002Fcode>","\u003Ccode>cdk deploy -c dashboardAccessIp=YOUR_IP\u002F32\u003C\u002Fcode>","Stream every event on an EventBridge bus to OpenSearch through Amazon Data Firehose for near real-time search, with payload flattening, daily index rotation and an S3 backup.","json","AWS CDK",{"template":22},{"patternPath":23,"templateFile":24},"eventbridge-firehose-opensearch\u002Ftypescript\u002Fcdk","cdk\u002Flib\u002Feventbridge-opensearch-stack.ts",{"headline":26,"text":27},"How it works",[28,29,30,31,32,33,34],"This pattern gives you full-text search over every event on an Amazon EventBridge bus, usually within about 60 seconds of the event being emitted. CloudWatch metrics tell you how many events flowed; this tells you what was in them.","A catch-all EventBridge rule matches every event on a custom bus and sends it to an Amazon Data Firehose delivery stream. The rule pattern matches on a source prefix of an empty string, because every EventBridge event carries a source and an entirely empty pattern is rejected.","Firehose buffers for 60 seconds or 1 MB, whichever comes first. That buffer is the floor Firehose allows and it sets the end-to-end latency of the pattern.","Before indexing, a Lambda transform flattens the EventBridge envelope: `detail-type` is renamed to `detail_type` so it needs no escaping in queries, and the fields inside `detail` are promoted to the top level so a dashboard can filter on `claimId` rather than `detail.claimId`. Envelope fields win on collision, so a business payload carrying its own `source` key is indexed as `detail_source` instead of masking the real event source.","Documents land in a daily-rotated index, `events-YYYY-MM-DD`, and every document is also written to an S3 bucket, which makes that bucket an audit trail rather than only a dead letter destination.","Firehose authenticates to OpenSearch with SigV4 using its delivery role. A managed domain authorizes each request against its own access policy, so the role is granted access on both sides: an identity policy on the role and a domain access policy naming that role as principal.","EventBridge delivery is at-least-once, so an event can occasionally be indexed twice; each document keeps the EventBridge event ID in its `id` field for deduplication at query time. The pattern adds no dead-letter queue on the rule target and no alarms. The README lists what to add for production.","TypeScript","200",{},{"icon1":39,"icon2":44,"icon3":48,"icon4":53,"icon5":57,"line1":61,"line2":65,"line3":68,"line4":71},{"x":40,"y":41,"service":42,"label":43},14,45,"eventbridge","Amazon EventBridge\nClassic bus",{"x":45,"y":41,"service":46,"label":47},37,"kinesis-firehose","Amazon Data\nFirehose",{"x":49,"y":50,"service":51,"label":52},63,20,"lambda-event-functions","AWS Lambda",{"x":49,"y":54,"service":55,"label":56},68,"s3","Amazon S3",{"x":58,"y":41,"service":59,"label":60},86,"opensearch","Amazon OpenSearch\nService",{"from":62,"to":63,"label":64},"icon1","icon2","All events",{"from":63,"to":66,"label":67},"icon3","Transform",{"from":63,"to":69,"label":70},"icon4","Backup",{"from":63,"to":72,"label":73},"icon5","Index",{"bullets":75},[76,79,82,85,88],{"text":77,"link":78},"Loading streaming data into Amazon OpenSearch Service with Amazon Data Firehose","https:\u002F\u002Fdocs.aws.amazon.com\u002Fopensearch-service\u002Flatest\u002Fdeveloperguide\u002Fintegrations-fh.html",{"text":80,"link":81},"Amazon Data Firehose data transformation with AWS Lambda","https:\u002F\u002Fdocs.aws.amazon.com\u002Ffirehose\u002Flatest\u002Fdev\u002Fdata-transformation.html",{"text":83,"link":84},"Amazon EventBridge event patterns","https:\u002F\u002Fdocs.aws.amazon.com\u002Feventbridge\u002Flatest\u002Fuserguide\u002Feb-event-patterns.html",{"text":86,"link":87},"Identity and Access Management in Amazon OpenSearch Service","https:\u002F\u002Fdocs.aws.amazon.com\u002Fopensearch-service\u002Flatest\u002Fdeveloperguide\u002Fac.html",{"text":89,"link":90},"Audit AWS service events with Amazon EventBridge and Amazon Data Firehose","https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Faudit-aws-service-events-with-amazon-eventbridge-and-amazon-kinesis-data-firehose\u002F","patterns\u002Feventbridge-firehose-opensearch-typescript-cdk",{"headline":93,"text":94},"Testing",[95,96,97,98],"See the GitHub repo README.md for detailed testing instructions.","Read the bus name from the stack outputs: \u003Ccode>BUS=$(aws cloudformation describe-stacks --stack-name EventMonitorStack --query \"Stacks[0].Outputs[?OutputKey=='EventBusName'].OutputValue\" --output text)\u003C\u002Fcode>","Emit a test event onto the bus: \u003Ccode>aws events put-events --entries '[{\"Source\":\"demo.test\",\"DetailType\":\"TestEvent\",\"Detail\":\"{\\\"message\\\":\\\"Hello OpenSearch\\\",\\\"claimId\\\":\\\"CLM-001\\\"}\",\"EventBusName\":\"'\"$BUS\"'\"}]'\u003C\u002Fcode>","Wait 60 to 90 seconds for the Firehose buffer to flush, then open the `DashboardsUrl` from the stack outputs and create an index pattern of `events-*` with `time` as the time field.","hcV65Rf1CmcE02yCMJ4lmtTJMUOGDrjy4v9_kc-guww",1795189150990]