Event-driven architecture with Amazon SQS and AWS Lambda
SQS to Lambda, event-driven architecture is a queue and made easier now that Amazon SQS can trigger AWS Lambda functions. Learn More!
Jun 08, 2023 • 6 Minute Read
The centerpiece of event-driven architecture is a queue — made easier now that Amazon SQS can trigger AWS Lambda functions.
You probably know that I am a huge fan of event-driven patterns. Instead of waiting and polling for changes, actions happen when changes in the system occur.
We have talked about this architectural pattern for years — but until recently, it was more of a theoretical idea rather than a simple-to-implement pattern.
However, AWS Lambda brought event-driven compute to everyone and took the event-driven paradigm to a whole new level. Companies like Netflix (Bless) and Capital One (Cloud Custodian) are now using event-driven architectures to perform real-time security, compliance and policies management.
The centerpiece of event-driven architectures is often a queue. On AWS, that central building block is taken care of by Amazon Simple Queue Service (SQS). In order to read information from an SQS queue, your lambda function had to poll for it — until now!
While drinking my coffee, I noticed something interesting in the list of triggers for my test lambda function. I was like — what?? Do I dream?
WHAAAAAT - This showed up in my AWS Console --- Yes my friends! This is the day when SQS becomes a trigger for AWS Lambda. #aws #Serverless pic.twitter.com/b79Ml7yLNv
— Adrian Hornsby @ArcticCircle (@adhorn) June 28, 2018
Let’s try this out!
- First, create an SQS queue. I called my test queue
TestMySQSLambda
which is the most original name I could come up before two cups of morning coffee. - I also used the default
Standard Queue
— which does not guarantee the order of the messages. - Click
Quick-Create Queue
— yes, we want it quick!
- Then create an IAM Role for your lambda function to be able to get triggered by SQS. This is a test, so let’s go wild and add that bellow policy to your basic lambda execution role — and save it as
SQSLambda
role.
- Then log into the AWS Lambda console and let’s create a function from scratch. Again, a very original name is required —
MySQSTriggerTest
.
- Click
create
andthen edit the Lambda function — I used the following super advanced python code.
- Now select the
Execution role
just created before.
- Then select the created SQS trigger from the long list of triggers — it is the last in the list.
- Finally, select the
TestMySQSLambda
SQS queue previously created. - Click
Add
and thenSave
. - If everything was done correctly, you should see something like that in the AWS Console — with
Saved
in green.
Time to test!
- Select
Configure test event
from the menu as shown below.
- Then search for
SQS
and you should see the test template as below.
- Click
Create
and then clickTest
.
All green — we are good! Now let’s test the whole pipeline by putting data directly into the SQS queue.
- Go into the SQS console and select the queue created before and click
Queue Actions
andthen selectSend a Message
from the dropdown.
- Type your message of hope to the world here. This is your chance to be a good person today!
- Click
Send Message
.
Now go check the Cloudwatch logs console and search for aws/lambda/MySQSTriggerTest
— you should be able to see the SQS to Lambda trigger!
Niiiice! This was a long-awaited feature which now allows developers to build truly event-driven architecture without needing to poll the SQS queue for data! Can’t wait to see what you are going to build with it :-)
Get the skills you need for a better career.
Master modern tech skills, get certified, and level up your career. Whether you’re starting out or a seasoned pro, you can learn by doing and advance your career in cloud with ACG.