DynamoDB Atomic Counters
Jun 08, 2023 • 0 Minute Read
DynamoDB has a lot of different features. One useful feature that is frequently overlooked is the ability to use atomic counters. Let’s see why we might want to use atomic counters and how we can use Python3 and the AWS SDK for Python (boto3) to implement atomic writes.Atomic counters can be useful in many situations where you want to consistently increment or reduce a value. For example, you might want to count website visits or count down from a certain number to determine a contest winner. But don’t take my word for it, let’s think about a potential scenario.Imagine we have a DynamoDB table called `siteVisits` that has a simple primary key and a single partition key attribute of `siteUrl`. Let’s also say the items in this table have a `visits` attribute. Finally, let’s say we add new items to the table whenever we decide to track another website’s visits. So an item that is just added to the table looks like this:
Now, say we put some JavaScript on all website pages in our database, and this JavaScript calls an API we created to count visits. How might we code the backend of the API to consistently record new visits? We could use a DynamoDB read operation to see what the previous `visits` were and then use some code to add an additional visit before using a DynamoDB write operation to write the new visits count. But, what if in between those two operations we got another visit? We might end up underreporting visit counts in our system because the new visits count hasn’t been written yet.Additionally, what if the write operation stalls or fails and then is re-tried? We might risk overwriting a newer visit count with an older one from the previous read of the table.So what is a better way to solve this problem?Well, with DynamoDB atomic writes you can avoid many of these issues entirely. Rather than reading and writing with multiple operations, you can use a single write operation to increment the `visits` value. Importantly, this can only be done if the `visits` attribute has a type of number.The code below is an example of what we could run each time a visit is logged to a website like https://www.flinuxacademy.com/. If this code fails and tries to run again at a later time, it will just do exactly what it was supposed to - increment the visits count by one for that visit.{ "siteUrl": "https://www.linuxacademy.com/", "visits": "0"}
Whenever the above code runs the `visits` attribute, it will be incremented by one from its current value. Because write operations are processed in order, there will never be a risk that these operations override each other or conflict.Additionally, if one or more `update_item` operations fail or are interrupted, this will not negatively impact the accuracy of the table by more than that single failed incrementation. This means you can retry the failed operations and continue to allow other increment operations to continue while you do so.What are you waiting for? Try this out on your own!import boto3dynamodb = boto3.client('dynamodb')response = dynamodb.update_item( TableName='siteVisits', Key={ 'siteUrl':{'S': "https://www.linuxacademy.com/"} }, UpdateExpression='SET visits = visits + :inc', ExpressionAttributeValues={ ':inc': {'N': '1'} }, ReturnValues="UPDATED_NEW")print("UPDATING ITEM")print(response)
