- Lab
- A Cloud Guru
Archive Cosmos DB Data Using the Change Feed and an Azure Function
The Cosmos DB for NoSQL change feed is a powerful feature that can be leveraged for both event-driven applications as well as data management tasks. In this hands-on lab, you get a chance to practice working with an integration between Cosmos DB for NoSQL and Azure Function Apps to enable DIY data archiving in Azure Blob Storage. Students with at least some experience in the Azure portal and scripting in JavaScript — or any modern programming language — will be best positioned to complete this lab quickly and successfully, but even if you are a novice, you should be able to follow along and take advantage of a few hints dropped along the way.
Path Info
Table of Contents
-
Challenge
Create a New Azure Function with Appropriate Bindings
In this objective, you will create a skeletal function in preparation to a little additional script using the Azure portal. Azure Functions are more typically coded in SDK code, but we wanted to make this a quick lab to help you get experience with the integration architecture, without the need to have a development environment spun up.
-
Log into the Azure portal using the credentials provided in the lab UI. If you have any trouble, check the housekeeping video for tips.
-
You should land in the resource group set up already. Take note of the location (region) for the resource group. You will need to use that same region when setting up the Azure Function App. Confirm that the following assets have already been deployed:
- A Cosmos DB for NoSQL account: Check that the Cosmos DB account has a database called LabDB, and a container called LabContainer already deployed.
- Azure Storage Account: Check that the Storage account already has a storage container called "lab-blobs" and confirm that the container is currently empty.
- Log Analytics Workspace: Confirm that a Log Analytics Workspace was created.
-
Create a new Azure Function App (Do not use an "integration" link from the Cosmos DB account; this may not bring you to the UI we want to use for this lab.). Helpful hints:
- Use the resource group already created in the lab environment.
- Provide any valid name you want.
- Select the same region/location as the resource group already created in the lab environment.
- Choose Node.js as the runtime stack.
- Confirm that the plan type is Consumption/Serverless.
- Leave everything else defaulted as-is.
- Choose the storage account deployed in the lab environment; the name starts with "labstorage...". Do NOT create a new storage account.
- Enable Application Insights.
- Leave all other categories of settings defaulted as-is.
-
Once the Function App is complete, go to the Function App resource, and create a new Azure Function.
-
For the input binding, choose a Cosmos DB trigger from the templated options and configure the binding based on your existing CosmosDB instance, using the container and database names you confirmed in step 2, above. Ensure you tick the option to create the defaulted "leases" database, if it doesn't exist. (Note that there may be an error in the UI where two labels both refer to the container name for leases. The first one is actually for the container to be monitored.)
-
Create a new output binding for the existing blob storage account. Alter the Path replace "outcontainer" with the name of the Azure Storage container you confirmed in step 2, above.
-
Test the Azure function, passing in a valid array with a single object.
-
-
Challenge
Update the Azure Function Code to Archive Data
In this objective, you will update the sample code with script that takes each updated item from the Cosmos DB change feed and writes it to the the Blob Storage container configured in the output binding. In the first step of this objective, we provide a natural language description of the code you need to author, purposely avoiding the use of syntax, in order to guide you without providing the actual code.
- Alter the sample code within the
if
block to write the inbound data to the Blob Storage container, by setting the function's output binding parameter, named outputBlob, equal to the first (and only) document input by the Cosmos DB change feed — or when testing manually, in the "Input" pane of the testing fly-out dialog. - Save the code and test with a simple input, such as
[{"id": "abc123"}]
. - Check if your tests were fully successful by confirming that at least one or more documents (depending on how many tests you run) has been archived in the Blob Storage container.
- Check if the trigger on the change feed is working by adding a few items manually to the "LabContainer" in Cosmos DB.
- Check the Blob Storage container to confirm that more items have been added.
Hints and Tips
- Even though the Function takes a list, there will always only be one document in the list, so the code can be written to work with the first — and only — document in the list.
- Do not over-think this task. You only need one line of code to successfully code the function. The purpose of the lab is to get practice with the integration architecture, not a test of your awesome coding skills.
- If you get stuck, the full solution is in the lab guide and is reviewed in the solution video.
- When testing the integration, you can put in any valid JSON you want. The only property that is required is
id
, and it must be unique within the container.
- Alter the sample code within the
What's a lab?
Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.
Provided environment for hands-on practice
We will provide the credentials and environment necessary for you to practice right within your browser.
Guided walkthrough
Follow along with the author’s guided walkthrough and build something new in your provided environment!
Did you know?
On average, you retain 75% more of your learning if you get time for practice.