Queue triggered Function with Cosmos DB output binding

Let’s talk about serverless!!

What is serverless? Well it’s definitely not “No Servers!” that’s for sure. It just means the person using the service does not have to bother about the deployment and support of the servers. The servers actually still exist somewhere but Microsoft takes care of the provisioning for you.

Today we are going to use Azure functions that are going to be triggered by a storage queue and write the message into a Cosmos DB container. For this demo we will be doing nearly everything on the portal using C#. Finally we will attempt to pump a bunch of messages into the storage queue and see how it behaves. A simple diagram of what we will achieve is shown below.

Let’s first create a Cosmos DB container.

From the azure home page, click the “Create Resource” button.

Type in Cosmos in the search box and select “Azure Cosmos DB” from the drop list.

Click on “Create”.

Enter your resource group or click “create” to create one.

Enter an account name. I’ve set mine to “anucosmos123”.

Leave all other options as default.

Click on “Review and Create”.

After the validation is complete, click on “Create” and wait for the deployment to complete.

Once the Cosmos DB account has been created, click on “Go to resource”.

Click “Add Container”.

Click on the dropdown arrow next to “New Container” and then click on “New Container” in the drop down.

Enter the database id as “testdb”.

Enter the container id as “testcont”(Container was previously referred to as collection).

Switch off the “Provision database throughput” switch. If this is ON, the throughput will be provisioned at database level and it will be autoscale with default range of 400–4000 RU/s. But we would like to keep costs down for our demo, so let’s set this to OFF.

Set Throughput to “Manual” and set the throughput to 400.

In the Partition key field, enter “/userid”. For more reading on partition keys you can refer to this https://docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#choose-partitionkey

Then click “OK”.

You should be able to see your DB and container once the provisioning completes.

If you get stuck you can try and follow the Microsoft documentation for this https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-create-container

Now let’s create our storage account.

Go back to the Home page and click on “Create Resource”.

Enter “storage account” in the search box and click on “storage account” in the drop down list.

Click on Create.

Enter your resource group and enter a name for the storage account (I’ve chosen “anustore”).

Click on “Review+Create” and then click on “Create” when the validation is complete.

We will now create our storage queue within our storage account.

Once the deployment is complete, click on “Go to resource”.

Scroll down on the left side until you see Queue Service. Click on Queues underneath it.

Click on “+ Queue” and add a queue name.

Your queue is now created.

Let’s get the queue keys

Click on the “Access Keys” on the left pane of the storage account.

Click on “Show keys” and copy the Connection string of key 1. Store this for later use. We will use this to push messages into the queue.

*ALL connection strings and keys are extremely important and confidential. DO NOT share these details with anyone and be careful before posting these into any online forums.

The Microsoft documentation for creating a storage queue can be found here https://docs.microsoft.com/en-us/azure/storage/queues/storage-quickstart-queues-portal

Next we will create our function.

Go back to the Home page and click on “Create Resource”.

Enter “function” in the search box and click on “Function app” in the drop down list.

Click on Create.

Select your resource group. Enter a function name (I’ve selected anufunc1).

Set the runtime stack to .NET. Select your region (I’ve selected South India).

Set the other options as shown below.

Click on Review + Create. Once validation is complete, click Create and wait for the deployment to be completed.

Go to your function app and click on “Functions” on the left side pane. Click on “+Add”. As a template click on “Azure queue Storage trigger”. Click Add.

This will generate a function from a template to be triggered from a storage queue. We still need to point to a specific queue and set the output binding to our Cosmos DB.

In our new function, click on Integration and then click on “myQueueItem”. In the new right side pane, change the queue name to what we had created earlier (myqueue). Click on NEW to configure our storage queue.

Then select the storage acct where we had created the queue and click OK.

Click Save to save the settings.

Next click on +Add output

Change the Binding type to Azure Cosmos Db. Change the database name to the cosmos database which we had created(testdb). Set the Collection name to the container name we had created earlier (testcont).

Click on the New under the Cosmos DB account connection

Select the Cosmos DB account which we had created earlier from the drop down list and click on OK.

Click on OK to save the settings.

Now for some coding.

Click on the Code+ Test button on the left side panel. You will see some template code already generated for us.

Replace the default code with the below C# code. The below function code is expecting 4 fields in the message body. userid, mycomp, myamt and time. The function will also print the system time in IST when it processes a message and the timestamp on the message itself. This will help us understand the time difference between producing the message and it being processed.

Function code

#r “Newtonsoft.Json”

using System;

using Newtonsoft.Json;

using Microsoft.Extensions.Logging;

private static TimeZoneInfo INDIAN_ZONE = TimeZoneInfo.FindSystemTimeZoneById(“India Standard Time”);

public static void Run(string myQueueItem, out object outputDocument, ILogger log)

{

MyClass class1 = (MyClass)JsonConvert.DeserializeObject(myQueueItem,typeof(MyClass));

outputDocument= new{

id = Guid.NewGuid().ToString(),

userid = class1.userid,

mycomp = class1.mycomp,

myamt = class1.myamt,

time = class1.time

};

DateTime localDate = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, INDIAN_ZONE);

log.LogInformation(localDate.ToString());

log.LogInformation(class1.time.ToString());

}

public class MyClass

{

public int userid {get;set;}

public string mycomp {get;set;}

public int myamt {get;set;}

public string time {get;set;}

}

Click on SAVE.

If your function compiles successfully, you should get a screen like the below.

Time or some sample testing! Let’s manually put in a message into our queue.

Open separate window/tab and go to our storage account. On the left side pane, click on the “Queues” tab under the Queue service. Then click on the queue(myqueue) which you had created earlier.

dd message” and enter the following message in the Message text box.

{“userid”:1,

“mycomp”:”laptopa”,

“myamt”:1000,

“time”:”2021–02–27T05:25:46.0312345+05:30"

}

Click on OK.

If you look at the screen which had your function open, after a few seconds(0–20s), you should see something like below.

This indicates that the function was triggered successfully.

Let’s go to our Cosmos DB container and check if it has any items.

Open a new window/tab and go to your container in your Cosmos DB. You should be able to see the one item that was inserted from our function.

As a final step of the demo, let’s pump in a bunch of messages into our queue and see if everything works as it should. You can use any took/language to try this but for this demo we will use Powershell code to write into our storage queue

$storageAccount = “anustore” # Your storage account name

$accesskey = “XXXXXXXXXXXXXXXXXXXXXXXXXXX” # Replace this with your storage key

$QueueName = “myqueue” # Your queue name

function UploadToQueue($QueueMessage,$QueueName){

$method = “POST”

$contenttype = “application/x-www-form-urlencoded”

$version = “2017–04–17”

$resource = “$QueueName/messages”

$queue_url = “https://$storageAccount.queue.core.windows.net/$resource"

$GMTTime = (Get-Date).ToUniversalTime().toString(‘R’)

$canonheaders = “x-ms-date:$GMTTime`nx-ms-version:$version`n”

$stringToSign = “$method`n`n$contenttype`n`n$canonheaders/$storageAccount/$resource”

$hmacsha = New-Object System.Security.Cryptography.HMACSHA256

$hmacsha.key = [Convert]::FromBase64String($accesskey)

$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))

$signature = [Convert]::ToBase64String($signature)

$headers = @{

‘x-ms-date’ = $GMTTime

Authorization = “SharedKeyLite “ + $storageAccount + “:” + $signature

“x-ms-version” = $version

Accept = “text/xml”

}

$QueueMessage = [Text.Encoding]::UTF8.GetBytes($QueueMessage)

$QueueMessage =[Convert]::ToBase64String($QueueMessage)

$body = “<QueueMessage><MessageText>$QueueMessage</MessageText></QueueMessage>”

$item = Invoke-RestMethod -Method $method -Uri $queue_url -Headers $headers -Body $body -ContentType $contenttype

}

for($i = 0; $i -lt 100; $i++) # change for the number of messages to push into the queue

{

$id = Get-Random -Minimum 1 -Maximum 1000

$comp = ‘laptopa’, ‘laptopb’, ‘laptopc’| Get-Random

$amt = Get-Random -Minimum 1 -Maximum 5000

$time = (Get-Date -Format o).ToString()

$payload = @{

“userid” = $id

“mycomp” = $comp

“myamt” = $amt

“time” = $time

}

$Message = @($payload) | ConvertTo-Json

#Write-Output $Message

UploadToQueue -QueueMessage $Message -QueueName $QueueName

}

You should be seeing a lot of action in your function window if you still have it open.

What the code does is push 100 messages into your storage queue. Your function is then picking up the messages and writing them into your Cosmos DB via the Output Binding.

If you open up your Cosmos DB, you should now see 101 items.

Hope this Demo was helpful to understand the basics of Azure functions and input and output bindings

If you have any questions or comments please feel free to leave a comment.

Until next time.

IT Solution architect with an interest in cloud computing and anything geeky and techy.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store