Sample Scenario: Backend for IoT Devices

In this chapter, we want to implement a backend infrastructure based on Durable Functions (and durable entities) to manage the actual state of a set of IoT devices.

The following figure shows you the scenario.

Figure 31: The architecture schema for the IoT backend platform

The scenario shown in Figure 31 is a classic IoT scenario. A set of devices produce telemetries (in our example, we suppose that they produce temperature telemetries, but we try to generalize the kind of telemetry). These telemetries must be ingested in our cloud platform to store and use data. We use an IoTHub service to ingest data, and we must implement a dispatcher layer to get telemetries from IoTHub and dispatch them to the correct device in the backend layer.

We don't care about the IoTHub and the dispatcher in this example because they aren't topics related to Durable Functions. You can find several implementations of these components in Microsoft documentation.

Let’s suppose that the backend layer must have these requirements:

• The backend must expose one or more REST endpoints to send telemetries to the devices.
• The backend must expose one or more REST endpoints to retrieve the list of the devices in the platform, the last telemetries received by a single device. A control dashboard can use these endpoints to show the device status.
• The backend must expose one or more REST endpoints to configure every single device in terms of the amount of telemetry to be kept in memory or the thresholds to use to send alarms to external services.
• The type of devices the platform supports can change over time and adding new types of devices must be easy. For example, in the second version, we could add devices that manage telemetry containing humidity values.
• The backend must interact with external services to start workflows (for example, sending an SMS when the temperature of a device goes over a threshold you set in the configuration). This interaction can change over time, and its evolution must be simple.
• The runtime we use must abstract the persistence operations for saving status for the devices.

During the example explanation, we try to emphasize how we implement every single requirement.

We can design our solution with three layers, as the following figure shows.

Figure 32: The layers of the IoT platform

The front-end layer contains all the REST endpoints to expose all the features requested in the requirements. This layer is composed of durable clients that interact with the stateful layer.

The stateful layer contains all the objects that manage the status of the devices. This layer is composed of durable entities, and each entity manages a device in the field.

Finally, the integration layer contains all the workflows invoked by the stateful layer and manages the integration with the external services. In this layer, you can find orchestrators and activities.

Front-end layer

The front-end layer contains the client functions that implement the APIs you can use to send telemetry to the devices or retrieve information from them.

These are the clients in the front-end layer:

• SendTelemetryToDevice: Allows you to send telemetry to one of the devices in the stateful layer.
• GetDevices, GetDevice: Allows you to retrieve the list of the devices in the solution or to get information about a specific device.
• SetConfiguration: Allows you to configure a specific device, setting the telemetry thresholds or alert details.
• GetDeviceNotifications: Allows you to retrieve the list of notifications for a single device.

Each of those client functions uses one of the previous chapters' communication patterns to communicate with the stateful layer composed of entities.

The following snippet of code shows you the implementation of SendTelemetryToDevice client functions.

Code Listing 46: SendTelemetryToDevice function

The TelemetryDispatcher component in our solution retrieves telemetry from the IoTHub, creates a DeviceTelemetry object, and sends it to the front-end layer using the POST API exposed by this function.

In the following figure, you can see the structure of the DeviceTelemetry class.

Figure 33: DeviceTelemetry class structure

You can add all the properties you need to enrich the information to send to the devices:

• DeviceId: This is the string that identifies a specific device. The SendTelemetryToDevice function uses it to create the IdentityId to signal data to the particular device.
• DeviceName: This is the name of the device.
• Timestamp: This is the date and time when the device sent the telemetry.
• DeviceData: This property contains the telemetries. It is a dictionary with a string key (the name of the telemetry, for example, temperature or humidity) and a double value (the value of the telemetry). Using a dictionary, you can have different devices with different telemetries managed in the same solution.
• DeviceType: This defines the type of the device. You have only a temperature device in the sample, but you can add other kinds of devices. The SendTelemetryToDevice uses this property (with the DeviceId) to generate the right IdentityId for the device to interact with.

The SendTelemetryToDevice client is easy, and its flow is the following:

1. Retrieves the DeviceTelemetry object from the request body.
2. Generates the EntityId using a factory class. The factory class (you can find its implementation in the GitHub repo) uses the DeviceId and the DeviceType values to instantiate an EntityId object. With this approach, you can add a new type of device simply adding a new value to the DeviceType enumeration, implementing the new device, and changing the behavior of the factory.
3. Signals the telemetry to the device using the EntityId created in step 2 and the operation TelemetryReceived exposed by the entity.

In this scenario, the signaling approach is the right choice because the client must send the telemetry to the device without waiting for the response. This way, it can immediately close and free its resources for the other telemetries sent by the TelemetryDispatcher.

The GetDevices client function allows you to search devices and retrieves information about them. The implementation code of this client is the following.

Code Listing 47: GetDevices function

The function expects a parameter in the query string of the request to define the device type. If this parameter is not present, the function returns an HTTP 400 (bad request) error.

The function code consists of two main steps.

The first step consists of creating an EntityQuery object to control the behavior of the ListEntitiesAsync method used later in the code. The EntityQuery class allows you to configure the page size of the list of entities for each request (PageSize property); to define if you want to retrieve only the device metadata or also the state (FetchState property); or to add a filter on the entity name (EntityName property).

If you don't set the EntityName, the ListEntitiesAsync method retrieves all the entities in the platform. Our sample generates the entity's name using a factory similar to the factory used in the SendTelemetryToDevice function to generate the EntityId.

The next logical step in the code is retrieving the entities data from the platform. For this purpose, you can use the ListEntitiesAsync method. Every time you call it, you retrieve a page of entities based on the setting you define in the EntityQuery class mentioned earlier. The return value of the ListEntitiesAsync method contains the page of results (property Entities) and the continuation token (ContinuationToken property).

If you want the next page of the query, you need to call the ListEntitiesAsync method again, passing EntityQuery with the continuation token retrieved in the previous call. You reach the last page when the continuation token returned by the ListEntityAsync method is null.

The Entities property is a list that contains the entity information. It includes the entity ID, the last operation timestamp, and the entity's status if you enable the FetchState property in the EntityQuery. This status is a JSON object stored in a JToken class.

In our example, we deserialize the JSON object to fill the list of the result to return to the client caller.

You can add other filters to your client functions (such as the device's name or the last operation timestamp) before adding the state to the return list.

The GetDevice client function allows you to retrieve information about a specific device, and its code is the following.

Code Listing 48: GetDevice function

The function's URL expects the ID of the device to be recovered and the deviceType field as a parameter in the query string. The code uses device ID and device type to generate the entity ID and uses this value to call the ReadEntityStateAsync method exposed by the IDurableEntityClient.

This method allows you to retrieve the entity's status and serialize it in the object used as generic. The method returns an EntityStateResponse object. This object contains the serialized status (EntityState property) and a Boolean property, which indicates if the entity exists or not (EntityExists property). If the entity exists, we create the response object and return it to the caller. Otherwise, the function returns a NotFoundObjectResult response (Not Found response, error code HTTP 404).

The SetConfiguration function allows you to configure a single device signaling the configuration object to the entity, as shown in the following code.

Code Listing 49: SetConfiguration function

This function is similar to the SendTelemetryToDevice function: it retrieves the configuration object from the request body and signals it to the entity. The configuration object can be different for each device, and in this way, we can support different devices in the future.

Finally, the last client function, the GetDeviceNotifications function, retrieves all the devices' alert notifications. It is like the GetDevices function: it calls the ListEntitiesAsync method, but this time, the EntityName property of the EntityQuery object is always the same, and it is the name of the entity that stores the notifications for each device.

Code Listing 50: GetDeviceNotifications function

Stateful layer

The stateful layer contains the entities of our solution.

We have two kinds of entities:

• DeviceEntity: These entities implement the IDeviceInterface and represent the physical device of our IoT platform. In the example, we implement only one device, called TemperatureDeviceEntity, but you can extend the platform by adding other implementations of the same interface.
• DeviceNotificationEntity: Every time a device needs to send a notification, it uses one of these classes to store the event before calling an external system using an orchestrator.

The following figure shows the definition of the IDeviceInterface interface.

Figure 34: The IDeviceEntity interface

The interface exposes the following methods:

• TelemetryReceived: Allows you to send telemetry to a device.
• SetConfiguration: Allows you to change the device configuration (for example, the thresholds for notification) of a device.
• GetLastTelemetries: Allows you to retrieve the last telemetries stored in the device status.

The following snippet of code shows you an implementation of a device that manages telemetries containing a temperature value.

Code Listing 51: The TemperatureDeviceEntity class

We will analyze the most important part of the code shortly, but you can find the whole class in the GitHub repo of this book.

The inner class, called DeviceConfiguration, contains the configuration you can set using the SetConfiguration method of the entity.

Code Listing 52: The DeviceConfiguration class

The properties of this class allow you to configure:

• The amount of telemetry to be stored by setting the time interval beyond which the telemetries are deleted (HistoryRetention property, by default 10 minutes).
• The thresholds for the temperature to be used to send notifications (the TemperatureHighThreshold and TemperatureLowThreshold properties).
• The telephone number to use to send notifications (the NotificationNumber property).

The following snippet of code shows you the implementation of the TelemetryReceived method.

Code Listing 53: The TelemetryReceived method

The method checks if the telemetry received by the caller must be stored (if it is older than the persistence interval set in the configuration, then it is discarded) and stores it in the HistoryData dictionary.

Finally, the method removes the expired telemetries (ClearHistoryData method) and checks if a notification must call (CheckAlert method).

If the device must throw a notification, it uses a method called SendAlert, which is shown in the following snippet of code.

Code Listing 54: The SendAlert method

This method signals to another durable entity, called DeviceNotificationEntity, the information about the notification. Every device has a notification entity that stores all its alerts, even if these didn't generate a real notification against an external service.

We implement the SignalNotification method to centralize the creation of the notification entity and the signal process. This method is an extension method of the IDurableEntityContext interface, and the code is shown in the following snippet.

Code Listing 55: The SignalNotification method

The name of the notification entity used by the device to store its notifications is generated using the device entity name and the device entity ID. In this way, you have a pair of entities: one for the telemetries (the device), and one for all the notifications thrown by the device (the notification entity).

Using the signal approach, the device sends the information to the notification entity, but it doesn't wait. It continues immediately with the next instruction. This approach helps you in scalability.

The last part of the method starts; if you set the phone number for the notification, an orchestration to send the alarm. In this case, the orchestration start is asynchronous, and the device can immediately manage the next telemetry.

If you change the behavior of the orchestrator, you can change the way the notification will be sent. In the next section, we will analyze the orchestrator.

The following snippet of code shows the DeviceNotificationEntity implementation.

Code Listing 56: The DeviceNotificationEntity class

Integration layer

The integration layer contains all the durable functions used for the interaction with external services.

When an entity device needs to interact with external services, it starts an orchestrator. The orchestrator manages the flow among all the activity functions used for interaction between our platform and the external services.

This way, as we saw for the notification approach between device entity and notification entity in the previous paragraph, it allows us to implement an async interaction between the entity and the orchestrator. We don't block the device while we wait for the orchestration to be completed.

In the following snippet, you can see the implementation of the SendTemperatureNotification orchestrator.

Code Listing 57: The SendTemperatureNotification orchestrator

We implement an orchestrator even if we have only one activity in the notification process (as we do in the previous snippet of code).

It is useful if you remember that an entity can signal another entity or start an orchestrator, and you cannot call activity directly from the entity.

The SendTemperatureNotification orchestrator calls a single activity (the TwilioActivities_SendSMS activity) to send an SMS using Twilio. The code for that activity is shown in the following snippet.

Code Listing 58: The SendMessageToTwilio activity

All the configurations for the Twilio account are stored in the configuration file, as shown in the following JSON.

Code Listing 59: The configuration file for our function app

Note: Twilio is a full solution for enterprise communications. One of the services provided by Twilio is SMS (Twilio Programmable Messaging). You can find the documentation here.

Summary

This concludes our tour of Azure Durable Functions.

Durable Functions is a powerful technology that you can use in different scenarios to solve various problems. As with all technologies, there are scenarios where it makes sense to use this technology, and others where it would be counterproductive. I hope this book gives you the tools to understand if Durable Functions is the right tool for you.

Good luck and buona fortuna!