When designing the architecture of an IoT project, there are a few things one needs to keep in mind. The amount of collected data, the technologies used on frontend and backend or the synchronisation between hardware and software applications are some of them.
In this article, I’m going to show how we came to use TICK stack for the IoT architecture in one of our projects, why it was different from what we expected at the beginning and what we learned while using it.
What you’ll learn about IoT architecture and TICK stack in this article:
- What’s a simple IoT architecture
- The proof of concept for IoT architecture
- How to use TICK stack in IoT architecture
- What can be challenging in using TICK stack in your IoT project
Simple IoT architecture
A simplified IoT infrastructure consists of four major components:
that collect data from the environment, e.g. thermometers, hygrometers and allows them to act (e.g. set a given temperature or turn the lights on/off)
The project our software team worked on was a fully custom solution. Our client designed their own hardware for smart heating control. This way we became an actual R&D team. We worked together with the hardware team to design the communication between the devices and the IoT software applications we were building.
The proof of concept for IoT architecture
The first thing we started with was a proof of concept for the project architecture. Our engineering team researched and evaluated different solutions and technologies to be used further in the project.
Why did we do it? Because building a proof of concept and failing takes much less time and much less money than building a full solution. Luckily, in most cases, you can make a proof of concept with ready-to-use solutions such as Raspberry Pi or Arduino hardware and Kibana data visualisation tool.
The initial architecture we started our research with looked as follows:
Before jumping into development, our tech team researched a few protocols, brokers, middleware and time-series databases.
Eventually, we decided on the following IoT architecture:
- MQTT as our communication protocol.
- Emqttd Broker which was Erlang based and supported both server-side and client-side authentication.
- InfluxDB as the time-series database.
- Elixir / Phoenix for the backend application.
- The communication between the backend and the IoT Broker handled by TICK stack supported by Instream library used for sending and receiving InfluxDB queries and responses
- Websockets for real-time communication between the backend server and client applications
What is TICK Stack
TICK stack is an open-source collection of technologies used to store, capture, monitor and visualize data in time series.
- Telegraf collects the data and writes it to Influx
- Influx is a time-series database, i.e. everything that gets to the database receives a timestamp so we know exactly when the data was collected
- Chronograf visualises real-time data from Influx database
- Kapacitor allows defining alerts and notifications in case the system detects an anomaly (e.g. an open window when the heating is on and there’s no one in the room)
Benefits of using TICK stack
To me, as a developer, the biggest benefit of TICK stack is its open-source nature. While working on the project I could see that the community put tremendous effort in keeping the stack up to date. And new, useful features were added!
What’s more, the structure of TICK stack makes it a handy tool to monitor IoT applications. Let’s take Kapacitor, for instance. When a device disconnected and stopped sending data, Kapacitor notified us by instantly sending the information to two separate channels: to the Phoenix backend and to our Slack channel. This way we could not miss it.
Chronograf’s interface is intuitive and easy to work with. I liked that the set contained more than 20 pre-defined dashboards we could start with. With Chronograf, it was easy to generate queries and visualise the results in the form of a chart. In fact, we were able to generate any query to Influx database.
Challenges in IoT architecture with TICK stack
Despite the obvious benefits of using TICK stack, we faced some challenges as well, especially when it came to authentication and data load.
In theory, the broker could handle up to 2 million parallel connections. However, the number dropped significantly when we used mutual authentication on the server and the client’s side. Every once in a while the connection used to restart and we had to confirm that the device was still visible in the system. With 500 devices, this procedure was a heavy load for the processor and RAM.
Telegraph also challenged us a bit. In our architecture, the broker sent messages organised in potentially hundreds of topics. On the other hand, the Telegraph used to write the topics as separate tags in InfluxDB. Tags were indexed in Influx’s memory. As you could imagine, bringing more data to the database could cause serious database overflow. We had to reduce the number of tags sent to Influx by implementing our own solution. The TICK stack plugin did not allow to change the way indexing is done.
Even though we did not use all features coming with TICK stack, there are certain observations we made:
- TICK stack does not depend on the frontend and backend technologies
- It monitors the data in real-time and creates alerts if something does not work correctly
- TICK stack has an easy-to-use interface to visualise data from a time-series database
- TICK stack can manage IoT architecture that relies upon many IoT devices
- Double server and client authentication can be a challenge for TICK stack