Configuring end-to-end IoT environment on Raspberry Pi

by Lakshmi Deepa S | Jun 30, 2020 | MQTT Broker | 0 comments

Over the past few years, MQTT has begun to establish itself as the most commonly used messaging protocol for IoT projects. This document provides developers with a complete guide on how to set up and run an MQTT Broker on Raspberry Pi in just a few minutes. You will also see how easily you can add intelligence to your Edge device using ESP8266. Once you have gained the knowledge of the MQTT Broker and how to use it, you will be able to make your own IoT projects.

Overview:

The steps involved in setting up a MQTT Broker on Raspberry Pi are

Installing MQTT Broker on Raspberry Pi
Configuration of MQTTBroker
Setting up the ESP8266
Flashing ESP8266
Connecting ESP8266 with MQTT Server
MQTTRoute User Interface

MQTT

MQTT is a lightweight messaging protocol that allows you to send arbitrary messages across a network to any interested device. It uses the publish/subscribe method to exchange data among clients and the server.

Installing MQTT Broker on Raspberry Pi

We are going to use Bevywise MQTT Broker and the Raspberry Pi 3 running the latest version of Raspbian Jessie in this tutorial.

To install the Bevywise MQTT Broker and its client packages follow the below steps

You can download the broker from

In the download form, provide all your required details, choose the Raspberry OS, and download the MQTT Broker.
The downloaded file will be in a zip format.
Unzip the downloaded zip folder using the command
unzip Bevywise_MQTT_Route_Raspbian.zip
Once you unzip the file, the Bevywise folder will be created.

Configuration of MQTTBroker on Raspberry Pi

To configure the broker go the Downloads -> Bevywise -> MQTTRoute -> Conf folder.
In the conf folder, there will be two conf files.

broker.conf
data_store.conf

broker.conf

To configure the MQTTRoute, use the broker.conf file inside the conf folder of the Bevywise package.
By default, the broker will run in non-TLS mode with the port number 1883, and the authentication is disabled.
MQTTRoute provides an option to enable encrypted data transfer and authentication.
To run your product with SSL/TLS support, change TLS_ENABLED = TRUE and change the port number to 8883.
You can also connect your edge devices more securely with MQTT authentication.
To enable the authentication change AUTHENTICATION_ENABLED = YES.
By enabling this, you need to provide an MQTT username and password when the edge device is connected.
The username and the password are available in the Broker User interface.
Once you run the broker, the user interface will appear.
In the User Interface below the “Authentication” tab, there will be a username and password.
You can use that username and password for a secure device connection.

data_store.conf

SQLite is the default data storage in MQTTRoute.
For storing a large amount of data, you can change the storage configuration to MySQL or MSSQL.
For powerful visualization and analysis of data, you can store the data in Elastic Search or any other Big Data engine using a custom implementation.
These options configured using the data_store.conf file inside the conf folder.
To use MySQL as your data storage change, DB_SERVER= MySQL.
And provide the required username and password for MySQL.
Now the data will be stored inside the data folder in the name of the file specified for MYSQL_DB.

After the broker configuration, now you can start the MQTT Broker.
To run the MQTTBroker as service, use the Monit version 5.25.

The procedure to set up the monit:

Download the Monit for Raspberry Pi 3, based on your servers.
Extract the archive using tar -xzf < downloaded file >.
Go to monit-5.25.2/bin.
Copy monit to /usr/bin folder by using the command
sudo cp monit /usr/bin
Go to monit-5.25.2/conf.
Using the below command copy the monitrc file to “/etc/”
sudo cp monitrc /etc/.

Monit configuration:

Modify the monit conf file.
Monit comes with its web server running on port 2812.
To configure the web interface, uncomment the section that begins with set httpd port 2812 in /etc/monitrc file.

Then use

sudo vi /etc/monitrc.
And, change the username and password as per your need.

Add Monitoring services:

At the end of the /etc/monitrc file , add the following.

Check process MQTTRoute, with pidfile Bevywise/MQTTRoute/Broker.pid.

start program = “Bevywise/MQTTRoute/bin/runbroker.sh” with timeout 2 seconds. stop program = “Bevywise/MQTTRoute/bin/stopbroker.sh” with timeout 2 seconds.

To reload the configuration changes, run

sudo monit reload

To add the MQTTRoute process to monitoring, use

sudo monit start MQTTRoute

Restart the Raspberry Pi.
Now, the broker will start running in the background.

Setting up the ESP8266

The below steps explains how to flash a client library in ESP8266 by using the Arduino IDE.

Step 1: Connect your PC with internet

Connect your PC to the internet in which you are going to flash the ESP8266.
You can connect it using an Ethernet cable or WiFi.

Step 2: Install the Latest Arduino IDE

To install the latest Arduino IDE, navigate to the Arduino the page using the following link https://www.arduino.cc
Select your operating system and download the latest software.

Step 3: Installing the ESP8266 Board

Once you install the Arduino software, open it.
Navigate to File -> Preferences -> Additional Boards Manager URLs.
In this text box, copy and paste the following link http://arduino.esp8266.com/stable/package_esp8266com_index.json

Click “ok” to save your changes.
Now open the Board Manager.
Go to Tools -> Board -> Board Manager.

Select the esp8266 by ESP8266 Community package and install it.

Then choose your ESP8266 board from Tools -> Board -> Generic ESP8266 Module (ESP8266-01).

Tools -> Board -> NodeMCU 1.0 ESP-12 E Module (ESP8266-12 E & Development Board)
Finally, re-open your Arduino IDE.

Step 4: Install the Device Driver Software – CDM21226

Download driver software for FTDI USB to TTL Serial Adapter Module from the zip file.
Now unzip downloaded folder.
Run the CDM21226_setup.exe file and install it.
Now, Connect the ESP8266 to your PC’s USB.

Step 5: Download and Install the needed Libraries

Before programming the ESP8266, you need to install the libraries in the Arduino library manager.
There are quite a several Arduino-compatible MQTT clients available now.
One of the best-known clients is PubSubClient.
It works very well and is simple to use.
Now, download the Library from GitHub.
To install the PubSubClient Library, copy and paste it in Documents -> Arduino -> Library.
And restart Arduino IDE.

Step 6: Compile the main program

Once you have installed those libraries, you will be able to run the code.
Presently, before beginning the programming, we have to choose a couple of various alternatives.
In the Tools menu choice, select ESP8266 for Board, 80 MHz for the CPU Frequency, 4M for the Flash Size, and 115200 for the Upload Speed.
Additionally, try to choose the COM port that you are using (this will rely upon your setup).
Now, open the MQTT Broker user interface to download the default library of Bevywise MQTT Broker.
Below the MQTT Clients tab, download the client code that is compatible with your OS.
Once you start the Arduino, Open file -> new in Arduino IDE.
Copy the program in ESPMQTT_Client.
Paste it on a new file.
To compile the main program, go to ArduinoIDE Sketch -> verify/compile.

ESPMQTT_Client

ESPMQTT_Client #include <ESP8266WiFi.h>
#include <PubSubClient.h>
const char* ssid = “wifi name”;
const char* password = “wifi password”;
const char* mqtt_server = “localhost”;
WiFiClient espClient;
PubSubClient client(espClient);
int RECV_PIN = 5;
void reconnect()
{
while (!client.connected())
{
Serial.print(“Attempting MQTT connection…”);
if(client.connect(“ESP8266Client”))
{
Serial.println(“connected”);
client.subscribe(“ESP8266/LED”);
client.publish(“ESP8266/LED”,”on”);
}
else
{
Serial.print(“failed, rc=”);
Serial.print(client.state());
Serial.println(” try again in 5 seconds”);
delay(5000);
}
}
}
void setup()
{
pinMode(RECV_PIN,OUTPUT);
Serial.begin(115200);
digitalWrite(RECV_PIN,HIGH);
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
reconnect();
}
void setup_wifi()
{
delay(10);
Serial.print(“Connecting to “);
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED)
{
delay(500);
Serial.print(“.”);
}
Serial.println(“WiFi connected”);
Serial.println(“IP address: “);
Serial.println(WiFi.localIP());
}
void callback(char* topic, byte* payload, unsigned int length)
{
Serial.print(“Message arrived [“);
Serial.print(topic);
Serial.print(“] “);
for (int i = 0; i < length; i++)
{
Serial.print((char)payload[i]);
}
if((char)payload[0] == ‘o’ && (char)payload[1] == ‘n’)
{
digitalWrite(RECV_PIN,LOW);
}
}
void loop()
{
if (!client.connected())
{
reconnect();
}
client.loop();
}

Below we have explained the code to help you understand what is going on.

Here, we have configured WIFI:

const char* ssid = “wifi name”;
const char* password = “wifi password”;

Then the MQTT Configuration:

const char* mqtt_server = “localhost”;
client.setServer(mqtt_server, 1883);

In the client.setServer, provide the broker port number to configure the MQTT.

Now, we have created the MQTT and WiFi stacks:

WiFiClient espClient;
PubSubClient client(espClient);

We have created the PubSubClient library, which we have downloaded before.

Arduino Setup Function:

void setup()
{
pinMode(RECV_PIN,OUTPUT);
Serial.begin(115200);
digitalWrite(RECV_PIN,HIGH);
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
reconnect();
}

This is the Arduino setup function.
First, the LED configured, then the serial port is set up, and the WiFi configured.
Finally, the client sets the MQTT server address as 1883, and the callback function.

Setting up an MQTT Publish/Subscribe:

void reconnect()
{
while (!client.connected())
{
Serial.print(“Attempting MQTT connection…”);
if(client.connect(“ESP8266Client”))
{
Serial.println(“connected”);
client.subscribe(“ESP8266/LED”);
client.publish(“ESP8266/LED”,”on”);
}
else
{
Serial.print(“failed, rc=”);
Serial.print(client.state());
Serial.println(” try again in 5 seconds”);
delay(5000);
}
}
}

In this phase, the publish and subscribe process will occur.
The client.connect will attempt to establish the connection.
The client.publish will publish the topic “ESP8266/LED” with the value “on”.
If successful, the client publishes the “on” message.
In client.subscribe, the client will subscribe to the topic “ESP8266/LED”.
If a failure occurs, it will attempt to reconnect.

Flashing ESP8266

To flash the ESP8266, go to ArduinoIDE -> Sketch -> upload.

Circuit Diagram for Flashing the program in ESP-12E

Connecting ESP8266 with MQTT broker on Raspberry Pi

Once the program uploaded, the ESP8266 will be started and connected to the MQTT Server.
To make sure it is connected, go to the MQTT user interface.
Check whether the client is connected or not.

MQTTRoute User Interface

Once you get connected to the MQTT Route, you can view it through the user interface.
Use your browser and open the User interface with the URL “http://localhost:8080”.
On the login page, you will see the default “Username” and “Password” as “admin”, use it to log in to the MQTT broker user interface.
Users can also change the username and password via the user interface.

The Dashboard of MQTTRoute will give a quick view of the connected device.
The individual device details can be seen by visiting the Devices tab.
The connection status and the messages published will be dynamically updated.

Refer our Broker help documentation for more information about our MQTT Broker User Interface.

MQTTRoute 3.3 is now available. To know more about the features, visit the MQTT Broker page. Also, try controlling your MQTT devices with MQTT Broker app. Get it on play store.

Note : Are you the technology person looking to automate your house on a Raspbery pi box? Here is your FOREVER FREE MQTT Broker with all the widgets and dashboard needed to monitor and control your MQTT devices.