jump to navigation

MtigOS – Plotting sensor data from ESP8266, where all of the work has been done for you March 2, 2020

Posted by GuySoft in Android, diy, Electronics, linux.
Tags: , , ,
trackback
Raspberry Pi and esp8266 to give fancy graphs

Raspberry Pi and esp8266 to give fancy graphs

Hey all,

This project started from a hackathon, where the organiser dumps a pile of esp8266 chips and sensors on a table that were up for grabs. I wanted build something that would take about 10 seconds to assemble and flash, and would be functional anywhere, no hardcoding of wifi settings and servers. Since I am the developer of CustomPiOS I also made a Raspberry Pi distro that has all the fancy dashboards pre-installed.

This work is also modular and written in a way you could build long lasting solutions, the code is all arranged so it’s easy to contribute and have a monitoring solution which is easy to customize.

What I am going to provide here is a set of tools, that would let you get pretty graphs from sensors with:

  1. NO coding required
  2. NO hardcoded wifi passwords or server settings
  3. NO soldering
  4. About 2-5 minutes assemble the hardware
  5. You get a standard which is supported in my dsitro and other devices. So you can use MQTT apps that are out there.
  6. Raspberry Pi image that sets up all the software that lets you see fancy graphs right away. Calling it MtigOS. It also automatically updates and is maintained.
  7. You are going to use Docker, without the need to know how to use Docker.

Cool tricks I am going to use:

  1. You press the reset button a couple of times to enter a web config interface available in an access point so you can set up your wifi and MQTT server
  2. I am saving you hosting this on a server and giving you the option to host this locally using a Raspberry Pi image.
  3. docker + docker-compose that sets up a database and graphing tool – everything is already all set up. So if you want a server instead of a Raspberry Pi you can use that too
    no need to know docker.

It looks like this in the end:

Pre-shipped Grafana Dashboard

Pre-shipped Grafana Dashboard

Temperature and PH sensor in an ice cream box, with a USB hub powering multiple sensors

Temperature and PH sensor in an ice cream box, with a USB hub powering multiple sensors

I am using this solution for hydroponics system for PH and temperature, but can also use it to measure the temperature in my oven, soil moisture, basically anything you want. I would be happy to add additional sensors.

Hardware

All the firmware is available in this github repo

The hardware used is:

  1. ESP8266 – Any setup, I used both the WEMOS and NodeMCU 0.9/1.0 – Just make sure it has a reset button because some models don’t.
  2. Your local home network, basically you need to give the ESP8266 access to the Internet
  3. Sensors – Current sensors supported:
    1. Temperature sensor using DS18B20
    2. PH sensor – similar to this one
    3. Moisture sensor – get a capacitor sensor that looks like this, the cheap ones die after about 24 hours

Assembly

1. Temperature sensor

ESP and DS18B20 sensor

ESP and DS18B20 sensor

2. PH sensor

First, you need to calibrate the driver of the PH sensor as explained in the “PH probe module Offset and how to use it” section of this guide. You need to do it once.

Then assemble it as explained here:

ESP and PH sensor

3. Moisture sensor

Firmware

You need the following Arduino libraries installed, all can be installed from the LibraryManager:

You also need the esp8266 boards from the Board manager as well.

Flash the firmware for the sensor you are using:

1. Temperature sensor inoraw

2. PH sensor ino raw

3. Moisture sensor inoraw

Important: Make sure you set the Flash size to a number larger than 0.
You can set it by going in the Arduino IDE to Tools>Flash Size>4MB (FS:2MB).

Adduino set filesystem

How to set The ESP8266 to save the settings to its filesystem

That’s all the installation on the Arduino side of things. Lets move to the server side, we will return to what to do with your ESP8266 once we have a server set up.

Software

I am going to go a bit in to detail of what is going on below, you can ignore it. All you really need to know in this step is that if you flash the Raspberry Pi image, you will have all of this pre-installed and set up.

Raspberry Pi Method – Recommended

Official download mirror

  1. Unzip the image and install it to an SD card like any other Raspberry Pi image
  2. Configure your WiFi by editing mtigos-wpa-supplicant.txt at the root of the flashed card when using it like a flash drive
  3. Boot the Pi from the SD card
  4. Hostname is mtigos (not raspberrypi as usual), username: pi and initial password is: raspberry
  5. After a few minutes you should be able to access http://mtigos.local/

Now that you have a server you need to set up your ESP8266 to connect to it, you can skip to that section below.

MtigOS github page with instructions how to flash a Pi image and set up wifi

Rather use your own server or anything with Docker and docker-compose installed

Then you can just use the docker-compose from the repo. I am going to assume you already have docker and docker-compsose installed. If you don’t know how to use them – just use the Raspberry Pi MtigOS image.

Download this repo and you will have all the docker-compose.yaml and setting files. Note that unlike the Raspberry Pi the port is port 3000 and not port 80 for the web service.

Some detail about what is installed

To display nice graphs we are going to use Grafana. I am going to use a popular solution which is:

  1. Sensor pushes data to MQTT
  2. Telegraf pushes data from MQTT to InfluxDB
  3. Grafana displays the data from InfluxDB

Installing and setting all this up is something I am going to save all the work for you. Just flash a RasperryPi with a distro with everything pre-installed.

I am calling it MtigOS ( Mosquitto, Telegraf, InfluxDB and Grafana ) OS.

Set Up the ESP8266 to Connect to your Network and MQTT server

Note that this step can be done by someone without any code of technical understanding this point. So you can give it to your friend, student etc.

Here is a video how to set up this phase (guide has an upgraded moisture sensor):

Text instructions:

  1. Press the ESP8266 button several times
  2. Search on your smartphone or phone for a wifi access point starting with ESP…
  3. Connect to it and then find the IP that you have. On android device you can do this by looking in the about section for the IP.
  4. Use the IP you found, for example 192.168.4.2 and replace the last digit with 1. So in that example it would be 192.168.4.1. Put that IP in your browser, you should get this page:
  5. Click on the button “configure wifi”
  6. Select you wifi SSID, insert your password
    1. In the same page fill in also your MQTT server address and the topic you want to publish to if you are using the Raspberry Pi image the server should be should be mtigos.local. and the topic should be to esp/moisture, esp/temp or esp/ph
    2. If you are using the PH sensor, here you will have an extra field that lets you set the calibration value of it. That is as explained in the calibration instruction I linked to above.
  7. This is important. Mtig/MtigOS has the default channels it listens to set as esp/moisture, esp/temp and esp/ph. So you must use these in the topic field. I put it as a setting because you can change this in grafana, or add multiple sensors this way. And because I am trying to avoid hardcoding. You can change which topics are saved to the database in telegraf.conf. And you will need to update your dashboards.

That’s it, you should have the Arduino publishing to MQTT, and Telegraf saving the data to InfluxDB.

Testing Tour Sensor is Reporting to the MQTT Server

This is a helpful step to make sure your sensor is indeed reporting to the server. You can look at it directly without looking at the graph, and it will confirm you got the previous steps right.

Its also convenient because it means you can check your smartphone for live status of the sensor readouts.

You can use a desktop tool such as MQTT.fx or an Android app such as IoT MQTT Panel. Anything you can put the Raspberry Pi/server IP address and then subscribe to the topic your ESP is publishing to (the topic you set it step 6.2 in the previous section.

IoT MQTT Panel

IoT MQTT Panel from an android phone

MQTT.fs settings dialog

MQTT.fx settings dialog

MQTT.fx receiving moisture information

MQTT.fx receiving moisture information

Accessing the Grafana Dashboard

If you are using the MtigOS Raspberry Pi image, you need to set to open in a browser (port 80) the Raspberry Pi IP address. Or hostname http://mtigos.local/ . It might take a few minutes to start because the Raspberry Pi needs to connect to the internet and download the latest software. It does that automatically, and you must have internet available for the Pi for it to install the software.

The default user is “admin”. And the default password is also “admin”. After first login you will be asked to set a new password.

To reach the dashboard, click on “Home”, you should see under “general” a dashboard named “sensors”. The dashboard and telegraf are listening to the topics esp/moisture, esp/temp and esp/ph. You can edit and save anything you want in the dashboard.

Sensors dashboard once you click on

Sensors dashboard once you click on “Home”

You should see a dashboard like this:

Sensors Dashboard

Sensors Dashboard

Note you can edit and save any changes in the dashboard and create new ones. If you made useful ones you can export them and contribute to the default ones.

That’s it! You have a modern customizable IOT monitoring service, that is storing your esp data in to a database, which you can build a set up anywhere you like, and let other people use. Its using open source components. If you have the coding ability it’s easy to add support for new sensors.

I am using a setup of this for my hydroponics. Here is a photo of it on my balcony:

My hydroponics system in the first month

My hydroponics system in the first month

My hydroponics system today, 5 months later

My hydroponics system today, 5 months later

Code contributions are welcome!

Comments»

1. Yigal - March 2, 2020

Looks like a nice project, Well detailed. I am certainly going to try it. Probably with docker since I don’ t have a PI yet (doing Arduino projects so far). Any recommended PI model?

GuySoft - March 2, 2020

All pi models should work. I’d say 3 and above will give you better results with lots of data and web responsiveness. But even a zero should be fine.

2. tal - May 20, 2020

Would you please mind sharing how you built your hydroponics system ?

GuySoft - May 20, 2020

Hey, yes sure. I followed this guide, unfortunately its in Hebrew so I doubt most will know how to read it. But you are welcome to take a look here.

tal - May 20, 2020

Thank you !


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: