Monitor and visualise "everything" with Grafana
Over the past 6 months or so, the WLAN Pi team has been busy designing and building features that will allow the WLAN Pi to more easily be deployed as a remote network sensor.
Deploy the WLAN Pi, collect data and metrics, then remotely monitor and visualize KPIs that matter, using a web-based tool called Grafana.
WLAN Pi + Grafana will enable you to visualize and analyze data collected, monitor your environment and optimize your network.
Grafana is an open-source web-based tool for monitoring and observability. It's commonly used to visualize time series data for infrastructure and application analytics. It's also flexible enough to support other types of data visualizations.
Here is an example Grafana dashboard. Dashboard is an easily customizable web page. No coding skills required, don't worry 😊 You can create comprehensive dashboards with panels that visualize data in various ways, such as graphs, histograms, heat maps, and tables. These dashboards can be tailored extensively, allowing you to present your data in an aesthetically pleasing format to your customer.
Grafana supports a wide range of data sources, including InfluxDB, Prometheus, Graphite, Elasticsearch, MySQL, PostgreSQL, and many more.
During this lab, we will not use a database to store data. Instead we will stream the data directly into Grafana.
"I find Grafana a little overwhelming"
-an actual quote from Nick Turner
A great deal of the fiddly work has already been completed, that we can focus on the visualizations and building graphs that just look great.
Grafana provides a feature for alerting based on specific thresholds. If a metric breaches a particular condition, alerts can be sent out via various notifiers, such as email, Slack, or PagerDuty. We won't explore this in a great level of detail, but if you are interested in this, definitely spend some time after this Deep Dive on this.
The platform is extensible, with a plethora of plugins available to extend its capabilities. If there is a specific need that is not addressed by the core features, there is a good chance that a plugin exists (or can be created).
Users can annotate graphs with event logs, which helps in correlating data changes or anomalies with events, such as code deployments or infrastructure changes. So if your DNS goes down, you can add a little note in the actual graph as an explanation or reminder of what happened.
Learn more about Grafana:
Let's start with a little housekeeping. It is critical that your WLAN Pi system time is correct.
WLAN Pi as an internet monitor
When an internet link goes down, most users will report that the “Wi-Fi doesn't work!”
How reliable is your internet link? The next dashboard can help you quantify that and find the smoking gun. Was it the Wi-Fi, was it DNS, or was it actually your internet provider's planned maintenance early Sunday morning?
Open the Internet Monitoring dashboard.\
It feels a bit lonely here, doesn't it? Don't you worry, that's because our WLAN Pi has not started collecting data yet.\
The purpose of the Data Stream is to collect data and feed it to the associated dashboard. In this instance, our Data Stream script measures the speed of your internet link, runs a few ping tests, and checks how fast your DNS server responds.
Head over to the top Grafana menu bar item, click the play (▶) button next to Internet Monitoring.\
The Data Stream will now become active. Every 5 minutes, it streams data to the Internet Monitoring Grafana dashboard.
How is the internet connection in the room holding up?\
If you find the 5-minute interval too great, try restarting the “Internet Monitoring” data stream script
All the data you are viewing is only stored in the browser cache, if you navigate away from the dashboard page, all data collected so far will be lost. This can be addressed by using an external data store (not in-scope of this lab).
Out of the box, Internet Monitoring dashboard uses librespeed-cli
an open-source package for internet speed testing. In some scenarios, we have found the proprietary speedtest CLI
tool developed by Speedtest.net to deliver greater accuracy.
Add the Ookla package repository to your WLAN Pi by executing this command in Terminal:
Install the Ookla speedtest package:
After the installation has finished, execute the command:
Now is the time when you call your lawyer and ask for some free advice. Or simply close your eyes and type YES
in capital letters and press enter key. On serious note, skim through the license terms and make sure you are comfortable.
Secondly, there is a similar, this time GDPR related, prompt. Again, if you want to proceed, type YES
in capitals followed by enter.
Speedtest will automatically run a new test. Wait until it has completed.
Finally, we need to instruct the Internet Monitoring dashboard data stream that you wish to use Speedtest.net. Do so by opening this file in nano text editor:
Add this new line to the end of the file:
To save changes, press Ctrl + O (that's an "o" for omelette). To exit editor, press Ctrl + X.
If the Internet Monitoring data stream has been enabled, disable it using the X and then start it again using the play (▶) button.
Special shout out to Bryan Ward and Kjetil Teigen Hansen, thanks to these guys, we can bundle up some remarkable dashboards as part of our WLAN Pi image. This equips you with pre-built dashboards that are good to go, but, naturally, you can further modify to suit your requirements.
Addictive Activity Warning!
Once you begin creating your own dashboards, you may experience difficulty stopping.
You have been warned 😊
If you build something cool with Grafana and you would like to share, please get it in touch with the WLAN Pi team.
Before we start exploring the dashboards, it is important to understand how the WLAN Pi collects the data to be visualised.
The WLAN Pi team has created several 'Data Stream' scripts that collect values such as RSSI, BSSIDs, SSIDs etc. You will find the list of pre-built “Data Stream” scripts within the Grafana menu item, these scripts may be started and stopped from this menu.
If Grafana is not running, please start it now.\
You should be able to see the available Data Streams:\
Start Scanner WLAN0 Data Stream using the play (▶) button.\
From the Grafana home page, open the Scanner dashboard\
Voilà! Here is a Wi-Fi scanner built using Grafana.\
Before you move on to the next lab, stop the Data Stream script, in the same way you started it, from the Grafana menu drop down.
Let's build something together
So far, we have looked at pre-built dashboards, in this lab you get to construct your own.
We have a suggested WLAN Pi health monitoring dashboard, this would enable you to keep an eye on key performance indicators of the WLAN Pi. How much space is there left on the SD card? What the CPU load is? What is the CPU temperature, is this WLAN Pi running too hot?
Here are the KPIs we would like to monitor and the proposed placement of the visualization panels.
Navigate to GRAFANA > OPEN GRAFANA IFRAME > "Build your own WLAN Pi Health"
dashboard.\
As you can see, we are not quite there yet. The visionary did not finish their job.\
... know that feeling when you move into a new house?
We already built the Data Stream service for you. As soon as you start it, the backend scripts will collect data and send them to the dashboard every 60 seconds.
Enable the Data Stream
GRAFANA > DATA STREAMS > WLAN Pi Health
\
You should now see CPU Utilisation
in your dashboard\
Now that Grafana is receiving data, it is up to you to add new visualizations.
Next up, add CPU Utilisation
graph.\
Add > Visualization
\
The default Time series
visualization is perfect for our needs.
Rename it to CPU Utilisation
Set Query Type > Live Measurements
Select Channel > stream/pihealth/pihealth
Finally, add Fields >
time
and cpu_util
\
Under Standard Options > Unit > select (Misc) > Percent (0-100)
Click the Apply
button.\
Drag & drop the new graph into position.
It is possible to resize panels by clicking and dragging the bottom right-hand corner.
When you are happy, hit the floppy disk Save
button.\
Open a new browser window, navigate to the WLAN Pi Web UI, and go to COCKPIT
Open Terminal
We can create CPU load with a command entered into Terminal
Type timeout 300 yes
and press the Enter
key\
This will keep saying yes to whatever the question is 😅 and it will stop after 300 seconds.
You can stop the yes command manually using"ctrl + c"
Head back to the Build your own WLAN Pi Health
dashboard and watch the CPU stats for a couple of minutes.
Mr. CPU, don't say yes to everything, it will keep you really busy.
Hopefully, we have helped you understand the concept of dashboards, visualizations and how they can be customised with a few clicks.
You would apply a similar concept to the other missing graphs. Eventually, you will build a dashboard that looks like ours.
Please head over to the Grafana home page, that's GRAFANA > OPEN GRAFANA IFRAME
Open the complete dashboard called WLAN Pi Health
\
Try to reverse-engineer the dashboard, edit values, add, move panels around, etc. To edit each panel, simply click the three-dot symbol
and select Edit
.
If you push the CPU to its limits, which actually takes some effort, you might even notice that CPU throttling is now active. You can tell by the Soft Temperature Limit Active
message.\
Try to open the Web UI and Grafana on your tablet. It is mobile-friendly and, it looks great on a smaller screen.
Absolutely, we can. So far, we have relied on existing Data Streams. They are BASH and/or Python scripts executed by a Linux service. When you start or stop a Data Stream in the WebUI, you are starting or stopping the respective service.
Scripts of the WLAN Pi Health Data Stream are located in /opt/wlanpi-grafana/health directory and they send fresh data to Grafana every 60 seconds. Let's explore them and change this setting to every 30 seconds.
Open Terminal in Cockpit and execute these commands.
Enter the directory by: cd /opt/wlanpi-grafana/health
List all files in this directory: ls -l
Execute the data collection script manually: ./pihealth.sh
\
After few seconds we get the output. This is the data in so called 'Line Protocol' format. The pihealth.sh script itself only collects and outputs data, but it doesn't send them to the dashboard. That's job of the stream-pihealth.sh
script.
Let's see how it works: cat stream-pihealth.sh
It takes the output of the pihealth.sh data collection script and sends it using to-grafana.sh script and sends it via Grafana API every 60 seconds.\
All we need to do is edit this script and change 60 to 30. Open the file by sudo nano stream-pihealth.sh
Enter your password and in the nano editor navigate using arrow keys to the right place. Delete 60 and replace it with 30.\
Save the file by pressing ctrl + o
(that's lowercase o for orange), press enter
, and press ctrl + x
to exit the editor.
We are nearly there, restart the Data Stream by executing sudo service wlanpi-grafana-health restart
. This is equivalent of stopping and restarting the Data Stream from the web interface using the 'X'
and Play buttons
. \
Perfect! Now, go to the WLAN Pi Health
dashboard and verify that we receive a fresh set of data every 30 seconds. \
Each dashboard has a Auto Refresh Interval
setting. When you change it to 30 seconds
, it will update the axis representing time twice a minute.
Grafana does not automatically start at boot. The Grafana process must be started manually!
Within the Web UI, hover over GRAFANA and click START.\
Please be extra patient while the Grafana service starts. It will take in the region of 30 seconds to start-up. Take the opportunity to meet a new, amazing Wi-Fi engineer sat next to you.
The Grafana menu will dynamically update and display new options when the service is operational. Hit OPEN GRAFANA IFRAME. This will open Grafana within the WLAN Pi Web UI.\
If you want to run Grafana in full-screen mode, use LAUNCH GRAFANA NEW TAB.
Log into Grafana using the default credentials wlanpi/wlanpi
.\
Welcome to the Grafana home dashboard.\
If we could only view RF spectrum in a web browser. Oh, wait!
WLAN Pi already supported USB analysers when paired with WiFi Explorer Pro, but now, you are equipped with Grafana. If you connect a supported USB spectrum analyser like Oscium WiPry Clarity or MetaGeek Wi-Spy DBx to your WLAN Pi, these dashboards work out of the box, and you do not need to install any software.
WiPry Clarity is a compact tri-band (2.4, 5, 6 GHz) spectrum analyser designed by Oscium. We should have few in the room. If you have reached this section, speak to one of our handsome instructors and ask to borrow one.
Connect the WiPry to one of the USB ports.
Open Grafana home page.
From the GRAFANA menu bar item, start the Oscium WiPry Clarity 5 GHz Data Stream.
Open Oscium WiPry Clarity dashboard.
Voilà! There is your live 5 GHz spectrum view and waterfall diagram just below. Magic? No, just amazing Wi-Fi people doing what they love ❤️ 🛜
Click, drag a box to zoom in on a section of the spectrum.
Double click to zoom back out again.
What does the spectrum look like in the room? Can you ask a friend next to you to generate some Wi-Fi traffic and see how it impacts the spectrum?
Each of the WLAN Pis has iperf3 server enabled. One of you can watch the spectrum and the other one can start an iperf3 throughput test.
Download: iperf3 -c``
<name or IP address of WLAN Pi server>
``-R
Upload: iperf3 -c``
<name or IP address of WLAN Pi server>
DBx is a dual-band spectrum analyser with 2.4 and 5 GHz support. The instructions are the same as for WiPry Clarity.
Connect Wi-Spy to one of the USB ports of your WLAN Pi.
From GRAFANA menu, start the MetaGeek Wi-Spy DBx 5 GHz Data Stream.\
Open the MetaGeek Wi-Spy DBx dashboard. Our 5 GHz spectrum looks very healthy.\
There is much more going on 2.4 GHz. To switch between bands, go to GRAFANA > Data Streams > disable MetaGeek Wi-Spy DBx 5 GHz and enable MetaGeek Wi-Spy DBx 2.4 GHz. Do a hard reload of the dashboard web page in your browser to start seeing spectrum data from the new band.\
Can you think of any other dashboards that would make your life easier? Perhaps you want to monitor channel utilization of your access points and see the top 10? Same with client count. Or keep an eye on DFS event frequency if that's a known troublemaker in your office?