Distributed weather monitoring in gardens
The Things Network
The Things Network provides a public network server for LoRaWAN applications, registration services for LoRaWAN nodes (sensors), security key generation, packet examination facilities, and, most important, an API for data extraction for several languages. Use of this service is free, but by default, The Things Network does not store your data. If you don't extract your data in a timely manner, it is simply discarded.
The first step is to register your gateway with The Things Network in the gateway console [16]. You can give your gateway any name you like (Figure 6) and choose to make it public, so other people can use it, or private, for your own use only. All data is strongly encrypted, so it is secure, even on a public gateway.
Figure 6: Gateway console and application registration in The Things Network console.
The next step is to register your application. The application unique ID (EUI) that is generated must be added to the sensor's code for the device to be considered for registration.
Finally, you can register one or more devices with a name you select by selecting register device in the Devices pane: I call the devices for this application weatherstation1, weatherstation2, …, and so on. Devices on The Things Network must have a unique ID (in the case of my microprocessor, the EUI is a unique 64-bit number programmed by the manufacturer into its flash memory), which is entered in the Device EUI field on the registration page (Figure 7).
Figure 7: The settings for weatherstation1.
Once registered, the generated application key must be added to the device's code, as well. When the device is powered up, it sends a registration request with a tuple of device EUI, application EUI, and application key.
To view the data, use the Data tab (Figure 8). The payload (application data) has been decrypted and is displayed in hexadecimal format. In this case, the data is a comma-separated list of five floating-point numbers representing temperature, humidity, pressure, intensity, and battery voltage. The metadata provides some details about the packet transmission, the most useful of which is timestamp, representing the time at which the packet was received.
Figure 8: Data view.
Extracting the Data
I chose to write my client for The Things Network in golang, although SDKs for many other languages are provided, as well [17]. Data is made available through an MQTT protocol [18], and an SDK for golang is provided. A simple example and GoDoc is provided online [19]. Only a few lines of golang subscribes to the service, lists the registered devices, and waits for data.
In the first example (Listing 1), I just access my data in The Things Network and write it to a file. The program must provide the application ID and the application access key (lines 24 and 25) to register successfully with The Things Network. Note that the time stamp is extracted from the metadata, which will be important when it comes time to plot the data as time series graphs. To make the data truly useful, however, you need to store it in something more sophisticated than a flat file.
Listing 1
ttn-client.go
01 package main
02
03 import (
04 "os"
05 "fmt"
06 "time"
07 "strings"
08 ttnsdk "github.com/TheThingsNetwork/go-app-sdk"
09 )
10
11 const (
12 sdkClientName = "lora-weather-station"
13 )
14
15 func main() {
16
17 // open a file to write results to
18 f, err := os.OpenFile("weather.dat", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
19 if err != nil {
20 fmt.Println(err)
21 }
22
23 defer f.Close()
24
25 appID := "loramini2"
26 appAccessKey := "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx"
27
28 config := ttnsdk.NewCommunityConfig(sdkClientName)
29
30 // create a client to 'the things network. the access key has been obscured
31 client := config.NewClient(appID, appAccessKey)
32 defer client.Close()
33
34 devices, err := client.ManageDevices()
35 if err != nil {
36 fmt.Printf("lora-weather-station: could not get device manager\n")
37 }
38
39 // create a list of known devices
40 deviceList, err := devices.List(10, 0)
41 if err != nil {
42 fmt.Printf("lora-weather-station: could not get devices\n")
43 }
44 fmt.Printf("lora-weather-station: found device(s)\n")
45 for _, device := range deviceList {
46 fmt.Printf("%s\n", device.DevID)
47 }
48
49 // subscribe to events from all the devices
50 pubsub, err := client.PubSub()
51 if err != nil {
52 fmt.Printf("lora-weather-station: could not get application pub/sub\n")
53 }
54
55 defer pubsub.Close()
56
57 allDevicesPubSub := pubsub.AllDevices()
58 defer allDevicesPubSub.Close()
59 uplink, err := allDevicesPubSub.SubscribeUplink()
60 if err != nil {
61 fmt.Printf("lora-weather-station: could not subscribe to uplink messages")
62 }
63
64 // wait for data from each device and save it to the file. loop forever
65 fmt.Printf("waiting for data\n")
66 for message := range uplink {
67 timestamp := time.Time(message.Metadata.Time).Unix()
68 measurements := strings.Split(string(message.PayloadRaw), ",")
69
70 if len(measurements)<4 {
71 fmt.Printf("lora-weather-station: uplink message malformed, ignored")
72 continue
73 }
74
75 data := fmt.Sprintf("%s %d %s %s %s %s %s\n", message.HardwareSerial,
76 timestamp,measurements[0],measurements[1],measurements[2],measurements[3],,measurements[4])
77
78 fmt.Printf(data)
79
80 if _, err := f.WriteString(data); err != nil {
81 fmt.Printf("lora-weather-station: could not write data")
82 }
83 }
84 }
Prometheus
Prometheus is a free and open source multidimensional time series database for event monitoring and alerting that runs on Linux. You can find many excellent tutorials about Prometheus installation online, so you should choose one that is most appropriate for your distro. (For my Ubuntu setup, I chose a video tutorial on the LinOxide website [20].) Once installed, you should be able to browse to the Prometheus home page. However, to be of real use, you need to be able to insert data into Prometheus, so it's back to the golang code to add Prometheus metrics (Listing 2).
Listing 2
ttn-client-prom.go
001 package main
002
003 import (
004 "flag"
005 "fmt"
006 ttnsdk "github.com/TheThingsNetwork/go-app-sdk"
007 "github.com/prometheus/client_golang/prometheus"
008 "github.com/prometheus/client_golang/prometheus/promhttp"
009 "net/http"
010 "strconv"
011 "strings"
012 "time"
013 )
014
015 // create a bunch of prometheus vectors, one for each measurement
016 var (
017 temperatureGauge = prometheus.NewGaugeVec(
018 prometheus.GaugeOpts{
019 Name: "temperature",
020 Help: "temperature reported by a weather station over time in degrees Celcius",
021 },
022 []string{"hardware_id"})
023
024 humidityGauge = prometheus.NewGaugeVec(
025 prometheus.GaugeOpts{
026 Name: "humidity",
027 Help: "humidity reported by a weather station over time in percent",
028 },
029 []string{"hardware_id"})
030
031 pressureGauge = prometheus.NewGaugeVec(
032 prometheus.GaugeOpts{
033 Name: "pressure",
034 Help: "barometric reported by a weather station over time",
035 },
036 []string{"hardware_id"})
037
038 intensityGauge = prometheus.NewGaugeVec(
039 prometheus.GaugeOpts{
040 Name: "intensity",
041 Help: "light intensity reported by a weather station over time",
042 },
043 []string{"hardware_id"})
044
045 batteryGauge = prometheus.NewGaugeVec(
046 prometheus.GaugeOpts{
047 Name: "battery",
048 Help: "battery voltage reported by a weather station over time",
049 },
050 []string{"hardware_id"})
051
052 addr = flag.String("prom-server-address", ":8080", "the address of the server for prometheus metrics")
053
054 appID = "loramini2"
055 appAccessKey = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
056 sdkClientName = "lora-weather-station"
057 )
058
059 func main() {
060
061 flag.Parse()
062
063 // register the prometheus vectors
064 prometheus.MustRegister(temperatureGauge)
065 prometheus.MustRegister(humidityGauge)
066 prometheus.MustRegister(pressureGauge)
067 prometheus.MustRegister(intensityGauge)
068 prometheus.MustRegister(batteryGauge)
069
070 // publish the prometheus vectors to the premetheus can 'scrape' them for recording
071 // this starts a mini http server
072 go func() {
073 http.Handle("/metrics", promhttp.HandlerFor(
074 prometheus.DefaultGatherer,
075 promhttp.HandlerOpts{},
076 ))
077
078 fmt.Printf("starting prometheus server on %s\n", *addr)
079
080 err := http.ListenAndServe(*addr, nil)
081 if err != nil {
082 fmt.Printf("unable to start prometheus server\v")
083 }
084 }()
085
086 // connect to 'the things network'
087 config := ttnsdk.NewCommunityConfig(sdkClientName)
088
089 client := config.NewClient(appID, appAccessKey)
090 defer client.Close()
091
092 devices, err := client.ManageDevices()
093 if err != nil {
094 fmt.Printf("could not get device manager\n")
095 }
096
097 // List the first 10 devices
098 deviceList, err := devices.List(10, 0)
099 if err != nil {
100 fmt.Printf("could not get devices\n")
101 }
102 fmt.Printf("found device(s)\n")
103 for _, device := range deviceList {
104 fmt.Printf("%s\n", device.DevID)
105 }
106
107 fmt.Println()
108
109 // subscribe to events from the devices
110 pubsub, err := client.PubSub()
111 if err != nil {
112 fmt.Printf("could not get application pub/sub\n")
113 }
114
115 defer pubsub.Close()
116
117 // listen for measurement events and update the prometheus counters. loop forever
118 allDevicesPubSub := pubsub.AllDevices()
119 defer allDevicesPubSub.Close()
120 uplink, err := allDevicesPubSub.SubscribeUplink()
121 if err != nil {
122 fmt.Printf("could not subscribe to uplink messages\n")
123 }
124 fmt.Printf("waiting for data\n")
125 for message := range uplink {
126 timestamp := time.Time(message.Metadata.Time).Format(time.RFC3339)
127 measurements := strings.Split(string(message.PayloadRaw), ",")
128
129 if len(measurements) < 4 {
130 fmt.Printf("uplink message malformed, ignored\n")
131 continue
132 }
133
134 fmt.Printf("measurement from %s at %s : %s %s %s %s %s\n", message.HardwareSerial, timestamp, measurements[0], measurements[1], measurements[2], measurements[3], measurements[4])
135
136 temperature, err := strconv.ParseFloat(measurements[0], 64)
137
138 if err == nil {
139 temperatureGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(temperature)
140 } else {
141 fmt.Printf("error parsing temperature: %s\n", err)
142 }
143
144 humidity, err := strconv.ParseFloat(measurements[1], 64)
145
146 if err == nil {
147 humidityGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(humidity)
148 } else {
149 fmt.Printf("error parsing humidity: %s\n", err)
150 }
151
152 pressure, err := strconv.ParseFloat(measurements[2], 64)
153
154 if err == nil {
155 pressureGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(pressure)
156 } else {
157 fmt.Printf("error parsing pressure: %s\n", err)
158 }
159
160 intensity, err := strconv.ParseFloat(measurements[3], 64)
161
162 if err == nil {
163 intensityGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(intensity)
164 } else {
165 fmt.Printf("error parsing intensity: %s\n", err)
166 }
167
168 battery, err := strconv.ParseFloat(measurements[4], 64)
169
170 if err == nil {
171 batteryGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(battery)
172 } else {
173 fmt.Printf("error parsing battery: %s\n", err)
174 }
175 }
176 }
Immediately after the metrics section, the code declares a set of two-dimensional Prometheus vectors, one for each measurement: hardware_id and time are two dimensions. Therefore, when you insert data, it is keyed on the ID of the sensor in question, so you can plot time-based curves from all the sensors on one graph. As you add sensors, their data will be stored without the need to change this code. The program's entry point, main (line 58), registers these vectors with Prometheus (lines 61-65) and starts an HTTP server to serve the metrics (lines 67-71).
Prometheus will query this HTTP endpoint on a regular basis to scrape data. The rest of the code is similar to the previous version, but instead of saving data to a file, the data is written to the Prometheus vectors. Again, note that you have to supply the hardware ID along with the measurement (e.g., line 131 for temperature), so that the data ends up in the correct time series.
Once this program is started (on the same machine as Prometheus), you can query and visualize the data (Figure 9). One of the vectors is simply called temperature, so if you type that in the query box, press Execute, and then select the Graph tab, you can see measurements from three sensors for the last hour. To see a different time range, you can play with the time range slider.
Figure 9: Querying the temperature data.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
US / Canada
UK / Australia
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.