Distributed weather monitoring in gardens
The Things Network
The Things Network provides a public network server for LoRaWAN applications, registration services for LoRaWAN nodes (sensors), security key generation, packet examination facilities, and, most important, an API for data extraction for several languages. Use of this service is free, but by default, The Things Network does not store your data. If you don't extract your data in a timely manner, it is simply discarded.
The first step is to register your gateway with The Things Network in the gateway console [16]. You can give your gateway any name you like (Figure 6) and choose to make it public, so other people can use it, or private, for your own use only. All data is strongly encrypted, so it is secure, even on a public gateway.
Figure 6: Gateway console and application registration in The Things Network console.
The next step is to register your application. The application unique ID (EUI) that is generated must be added to the sensor's code for the device to be considered for registration.
Finally, you can register one or more devices with a name you select by selecting register device in the Devices pane: I call the devices for this application weatherstation1, weatherstation2, …, and so on. Devices on The Things Network must have a unique ID (in the case of my microprocessor, the EUI is a unique 64-bit number programmed by the manufacturer into its flash memory), which is entered in the Device EUI field on the registration page (Figure 7).
Figure 7: The settings for weatherstation1.
Once registered, the generated application key must be added to the device's code, as well. When the device is powered up, it sends a registration request with a tuple of device EUI, application EUI, and application key.
To view the data, use the Data tab (Figure 8). The payload (application data) has been decrypted and is displayed in hexadecimal format. In this case, the data is a comma-separated list of five floating-point numbers representing temperature, humidity, pressure, intensity, and battery voltage. The metadata provides some details about the packet transmission, the most useful of which is timestamp, representing the time at which the packet was received.
Figure 8: Data view.
Extracting the Data
I chose to write my client for The Things Network in golang, although SDKs for many other languages are provided, as well [17]. Data is made available through an MQTT protocol [18], and an SDK for golang is provided. A simple example and GoDoc is provided online [19]. Only a few lines of golang subscribes to the service, lists the registered devices, and waits for data.
In the first example (Listing 1), I just access my data in The Things Network and write it to a file. The program must provide the application ID and the application access key (lines 24 and 25) to register successfully with The Things Network. Note that the time stamp is extracted from the metadata, which will be important when it comes time to plot the data as time series graphs. To make the data truly useful, however, you need to store it in something more sophisticated than a flat file.
Listing 1
ttn-client.go
01 package main
02
03 import (
04 "os"
05 "fmt"
06 "time"
07 "strings"
08 ttnsdk "github.com/TheThingsNetwork/go-app-sdk"
09 )
10
11 const (
12 sdkClientName = "lora-weather-station"
13 )
14
15 func main() {
16
17 // open a file to write results to
18 f, err := os.OpenFile("weather.dat", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
19 if err != nil {
20 fmt.Println(err)
21 }
22
23 defer f.Close()
24
25 appID := "loramini2"
26 appAccessKey := "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx"
27
28 config := ttnsdk.NewCommunityConfig(sdkClientName)
29
30 // create a client to 'the things network. the access key has been obscured
31 client := config.NewClient(appID, appAccessKey)
32 defer client.Close()
33
34 devices, err := client.ManageDevices()
35 if err != nil {
36 fmt.Printf("lora-weather-station: could not get device manager\n")
37 }
38
39 // create a list of known devices
40 deviceList, err := devices.List(10, 0)
41 if err != nil {
42 fmt.Printf("lora-weather-station: could not get devices\n")
43 }
44 fmt.Printf("lora-weather-station: found device(s)\n")
45 for _, device := range deviceList {
46 fmt.Printf("%s\n", device.DevID)
47 }
48
49 // subscribe to events from all the devices
50 pubsub, err := client.PubSub()
51 if err != nil {
52 fmt.Printf("lora-weather-station: could not get application pub/sub\n")
53 }
54
55 defer pubsub.Close()
56
57 allDevicesPubSub := pubsub.AllDevices()
58 defer allDevicesPubSub.Close()
59 uplink, err := allDevicesPubSub.SubscribeUplink()
60 if err != nil {
61 fmt.Printf("lora-weather-station: could not subscribe to uplink messages")
62 }
63
64 // wait for data from each device and save it to the file. loop forever
65 fmt.Printf("waiting for data\n")
66 for message := range uplink {
67 timestamp := time.Time(message.Metadata.Time).Unix()
68 measurements := strings.Split(string(message.PayloadRaw), ",")
69
70 if len(measurements)<4 {
71 fmt.Printf("lora-weather-station: uplink message malformed, ignored")
72 continue
73 }
74
75 data := fmt.Sprintf("%s %d %s %s %s %s %s\n", message.HardwareSerial,
76 timestamp,measurements[0],measurements[1],measurements[2],measurements[3],,measurements[4])
77
78 fmt.Printf(data)
79
80 if _, err := f.WriteString(data); err != nil {
81 fmt.Printf("lora-weather-station: could not write data")
82 }
83 }
84 }
Prometheus
Prometheus is a free and open source multidimensional time series database for event monitoring and alerting that runs on Linux. You can find many excellent tutorials about Prometheus installation online, so you should choose one that is most appropriate for your distro. (For my Ubuntu setup, I chose a video tutorial on the LinOxide website [20].) Once installed, you should be able to browse to the Prometheus home page. However, to be of real use, you need to be able to insert data into Prometheus, so it's back to the golang code to add Prometheus metrics (Listing 2).
Listing 2
ttn-client-prom.go
001 package main
002
003 import (
004 "flag"
005 "fmt"
006 ttnsdk "github.com/TheThingsNetwork/go-app-sdk"
007 "github.com/prometheus/client_golang/prometheus"
008 "github.com/prometheus/client_golang/prometheus/promhttp"
009 "net/http"
010 "strconv"
011 "strings"
012 "time"
013 )
014
015 // create a bunch of prometheus vectors, one for each measurement
016 var (
017 temperatureGauge = prometheus.NewGaugeVec(
018 prometheus.GaugeOpts{
019 Name: "temperature",
020 Help: "temperature reported by a weather station over time in degrees Celcius",
021 },
022 []string{"hardware_id"})
023
024 humidityGauge = prometheus.NewGaugeVec(
025 prometheus.GaugeOpts{
026 Name: "humidity",
027 Help: "humidity reported by a weather station over time in percent",
028 },
029 []string{"hardware_id"})
030
031 pressureGauge = prometheus.NewGaugeVec(
032 prometheus.GaugeOpts{
033 Name: "pressure",
034 Help: "barometric reported by a weather station over time",
035 },
036 []string{"hardware_id"})
037
038 intensityGauge = prometheus.NewGaugeVec(
039 prometheus.GaugeOpts{
040 Name: "intensity",
041 Help: "light intensity reported by a weather station over time",
042 },
043 []string{"hardware_id"})
044
045 batteryGauge = prometheus.NewGaugeVec(
046 prometheus.GaugeOpts{
047 Name: "battery",
048 Help: "battery voltage reported by a weather station over time",
049 },
050 []string{"hardware_id"})
051
052 addr = flag.String("prom-server-address", ":8080", "the address of the server for prometheus metrics")
053
054 appID = "loramini2"
055 appAccessKey = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
056 sdkClientName = "lora-weather-station"
057 )
058
059 func main() {
060
061 flag.Parse()
062
063 // register the prometheus vectors
064 prometheus.MustRegister(temperatureGauge)
065 prometheus.MustRegister(humidityGauge)
066 prometheus.MustRegister(pressureGauge)
067 prometheus.MustRegister(intensityGauge)
068 prometheus.MustRegister(batteryGauge)
069
070 // publish the prometheus vectors to the premetheus can 'scrape' them for recording
071 // this starts a mini http server
072 go func() {
073 http.Handle("/metrics", promhttp.HandlerFor(
074 prometheus.DefaultGatherer,
075 promhttp.HandlerOpts{},
076 ))
077
078 fmt.Printf("starting prometheus server on %s\n", *addr)
079
080 err := http.ListenAndServe(*addr, nil)
081 if err != nil {
082 fmt.Printf("unable to start prometheus server\v")
083 }
084 }()
085
086 // connect to 'the things network'
087 config := ttnsdk.NewCommunityConfig(sdkClientName)
088
089 client := config.NewClient(appID, appAccessKey)
090 defer client.Close()
091
092 devices, err := client.ManageDevices()
093 if err != nil {
094 fmt.Printf("could not get device manager\n")
095 }
096
097 // List the first 10 devices
098 deviceList, err := devices.List(10, 0)
099 if err != nil {
100 fmt.Printf("could not get devices\n")
101 }
102 fmt.Printf("found device(s)\n")
103 for _, device := range deviceList {
104 fmt.Printf("%s\n", device.DevID)
105 }
106
107 fmt.Println()
108
109 // subscribe to events from the devices
110 pubsub, err := client.PubSub()
111 if err != nil {
112 fmt.Printf("could not get application pub/sub\n")
113 }
114
115 defer pubsub.Close()
116
117 // listen for measurement events and update the prometheus counters. loop forever
118 allDevicesPubSub := pubsub.AllDevices()
119 defer allDevicesPubSub.Close()
120 uplink, err := allDevicesPubSub.SubscribeUplink()
121 if err != nil {
122 fmt.Printf("could not subscribe to uplink messages\n")
123 }
124 fmt.Printf("waiting for data\n")
125 for message := range uplink {
126 timestamp := time.Time(message.Metadata.Time).Format(time.RFC3339)
127 measurements := strings.Split(string(message.PayloadRaw), ",")
128
129 if len(measurements) < 4 {
130 fmt.Printf("uplink message malformed, ignored\n")
131 continue
132 }
133
134 fmt.Printf("measurement from %s at %s : %s %s %s %s %s\n", message.HardwareSerial, timestamp, measurements[0], measurements[1], measurements[2], measurements[3], measurements[4])
135
136 temperature, err := strconv.ParseFloat(measurements[0], 64)
137
138 if err == nil {
139 temperatureGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(temperature)
140 } else {
141 fmt.Printf("error parsing temperature: %s\n", err)
142 }
143
144 humidity, err := strconv.ParseFloat(measurements[1], 64)
145
146 if err == nil {
147 humidityGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(humidity)
148 } else {
149 fmt.Printf("error parsing humidity: %s\n", err)
150 }
151
152 pressure, err := strconv.ParseFloat(measurements[2], 64)
153
154 if err == nil {
155 pressureGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(pressure)
156 } else {
157 fmt.Printf("error parsing pressure: %s\n", err)
158 }
159
160 intensity, err := strconv.ParseFloat(measurements[3], 64)
161
162 if err == nil {
163 intensityGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(intensity)
164 } else {
165 fmt.Printf("error parsing intensity: %s\n", err)
166 }
167
168 battery, err := strconv.ParseFloat(measurements[4], 64)
169
170 if err == nil {
171 batteryGauge.With(prometheus.Labels{"hardware_id": message.HardwareSerial}).Set(battery)
172 } else {
173 fmt.Printf("error parsing battery: %s\n", err)
174 }
175 }
176 }
Immediately after the metrics section, the code declares a set of two-dimensional Prometheus vectors, one for each measurement: hardware_id and time are two dimensions. Therefore, when you insert data, it is keyed on the ID of the sensor in question, so you can plot time-based curves from all the sensors on one graph. As you add sensors, their data will be stored without the need to change this code. The program's entry point, main (line 58), registers these vectors with Prometheus (lines 61-65) and starts an HTTP server to serve the metrics (lines 67-71).
Prometheus will query this HTTP endpoint on a regular basis to scrape data. The rest of the code is similar to the previous version, but instead of saving data to a file, the data is written to the Prometheus vectors. Again, note that you have to supply the hardware ID along with the measurement (e.g., line 131 for temperature), so that the data ends up in the correct time series.
Once this program is started (on the same machine as Prometheus), you can query and visualize the data (Figure 9). One of the vectors is simply called temperature, so if you type that in the query box, press Execute, and then select the Graph tab, you can see measurements from three sensors for the last hour. To see a different time range, you can play with the time range slider.
Figure 9: Querying the temperature data.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
US / Canada
UK / Australia
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs