Many home automation devices can be controlled through WiFi, but often these devices have limitations. For example, they might only work through the manufacturer's cloud service, they might be difficult to integrate with your own home automation system if you prefer to do everything local, they might lack advanced functionality, or they might be difficult to update.

Luckily, you can install alternative firmware on many existing or homemade devices. In this article, I introduce you to ESPHome [1], which supports numerous devices with an ESP32, ESP8266, or RP2040 microcontroller (the chip in the popular Raspberry Pi Pico W), although ESPHome support for the RP2040 is still in development. In the examples in this article, I'll use the Raspberry Pi Pico W. However, if you encounter any issues with your own projects, I recommend an ESP32 development board.

With ESPHome, you can create your own home automation devices with a supported microcontroller board that you connect to LEDs, sensors, or switches. What sets ESPHome apart from other solutions like Arduino [2] or MicroPython [3] is that you don't need to program. Instead, you configure which components are connected to which pins on the board. ESPHome then generates the necessary C++ code and compiles it into firmware that you can install on the device (see also the "Replacing Firmware on Commercial Devices" box).

Replacing Firmware on Commercial Devices

You can replace the existing firmware on commercial devices with ESPHome to gain full control over a device and use it in ways that the manufacturer hasn't anticipated. Two popular brands that have easy-to-flash devices are Shelly [4] and Sonoff [5]. A website [6] hosts more than 300 ESPHome device configuration templates that can help you get the most out of them. Note that often you'll need special hardware to flash your own firmware to these devices, at least the first time – afterward you can update them through WiFi. You'll need a USB-to-TTL adapter and to connect the pins of the adapter to the appropriate GPIO pins on the device. This isn't always a straightforward process (Figure 1).

Figure 1: Crocodile clips and cut resistor leads saved the day when I wanted to flash ESPHome to this Shelly RGBW2 WiFi LED controller.

Installing ESPHome

ESPHome is a Python program, and most Linux distributions already have Python installed by default. You should first confirm that you have at least version 3.9 installed, by running the command

$ python --version
Python 3.9.15

If your Python version is older, consider upgrading your distribution, or deploy the ESPHome Docker image [7].

If the Python version looks good, create a virtual environment to contain ESPHome and its dependencies:

$ python -m venv esphome_venv
$ source esphome_venv/bin/activate

Once you're in the Python virtual environment, install the ESPHome package from PyPI:

$ pip install esphome

After the installation is complete, enter

$ esphome version
Version: 2023.6.5

to confirm that ESPHome has been installed successfully.

Creating a Project with the Dashboard

A directory in which you store all of your ESPHome projects is recommended. Suppose you call this directory config. To start the ESPHome dashboard and point it to this directory, run:

$ esphome dashboard config/

This command starts a web server on http://0.0.0.0:6052, which you should be able to open in your web browser. If you already have ESPHome devices on your network, the dashboard will automatically discover them.

Next, click New Device at the bottom right corner, and then Continue. Give your device a name and enter the SSID and password for the WiFi network to which you want your device to connect, then click Next and select your device type (Figure 2).

Figure 2: Select the type of device on which to install ESPHome.

In this example, choose Raspberry Pi Pico W; for an ESP32 or ESP8266 you also need to select the specific board. The dashboard then creates a minimal configuration and shows an encryption key that you can use to allow the ESPHome device to communicate with Home Assistant [8], a popular open source home automation gateway developed by the same team behind ESPHome. Finally, click Install.

You can use several methods to install ESPHome to your device, but not all of them are supported by every device. Because no ESPHome firmware is running on the device yet, the first method (over WiFi) is not possible; the Plug into the computer running ESPHome Dashboard choice isn't available either. You can always choose Manual download, which has instructions on how to accomplish the installation (Figure 3).

Figure 3: The ESPHome dashboard has excellent instructions for every step of the installation.

For the Raspberry Pi Pico W, you'll need to disconnect the board from USB, hold the BOOTSEL button while reconnecting the board, and then release the button, which causes a USB drive named RPI-RP2 to appear in your file manager. Now, click Download project and drag the .uf2 file to the USB drive. Once the drive disappears, the board runs your ESPHome firmware, and you can click Close.

Default ESPHome Configuration

In the ESPHome dashboard, click Edit in the box representing your device to open your device configuration in a web editor. The configuration file is written in YAML [9], with various key-value pairs for different options (Listing 1).

esphome:
  name: linuxmag
  friendly_name: linuxmag
rp2040:
  board: rpipicow
  framework:
    # Required until https://github.com/platformio/platform-raspberrypi/pull/36 is merged
    platform_version: https://github.com/maxgerhardt/platform-raspberrypi.git
# Enable logging
logger:
# Enable Home Assistant API
api:
  encryption:
    key: "7wFO19sSMAkGX008l+wX4u53hBz/a1Ha+9bAdouUjo8="
ota:
  password: "20e3778465f1c5b147f8645dc237b146"
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  # Enable fallback hotspot in case wifi connection fails
  ap:
    ssid: "Linuxmag Fallback Hotspot"
    password: "DVaDAPFJN5cA"

As you can see, this configuration file sets the device name and its friendly name, as well as the platform and board. It then enables logging and the Home Assistant API, sets a password for over-the-air (OTA) updates, and configures WiFi credentials and a fallback hotspot in case the WiFi connection fails. If a failure happens, you can connect with your mobile phone to the hotspot of the device to reconfigure the network. The WiFi credentials are stored in a separate file, secrets.yaml, which prevents accidental exposure of sensitive information when sharing your device configuration with others.

Note that if you don't use Home Assistant, you should remove the api line and the two lines that follow; otherwise, your ESPHome device keeps waiting for a connection from Home Assistant. If no connection is established within 15 minutes, the device will assume that something's wrong and reboot.