With all the discussion about the application of artificial intelligence (AI) in cybersecurity, we are reminded that criminals are paying close attention to AI's advances. New functionality identified by British researchers [1] involves training a deep learning model to listen in on the acoustic sounds made by keyboards when a user is typing. The model then records the audio from the typing and determines what was typed. Applications include recording users logging in to sensitive online accounts or entering payment details.

However, this type of attack does not require AI to do damage. Keylogging tools already exist that can listen in on your typing. While it might sound paranoid, you might be surprised how advanced such tools have become, even without machine learning (ML) removing much of the required "training" time for an acoustic keylogger to fully recognize keyboard sounds.

To get you up to speed on keylogging, I will explain how keylogging works and look at some of the tools currently available on Linux.

What's All the Fuss?

Popularized in movies, the logging of keystrokes often involves malware being installed on a target machine with a USB drive. Once installed, anything typed on the keyboard attached to the infected computer is saved and forwarded to the attacker, giving them access to passwords, credit card numbers, bank account information, and more. Of course, today the malware payload can be just as easily delivered by unwelcome JavaScript unsuspectingly executed by the browser when you visit an infected web page.

There are some legitimate (though contentious) uses of this technology. For example, parents might monitor their child's tablet usage or a corporate employer might keep tabs on an employee's computer usage.

A recent article on the Bleeping Computer website [1] regarding the British deep learning acoustic attack study makes two fascinating points. Firstly, the study outlines the baseline where a training algorithm receives enough training data to recognize the sound of each keystroke. Bleeping Computer noted: "The researchers gathered training data by pressing 36 keys on a modern MacBook Pro 25 times each and recording the sound produced by each press." Devices such as phones, or anything with a reasonable quality microphone (also infected by malware, most likely) are used for the recording. The study also used videoconferencing software (specifically Zoom) to record keystrokes when attendees logged into various accounts during the meeting.

Secondly, when presented with the above training data, the AI's success rate was incredible. Overall the success rate was a staggering 95 percent. Zoom calls achieved a 93 percent success rate and Skype managed 91.7 percent accuracy, according to the Bleeping Computer article.

Keyloggers can be deployed in many different ways. For instance, Endpoint Detection and Response (EDR) technology was found to have missed the presence of BlackMamba keylogging malware. According an article in Dark Reading [2], such an attack "demonstrates how AI can allow the malware to dynamically modify benign code at runtime without any command-and-control (C2) infrastructure, allowing it to slip past current automated security systems that are attuned to look out for this type of behavior to detect attacks."

The Dark Reading article concludes that without extensive research combined with effort from the security industry, solutions will struggle to keep us secure.

Now that your fight-or-flight senses are tingling suitably, I will show you some tools in action.

Can't Hear You

Before looking at acoustic keylogging tools, I'll cover a non-acoustic keylogger, logkeys [3], to show how older tools work as well as some of the jigsaw pieces involved with keyloggers. The logkeys tool records all common character and function key presses as well as recognizes the Shift and AltGr modifiers.

To use logkeys, you first need to install the build tools required on Debian Linux derivatives (such as Ubuntu Linux):

$ apt install build-essential autotools-dev autoconf kbd

Next, you need to clone the repository and move into the logkeys directory with the following command:

$ git clone https://github.com/kernc/logkeys.git
$ cd logkeys

You are now ready to create some build files with the following script and enter the directory:

$ ./autogen.sh
$ cd build

Then, you need to check that the build environment is sound before compiling the logkeys software with the following command (note the two dots):

$ ../configure

Finally, you need to compile logkeys for your system as follows, before installing it:

$ make
$ make install

The command compiles with ease on my Ubuntu Linux 22.04 laptop (if you want to remove logkeys, see the "Uninstalling logkeys" box). In order to check that the logkeys binary is available to my user's path (or the root user in this case), I start typing the logkeys command and then tab-complete it as shown in Listing 2. Happily, the logkeys help file output appears as hoped.

$ make uninstall
Making uninstall in src
make[1]: Entering directory '/root/logkeys/build/src'
  ( cd '/usr/local/bin' && rm -f logkeys llk llkk )
make[1]: Leaving directory '/root/logkeys/build/src'
[...]

$ logkeys --help
Usage: logkeys [OPTION]...
Log depressed keyboard keys.
  -s, --start          start logging keypresses
  -m, --keymap=FILE    use keymap FILE
[...]

Now I will run a test to see if I can get logkeys to work using instructions from the documentation [4]. For this test, I will need two terminals. In the first terminal, I will move into the /tmp directory to keep the root user's home directory tidy and then create an empty logfile with the following commands:

$ cd /tmp
$ echo "" > watching_you.log

Next I will start logging output with:

$ logkeys --start --output watching_you.log

Then, in the second terminal window, I'll move into the /tmp directory and follow the output with the tail command using:

$ tail -f watching_you.log

Listing 3 shows that something is recorded from each keystroke.

Logging started ...
2023-08-13 12:58:05+0100 > <LShft><LCtrl>E<LAlt><Tab>
2023-08-13 12:58:13+0100 > <Enter><Up><LShft>Xu  <BckSp>v yx [ ]q?ux? eqwcyx[g  yx?u v <LShft>Yjgg cuq <BckSp><BckSp>#q yxeu esq <LShft>"neci<LShft>" ?ywq?euwr [x? e[yg esq gua aygqv<BckSp><BckSp><BckSp><BckSp><BckSp>?ygq [] ]u<LShft>H
2023-08-13 12:58:41+0100 > <Enter>
2023-08-13 12:58:41+0100 > <Enter><LShft>$ e[yg -<BckSp>?? neci
2023-08-13 12:58:52+0100 > <Enter>?<BckSp>e[yg -?  [<Tab>?<Tab>
2023-08-13 12:58:56+0100 > <Enter><LCtrl><LShft>?
[...]

You'll notice that Listing 3 isn't very easy to read thanks to the fact that I use a UK keyboard. If you use a standard US keyboard, then the following command should work for you:

$ logkeys --start --us-keymap --output watching_you.log

I need to stop logkeys in order to change the keyboard mapping. During testing, I used the pkill command to stop logkeys while it was running; there's almost certainly a more graceful way of stopping the daemon however.

For those not familiar with pkill, it's a simple route to take instead of using the kill command. Be very careful how you use it as the root user. It essentially saves time spent looking up a process's PID to terminate it. Its purpose is to match the human-readable name of a process before stopping it ungracefully. For more information on pkill, run man pkill.

For logkeys, simply enter:

$ pkill logkeys

Once stopped, I use the UK keyboard layout with logkeys via (note the -m switch):

$ logkeys --start --output /tmp/watching_you.log -m /root/logkeys/keymaps/en_GB.map

The keyboard layouts (shown in Listing 4) are included in /root/logkeys/keymaps, so you don't need to customize them.

ca_FR.map cs_CZ.map de_CH.map de.map en_GB.map en_US_dvorak.map en_US_ubuntu_1204.map es_AR.map es_ES.map fr_CH.map
fr-dvorak-bepo.map fr.map hu.map it.map no.map pl.map pt_BR.map pt_PT.map ro.map ru.map sk_QWERTY.map sk_QWERTZ.map sl.map sv.map tr.map

Using the UK keyboard layout, Listing 5 now displays actual typed text instead of gobbledygook. It's a little scary how accurate logkeys is if you look through the logfile for unusual key combinations, that you don't realize you regularly use.

chris@Xeo:/tmp$ tail -f watching_you.log
Logging started ...
2023-08-13 13:35:23+0100 > <PgUp>
2023-08-13 13:35:29+0100 > <Enter><LShft>I am watching you very closely
2023-08-13 13:35:42+0100 > <Enter>

I will leave you to experiment with logkeys. If you want to compare features with other keyloggers, see lkl [5] and uberkey [6].

An important takeaway: Keyboard layout can be important to more traditional keyloggers as well as the logfile setup.

Sorry, I Missed That

Acoustic keylogging, which is a form of a side-channel attack [7], uses the audio signal to determine what the user is typing. Shoyo Inokuchi created acoustic-keylogger [8] as part of his undergraduate studies and it offers a good way to see how tools like this work.

Although Inokuchi's GitHub repo hasn't been updated for three years, the slick installation process (I chose the Docker route) worked flawlessly. However, I wasn't sure how to view and analyze the results afterwards.

To install acoustic-keylogger (which takes about 594MB of disk space), enter:

$ git clone https://github.com/shoyo/acoustic-keylogger.git
Cloning into 'acoustic-keylogger'...
[...]
$ cd acoustic-keylogger/

To use Docker Compose, you need to install it as follows:

$ apt install docker-compose

which then installs the following new packages:

bridge-utils containerd docker-compose docker.io pigz python3-attr python3-docker python3-dockerpty python3-docopt python3-dotenv python3-jsonschema python3-pyrsistent python3-texttable python3- websocketrunc ubuntu-fan

This added another 295MB of disk space. You also need to check that Docker Engine installed successfully with:

$ apt install docker.io

Now, you are ready to run the Docker Compose build command, whose output is shown in Listing 6. This installs all of env's dependencies (i.e., Jupyter, Tensorflow, NumPy, etc.) and mounts your local filesystem alongside the env Docker container filesystem. After running the build command, you need to confirm that two container images are present as shown in Listing 7.

$ docker-compose build
[...]
     ???????????????????????????????????????? 9.7/9.7 MB 9.8 MB/s eta 0:00:00
Collecting psycopg2-binary==2.8.6
  Downloading psycopg2_binary-2.8.6-cp38-cp38-manylinux1_x86_64.whl (3.0 MB)
     ???????????????????????????????????????? 3.0/3.0 MB 9.3 MB/s eta 0:00:00
Collecting pytest==6.2.2
  Downloading pytest-6.2.2-py3-none-any.whl (280 kB)
     ??????????????????????????????????????? 280.1/280.1 kB 3.1 MB/s eta 0:00:00
Collecting scikit-learn==0.24.1
  Downloading scikit_learn-0.24.1-cp38-cp38-manylinux2010_x86_64.whl (24.9 MB)
     ???????????????????????????????????????? 24.9/24.9 MB 9.0 MB/s eta 0:00:00
Collecting scipy==1.6.0
  Downloading scipy-1.6.0-cp38-cp38-manylinux1_x86_64.whl (27.2 MB)
     ???????????????????????????????????????? 27.2/27.2 MB 9.5 MB/s eta 0:00:00
[...]

$ docker images
REPOSITORY               TAG       IMAGE ID       CREATED         SIZE
acoustic-keylogger_env   latest    3608317074d5   4 minutes ago   4.06GB
python                   3.8       d114ab2cf5bc   3 weeks ago     997MB

Then you bring up the database, along with the Python environment, as shown in Listing 8. The Python environment makes use of the Jupyter Notebook [9]. Listing 9 shows additional output notes for accessing the Jupyter Notebook.

$ docker-compose up
Creating network "acoustic-keylogger_default" with the default driver
Pulling db (postgres:11)...
11: Pulling from library/postgres
bff3e048017e: Pull complete
[...]

env_1  |     To access the notebook, open this file in a browser:
env_1  |         file:///root/.local/share/jupyter/runtime/nbserver-1-open.html
env_1  |     Or copy and paste one of these URLs:
env_1  |         http://09815db9c0c3:8888/?token=c55389826f2c1a66819428bad3e6d75a9f91eda5deccded7
env_1  |      or http://127.0.0.1:8888/?token=c55389826f2c1a66819428bad3e6d75a9f91eda5deccded7

To keep the environment up and running, you must leave the terminal open and untouched. The final URL in Listing 9 shows the web interface (Figure 1).

Figure 1: The dashboard courtesy of Jupyter.

To run tests, make sure that you are still in the cloned repository directory (acoustic-keylogger/) and use the following command:

$ docker-compose run env pytest -q tests

Figure 2 shows the output, with some user input at the top (where I randomly typed keys). Incidentally, my CPU load went up and fans started whirring on my old (8-core CPU) laptop when running this command.

Figure 2: acoustic-keylogger listens for keystrokes.

I looked around the container's filesystem and in Jupyter but couldn't find where the analysis of the data was stored. Inokuchi states that reading the external documentation isn't a priority for using acoustic-keylogger, but it's relatively clear that the process is working as expected.

In the process of using acoustic-keylogger, I discovered that the way that keys are depressed and how keys spring back affects how the audio is captured. According to Inokuchi's research, the sounds emitted by the keys can be clustered by their position on the keyboard. Figure 3 shows the results for a MacBook Pro 2016.

Figure 3: Keystroke sounds generated by a MacBook Pro 2016 (source: https://github.com/shoyo/acoustic-keylogger).