Tutorial – Obsidian

Our minds work in mysterious ways. No matter how hard we try, we cannot always direct our thoughts in an ordered succession of steps, ignoring distractions. It can be equally hard to spot useful links between apparently unrelated pieces of data or to remember all the relevant information connected to a particular issue.

Software programs attempting to make these processes more efficient can take many forms, from desktop or personal "wikis," to "knowledge bases," or "mind mappers." But their general goal is the same: to help users document and connect their ideas as efficiently as possible. This tutorial describes one of these tools, the basic version of Obsidian [1], which runs on Linux and other platforms and is free of charge for personal and educational use.

Main Features

Obsidian, describes itself as a "knowledge base," a "second brain," and a "note multiplexer." It's an Electron application for making notes that makes it easier to catalog, connect, and even publish them. Each Obsidian note is a plain text file, formatted according to the Markdown syntax [2], which is very efficient for formatting text and recording ideas, and so straightforward to use it is almost self-explanatory. In the Obsidian interface, you can view and edit as many notes as your screen and eyesight allow, each in its own pane. Each separate collection of notes (text files) is stored in a "Vault," which is just a normal folder in your filesystem.

To try out Obsidian, I used it for two separate projects. The first, seen Figures 1-7, was created from scratch and includes the initial raw material for an essay on what could be the ideal Linux distribution. The second, discussed later, is visible in Figures 8 and 9 and shows Obsidian handling a full copy of my blog.

A main graph for the "Ideal Distribution project" (Figure 1), shows how the notes are connected to each other visualized as nodes on a graph. The list of all notes that mention the current note, or explicitly link to it (backlinks) is also easily available. For these reasons, some people call Obsidian a Markdown editor with mind-mapping support. Notes can be further organized with tags and other metadata.

Figure 1: An Obsidian Vault graph shows how your different notes are connected to each other.

The first time you start Obsidian, the only existing Vault is the one that contains all the program documentation. To browse its content, click on the question mark button in the lower left corner. Each Vault you create is independent from the other Vaults, meaning that you cannot link notes across Vaults. For the same reason, each Vault gets its own, separate configuration that defines everything from which plugins it can use, to visual settings like background color and graphic theme. All this data is stored in the .obsidian subfolder inside the Vault itself.

The Obsidian documentation says that this "per-Vault" configuration "is useful, for example, if you have one Vault where you keep notes but a different one in which you do long-form writing." That makes a lot of sense, not to mention that fully self-contained Vaults are completely portable from one computer to another. Personally, however, I found it a bit annoying that it seems impossible, from the graphical interface, to import all the customizations already applied to an existing Vault into a new one. The quickest way to do that seems to be to just copy the .obsidian folder from the old Vault to the new one and then restart Obsidian to load those settings.

Obsidian's Openness

In trying out Obsidian, I was most interested in questions that should be the first concern for everyone who understands the importance of open standards and data ownership: Even when all your data remains on your computer, how much of what you do with that program is reusable outside it, and how easy is that to do? Is it possible, for example, to import or generate the initial data for that program automatically with some scripting? Can you edit the data with third-party tools? What about publishing it where and how you want?

With Obsidian, the answers to all these questions were positive, thanks to its "local folders of Markdown files" nature.

Format-wise, besides vanilla Markdown, Obsidian supports constructs for diagrams and math formulas, and the Markdown dialects called CommonMark [3] and GitHub Flavored Markdown (GFM) [4]. There is a plugin called "Markdown Format Importer" that I would not call exactly an importer, but which is useful anyway: it just replaces certain Markdown elements with the ones in the dialects it supports. Obsidian can also directly import notes from Roam Research [5] and zettelkasten-based [6] knowledge-management systems.

You can directly edit Obsidian notes with any text editor, from any platform, in any moment, and even rearrange them in subfolders with a normal file manager. Obsidian will notice the changes and import them automatically, without problems. Coupled with a file synchronization system, this makes it relatively painless to edit the content of a Vault even from smartphones, for which at time of writing there is no Obsidian app. On desktop systems, if you copy and paste text directly from a web page into an Obsidian note, the software will try to automatically convert it to Markdown! Copying the same text with browser plugins like MarkDownload [7] will even add metadata, like source URL and date, to the same note.

On the publishing side, you may export your notes as PDF files or publish them online very easily as one wiki on the Obsidian website through their paid service. However, you may never need any of those options. There are plenty of tools on Linux and any other operating system to convert Markdown files to many other formats and ways to put them online, for example with static website generators like Hugo or Jekyll.

Installation and Configuration

The free version of Obsidian is available for Linux in several package formats. On my Ubuntu desktop, Obsidian was up and running one minute after downloading the Snap package of version 0.11.0 from the website, and installing it with this command:

sudo snap install obsidian_0.11.0_amd64.snap --dangerous

The --dangerous option was necessary, at time of writing, because the package was not registered or digitally signed.

By default, Obsidian comes in two "base" themes, light and dark. The dark mode is the default, but if like me, you find it really hard to read, click on the Settings gear icon in the lower left corner, open the Appearance tab, and switch it to light. Whatever base mode you choose, you can further customize Obsidian by enabling a theme, or even creating your own with CSS rules. Activating the Custom CSS feature allows you to install any of the graphic themes provided by the Obsidian community from the configuration tab. In this tutorial, Figures 1-7 show the Obsidian Solarized theme, and Figures 8 and 9 the default Obsidian look and feel, in the light base mode.

Even before setting the appearance and location of a Vault, you may want to configure how to delete notes, because there are three distinct ways to do it. You can send deleted notes to an Obsidian-only trash folder, send them to the system trash bin, or actually delete them right away.

Most Useful Plugins

The other thing to do, ideally before starting to use Obsidian, is to figure out which plugins you may need and activate them if necessary. The configuration panel lists the official plugins, most of which are enabled by default, separately from the many more provided by Obsidian users. Before installing any of the latter, however, you must explicitly disable the Obsidian "safe mode."

After a quick look at the available plugins, I immediately installed some that I thought would likely be useful for a large number of users. I started with a calendar widget, a mind mapper, and an outliner. After a second look at the plugins list, I also activated a tool to quickly find notes not connected to any other, and a "Quick switcher," to create notes, or move from note to note, without using the mouse. There are many more possibilities, however. To mention just a few examples, there are plugins to convert notes to slide shows (with additional, but minimal Markdown formatting), save audio recordings as notes, highlight syntax in software code, or open a random note every time, to stimulate creativity and serendipity.

I would also like to mention two plugins that I am using very little myself, but may be reason enough to try Obsidian for others: Workbench, and Text {{expand}}. Workbench may be described as a special area where you may temporarily "remix" links and text snippets among notes, or from external sources, more quickly than you may do otherwise, that is in a standard Obsidian note. Text {{expand}}, instead, pastes the result of a search done with the Obsidian search function into the current note.

Working with Obsidian

To start working in Obsidian, you must create at least one Vault by clicking on the button right above the Settings one, in the left border of the main window. Vaults are just folders that can stay anywhere in your filesystem. The only thing to avoid is creating a Vault inside another Vault. However, a Vault may certainly contain subfolders to help you keep your files organized, and also easier to navigate with normal file managers.

As I said, each note gets at least one separate pane. You can pin the most important panes in fixed positions, combine them in different workspaces, or link together all the panes that contain different views (e.g. Markdown source, Mind Map, and outline) of the same note.

The text editor embedded in Obsidian is as simple as it is efficient. No menus or rows of icons, just an area to type in, with three small buttons in the top right corner: Preview, Close, and More Options. Depending on which plugins you have activated, other buttons or options may appear, for example to list or search tags.

The Preview button is necessary because the editor is not What You See Is What You Get (WYSIWYG). Click there, and Obsidian will show you, in another pane, how the text will actually look, clickable hyperlinks included, when converted to HTML or other rich text formats. Figure 2 shows the editing and preview pane of the same note, side by side.

Figure 2: Markdown editing, Obsidian style: The left pane contains the editable source of a note, and the right one shows how it will look in HTML format.

Of course, the real power of Obsidian is creating and displaying links and backlinks. Just type two left square brackets ([[). If there already are other notes in the same Vault, Obsidian will list them in a pop-up window (Figure 3), to let you select the one you want. Otherwise just type whatever you want in the brackets. The first time you click on the link to a note that doesn't exist yet, Obsidian will automatically create the corresponding file and open it in its editor. Should you change the title of that file, no problem: Obsidian will automatically update all the links to it.

Figure 3: Adding links is as simple as in all wikis, if not simpler. Type two left square brackets, and then add a new link or choose an existing node.

To issue commands inside any pane, type Ctrl+P, and an autocompleting list of available commands will appear (Figure 4). If you want to keep some notes always at hand, you can activate the "Starred notes" plugin, and all the notes you mark with a star will be listed in a dedicated panel.

Figure 4: Autocompletion also works for commands. As shown here, you can easily find and run the Mind Map generator for the current node.

Figure 5 shows the mind map generated by Obsidian for the note in Figure 2. It is clean and readable, but I was disappointed to find that, just like the outline generated by the plugin, it is static. You can click on an outline heading, or mind map element, to view the corresponding text, but you cannot drag and drop elements around to change the structure of a note. That functionality is available, but hidden among other Obsidian settings that would take more space than available to explain here.

Figure 5: The Obsidian mind map of the note shown in Figure 2.

Besides the basic formatting features, the ones that are particularly useful in a knowledge manager are dynamic task lists and footnote support. If you write something like the list shown in Listing 1, Obsidian will render it as shown in Figure 6. If you check one of the unmarked boxes, it will automatically fill the empty brackets of the item with an X. The same code sample also shows how to insert footnotes. Other markup rules let you add tables or even formulas in LaTeX format.

- [x] done
- [ ] not done
- [ ] done^[This is a footnote]
- [x] maybe done

Figure 6: This is how Obsidian renders footnotes and (clickable!) checklists.

Another very useful markup that I may use very heavily in the future is the one that embeds images or other text files in a note. If you prepend a link to a note (or image) with an exclamation mark (![[Filename]]), the Obsidian preview will replace that link with the actual content of that file.

If you intend to reuse your notes outside Obsidian with other Markdown-compatible software, you need to know that the format with the two square brackets, called "Wikilink," that Obsidian uses by default, is slightly different from the standard Markdown syntax for links, that looks like this:

[text linking to some file](location of the file)

So, if you plan to heavily reuse Obsidian notes in other Markdown systems, you may want to select Use Markdown Links in the Obsidian editor settings.

Tags, Metadata, and Aliases

No software intended to organize and connect bits of knowledge can do it just with direct links. That's why you can also organize your notes by assigning tags to them, and of course browse notes by tag, or use tags as links.

Tags are really simple to add and use: just prepend a hash character (#) to a word, and Obsidian will highlight and use it as a tag. You can search by tag and then browse the related notes or examine all the tags you are using in a dedicated pane, sorted by name or frequency (Figure 7). In this way, Obsidian also facilitates the identification and removal, of inconsistencies in your tags. For example, if you tagged almost all your notes about smartphones with the #smartphone (singular) tag and just a few with #smartphones (plural), this will be very easy to spot and fix.

Figure 7: Obsidian has powerful functions for tag management, display, and search.

I also like how Obsidian lets me organize tags hierarchically. In a Vault about free software, for example, you could have a #Linux tag, as well as one tag per distribution (e.g., #Centos, #Debian, #Ubuntu, and so on). However, writing tags separated by slashes, such as:

#Linux/Debian
#Linux/Ubuntu
#Linux/Centos

would make Debian, Ubuntu, and Centos sub-tags of Linux. Then, searching for #Centos would find just the notes about Centos, but searching for #Linux would also return all your notes about any of those distributions.

In addition to tags, Obsidian also supports metadata in a format widely used by many Markdown-processing tools, called YAML. Nothing difficult here! YAML is a recursive acronym for YAML Ain't Markup Language. What it is however, is an extremely simple way to declare all of a file's properties. Listing 2 shows what a YAML-formatted description of this tutorial might look like.

---
title: An Obsidian tutorial
date: 2021-02-28
author: M. Fioretti
tag:
- linux
- knowledge bases
- writing
- productivity
url: <actual URL here>
magazine: Linux Magazine
---

In the Markdown world, and in many other markup systems too, metadata in YAML or similar formats is called "frontmatter," because it is always placed at the very beginning of a file, delimited by two lines, each containing just three dashes. When a Markdown application finds a YAML key that it does not recognize inside frontmatter, it just ignores it. This is good, because it makes the format extremely customizable without breaking compatibility. Obsidian exploits this flexibility with an aliases key. If you place a key like this:

aliases: ["Internet of Things", IoT]

in the frontmatter of a note titled "What is the Internet of Things?" then Obsidian will understand that every occurrence of strings like [[Internet of Things]] or [[IoT]] in other notes is a link to that specific note, even if you change its title afterwards. Very convenient, isn't it? In practice, the only problems with YAML metadata happen if you must use two Markdown applications that do not parse the same key in exactly the same way. This, as I will show in a moment, is a problem you may encounter with Obsidian.

New Insights into Past Work

Here is the other issue I wondered about when I came across Obsidian: Taking new notes for new projects is all well and good, but what about past work? Can Obsidian give me more insights than I could discover myself about my previous writing activities? That is, can Obsidian help me to write more efficiently or to find new uses for my past work?

Right now, I still do not have a final answer to these questions. But I already know enough to believe that the answers may be at least partially positive.

As of February 2021, my main blog [8] contains over 1,200 posts, written over 13 years. The posts contain more than 700K words total, and are published online with the Hugo static site generator [9]. Each post is a Markdown file, with tags and other metadata in its own frontmatter, and all links among posts are relative URLS (that is, the URL of the About page is just /about, without the full domain name).

After practicing with the Ideal Linux Distro Vault, I copied all the Markdown files of that blog in another folder, told Obsidian to open it as Vault, and began to explore the result. On one hand, I was pleased to see that Obsidian correctly recognized all the links between my posts, as well as most of the tags in the frontmatter. On the other hand, I was disappointed to find that Obsidian and Hugo do not parse frontmatter tags in exactly the same way. In Hugo, a frontmatter line like this:

- open source

defines one single tag, open source. Obsidian, instead does recognize such lines as tags, but seems to stop parsing at the first whitespace, thus treating tags like open source or open hardware as if they all were the same tag, called open (Figure 8). Enclosing words between quotes seemed to make no difference. For other users, this may be a totally irrelevant issue. For me, so far this mismatch has made analyzing my tags with Obsidian much less effective than it could be. But it's just plain text files, remember? Should I find that I really need it, I could probably patch the problem with some script that automatically removes all the spaces inside tags.

Figure 8: Obsidian recognizes the tags presents in files generated with other tools … as long as they don't contain spaces!

Apart from tags, I have found the way Obsidian shows me my own past work intriguing, at a minimum, and potentially very useful, even if it requires time. Figure 9 shows what my whole corpus of more 1,200 blog posts looks like through Obsidian. Some groups of posts (e.g. those numbered 1 to 5 in Figure 9) rightly appear as mostly independent clusters. (Number 6 does not really matter, because it is the cluster of all the posts that link to the About page of the whole blog.)

Figure 9: More than 1,200 blog posts, all connected in one zoomable graph!

Those five clusters, in fact, are long essays that at some point I republished on the blog after splitting them into multiple short posts, with an "index" post that is now the hub of the cluster. Further study of the Obsidian graph will help me to see if those clusters should also be linked from other nodes. Then there are many posts without any link to or from the others. Obsidian can list them in a special pane, but the graph (see number 7 in Figure 9) really makes them stand out. The last marker in Figure 9 indicates the graph menu, which offers many options to customize the graph. Hopefully, those graph visualization options, combined with the backlink listing function of Obsidian, will give me many suggestions to make my blog more interesting, and more useful, for all its readers.

Conclusions

Let's be clear: It takes a lot of self-discipline to really take advantage of any tool like Obsidian, but if you can gather that discipline, it is really worth it. While this software is not open source, it is "open" in the sense explained above (i.e., highly interoperable with other tools and without lock-ins). For these reasons, plus its graphic presentation, link management, and tagging functions, Obsidian can be quite helpful for anyone who has a lot of mainly textual material to record, organize, and reuse with the smallest possible effort, and that doesn't mean just students, academics, and other professionals of the written word, from lawyers to journalists. I half suspect, half hope, that Obsidian can really help to discover what an essay I read calls "the adjacent possible" [10].

Its web page says that "Obsidian works better if you have large screens and atomic short notes." As true as this is, surely it is not the only valid setting for Obsidian.

Personally, I will continue to test Obsidian as an organizer and assistant for preparing ebooks and other long-form texts or data catalogs. But the best way for you to use Obsidian depends on how your own brain works and on the kind of material you need to organize and analyze. I encourage everybody to try Obsidian to discover what their way may be.

The Author

Marco Fioretti (http://mfioretti.com) is a freelance author, trainer, and researcher based in Rome, Italy. He has been working with free/open source software since 1995 and on open digital standards since 2005. Marco also is a Board Member of the Free Knowledge Institute (http://freeknowledge.eu) and blogs about digital rights at https://stop.zona-m.net.