Hobby Time

Everyone who works professionally with Linux is used to building software from the source code, perhaps implementing small changes, automating routine work with shell scripts, or developing their own software in one of the many current programming languages. Tinkering with your own, completely new operating system, on the other hand, is a pretty unusual pastime. If you start from scratch, it will take a long time before your system is useful for anything.

If you are looking for a challenging amateur project (for example, if you are a computer science student) or you want a better understanding of the theoretical basics of interrupts, memory management, scheduling, and other OS features, working on your own kernel can provide valuable insights. Linus Torvalds actually created Linux through a similar tinkering project. In 1991, he posted in a Minix news group, "I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones." [1]

A development environment for an operating system is more complex than one for an application, because you cannot simply compile the source code and run it on a trial basis. Instead, you need to create a bootable disk that can be used to boot a VM or emulated PC. It makes the work easier if there are debugging possibilities. The Qemu [2] and Bochs [3] emulation tools have proven useful for building a virtual environment.

One way to create a bootable disk with as little overhead as possible (and allow it to receive regular updates) is to use a FAT-formatted disk image with the Grub 1 boot manager [4] (Figure 1).

Figure 1: The easily configurable Grub 1 boot manager doesn't just boot Linux.

The process of building an operating system is quite rigorous and fills up entire books. This article is intended as a roadmap for taking your first steps. Needless to say: the task of building an operating system is not for beginners. You will need some background in computer science and programming to undertake this kind of a project. But even if you are new to the subject, this article will give you some insights into how developers think about operating systems and the OS programming environment.

If you're ready for the journey, I will introduce some tools that can help you with building your hobby OS. I have also collected online sample files and links to more detailed explanations of the theoretical principles [5].

Basics

The classic programming language for operating system development is C, not C++ or C#, but the old procedural C. In addition, some parts have to be written in assembler, either in Intel or AT&T syntax [6].

Parallel to the operating system, a collection of test and utility programs must be created to test newly developed OS features; you'll also have to create a shell (at least a rudimentary one). It is not a good idea to program a graphical user interface before the most important basic functions around process management and the file system are stable.

"Process management" already implicitly assumes that the operating system will support multitasking – but this is not mandatory; even building an MS-DOS clone can be an informative exercise.

Although you can give free rein to your creativity when designing and implementing your own operating system, there are some places where the developer has to pedantically implement the descriptions from standards documents. Two examples are storage management and file systems.

Segments and Pages

In order to implement the modern memory management method of paging [7] on a 32-bit Intel processor, it is mandatory to first activate the older method of segmentation [8]. The Intel documentation describes how to define segments in the Global Descriptor Table (GDT) and then switch the processor into Protected Mode and persuade it to use this table. Only then can a two-level page table be created and activated.

Figure 2 shows the structure of an eight-byte (64-bit) entry in the GDT. In addition to any special options, two important values are stored here: the 32-bit base address as the start address of a segment and the 20-bit limit, from which the size of the segment is calculated. However, these values are in a format that takes getting used to: limit and base address are interrupted by other attributes. This has historical reasons: the Global Descriptor Table (GDT) used to define the characteristics of the memory area used in program execution was created from a smaller data structure of older Intel CPUs.

Figure 2: Many data structures in the kernel are predefined by the hardware; this figure shows the Intel x86 segment table (GDT).

It is crucial that the operating system generates and fills these data structures with content exactly in the form prescribed by Intel. There is no scope for creative deviations or improvements, because the processor only accepts exactly the given format.

File System

The developer has more freedom in the conception of a (logical) file system, i.e., in the question as to which blocks on a the storage medium store which administrative and user data and which features the file system offers. For a Unix-like system, at least the features of the Minix file system [9] are required; there should therefore be inodes with Unix access permissions, as well as user and group IDs, directories, symbolic and hard links, and special files (block and character devices, pipes, sockets).

If you are not looking for compatibility with other file systems, there is plenty of scope for new approaches. If, on the other hand, the operating system has to be able to read and write data from another OS, you'll need to pay close attention to the standards.