Let Tube Archivist organize your YouTube collection

And … Action!

Article from Issue 276/2023
Author(s):

Tube Archivist indexes videos or entire channels from YouTube in order to download them with the help of the yt-dlp tool.

YouTube videos have become an important part of many users' leisure and educational activities in recent years. Many videos now also end up on free video platforms such as PeerTube [1], LBRY [2], Invidious [3], or on NewPipe [4] for Android, but by no means do all of them. This is why, no matter what users think of Google, most will continue to rely on Google's video platform.

To archive videos and preempt deletions by content creators or Google, many YouTube fans download the videos to hardware they control. Downloading videos is prohibited by default in the terms of service for the free version of YouTube, however, content creators can grant permission through a Creative Commons license. Also, downloads for offline viewing are allowed for subscribers to the YouTube Premium edition.

Video collections have a habit of growing uncontrollably over time, to the point where things just run wild. That's why it makes sense to counter the threat of chaos by organizing everything right from the outset. This is where Tube Archivist (TA)comes in [5], a self-hosted media server for YouTube videos.

Functionality

TA lets you subscribe to YouTube channels, download videos, and index your archive so the collection can be browsed and streamed offline from any device on the local network. You can download both manually and automatically. Yt-dlp [6] runs in the background for this purpose. Searches against the collection are handled by the Elasticsearch [7] tool. For Firefox and Chrome there is the TA Companion [8], an extension that displays a download button when you open a YouTube video. In the case of a YouTube channel, a subscribe button also appears in TA.

TA gives you full control over the content you want to store. You can rescan the subscribed channels at any time and decide if you want to archive the new content. This is also a good way to manage the kind of YouTube content your kids are allowed to watch. The stored videos can be played with TA's built-in player, but also – given a little extra effort – output to Plex, Emby, or other media platforms.

Before I go into more detail about the operation and other functions, I'll briefly look at the installation. The hardware requirements, according to the developers, are a 4-core CPU and 2GB RAM for smaller collections, or 4GB for medium to large collections, but more RAM never hurts, as we all know. The size of the required hard disk is primarily driven by the size of the archive you intend to keep. You can install TA as a Docker container; Docker Compose largely automates the process. The "Installing Docker and Docker Compose" box explains how to set up the current versions of the two components.

Installing Docker and Docker Compose

You should always install Docker and Docker Compose directly and not through the distribution archives [9]. If you are familiar with Docker Compose: Previously, the command line was docker-compose, but newer versions use docker compose. The hyphenated variant just provokes an error message.

For testing, I installed TA on Proxmox. However, any hardware that meets the requirements can be used just as easily. Hardkernel single board computers (SBCs) such as ODROID-H3 or ODROID-H3+ [10] are ideal as a hardware basis for TA and can also host other services. Instructions for installing TA with Podman (via Docker Compose), Unraid, TrueNAS Scale, or on a Synology NAS can be found on GitHub [11]. If using an ARM64-based device, TA is available as a multi-arch container, and the same applies to Redis. For Elasticsearch, simply use the official image with ARM64 support. The software does not support other architectures.

Preliminary Work

After installing Docker and Docker Compose, copy the contents of the YAML file for Docker Compose (Figure 1) to a file in your home directory and name it docker-compose.yml [12].

Figure 1: Copy the contents of docker-compose.yml from the GitHub page and save them to a file of the same name in your home directory.

Then edit the file and enter the IP address or domain name of your TA system for TA_Host. TA_USERNAME and TA_PASSWORD are where you store the matching access credentials. Also replace the default password in Elastic_Password and set the time zone correctly.

Use the docker ps command to check that ports 8000 for TA and 9200 for Elasticsearch are available. If not, change the first part of the 8000:8000 port specification to a different port in the YAML file. Changing the port for Elasticsearch is a bit more complex; it is described in the documentation in the wiki.

HOST_UID and GID_UID are equivalent to the ID of 1000 specified in the YAML file on Ubuntu. For other distributions, check this with the id command and edit the details of the YAML file if needed. volumes: is where you specify where TA stores the videos and where the cache is located.

After saving the file, set up the containers with the docker compose up -d command. Then launch TA by typing http://<TA-IP>:8000 or http://<TA-FQDN>:8000 in a browser and logging in with the credentials created previously. If you want to access TA from outside your home network, you will also need to set up a reverse proxy.

Getting Started

After the launch, you are taken to TA's user interface in Dark Mode; you can swap this for a brighter desktop in the settings. The interface is clearly laid out and visually appealing (Figure 2). In the upper part you will see four sections Home, Channels, Playlists, and Downloads along with three icons for searching, settings, and logging out. The Home [13] section gives you an overview of the videos you already downloaded, which can be played back there by the integrated player. TA shows separately videos that have not been viewed completely to let you continue viewing. The software can also hide viewed videos if so desired.

Figure 2: TA comes with a clear-cut design. As you can see, I have already subscribed to some channels; some of the videos are partially viewed.

The Channels [14] section lists YouTube channels you subscribe to (Figure 3). To subscribe to a channel, enter either the 25-digit YouTube channel ID or the channel's URL in a field that you open by pressing the plus button. Alternatively, enter the URL of a video from the channel and let TA find the channel's URL. Subscribing to a channel does not download any videos.

Figure 3: The Channels section contains the subscribed channels in a list or grid view. You can use the Unsubscribe button to delete subscriptions you no longer need.

If you click on one of the subscribed channels, the list of videos already downloaded from that channel will appear. You can use the Unsubscribe button to terminate a subscription to a channel. About gives you more information and configuration options for the respective channel (Figure 4). These settings override the entries in the global settings dialog. For example, you can specify a video format that differs from the global settings and specify whether and when TA automatically deletes videos you have played back. In addition, you can decide whether you want to see the sponsor block for this channel.

Figure 4: After clicking on the label of a channel you will see the Show more button, among other things. It provides more detailed information and supports individual configuration of the channel.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • An open source, federated video sharing platform

    With PeerTube you can self-host your videos without the limitations embedded in YouTube and similar platforms.

  • Legal Video Downloads: YouTube goes Offline

    Until now, watching YouTube videos offline has been an awkward business, possible only with Firefox plugins or similar contraptions. Now, with the help of Creative Commons licenses, this could soon change.

  • YouTube Players

    YouTube offers more than just funny kitten movies; you will also find more than 60 million music videos. With a native YouTube client for Linux, you can use this online jukebox as conveniently as your local music collection.

  • Downloading Web Video

    With the right tools, you can store YouTube movies on your hard disk and view them when Internet access is unsatisfactory or unavailable.

  • Google Launches New Channel On YouTube

    Ellen Ko of Google's Open Source Team announces the creation of a new Google YouTube channel, googleOSPO, created to house all the Google and open source related videos in one place.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News