Access your servers securely with a Magic URL

Magic Act

© Lead Image © bowie15,

© Lead Image © bowie15,

Article from Issue 190/2016

With a highly secure Linux server, you don't need a fixed IP address to connect over a cellphone network from anywhere on the planet if you have a Magic URL.

Much of my time I work with enterprise technologies, but when I get the chance to view Linux as a hobby again, I find myself solving problems by using only a simple script and a limited number of tools. My last foray into customizing Linux was securing a CCTV (closed circuit television) server that helps protect my home. In this article, I'll demonstrate how I solved a problem by using what I'm going to call a "Magic URL."

By Magic URL I mean that despite having what some would consider to be a highly secure Linux server (a firewall, two-step verification, fail2ban, TCP Wrappers, and htaccess), I don't need a fixed IP address to connect over a cellphone network from anywhere on the planet, just a spare 30 seconds, an SSH client, and access to a valid two-step verification code. I'll assume you have an Apache web server installed, preferably with HTTPS for a little more htaccess password secrecy. An htaccess configuration file is used to add simple password controls and is loaded and executed by the Apache web server software [1].

It's worth saying that I'm sure you can solve similar problems in other ways, and some will suit your needs to a greater extent; however, this solution works for me, and you might find you can apply it or modify it to work for you in a future scenario. I've no doubt that after tinkering with this solution, I'll end up using something I've learned here on another project.


How I arrived at having a Linux server hosted in the cloud and locked down tightly to host my CCTV footage has a significant background, but I don't want to lose track of how to set up a Magic URL, so I'll briefly just explain the tools I'm using before beginning.

None of these tools are mandatory, and once you see how they integrate with each other, you should be able to choose which tools you can use with relative ease. For those who prefer a quick solution, I'll also offer details of the bare minimum required to achieve functionality.

For clarity, all the CCTV server does is receive uploaded footage (images and videos) of any motion-triggered events at my home. For obvious reasons, I won't go into too many details, but in essence, it's an image repository and a simple viewing platform to watch for unusual events at home day or night. Generally it will pull about 20GB of data off a single camera in a month. Incidentally, it costs pennies to run on a monthly basis, and camera hardware (even running glorious HD) is very affordable.


I'm a big fan of a super-slick tool called fail2ban [2]. It cleverly provides a mechanism to add firewall rules if it spots repeated failed logins in system logfiles. Netfilter's iptables is used in most cases on Linux, because it's so powerful and highly accessible.

In my case, on the CCTV server, I watch fastidiously and block anyone (automated or otherwise) from triggering an HTTP 404. Here's the slightly unusual bit: I ban an IP address for a very long period of time for even triggering a single 404. This action might seem a bit draconian, but it's for good reason: I'm the only person accessing the CCTV server, and I know where to view the latest events with impunity. If you're completely new to fail2ban, it's well worth a look at an article I wrote in Linux Magazine [3] about how to keep pesky robots from filling your logs with noise when they generate HTTP 404s and, indeed, anything above a friendly HTTP 200.

On the CCTV server, I've possibly gone a little overboard with my fail2ban configuration, because sometimes troubleshooting a problem can take a little longer than it should; however, I have three iptables-driven fail2ban configs running. First, I use a config to deal with a popular remit for fail2ban: protecting against repeated SSH login attempts. In Figure 1 you can see how iptables reports it, the label is fail2ban-ssh.

Figure 1: Fail2ban stopping repeated incorrect password attempts over SSH.

Second, as you can see in Figure 2, I have a very strict set of rules for a non-existent page that I call fail2ban-apache-chris. The output in Figure 2 is abbreviated because of all the offenders (ranging from search engines to automated attacks). Admittedly, the rule doesn't have the best of names, but I'm the only sys admin on the server, so I know what it does.

Figure 2: I'm being really strict with 404 and other errors by banning every offender for a long time.

You might want to compare the config examples in the fail2ban article mentioned earlier [3] with the fail2ban-apache-chris rules I run here (Figure 3). Notice the mention of MAGIC-URL-PATH within the ignoreregex field, because I talk about that later.

Figure 3: Fail2ban is catching the IP addresses of those requesting any non-existant pages on the web server and banning them with the fail2ban-apache-chris filter.

The final fail2ban ruleset was originally running within fail2ban-apache-chris to look for HTTP 401 login errors (i.e., users who don't know the password to .htaccess pages), but I found it too cumbersome to include alongside the Magic URL functionality, so I separated it out. In Figure 4 you can see the filter fail2ban-apache-denied, which deals solely with HTTP 401 errors. Anyone who attempts to access my Magic URL subdirectories with the wrong login and password (also known as HTTP 401 Unauthorized errors) is promptly struck down by lightning. It took a little while to get this ruleset just as I wanted, but the rules essentially catch login failures over htaccess (Figure  5).

Figure 4: To make scripting easier, fail2ban has a separate filter just for HTTP 401 errors.
Figure 5: The fail2ban-apache-denied filter catches HTTP login errors.


As mentioned, fail2ban, like the other security tools used in this setup, isn't absolutely necessary to get the Magic URL functionality working, but it certainly adds to my peace of mind. It's also highly configurable, and I would suggest getting the other aspects of the Magic URL working first (so you're sure they're running for you as expected) and then adding fail2ban and two-step authentication with Google Authenticator afterward.

The CCTV images and video footage are uploaded by FTP with vsftpd into a user's home directory. The CCTV cameras are locked down by IP address thanks to TCP Wrappers, a host-based networking ACL system for *nix systems used to filter network access to Internet protocol servers. If you're new to TCP Wrappers and don't want to use iptables to lock down the likes of SSH or other network-accessible services, you might want to read an article I wrote for ADMIN magazine on that topic [4]. I use TCP Wrappers on this server to limit access to sshd, vsftpd, and the Magic URL script to a few static IP addresses. How it integrates with that functionality will become clear soon.

Within a subdirectory off the web root lives my main viewing page for the latest image (captured during the most recent motion-detected event). If I want to dig deeper, I can download an MPEG video or JPG images onto my smartphone to look at an event in more detail, having been presented with a prettified directory listing of all of the footage residing under capture dates and timestamp subdirectories.

Those who don't approve of security through obscurity might debate this next approach. I thought for a while about making this aspect much more sophisticated, and I might improve it in the future, time permitting. My issue with the core of this simple design is that, essentially, my Magic URL is statically named. Don't get me wrong: In addition to just being impossible to guess, the URL is secured by fail2ban, which would make short shrift of attackers' attempts to access it. (They'd be banned for more than a week if they got the password wrong once.)

To my mind, however, the best security comprises of a number of carefully considered layers. In the future, I might change my unguessable static URL every hour. I could generate a value, along the same vein as two-step authentication does, or use some other apparently random value known only to me, or I might opt for simplicity and just append the current hour of the day to the end of the URL.

Say, for example, that under the web root of my Linux server I wanted to connect to my server at two o'clock in the afternoon by calling a subdirectory for my Magic URL (e.g., chris-magic-url/). I would have a cron job make sure my directory was renamed chris-magic-url-14/ at 1400hr.

For the rest of this article I'll assume that chris-magic-url/ is the name of my subdirectory. If you're diving straight into using fail2ban from the start, you should look again at the ignoreregex line in Figure 3, which I will use to replace MAGIC-URL-PATH with chris-magic-url.

I'll try not get too bogged down with the many security options available, but I do want to mention two-step authentication (also known as two-step auth or two-step verification) at this stage. It's a fantastic way of bolstering SSH security immediately after you've entered your password. You have to enter a time-limited code (as with a dongle or RSA token).

Figure 6 shows how to view Google Authenticator codes on a smartphone once it's set up. In case you're wondering, I don't ever name my authentication services by their real name in case my phone is lost or stolen. There's nothing wrong with some security through obscurity in my book; it all helps to slow an attacker down or make things more difficult to figure out.

Figure 6: Ephemeral Google Authenticator codes on my smartphone.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95


njobs Europe
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia