Analyzing network flow records
Data Security
The method of detecting downloads directly from the metadata presented here obviously has a considerable effect on the general security of the devices on your network. Many updates are not intended for all versions of an operating system, so you can relatively quickly find out exactly what version is in use. Various manufacturers of security software offer countermeasures that prevent complete scanning of the Internet or your own subnet.
But because metadata is passive by nature, it cannot be filtered by these protection systems – and it is also impossible to get rid of metadata. This method is far less suitable for use on large networks because, on the one hand, the detection rate drops severely the farther away you are from the download server. On the other hand, the alternative approaches that many providers now use to deliver updates also takes its toll.
In particular, the peer-to-peer update function in Windows 10, as well as Content Delivery Networks (CDNs), which manage a globally distributed network with a correspondingly large number of different IP addresses, should be mentioned. These large providers in particular offer so many files of different types that identification based solely on the size of the file seems to be fairly meaningless. Without a knowledge of the DNS requests, it is impossible – especially in the case of CDNs – to identify the domain that was originally the target of the request.
That said, it is important to note that the general lack of attention paid to metadata can become problematic if worst comes to worst and an attacker is able to exploit an unpatched vulnerability on a system footprinted using this method.
Reference Downloads
To be able to assign flows to downloads, it is important to know what possible downloads exist. This task is impossible to handle manually because of the sheer volume of possibilities, so you need to think about an automation strategy. Two methods turn out to be very useful here.
The first method is based on grabbers, which work much like the classical search engine grabbers that index the Internet for fast searching. This involves searching the Internet or a suitable selection of websites for download links. Because the detection rate is very poor in the lower kilobyte range of numbers, administrators will only want to consider downloads whose size exceeds a certain threshold for indexing.
However, this method has the massive disadvantage that it prevents the detection of incremental updates because the download links typically offer the full download. Additionally, the collection can become unmanageably large even after a very short time.
The second method is based on honeypots equipped with a software configuration similar to those used on enterprise networks. By monitoring the network traffic to these honeypots, administrators can now directly observe update sequences. Additionally, it is possible to start downloads directly from the systems, making it easy to map the flows because the honeypot systems are not used for any other purpose.
The major advantage offered by this method is that the recorded packet sizes lead to good detection rates, especially if the honeypots are located on the same subnet as the systems you want to protect. Moreover, it is easier to emulate and analyze special update mechanisms. These benefits come at a price, in that you can only monitor known software versions and combinations and you are relying on honeypot systems that need to work with full, licensed versions of the software you deploy.
Conclusions
For IT staff who want to keep track of their own IT infrastructure and do not have, or are not allowed to have, access to all of the systems, the method introduced here is an additional option that supplements classical penetration tests to provide better asset protection. It also draws attention to the value of metadata. If you log flow records directly on the switches and backbone routers on your network, you will also ensure that the distance to the systems you are monitoring is not too large, which means that the variance in the monitored download sizes remains manageable.
Infos
- Shadow IT: https://en.wikipedia.org/wiki/Shadow_IT
- "Managing port scan results with Dr. Portscan" by Wolfgang Hommel, Stefan Metzger, Michael Grabatin, and Felix von Eye, Linux Pro Magazine, issue 155, October 2013, pg. 20, http://www.linuxpromagazine.com/Issues/2013/155/Dr.-Portscan
- Bernhard, Andreas, Netzbasierte Erkennung von Systemen und Diensten zur Verbesserung der IT-Sicherheit [Network-Based Detection of Systems and Services to Improve IT Security], Bachelor thesis, Ludwig-Maximilians-University, Munich, March 2014, http://www.mnm-team.org/pub/Fopras/bern14/PDF-Version/bern14.pdf [in German]
- Softflowd: http://www.mindrot.org/projects/softflowd
- Flow-tools: https://code.google.com/p/flow-tools
- Data retention laws: https://en.wikipedia.org/wiki/Telecommunications_data_retention
- Pandas: http://pandas.pydata.org
- Sklearn: http://scikit-learn.org
« Previous 1 2
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.