Elasticsearch, Logstash, and Kibana – The ELK stack
ELK Hunt
© Lead Image © welcomia, 123RF.com
A powerful search engine, a tool for processing and normalizing protocols, and another for visualizing the results – Elasticsearch, Logstash, and Kibana form the ELK stack, which helps admins manage logfiles on high-volume systems.
Even a single, small LAMP server will produce a number of logfiles, and if you have a large array of servers, you can generally look forward to a volume of logfiles that is likely to exceed the capabilities of most built-in log management tools – if you want to analyze the data in your logs, that is. The different file formats output by the typical zoo of applications also add complexity.
The ELK stack, which is a combination of Elasticsearch [1], Logstash [2], and Kibana [3] addresses these difficulties. Elasticsearch is an extremely powerful search server that receives its data from Logstash, an application that extracts the data from server protocols, normalizes them, and dumps the results in an Elasticsearch index. Finally, the Kibana analytics and data visualization tool offers extremely flexible views of the information.
The lab environment consisted of several Debian Jessie servers, one running an ELK stack, as well as Filebeat [4], a service that acquires the local logs and sends them to Logstash. Filebeat can also collect logs from remote sources; we used it on another server that was already set up and upgraded as a central log host. The server also takes care of Syslog forwarding.
[...]
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Digital Autonomy
• The Answer Was Already on the Shelf
• Changing the Chip Industry: How Public Investment Has Grown Open Silicon
• United Nations Open Source Portal Goes Live
• EU Open Source Strategy Plays Key Role in Tech Sovereignty Package
• France Says “Au Revoir” to Microsoft
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Dutch Government Turns to Open Source for Digital Sovereignty
Following in the footsteps of European countries, the Netherlands is looking to increase its sovereign digital infrastructure with open source software.
-
Linux May Soon Work on Snapdragon X2-Based PCs
If you're thinking about buying a Snapdragon X2-based desktop or laptop, Linux may soon be an option.
-
KDE for People Initiative Calls for AI Ban in Plasma
A new community of KDE wants the project to adopt a strict no-AI policy for KDE Plasma.
-
KDE Sets Ambitious Goals for 2026 and Beyond
KDE Connect reveals the goals for the Linux desktop darling, with one of those goals long overdue.
-
It’s Time to Test Fedora 45 Beta
Fedora 45 beta has been released with updated Gnome, KDE Plasma, and kernel.
-
Ubuntu Stonking Stingray Gets Even Rustier
Ubuntu 26.10 has completed its migration to the Rust-based coreutils.
-
AI Fixes Linux Bottlenecks Using “Hideous” Code
A Linux developer used AI to fix bottlenecks that caused problems when building the kernel. The resulting AI-generated code needed a lot of work.
-
Advanced Video Coding Still Under Patent
Brazilian patent BRPI0109962B1 has expired, which means that DivX and Xvid are no longer problematic for Linux, but it doesn’t mean that video is all of a sudden fully unleashed.
-
2,000 Vulnerabilities per Linux Release
Thanks to AI bug hunters, the Linux kernel is seeing record numbers of vulnerabilities, and it's overwhelming developers.
-
Linux Exempt from California’s Age Verification Law
So long as Linux is distributed under the GPL, MIT, BSD, and Apache licenses, the OS is exempt from being required to verify the age of its users in California.