The Elastic Stack

What is the Elastic Stack?

Elastic Stack (formally known as ELK stack) is an open source tool set used to aggregate and analyze data. One of the most common uses for Elastic is for logging data. Logs can be aggregated across multiple hosts and can do many useful things with the data like monitor the logs, make dashboards of the data and allow for querying of the data.

The Elastic Stack has various components that it is comprised of. Here is a list of some of the main components:

• Elasticsearch - A JSON based search and analytics engine. The logs are sent here for processing and possibly formatting into JSON if they were not sent in JSON format already.
• Kibana - The user interface to view the data sent to Elasticsearch. This is the front end to view the logs, setup dashboards and alerts etc.
• Logstash - A dynamic data collection pipeline. This is what sends the logs to Elasticsearch.
• Beats - A collection of lightweight data shippers. This is similar to Logstash but has a lot of various plugins for sending data in a more structured way. For example use a Redis plugin with beats to send Redis logs so they are structured more uniformly for Elasticsearch.

...and many more (see External Content section below)

Viewing Logs

The logs are broken up into different indices, a logical separation of how the log data being viewed, and structured in JSON. This is very helpful as the metadata with the logs is structured for things like timestamps, the message and various Kubernetes information such as pod or container. With the JSON metadata automatically added to the logs these fields can be used in searches or filters.

External Content

• List of all Elastic Products
• Elastic Documentation