Do you like what I did there with the picture of a car dashboard? I figured if we were going to talk about dashboarding and metrics, the most anachronistic and least relevant Unsplash hit should be the cover page for the article.
The "TIG" stack comprised of Telegraf, InfluxDB, and Grafana is a collection of powerful open-source monitoring tools.
Telegraf is an agent that can collect and ship metrics or events from databases, systems, Pub/Sub (e.g. Kafka), and pretty much anything else that you can think of that has data to ship.
InfluxDB is a time series database similar to Elasticsearch and serves as a datastore for events, real-time analytics, and metrics. Make sure to use the 1x branch for Grafana to save yourself some headaches.
Grafana is a visualization tool with monitoring functionality and supports time series datastores like Prometheus, Elasticsearch, and InfluxDB.
I've created a GitHub Repository with Terraform code to instantiate this stack on an Ubuntu 20.04 LTS EC2 instance with versions pinned to at least today's most current versions for members of the application stack. Actually running the Terraform code whether locally, via GitLab CI, or Terraform Cloud is outside the scope of this particular tutorial, but if you need help - hit me up on the comments, via my social links, or the DevOps Lounge Discord and I will set up a time to help you out!
If you are using this in production or intend to have this facing production infrastructure even within your own VPC let alone the public internet, please note that it is important to change the password in the
cloudinit.yml file on line 33 and 48.
Once you have successfully instantiated the instance and given it a couple minutes to do all the fetching/installation of packages, you should be able to access the public ip of the instance at port
3000 and will be presented with a login prompt; use
admin for both username and password:
You will then be able to select a new password:
Now you will be presented with the main Grafana interface. Select the "Data Sources" option to configure the InfluxDB connector:
For the host, choose
http://localhost:8086 and the credentials that you selected for the
telegraf database user:
You should receive a notification that the connection to InfluxDB was successful. Reach out for help if you get stuck here at all:
Now, it is time to create a dashboard that will connect to our Telegraf data in our
telegraf InfluxDB database. On the left-hand navbar, select Dashboards -> Manage:
You will have an option to "Import via grafana.com" - just type in
5955 which is the Dashboard ID and then click "Load":
The dashboard requires minimal configuration, just point it at our InfluxDB datasource and ensure that you select a unique identifier if you plan on shipping data/metrics/events from other sources to this stack:
Bingo Bammo dashboards with metrics!
Obviously this is just a quick little tutorial to get you started with a single host, but if you desire - this can scale up to multiple hosts and you can increase the instance sizing and/or figure out other scale up/out strategies with us over on Discord.