Install GitLab Runner manually on GNU/Linux | GitLab

Alternatively, you can follow the below steps to create a GitLab runner on the local machine:

curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
sudo dpkg -i gitlab-runner_amd64.deb
sudo gitlab-runner register

Note: The information in the above picture could be obtained from your GitLab account (settings /CI/CD/ runners/ specific runners)

Output can be seen below on GitLab:

Note: The runner shall always be green in color indicating that the runner is ready to accept the jobs.

*** THE END ***

1. Architecture

2. What is Prometheus?

3. Use cases of Prometheus

4. Configure Prometheus

5. Understanding Prometheus.yml file

6. Configuring Prometheus server + Alertmanager + Node exporter

7. Integrating Alertmanager with Slack

8. Integrating Prometheus with Grafana

\======================================================

Let's discuss this in detail:

The Architecture of Prometheus :

Understanding the architecture: Prometheus will pull the metrics from Prometheus targets (any application whose analytics has to be monitored) and store the data in its TSDB (Time series Database). We use PromQl(Prometheus Query language) to scrape the data at defined intervals from the above-set Prometheus targets. This data can be read in Prometheus web UI and visualized in third-party apps like Grafana. We shall also install an Alertmanager to push alerts to Email/slack etc.

What is Prometheus?

Prometheus is open-source software that collects metrics from targets by "scraping" metrics HTTP endpoints. Supported "targets" include infrastructure platforms (e.g. Kubernetes), applications, and services (e.g. database management systems). With its companion Alertmanager service, Prometheus is a flexible metrics collection and alerting tool.

Use cases of Prometheus:

One of the most popular use cases is gathering numeric metrics from a service that runs constantly. As long as the files are published on an HTTP server, you can query Prometheus by using the hostname, path, and port for the server to scrape the data you need. Monitoring the service metrics for an app can tell you important information about memory utilization, CPU utilization, the number of threads, and more.

Another use case for Prometheus involves monitoring the operating system. You’ll want to know when a server’s hard disk is full or when a CPU is always running at 100%. Memory usage, website uptime, and the number of HTTP requests for each page are some other metrics that users commonly track so they can make improvements as needed.

Configure Prometheus:

You may download and configure Prometheus from the URL given below

Download | Prometheus

We have to select the tar.gz file for Linux and extract it on our local system by using the below command

tar -xvf <file_name>.tar.gz

Similarly, an Alertmanager has to be downloaded and installed in the same manner to receive alerts over email/slack.

Below mentioned is a Prometheus configuration file (Prometheus.yml) which is made available for a better understanding :

NOTE: The heart of Prometheus is its configuration file so whatever configuration we would like to make or change it can only be done in the main configuration file.

# my global config

# we shall mention the scrape intervals in this block of code.That means we shall tell prometheus to fetch the data from targets at given time intervals as mentioned below.

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration

# Under this block of code we have to make configuration related to Alertmanager configuration

alerting:
  alertmanagers:
    - static_configs:
        - targets:
           - localhost:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.

# Under this block of code we shall provide the alerting rules file.

rule_files:
    - prometheus_rules.yml
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:

# In here we mention the metrics that we want to scrape. we mention the endpoint of the target.

# Below we have 3 jobs namely prometheus and node exporter running at different ports.That means the data is being scraped from 3 endpoints and displayed on prometheus UI.

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.

  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "node-exporter"

    static_configs:
      - targets: ["localhost:9100"]

  - job_name: "grafana"

    static_configs:
      - targets: [ "localhost:3000" ]

After extracting the Prometheus.tar.gz file which was downloaded from Download | Prometheus, we have to fire the command mentioned below to run Prometheus.

./prometheus

This will run the Prometheus on HTTP://localhost:9090

Here, we can see a search bar that says “Expression“That's where we write our queries, and the return values are displayed as shown below.

When we write a query “up“ it displays the status of the total jobs that we have mentioned in our scrape configs in the Prometheus.yml file (refer to sample yaml file above). Upon looking at the above image whether our job is running or not, it displays “1“ which means it's active and running and if it displays “0“ that means the job is not running to scrape the metrics.

• Exporters:

A Prometheus exporter acts as a proxy between an application and the Prometheus server. Exporters use a simple, text-based, key-value format to scrape and expose metrics over time, via HTTP, for aggregation and decision-making.

In simple words, it acts as a bridge between our target (where we would like to scrape the metrics from) and the Prometheus server.

In our scenario, we are using Node exporter just to showcase that when it's not responding/down it creates an alert that can be received on Email/slack.

Node exporter can be downloaded from Download | Prometheus and we must have to extract the files to make the node_exporter run by using the below-given command

./node_exporter

Now, we shall write a simple alert rule in Prometheus_rules.yml that informs us if our node exporter is running / down.

groups:

- name: alert_rules
  rules:
    - alert: NodeExporterDown
      expr: up{job="node-exporter"} == 0

      labels:
        severity: critical

we have to mention the Proemtheus_rules.yml file name in the rule_files section of our main Prometheus configuration file i.e, Prometheus.yml

• Alert Manager

To view the alerts and send them via Email/slack, we must have to install Alertmanager which is available here to download Download | Prometheus and we must have to extract the files to make the alert manager run by using the below-given command

./alertmanager

Alertmanager.yml file is written as below for slack configuration. However, we can configure the same for Email using an SMTP server.

route:
  receiver: admin

receivers:
- name: admin
  slack_configs:
  - channel: '#prometheus'
    api_url: 'https://hooks.slack.com/services/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'

• Slack Configuration

On Slack, we must add an app called Incoming WebHooks available in the Apps directory as shown below

use the webhook URL in the above Alertmanager.yml (api_url)

For illustration purposes, I shall make the node_exporter shut down which is currently running on HTTP://localhost:9100 so that we can see that an alert is generated.

on HTTP://localhost:9093 Alertmanager has also captured an alert regarding the node_exporter downtime and sent the information to Slack as shown below

since we have configured Slack, node_exporter downtime is being displayed

• Grafana:

Prometheus collects rich metrics and provides a powerful querying language and Grafana transforms metrics into meaningful visualizations. Both are compatible with many if not most, data source types.

It is a third-party tool to visualize beautiful graphs drawn out of data from Prometheus.

You may download Grafana from the below URL

Download Grafana | Grafana Labs

After configuring Grafana on your local system, enter the command shown below to make Grafana accessible at HTTP://localhost:3000

sudo systemctl start grafana-server

Grafana Homepage at HTTP://localhost:3000 is shown below

In configuration settings, we must integrate our Prometheus

In settings, we shall provide the HTTP URL of our Prometheus Server so that it pulls data from that endpoint

Grafana metrics are showcased here in which you can see whether your jobs are running or in firing state

Prometheus alerts showcasing the firing alerts of node_exporter which was down/not working

*** THE END ***