Monitoring Your Docker Resources and Containers Part 2

Overview

This guide builds on Part 1 by adding Node Exporter and cAdvisor to monitor Docker container and host system resources alongside Grafana and Prometheus.

Node Exporter Installation

Node Exporter runs as a service on the host system to collect system metrics. Begin by downloading the latest version:

wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz

Extract and install:

tar xvfz node_exporter-1.6.1.linux-amd64.tar.gz
sudo cp node_exporter-1.6.1.linux-amd64/node_exporter /usr/local/bin

Create a dedicated system user:

sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Clean up installation files:

rm -rf node_exporter-1.6.1.linux-amd64 node_exporter-1.6.1.linux-amd64.tar.gz

Create a systemd service file:

sudo nano /etc/systemd/system/node_exporter.service

Add this configuration:

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

Prometheus Configuration

Update prometheus.yml to add Node Exporter as a scrape target:

sudo nano /etc/prometheus/prometheus.yml

Add this job definition:

  - job_name: 'node_exporter'
    static_configs:
      - targets: ['172.17.0.1:9100']

The IP 172.17.0.1 represents Docker's default bridge network gateway, allowing Prometheus (running in a container) to reach the host's Node Exporter service.

Restart the Prometheus container via Portainer to apply changes. Verify in Prometheus's Targets page that node_exporter appears alongside prometheus.

cAdvisor Installation

cAdvisor monitors individual container resource consumption. Add it to your Docker stack by editing the existing configuration in Portainer.

First, add a dependency to Prometheus:

    depends_on:
      - cadvisor

Then add the cAdvisor service definition:

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    ports:
      - 8080:8080
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    networks:
      spacenet:
        ipv4_address: 172.18.0.101

Update prometheus.yml to include cAdvisor:

  - job_name: 'cadvisor'
    static_configs:
      - targets:
        - '172.18.0.101:8080'

Restart Prometheus. The Targets page should now display cAdvisor alongside your other monitoring services.