How to Set Up Prometheus and Grafana on Linux (Step-by-Step)

Why This Monitoring Stack Is Worth the Setup Time

Learning how to set up Prometheus and Grafana on Linux is one of those skills that pays for itself the first time something breaks at 2 AM. According to the Grafana Labs 2025 Observability Survey, 67% of organizations already use Prometheus in production. There’s a reason it became the standard: it’s free, it’s powerful, and it runs beautifully on Linux.

I learned this lesson the hard way. A couple years back, I had a VPS running a handful of services for my homelab. One morning I woke up to everything offline. SSH wouldn’t connect. The hosting panel showed the disk was 100% full. Some runaway log file had eaten every last byte, and I had zero visibility into what was happening before the crash. After that, I set up Prometheus and Grafana and never flew blind again. If you’ve ever been surprised by a full disk or a CPU spike you didn’t see coming, this guide is for you.

The whole setup takes under 30 minutes. By the end, you’ll have a live dashboard showing CPU, RAM, disk, and network metrics for any Linux server you want to monitor. Among the best Linux monitoring tools available today, the Prometheus and Grafana stack remains the most popular open source option for good reason.

How Prometheus, Node Exporter, and Grafana Work Together

Before we touch the terminal, let’s understand what each piece does. This is the part most tutorials skip, and it’s exactly why people get confused when something breaks later.

Get a VPS from as low as $11/year! WOW!

What Prometheus Does

Prometheus is the engine of this stack. It’s a time-series database that pulls (scrapes) metrics from targets on a configurable interval — every 15 seconds by default. Unlike older push-based tools like Nagios or Zabbix, Prometheus reaches out and grabs data from your services. The Prometheus official documentation covers the architecture in detail, but the short version is: it stores everything locally and lets you query it with PromQL.

What Node Exporter Does

Node Exporter is a lightweight agent that runs on each Linux host you want to monitor. It exposes system metrics — CPU usage, memory, disk I/O, network traffic — on port 9100. Prometheus then scrapes those metrics on its schedule. Think of Node Exporter as the translator between your Linux kernel’s stats and Prometheus’s database.

What Grafana Does

Grafana is the visualization layer. It connects to Prometheus as a data source and turns raw metrics into beautiful, interactive dashboards. You’ll see live graphs, gauges, and tables — all updating in real time. It’s where the “wow, I can actually see what my server is doing” moment happens.

“Prometheus was the first tool to allow you to dynamically detect and monitor workloads of arbitrary complexity and deployment — and do math with the data.” — Richard Hartmann, Director of Community at Grafana Labs

The flow is simple: Node Exporter exposes metrics → Prometheus scrapes and stores them → Grafana visualizes them. Three components, one clean pipeline.

What You’ll Need Before You Start

If you’re setting this up on a remote server or cloud VPS, make sure your provider’s security groups or VPC rules also allow traffic on those ports. I’ve wasted more time than I’d like to admit troubleshooting a “broken” setup that was really just a cloud firewall blocking the port.

Step 1 — Install Prometheus on Linux

Create the Prometheus System User and Directories

First, we’ll add a system user in Linux dedicated to running Prometheus. This is a security best practice — you never want services running as root.

sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Download and Extract the Prometheus Binary

Head to the Prometheus releases page and grab the latest version. Replace the version number below with whatever’s current:

cd /tmp
curl -LO https://github.com/prometheus/prometheus/releases/download/v2.53.0/prometheus-2.53.0.linux-amd64.tar.gz
tar xvfz prometheus-2.53.0.linux-amd64.tar.gz
cd prometheus-2.53.0.linux-amd64

Now copy the binaries and config files into place:

sudo cp prometheus promtool /usr/local/bin/
sudo cp -r consoles console_libraries /etc/prometheus/
sudo cp prometheus.yml /etc/prometheus/
sudo chown -R prometheus:prometheus /etc/prometheus

Create the Prometheus systemd Service File

If you want to understand systemd services in depth, I’ve got a full walkthrough on how to create a systemd service in Linux. For now, create the service file:

sudo nano /etc/systemd/system/prometheus.service

Paste this content:

[Unit]
Description=Prometheus Monitoring
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus   --config.file=/etc/prometheus/prometheus.yml   --storage.tsdb.path=/var/lib/prometheus/   --web.console.templates=/etc/prometheus/consoles   --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus

Open http://your-server-ip:9090 in a browser. If you see the Prometheus UI, you’re golden.

Step 2 — Install Node Exporter

Download Node Exporter and Create a systemd Service

Node Exporter is a separate binary. Grab the latest release from Node Exporter on GitHub and install it the same way we did Prometheus:

sudo useradd --no-create-home --shell /bin/false node_exporter
cd /tmp
curl -LO https://github.com/prometheus/node_exporter/releases/download/v1.8.1/node_exporter-1.8.1.linux-amd64.tar.gz
tar xvfz node_exporter-1.8.1.linux-amd64.tar.gz
sudo cp node_exporter-1.8.1.linux-amd64/node_exporter /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Create the systemd service file:

sudo nano /etc/systemd/system/node_exporter.service

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter

Open Port 9100 on Your Firewall

Prometheus needs to reach Node Exporter on port 9100. If you’re running a firewall, you’ll need to manage firewall rules with firewalld or use ufw:

sudo firewall-cmd --permanent --add-port=9100/tcp
sudo firewall-cmd --reload

sudo ufw allow 9100/tcp

Test it by visiting http://your-server-ip:9100/metrics. You should see a wall of plain-text metrics. That’s exactly what Prometheus will scrape.

Step 3 — Configure Prometheus to Scrape Node Exporter

Now we tell Prometheus where to find Node Exporter. Edit the Prometheus config file:

sudo nano /etc/prometheus/prometheus.yml

Add a new job under the scrape_configs section:

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node_exporter'
    static_configs:
      - targets: ['localhost:9100']

Restart Prometheus to load the new config:

sudo systemctl restart prometheus

Head back to the Prometheus UI at :9090 and go to Status → Targets. Both prometheus and node_exporter should show as UP. If node_exporter shows DOWN, double-check the IP and port — the most common mistake is using localhost when monitoring a remote host that needs the actual IP address.

Step 4 — Install Grafana on Linux

Add the Grafana APT or YUM Repository

Grafana provides official repositories so you get updates through your package manager. Here’s how to add them:

For Debian/Ubuntu:

sudo apt install -y apt-transport-https software-properties-common wget
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee /etc/apt/sources.list.d/grafana.list

For RHEL/Rocky:

cat <<EOF | sudo tee /etc/yum.repos.d/grafana.repo
[grafana]
name=grafana
baseurl=https://rpm.grafana.com
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://rpm.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
EOF

Install, Enable, and Start Grafana

# Debian/Ubuntu
sudo apt update && sudo apt install grafana -y

# RHEL/Rocky
sudo dnf install grafana -y

sudo systemctl enable grafana-server
sudo systemctl start grafana-server

Don’t forget to open port 3000 on your firewall the same way you opened 9100 earlier. Grafana’s default login is admin / admin — you’ll be prompted to change the password on first login.

Step 5 — Connect Grafana to Prometheus and Import a Dashboard

This is where everything comes together. You’re about to go from raw metrics to a live visual dashboard in about two minutes.

Add Prometheus as a Data Source in Grafana

1. Log in to Grafana at http://your-server-ip:3000
2. Go to Configuration → Data Sources → Add data source
3. Select Prometheus
4. Set the URL to http://localhost:9090 (or your Prometheus server’s IP)
5. Click Save & Test — you should see “Data source is working”

Import the Node Exporter Full Dashboard (ID: 1860)

You could build a dashboard from scratch, but why reinvent the wheel? The Node Exporter Full dashboard has over 4 million downloads and shows everything you need.

1. Go to Dashboards → Import
2. Enter ID 1860 and click Load
3. Select your Prometheus data source from the dropdown
4. Click Import

And just like that, you’ve got live graphs for CPU usage, memory, disk I/O, network traffic, and load average. I still remember the first time I saw my homelab metrics light up on a Grafana dashboard. It felt like going from driving without a speedometer to having a full heads-up display.

“It has a unique combination of how easy it is to operate, how consistent it is — there’s always one way to do something, and it’s also the most natural way.” — Frederic Branczyk, Prometheus Maintainer

Common Issues and How to Fix Them

Things don’t always go perfectly. Here are the issues I’ve seen the most, both on my own servers and from helping others in forums:

• Prometheus target shows DOWN: Check that node_exporter is running with systemctl status node_exporter. Verify the firewall isn’t blocking port 9100. Make sure the IP in prometheus.yml is correct.
• Grafana dashboard shows “No data”: Confirm the Prometheus data source URL is right. Give it a minute — the scrape interval needs time to collect a few data points first.
• Port 9090 or 3000 unreachable externally: Open the port on your local firewall and check your cloud provider’s security groups. I’ve lost an hour to this one more than once.
• Permission denied errors: Make sure the prometheus user owns /usr/local/bin/prometheus and /var/lib/prometheus/. Run ls -la to check.
• Binary not found after extraction: Confirm the binary is in /usr/local/bin/ and is executable (chmod +x).

Where to Go From Here

You’ve got a working Prometheus and Grafana monitoring stack on Linux. That’s a solid foundation. Here’s what to explore next:

• Alerting with Alertmanager: Send Slack or email notifications when disk usage crosses 80% or a service goes down. This is the natural next step.
• Docker monitoring: Add cAdvisor to monitor your containers. If you’re already running services with Docker Compose on Linux, cAdvisor plugs right in.
• More exporters: MySQL exporter, nginx exporter, blackbox exporter for uptime checks — there’s an exporter for almost everything.
• Log management: Pair Grafana Loki with your metrics for centralized logging. And while you’re thinking about logs, make sure you manage log rotation with logrotate so those logs don’t eat your disk.
• Storage scaling: If your Prometheus data directory starts growing, you can manage storage with LVM to expand the volume without downtime.
• HTTPS for Grafana: If your dashboard is accessible over the internet, secure Grafana with Let’s Encrypt so your login credentials aren’t flying around in plain text.
• Automated tasks: Use systemd timers to automate health checks or data cleanup scripts alongside your monitoring.

Monitoring is one of those things where once you start, you wonder how you ever managed servers without it. If you’re looking for a broader view of what’s available, check out our roundup of the best Linux monitoring tools to see how Prometheus and Grafana compare to other options. And if you’re still picking a distro for your next server build, our guide to the best Linux distro for servers can help you decide.

Got questions or ran into something not covered here? Drop a comment below — I’ve probably debugged it before.

Alexa Velinxs

I'm Alexa Velinxs, a cryptocurrency trading expert passionate about demystifying digital assets for both beginners and seasoned investors. Through my writing, I share actionable strategies, market insights, and practical tips to help you navigate the crypto landscape with confidence. Let's explore the future of finance together.

See Full Bio