💻 Custom Application Instrumentation

Instrument Your Applications

Learn to add custom metrics to your applications using Prometheus client libraries. Monitor business logic, performance, and user behavior with comprehensive instrumentation.

Prometheus Client Libraries

Node.js (prom-client)

Official Prometheus client for Node.js applications

Add custom metrics to your Node.js applications using the prom-client library.

Features:

Counter metrics
Gauge metrics
Histogram metrics
Summary metrics

Example:

const client = require('prom-client');
const counter = new client.Counter({
  name: 'http_requests_total',
  help: 'Total number of HTTP requests'
});

Python (prometheus_client)

Official Prometheus client for Python applications

Instrument Python applications with custom metrics using the prometheus_client library.

Features:

Counter metrics
Gauge metrics
Histogram metrics
Summary metrics

Example:

from prometheus_client import Counter, Gauge

REQUEST_COUNT = Counter('http_requests_total', 'Total HTTP requests')
REQUEST_DURATION = Histogram('http_request_duration_seconds', 'HTTP request duration')

Go (prometheus/client_golang)

Official Prometheus client for Go applications

Add Prometheus metrics to Go applications using the official client library.

Features:

Counter metrics
Gauge metrics
Histogram metrics
Summary metrics

Example:

import "github.com/prometheus/client_golang/prometheus"

var (
    httpRequestsTotal = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"method", "endpoint"},
    )
)

Java (micrometer)

Popular metrics library for Java applications

Use Micrometer to expose metrics from Java applications in Prometheus format.

Features:

Counter metrics
Gauge metrics
Timer metrics
Custom metrics

Example:

import io.micrometer.prometheus.PrometheusMeterRegistry;

@RestController
public class MetricsController {
    private final Counter requestCounter;
    
    public MetricsController(PrometheusMeterRegistry registry) {
        this.requestCounter = Counter.builder("http_requests_total")
            .description("Total HTTP requests")
            .register(registry);
    }
}

Prometheus Metric Types

Counter

Monotonically increasing counter

Use Cases:

Request counts, errors, events

Example:

http_requests_total{method="GET", status="200"} 1234

Characteristics:

Only increases
Reset on restart
Use for rates

Gauge

Single numerical value that can go up and down

Use Cases:

Current values, temperatures, queue sizes

Example:

memory_usage_bytes 1073741824

Characteristics:

Can increase or decrease
Current snapshot
Use for current state

Histogram

Distribution of values in buckets

Use Cases:

Request duration, response sizes

Example:

http_request_duration_seconds_bucket{le="0.1"} 95

Characteristics:

Multiple buckets
Percentiles
Use for distributions

Summary

Similar to histogram with configurable quantiles

Use Cases:

Latency percentiles, quantiles

Example:

http_request_duration_seconds_summary{quantile="0.5"} 0.1

Characteristics:

Configurable quantiles
Client-side calculation
Use for percentiles

Instrumentation Process

1

Add Client Library

Install the appropriate Prometheus client library for your language

Choose the official Prometheus client library for your programming language and add it to your project.

$ npm install prom-client
$ pip install prometheus_client
$ go get github.com/prometheus/client_golang
2

Define Metrics

Create metrics that represent your application's behavior

Define counters, gauges, histograms, and summaries that capture important application metrics.

const counter = new Counter({name: 'requests_total'});
const gauge = new Gauge({name: 'queue_size'});
3

Expose Metrics Endpoint

Create an HTTP endpoint that serves metrics in Prometheus format

Set up an endpoint (typically /metrics) that returns metrics in the Prometheus exposition format.

app.get('/metrics', (req, res) => {
  res.set('Content-Type', register.contentType);
  res.end(register.metrics());
});
4

Configure Scraping

Configure Prometheus to scrape your application metrics

Add your application to Prometheus configuration or create a ServiceMonitor resource.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-app-monitor

Sample Instrumented Applications

Node.js Express API

Complete Node.js application with Prometheus instrumentation

Metrics:

HTTP request count
Request duration
Error rate
Active connections

Dashboard:

Custom API Dashboard

Alerts:

High error rate
Slow response time
High memory usage

Python Flask Web App

Python Flask application with comprehensive metrics

Metrics:

Request count by endpoint
Database query duration
Cache hit ratio
User sessions

Dashboard:

Flask App Dashboard

Alerts:

Database connection errors
Cache miss rate
High CPU usage

Go Microservice

Go microservice with business metrics

Metrics:

Message processing rate
Queue depth
Processing latency
Failed operations

Dashboard:

Microservice Dashboard

Alerts:

Queue backlog
High latency
Processing failures

Best Practices

Instrumentation Guidelines

Follow these best practices for effective application instrumentation.

  • Use meaningful metric names that clearly describe what they measure
  • Include relevant labels but avoid high cardinality
  • Choose appropriate metric types for your use case
  • Document your metrics with help text
  • Use consistent naming conventions across your applications
  • Monitor your metrics endpoint health
  • Test your metrics in staging before production deployment

Quick Start Example

Node.js Express App

Complete example of instrumenting a Node.js Express application with Prometheus metrics.

const express = require('express');
const client = require('prom-client');

const app = express();
const register = new client.Registry();

// Add default metrics
client.collectDefaultMetrics({ register });

// Create custom metrics
const httpRequestDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of HTTP requests in seconds',
  labelNames: ['method', 'route', 'status_code'],
  registers: [register]
});

const httpRequestTotal = new client.Counter({
  name: 'http_requests_total',
  help: 'Total number of HTTP requests',
  labelNames: ['method', 'route', 'status_code'],
  registers: [register]
});

// Middleware to collect metrics
app.use((req, res, next) => {
  const start = Date.now();
  
  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000;
    const route = req.route ? req.route.path : req.path;
    
    httpRequestDuration
      .labels(req.method, route, res.statusCode)
      .observe(duration);
      
    httpRequestTotal
      .labels(req.method, route, res.statusCode)
      .inc();
  });
  
  next();
});

// Metrics endpoint
app.get('/metrics', async (req, res) => {
  res.set('Content-Type', register.contentType);
  res.end(await register.metrics());
});

// Your application routes
app.get('/', (req, res) => {
  res.json({ message: 'Hello World!' });
});

app.listen(3000, () => {
  console.log('App listening on port 3000');
});

Congratulations!

You've learned how to instrument applications with Prometheus. You now have a complete monitoring stack from installation to custom application metrics.

Back to Prometheus OverviewStart Over