Adding Prometheus metrics to an app

This guide shows you how to collect metrics of a Great Bear application you are developing. Metrics allow you to observe and monitor your application. The metrics are collected by Prometheus and they can be visualized on Grafana. For details, see Observing Great Bear.

The following steps extend the Bear Echo application with basic metrics, and assume that you have already followed the previous sample on Adding Liveness Probe to an app.

Add metrics endpoint to the application

First you need to add a new metrics endpoint to your application so when Prometheus scrapes the HTTP endpoint of the application, the client library sends the current state of all tracked metrics to the Prometheus server.

This example uses the official Python client library for Prometheus and its example on adding metrics to a Flask application.

  1. Add the metrics endpoint to the server.

    from werkzeug.middleware.dispatcher import DispatcherMiddleware
    from prometheus_client import make_wsgi_app
    app.wsgi_app = DispatcherMiddleware(app.wsgi_app, {
        '/metrics': make_wsgi_app()
  2. Add a metric to the application. This example uses a counter metric type, that is incremented by one every time the base url is accessed.

    from prometheus_client import Counter
    _COUNTER_METRIC = Counter('num_of_queries', 'Number of queries')
    @app.route(_BASE_URL, methods=['GET'])
    def echo():

    For reference, here’s the extended of the Bear Echo application:

    # bear-echo/
    import os
    import sys
    import importlib
    from flask import Flask
    from prometheus_client import Counter
    from werkzeug.middleware.dispatcher import DispatcherMiddleware
    from prometheus_client import make_wsgi_app
    _HTTP_PORT: int = int(os.getenv('HTTP_PORT') or '11777')
    _BASE_URL: str = os.getenv('BASE_URL') or '/'
    _ECHO_TEXT: str = os.getenv('ECHO_TEXT') or 'Hello World'
    _COUNTER_METRIC = Counter('num_of_queries', 'Number of queries')
    app = Flask(__name__)
    app.wsgi_app = DispatcherMiddleware(app.wsgi_app, {
        '/metrics': make_wsgi_app()
    @app.route(_BASE_URL, methods=['GET'])
    def echo():
    def healthz():
        return "OK"
    def _main():
        print('ECHO_TEXT: ' + _ECHO_TEXT)
        print(f'Started on{_HTTP_PORT}{_BASE_URL}')'', port=_HTTP_PORT)
    if __name__ == '__main__':

Update the Dockerfile

  1. The official example recommends using uwsgi to run the application. Update the Dockerfile of the application to do that.

    # bear-echo/Dockerfile
    FROM python:3.8-alpine
    WORKDIR /echo
    COPY requirements.txt .
    RUN set -e; \
        apk add --no-cache --virtual .build-deps \
        gcc \
        libc-dev \
        linux-headers \
        ; \
        pip install --no-cache-dir -r requirements.txt; \
        apk del .build-deps;
    COPY ./
    EXPOSE 11777
    CMD uwsgi --http --wsgi-file --callable app
  2. You can see that the Dockerfile now uses a bear-echo/requirements.txt file to keep track of the dependencies used by the application server. Create this file with the following content:

    # bear-echo/requirements.txt

Update the Chart to make metrics available

Providing metrics is optional, the user can enable metrics when deploying the app. If metrics are enabled, a podmonitor is deployed. Prometheus discovers the metrics endpoint of your app via the podmonitor. You can customize several settings related to the scraping.

  1. In order to add the podmonitor to the Chart, create the bear-echo/GBAP/templates/podmonitor.yaml file with the following content.

    {{- if eq .Values.config.MONITORING "on" }}
    kind: PodMonitor
      name: {{ include "app.fullname" . }}-prometheus-exporter
      labels: {{- include "app.labels" . | nindent 4 }}
        - interval: 15s
          port: http
          path: /metrics
      jobLabel: {{ template "app.fullname" . }}-prometheus-exporter
          - {{ .Release.Namespace }}
          {{- include "app.selector" . | nindent 6 }}
    {{- end }} 
  2. In order to allow users to decide whether to enable metrics when they are deploying the Bear Echo app, we can add a toggle switch for the podmonitor as a deployment parameter within the GBAP inside the gbear/appmetadata.yaml:

        - name: MONITORING
          title: Enable monitoring. Make sure you have enabled observability for the site you're deploying beforehand.
          value: "off"
          choices: ["on", "off"]

    For reference, here’s the full bear-echo/GBAP/gbear/appmetadata.yaml file of the Bear Echo application:

    displayName: bear-echo
    description: A simple HTTP echo service, based on Python Flask.
    - debug
    - sample    
    singleton: true    # Defines whether the app can be deployed to the same site several times
      - name: "ECHO_TEXT"
        title: "Message to say"
        description: "The application will echo this text"
        value: "A Hello from Great Bear."            # Default used if not overridden by the user
        type: String
      - name: "ACCESS_PORT"
        title: "Access HTTP port (30000-32767)"
        value: 31777                               # Default used if not overridden by the user
        type: NodePort
      - name: MONITORING
        title: Enable monitoring. Make sure you have enabled observability for the site you're deploying beforehand.
        value: "off"
        choices: ["on", "off"]
  3. Finally, you need to edit the bear-echo/GBAP/values.yaml file and add the monitoring variable to the config section:

      MONITORING: ""

    For reference, here’s the full Values file:

    replicaCount: 1
    strategyType: Recreate
      tag: develop
      pullPolicy: IfNotPresent
      - name: gbear-harbor-pull
      # Config parameters described in application metadata
      # Their values will be injected here by the Great Bear deployer
      GB_SITE_NAME: |
            Site name with "quotes" and 'quotes'
      GB_NODE_NAMES: |
            {"node1":"Display name 1 with \"quotes\" and 'quotes'","node2":"Display name 2 with \"quotes\" and 'quotes'"}
      ACCESS_PORT: |
      MONITORING: ""
      type: NodePort
      port: 11777
        port: 11777
        protocol: TCP
        nodePort: "{{ default 31777 (int .Values.config.ACCESS_PORT)}}"
    # ----
    nameOverride: ""
    fullnameOverride: ""
    nodeSelector: {}
    affinity: {}
      enabled: false
      istioGateway: ""
      servicePath: ""
    siteId: ""
  4. Update and redeploy your application for the changes to take effect.

Next steps

By following these steps you should now have an application with metrics delivered to Prometheus, which should also be visible in Grafana. For more information on Grafana, see Observing Great Bear.

Bear Echo Metrics Bear Echo Metrics