Links
Comment on page

Monitoring

Pool metrics exporter

The Mayastor pool metrics exporter runs as a sidecar container within every io-engine pod and exposes pool usage metrics in Prometheus format. These metrics are exposed on port 9502 using an HTTP endpoint /metrics and are refreshed every five minutes.

Supported pool metrics

Name
Type
Unit
Description
disk_pool_total_size_bytes
Gauge
Integer
Total size of the pool
disk_pool_used_size_bytes
Gauge
Integer
Used size of the pool
disk_pool_status
Gauge
Integer
Status of the pool (0, 1, 2, 3) = {"Unknown", "Online", "Degraded", "Faulted"}
disk_pool_committed_size
Gauge
Integer
Committed size of the pool in bytes
# HELP disk_pool_status disk-pool status
# TYPE disk_pool_status gauge
disk_pool_status{node="worker-0",name="mayastor-disk-pool"} 1
# HELP disk_pool_total_size_bytes total size of the disk-pool in bytes
# TYPE disk_pool_total_size_bytes gauge
disk_pool_total_size_bytes{node="worker-0",name="mayastor-disk-pool"} 5.360320512e+09
# HELP disk_pool_used_size_bytes used disk-pool size in bytes
# TYPE disk_pool_used_size_bytes gauge
disk_pool_used_size_bytes{node="worker-0",name="mayastor-disk-pool"} 2.147483648e+09
# HELP disk_pool_committed_size_bytes Committed size of the pool in bytes
# TYPE disk_pool_committed_size_bytes gauge
disk_pool_committed_size_bytes{node="worker-0", name="mayastor-disk-pool"} 9663676416

Stats exporter metrics

When eventing is activated, the stats exporter operates within the obs-callhome-stats container, located in the callhome pod. The statistics are made accessible through an HTTP endpoint at port 9090, specifically using the /stats route.

Supported stats metrics

Name
Type
Unit
Description
pools_created
Guage
Integer
Total successful pool creation attempts
pools_deleted
Guage
Integer
Total successful pool deletion attempts
volumes_created
Guage
Integer
Total successful volume creation attemtps
volumes_deleted
Guage
Integer
Total successful volume deletion attempts

Integrating exporter with Prometheus monitoring stack

  1. 1.
    To install, add the Prometheus-stack helm chart and update the repo.
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
Then, install the Prometheus monitoring stack and set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues to false. This enables Prometheus to discover custom ServiceMonitor for Mayastor.
helm install mayastor prometheus-community/kube-prometheus-stack -n mayastor --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false
  1. 2.
    Next, install the ServiceMonitor resource to select services and specify their underlying endpoint objects.
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: mayastor-monitoring
labels:
app: mayastor
spec:
selector:
matchLabels:
app: mayastor
endpoints:
- port: metrics
Upon successful integration of the exporter with the Prometheus stack, the metrics will be available on the port 9090 and HTTP endpoint /metrics.

CSI metrics exporter

Name
Type
Unit
Description
kubelet_volume_stats_available_bytes
Gauge
Integer
Size of the available/usable volume (in bytes)
kubelet_volume_stats_capacity_bytes
Gauge
Integer
The total size of the volume (in bytes)
kubelet_volume_stats_used_bytes
Gauge
Integer
Used size of the volume (in bytes)
kubelet_volume_stats_inodes
Gauge
Integer
The total number of inodes
kubelet_volume_stats_inodes_free
Gauge
Integer
The total number of usable inodes.
kubelet_volume_stats_inodes_used
Gauge
Integer
The total number of inodes that have been utilized to store metadata.