Prometheus 蒐集 node-exporter、kube-state-metrics

Devops > Prometheus K8S Garfana Prometheus

發布時間 : 2022-06-20 00:14

字數:517 觀看數 :

評論:

前言
建立 node-exporter 服務
建立 kube-state-metrics 服務
調整 prometheus.yml
匯入 Garfana Dashboard

前言

本篇記錄透過 Prometheus 蒐集 node-exporter、kube-state-metrics

建立 node-exporter 服務

K8S機器上建立 node-exporter 服務

cat <<EOF | kubectl create -f -
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    name: node-exporter
  name: node-exporter
  namespace: monitor
spec:
  # replicas: 1
  # revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: node-exporter
  template:
    metadata:
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: "9100"
        prometheus.io/scrape: "true"
      labels:
        name: node-exporter
    spec:
      hostNetwork: true
      hostPID: true
      containers:
      - image: prom/node-exporter:v1.0.1
        name: node-exporter
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9100
        resources:
          requests:
            cpu: 50m
            memory: 50Mi
          limits: 
            cpu: 50m
            memory: 50Mi
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: node-exporter
  name: node-exporter
  namespace: monitor
  # annotations:
  #   prometheus.io/scrape: 'true'
spec:
  ports:
    - port: 9100
      targetPort: 9100
      nodePort: 30010
      protocol: TCP
  type: NodePort    
  selector:
    name: node-exporter
EOF

開通防火牆 Port 30010

gcloud compute firewall-rules create test-node-port \
    --allow tcp:30010 \
    --source-ranges "35.194.154.x" \
    --target-tags=web-node-pool

建立 kube-state-metrics 服務

kube-state-metrics 版本參考 kube-state-metrics github kubernetes/kube-state-metrics
本篇使用 release-2.2 版本

git clone https://github.com/kubernetes/kube-state-metrics.git
git checkout release-2.2

調整一下 service.yaml 從 ClusterIP 改成 NodePort

vi .\kube-state-metrics\examples\standard\service.yaml

service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.2.4
  name: kube-state-metrics
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - name: http-metrics
    port: 8080
    targetPort: http-metrics
    nodePort: 30080
  - name: telemetry
    port: 8081
    targetPort: telemetry
    nodePort: 30081
  selector:
    app.kubernetes.io/name: kube-state-metrics

部署 kube-state-metrics

kubectl create -f .\kube-state-metrics\examples\standard\

開通防火牆 Port 30080-30081

gcloud compute firewall-rules create kube-state-metrics-port \
    --allow tcp:30080-30081 \
    --source-ranges "35.194.154.x"
    --target-tags=web-node-pool

調整 prometheus.yml

...略
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "k8s"
    
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["104.199.202.68:30010"]
  - job_name: "k8s-http-metrics"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["104.199.202.68:30080"]
  - job_name: "k8s-telemetry"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["104.199.202.68:30081"]

熱更新 prometheus

curl -X POST http://127.0.0.1:9090/-/reload

查看一下 prometheus targets 確認是否有抓到 metrics

匯入 Garfana Dashboard

node exporter - Garfana Dashboard

kube-state-metrics-v2 - Garfana Dashboard

Kubernetes Cluster Overall Dashboard - Garfana Dashboard

1 K8S for Prometheus Dashboard 20211010 - Garfana Dashboard
starsliao/Prometheus

轉載請註明來源，若有任何錯誤或表達不清楚的地方，歡迎在下方評論區留言，也可以來信至 leozheng0621@gmail.com
如果文章對您有幫助，歡迎斗內(donate)，請我喝杯咖啡