How to monitor KUBERNETES INVENTORY servers

What is Kubernetes Inventory?

Kubernetes — also known as k8s or kube — is a container orchestration platform for scheduling and automating the deployment, management, and scaling of containerized applications.

An accurate and up-to-date Kubernetes Inventory is critical to business success due to its ability to reduce inventory errors, improve an organization's ability to meet customer demands, and lower operational costs. Kubernetes inventory keeps track of the Nodes, Pod Containers, and Daemon Sets deployed in a Kubernetes cluster.

Monitoring Kubernetes Inventory

ServicePilot monitors a Kubernetes cluster by communicating with the Kubelet API from a ServicePilot Agent installed in a Pod. It is assumed that ServicePilot Agents are running in Pods as part of a DaemonSet within a Kubernetes cluster so that each ServicePilot Agent can report on its Cluster statistics.

The statistics gathered in this way include:

• DaemonSet
  • State and number of pods running the DaemonSet
• Deployments
  • Replicas state
• Node
  • Pods usage
  • Memory usage
  • CPU usage
• Pod Containers
  • State
  • Memory requests and limit
  • Restarts

Requirements

• Network Flows - It is necessary to ensure network connectivity between the monitored device and ServicePilot.

  • TCP/443 or 80 (HTTPS/HTTP): Between ServicePilot Agent deployed in Kubernetes Pods and ServicePilot Manager

• Third Party Configuration

  • Only for GKE: If you are using Google cloud GKE, you need to run the following commands as you need privileges to create cluster roles for ServicePilot Agent setup.

    ACCOUNT=$(gcloud info --format='value(config.account)')kubectl create clusterrolebinding owner-cluster-admin-binding \   --clusterrole cluster-admin \   --user $ACCOUNT

  • Deploy a ServicePilot Agent DaemonSet to the Kubernetes cluster. Note the SERVICEPILOT_* environment variables that need to match your ServicePilot deployment.

    # Create a namespace for ServicePilot componentsapiVersion: v1kind: Namespacemetadata: name: monitoring labels:   app: servicepilot-agent---# To have ServicePilot retrieve metrics from Kubelets with authentication and# authorization enabled (which is highly recommended and included in security# benchmarks) the following flags must be set on the kubelet(s):## --authentication-token-webhook# --authorization-mode=Webhook#apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: servicepilot-stats-viewer labels:   rbac.authorization.k8s.io/servicepilot-stats-viewer: "true"rules: - apiGroups: [""]   resources:     - nodes     - nodes/proxy     - nodes/metrics     - nodes/stats     - services     - endpoints     - pods     - ingresses     - configmaps     - persistentvolumes   verbs: ["get", "list", "watch"] - apiGroups: ["extensions", "networking.k8s.io"]   resources:     - ingresses/status     - ingresses   verbs: ["get", "list", "watch"] - apiGroups: ["metrics.k8s.io"]   resources:      - pods   verbs: ["get", "list", "watch"] - nonResourceURLs:     - "/metrics"   verbs: ["get"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: servicepilot labels:   app: servicepilot-agentaggregationRule: clusterRoleSelectors:   - matchLabels:       rbac.authorization.k8s.io/servicepilot-stats-viewer: "true"   - matchLabels:       rbac.authorization.k8s.io/aggregate-to-view: "true"rules: [] # Rules are automatically filled in by the controller manager.---apiVersion: v1kind: ServiceAccountmetadata: name: servicepilot labels:   app: servicepilot-agent namespace: monitoring---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: servicepilot labels:   app: servicepilot-agentsubjects: - kind: ServiceAccount   name: servicepilot   namespace: monitoringroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: servicepilot---# ServicePilot Agent deploymentapiVersion: apps/v1kind: DaemonSetmetadata: name: servicepilot-agent namespace: monitoring labels:   app: servicepilot-agentspec: selector:   matchLabels:     app: servicepilot-agent template:   metadata:     labels:       app: servicepilot-agent   spec:     tolerations:     # this toleration is to have the daemonset runnable on master nodes     # remove it if your masters can't run pods     - key: node-role.kubernetes.io/control-plane       operator: Exists       effect: NoSchedule     - key: node-role.kubernetes.io/master       operator: Exists       effect: NoSchedule     serviceAccountName: servicepilot     containers:       - name: servicepilot         image: servicepilot/agent         env:         - name: SERVICEPILOT_API_KEY           value: "10000000-0000-0000-0000-000000000000"         - name: SERVICEPILOT_IP           value: "sp1.company.com"         - name: SERVICEPILOT_PORT           value: "443"         - name: HTTPS           value: "1"         - name: NODE_NAME           valueFrom:             fieldRef:               fieldPath: spec.nodeName         - name: NODE_IP           valueFrom:             fieldRef:               fieldPath: status.hostIP         - name: HOSTNAME           value: $(NODE_NAME)

• ServicePilot Requirements

Add a server-kubernetes-inventory resource for each Kubernetes Cluster, selecting the ServicePilot Agent running on the associated Cluster.

Key field notes

1. General tab:

   1. Filter NameSpaces: Check to filter inventory by NameSpace
   2. NameSpace: Specify the NameSpace to monitor
   3. Add Selectors: Add selectors to the inventory monitoring
   4. Extend timeout: Extend Kubelet API response timeout
   5. Check Kubelet certificate: Check the validity of Kubelet API web certificate when using non-self signed Kubelet certificates. ServicePilot Pods will need to be restarted when CA certificate changes.

2. Resources tab:

   1. Filter Resources: Check to filter resources to monitor
   2. Daemon Sets: Monitor Daemon Sets
   3. Deployments: Monitor Deployments
   4. Endpoints: Monitor Endpoints
   5. Ingress: Monitor Ingress
   6. Nodes: Monitor Nodes
   7. Persistent Volumes: Monitor Persistent Volumes
   8. Persistent Volume Claims: Persistent Volume Claims
   9. Pods: Monitor Pods
   10. Services: Monitor Services
   11. Stateful Sets: Monitor Stateful Sets

3. In the Policies tab, specify the policy or policies to apply to the resource

License

How to install a kubernetes-inventory resource?

1. Use your ServicePilot OnPremise installation or a SaaS account.
2. Add a new kubernetes-inventory resource via the web interface (/prmviews or /prmresources) or via API (/prmpackages page), the default ServicePilot agent or another agent will be provisioned automatically.

Details of the kubernetes-inventory package are located in the /prmpackages page of the software.

Benefits

ServicePilot enables you to deliver IT services faster and more securely with automated discovery and advanced monitoring features.

By correlating the technology KUBERNETES INVENTORY with APM and infrastructure monitoring, ServicePilot is able to provide a more comprehensive view of an organization's IT environment.

This allows IT teams to quickly identify and diagnose issues that may be impacting application performance, and take corrective action before end-users are affected.

Start with a free trial of our SaaS solution. Explore ourplans or contact us to find what works best for you.