Edit

Share via

Kubernetes monitoring and logging

  • Article

This article compares Azure Kubernetes Service (AKS) monitoring and Amazon Elastic Kubernetes Service (EKS) monitoring. It describes options that you can use to monitor and manage the logs of an AKS cluster and its workloads.

Note

This article is part of a series of articles that helps professionals who are familiar with Amazon EKS understand Azure Kubernetes Service (AKS).

Amazon EKS monitoring and logging

Like other Kubernetes services, Amazon EKS has two main components, the control plane and worker nodes. Each layer has specific capabilities.

Amazon EKS control plane and cluster monitoring

Amazon EKS integrates with Amazon CloudWatch Logs to provide logging and monitoring for the Amazon EKS control plane. This integration isn't enabled by default. You must configure the integration to gather logs on the following components:

  • API server and API calls
  • Audit logs and user interactions
  • Authentication processes
  • Scheduler and controller activities

Amazon EKS exposes control plane metrics at the /metrics endpoint, in Prometheus text format. CloudWatch Container Insights collects and stores Prometheus metrics. You can deploy and self manage Prometheus on top of your EKS cluster, or use Amazon Managed Service for Prometheus.

Amazon EKS also integrates with Amazon Web Services (AWS) CloudTrail to track actions and API calls. For more information, see Log Amazon EKS API calls by using AWS CloudTrail.

Amazon EKS workload monitoring

CloudWatch Container Insights collects and aggregates metrics and logs from containerized applications that are deployed in EKS. To implement Container Insights on Amazon EKS, use a containerized version of the CloudWatch agent, or use AWS Distro for OpenTelemetry as a DaemonSet. You can use Fluent Bit to send logs.

AKS monitoring and logging

Like other Azure resources, AKS generates platform metrics and resource logs that you can use to monitor its basic health and performance.

Diagram that shows an AKS logging and monitoring solution.

Download a Visio file of this architecture.

Azure Monitor

AKS natively integrates with Azure Monitor. Azure Monitor stores metrics and logs in a central location called a Log Analytics workspace. Azure Monitor processes and analyzes the metrics and logs to provide insights and alerts. For more information, see Monitor AKS by using Azure Monitor.

Container insights is a feature of Azure Monitor that collects, indexes, and stores data that your AKS cluster generates. You can configure container insights to monitor managed Kubernetes clusters that are hosted on AKS. You can also monitor other cluster configurations. Container insights monitors AKS health and performance and presents that data via visualizations that are tailored to Kubernetes environments. Similar to EKS, when you enable container insights for your AKS cluster, it deploys a containerized version of the Log Analytics agent. The agent sends data to your Log Analytics workspace.

Container insights uses data from a Log Analytics workspace to power the visualizations in the Azure portal. Consider switching to Azure Monitor managed service for Prometheus, which provides a cheaper and more efficient approach to metrics collection. You can use container insights to visualize metrics by using only managed Prometheus data. For more information, see Switch to managed Prometheus visualizations for container insights.

To help ensure comprehensive monitoring of your Kubernetes clusters, use the following services and Azure Monitor features:

Microsoft Sentinel

Microsoft Sentinel delivers intelligent security analytics and threat intelligence across enterprises. Microsoft Sentinel provides a single solution for attack detection, threat visibility, proactive hunting, and threat response.

You must connect Microsoft Sentinel with AKS by using the AKS connector. Then you can stream your AKS diagnostics logs into Microsoft Sentinel to continuously monitor activity in your instances.

After you connect your data sources to Microsoft Sentinel, you can visualize and monitor the data. Microsoft Sentinel and Azure Monitor workbooks provide versatility to create custom dashboards.

AKS cluster and workload monitoring

An AKS deployment consists of cluster-level components, managed AKS components, Kubernetes objects and workloads, applications, and external resources. A common strategy to monitor an AKS cluster and workload applications consists of the following monitoring requirements.

Level Description Monitoring requirements
Cluster-level components Virtual machine scale sets represent AKS nodes and node pools Node status and resource usage, including CPU, memory, disk, and network
Managed AKS components AKS control plane components, including API servers, the cloud controller, and kubelet Control plane logs and metrics from the kube-system namespace
Kubernetes objects and workloads Kubernetes objects, such as deployments, containers, and replica sets Resource usage and failures
Applications Application workloads that run on the AKS cluster Architecture-specific monitoring, including application logs and service transactions
External External resources that aren't part of AKS but are required for cluster scalability and management Specific to each component

  • Cluster-level components: You can use existing container insights views and reports to monitor cluster-level components to understand their health, readiness, performance, CPU and memory resource usage, and trends.

  • Managed AKS components: You can use Azure Monitor metrics explorer to view the Inflight Requests counter. This view includes request latency and work queue processing time.

  • Kubernetes objects and workloads: You can use existing container insights views and reports to monitor the deployment, controllers, pods, and containers. Use the Nodes and Controllers views to see the health and performance of the pods that run on nodes and controllers. You can also view their resource consumption in terms of CPU and memory.

    The container insights Containers view shows the health and performance of containers. Or you can select an individual container, and monitor its events and logs in real-time. For more information, see Monitor your Kubernetes cluster performance by using container insights.

  • Applications: You can use Application Insights to monitor applications that run on AKS and other environments. Application Insights is an application performance management tool that provides support for many programming languages. Depending on your needs, you can instrument your application code to capture requests, traces, logs, exceptions, custom metrics, and end-to-end transactions. Then you can send this data to Application Insights. If you have a Java application, you can provide monitoring without instrumenting your code. For more information, see Autoinstrumentation for AKS.

  • External components: You can use Azure Monitor features to monitor Azure platform as a service (PaaS) solutions that your workload applications use, such as databases and other Azure resources.

Azure Monitor managed service for Prometheus

Prometheus is a popular open-source metrics monitoring solution from the Cloud Native Computing Foundation. Prometheus collects and analyzes metric data from Kubernetes clusters. Azure Monitor managed service for Prometheus is a fully managed Prometheus-compatible monitoring solution in Azure. It stores data in an Azure Monitor workspace that links to an Azure Managed Grafana workspace. You can use Azure Managed Grafana to analyze the data.

You can deploy Prometheus independently as a self-managed solution within AKS clusters. To integrate self-hosted Prometheus with Azure Monitor, configure container insights to collect Prometheus metrics. You can expose the Prometheus metrics endpoint through your exporters or pod applications. The containerized agent for container insights collects these metrics.

Azure Managed Grafana

Azure Managed Grafana is a data visualization platform that's built on top of Grafana. It's a fully managed Azure service that Microsoft operates and supports. Azure Managed Grafana has predefined Grafana dashboards to monitor Kubernetes and full-stack troubleshooting.

Azure Managed Grafana is optimized for the Azure environment, works with many Azure services, and provides simple integration features. You can also deploy Grafana independently as a self-managed solution. For more information, see Monitor your Azure services in Grafana.

AKS monitoring costs

The Azure Monitor pricing model is primarily based on the amount of data that your Log Analytics workspace ingests each day. The cost varies depending on the plan and retention periods that you choose.

Before you enable container insights, estimate costs and understand how to control data ingestion and its costs. For more information, see Estimate costs to monitor your AKS cluster.

Contributors

Microsoft maintains this article. The following contributors wrote this article.

Principal authors:

Other contributors:

To see nonpublic LinkedIn profiles, sign in to LinkedIn.

Next steps