Azure Kubernetes Service Logs

  • Updated on Sep 1, 2024
  • Published on Aug 29, 2024

Self Service Ingestion

Connect this data source on your own, using the Hunters platform.

Overview

Table name: azure_aks

Azure Kubernetes Service (AKS) is a managed container orchestration service provided by Microsoft Azure that simplifies the deployment, management, and operations of Kubernetes. AKS automates the provisioning, upgrading, and scaling of resources, allowing developers to focus on building and maintaining applications rather than managing the underlying infrastructure. It integrates seamlessly with other Azure services, enhancing capabilities such as monitoring, security, and identity management. AKS supports continuous integration and continuous deployment (CI/CD) workflows, making it an ideal choice for modern, microservices-based application development. The service ensures high availability and robust disaster recovery mechanisms, thus offering a reliable platform for running containerized workloads in the cloud.

Send data to Hunters

Hunters supports the ingestion of these logs using Azure Event Hub. Follow the steps below to complete the connection.

STEP 1: Set up Azure Event Hub

Before setting up the connection on the Hunters platform, you'll need to set up and create an Azure Event Hub.

Follow this guide to complete the set up.

STEP 2: Route logs to the Event Hub

  1. In the Azure portal home screen, search for and open Diagnostic Settings.
    Azure diagnostic settings
  2. Select the relevant Kubernetes resource whose logs you want to connect.
  3. Click Add Diagnostic setting.
    Resource group new diagnostic setting
  4. Under Logs, check the Kubernetes Audit checkbox.
  5. Under Destination details, check the Stream to an Event Hub option.
  6. Fill in the requested details and give the diagnostic setting a name.
  7. Click Save.

STEP 3: Set up the connection on Hunters

📘 Before you begin
To complete this process you will need the information gathered when following this guide.
To connect logs to Hunters:
  1. Open the Hunters platform and navigate to Data > Data Sources.
    Data sources1
  2. Click ADD DATA SOURCES.
    Add data source4
  3. Locate the Microsoft Azure panel and click Connect.
    The Add Data Flows window opens.
  4. Fill in the required Azure application details, as gathered here under STEP 2.
    Connect Azure logs on Hunters
  5. Under the Data Types section, activate the data types you want to connect.
  6. For each activated data type, fill in the required information, as gathered here:
    1. Under STEP 1 - Subscription ID
    2. Under STEP 3 - Resource group name and Event Hub namespace and
    3. Under STEP 4 - Event Hub name.
  7. OPTIONAL: Under the Consumer group field you can specify a specific Azure Event Hub consumer group, or leave this field empty to use the default consumer group.
  8. Click Test Connection to make sure everything was set up correctly.
  9. Once the connection is established, click Submit.

Expected format

Logs are expected in JSON format.

{
    "category": "kube-audit",
    "operationName": "Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
    "properties": {
        "containerID": "fasfawfaewgiuo2h3t98o2htoi23nj4g982n4gh",
        "log": "{\"kind\":\"Event\",\"apiVersion\":\"audit.k8s.io\/v1\",\"level\":\"Metadata\",\"auditID\":\"d0394b5f-4eee-45d3-89a9-8e5a9cc7e689\",\"stage\":\"ResponseComplete\",\"requestURI\":\"\/apis\/coordination.k8s.io\/v1\/namespaces\/test-namespace\/leases\/test-customers\",\"verb\":\"patch\",\"user\":{\"username\":\"system:serviceaccount:test-portal\",\"uid\":\"dac2a960-6bf3-49ff-baf2-79a9fffc904e\",\"groups\":[\"system:serviceaccounts\",\"system:serviceaccounts:test-portal\",\"system:authenticated\"],\"extra\":{\"authentication.kubernetes.io\/pod-name\":[\"test-customers-66d68fb56d-lxfpd\"],\"authentication.kubernetes.io\/pod-uid\":[\"19ea7935-c7b2-4fe0-b2ed-2f6adf0d77fe\"]}},\"sourceIPs\":[\"1.2.3.4\"],\"userAgent\":\"fabric8-kubernetes-client\/6.9.2\",\"objectRef\":{\"resource\":\"leases\",\"namespace\":\"test-portal\",\"name\":\"test-customers\",\"apiGroup\":\"coordination.k8s.io\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"metadata\":{},\"code\":200},\"requestReceivedTimestamp\":\"2024-03-13T10:00:00.702537Z\",\"stageTimestamp\":\"2024-03-13T10:00:00.707337Z\",\"annotations\":{\"authorization.k8s.io\/decision\":\"allow\",\"authorization.k8s.io\/reason\":\"RBAC: allowed by RoleBinding \\\"test-customers-leader\/test-portal\\\" of Role \\\"test-customers-leader\\\" to ServiceAccount \\\"test-customers\/test-portal\\\"\"}}\n",
        "pod": "kube-apiserver-86d9cdc87c-z5btp",
        "stream": "stdout"
    },
    "resourceId": "/SUBSCRIPTIONS/AFOIWJAWF-ASDASD-ASDASD/RESOURCEGROUPS/KUBERNETES/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/TEST-CLUSTER",
    "serviceBuild": "na",
    "time": "2024-03-13T10:00:00.707455698Z"
}