Ingress Log Data from Azure Kubernetes Service to Axiom

In this tutorial, I will show you how to ingress your logs from Microsoft Azure Kubernetes Service (AKS) to Axiom. You will be able to stream and analyze your logs.

We’ll set up an AKS deployment from scratch and configure it to log directly to Axiom. If you already have a Microsoft AKS deployment with log analytics workspace, feel free to skip to step 7.

Prerequisites

For this tutorial, you will need:

• Access to an Axiom deployment;

• Azure account, create one here if you don’t have one;

• Azure CLI installed.

Azure Kubernetes Service (AKS) is a robust and cost-effective container orchestration service that helps you deploy and manage containerized applications in seconds where additional resources are assigned automatically without the headache of managing additional servers and workflows.

Using Axiom for log management means you can log more data from AKS and keep it for longer. To summarize its benefits and to express them in simple words, you and your team will have every piece of monitoring data delving right into your Azure log analytics. These metrics can also help you to gain application insights, telemetry, or react should something go wrong. Or if you just want to see how well your services are running!

With this, you can deploy your Azure monitor logs analytics from AKS directly to Axiom.

Let's get to it 🔥

1. Make sure you’ve logged into Azure.

2. Create a resource group you will use for your AKS Configuration.

Azure resource group is a container that holds related resources for your Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization.

3. In your resource group dashboard, click Add.

4. Select Kubernetes service.

5. Create your Kubernetes Cluster:

• Enter your Kubernetes cluster name;
• Choose your Region;
• Select the Kubernetes version you want to use.

Review your configuration and click Create.

6. When your deployment is complete, click Connect to a cluster.

This gives you the commands you need to configure and interact directly with your cluster using Azure Kubernetes Service command-line tooling. Run and paste the commands in your terminal.

7. Now, let’s switch back to Axiom for a section. Head to Settings → Datasets and create a new dataset for your AKS logs.

To create an ingest Token API key for authentication, perform the steps below:

• On the Axiom UI, under Settings, select Ingest token.
• Select Add ingest token.
• Enter a name and description and select ADD.
• Copy the generated token to your clipboard. Once you navigate from the page, token can be seen again by selecting Ingest Tokens.

8. Axiom uses Filebeats to forward logs from your Kubernetes cluster data sources. Setting it up is as simple as copying the daemonset’s yaml file from the Axiom Docs and configuring it by adding the environment variable values for your Axiom host, dataset, and the ingest token. See the example template below:

env:
	- name: AXIOM_HOST
	  value: https://app.axiom.co // replace with your self-host url if needed
	- name: AXIOM_DATASET_NAME
	  value: aks-logs
	- name: AXIOM_INGEST_TOKEN
	  value: xait-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

9. Apply and run your .yaml file using kubectl apply -f (NAME OF FILE).yaml. The Filebeats pods will be created and you should see them running on the Azure portal.

10. Back in Axiom, the data collection begins. You will see your logs from AKS streaming in live to the dataset you created earlier. You can now run queries and gather insights on your resource logs and run aggregations across all the matching data to produce various types of visualizations as well as a table of results.

11. You can also stream your logs on Axiom, this allows you to inspect individual events from Azure Kubernetes Service and watch their ingestion live. Once configured, you can query results with Axiom Data Explorer and define alert rules to trigger notifications. If you are new to Axiom Processing Language (APL), which is very similar to Kusto Query Language (KQL), see the APL by Example interactive section of Axiom documentation. In this section, you can run simple sample queries and create your own example queries in the built-in APL editor!

Merci 🎀

Whew, you did it! Thanks for following along. You can read more about Axiom Kubernetes configuration on our docs.

If you have specific questions on pricing, troubleshooting, or issues configuring the file, I’d love to hear about them. Contact us here or ask a question in our Discord community!