DigitalOcean Kubernetes (DOKS) is a managed Kubernetes service that lets you deploy Kubernetes clusters without the complexities of handling the control plane and containerized infrastructure.
In this tutorial, I will show you how to:
- Create a Kubernetes Cluster on DigitalOcean
- Ingress logs from DigitalOcean Kubernetes Service to Axiom
- Create a dataset & Ingest token on Axiom.
- Analyze and Stream your Kubernetes logs on Axiom.
Axiom centralizes all your logs from all your log sources, so you have complete visibility into your entire system. Using Axiom for log management means that you can monitor your event logs more effectively and gain deeper insight into your system metrics.
Create your account on DigitalOcean, and enter your details.
Select Kubernetes in the dashboard menu/control plane, select the Kubernetes version you want to work with (the latest version is selected by default), and choose the data center region you want to create your cluster in.
- Customize the default node pool, you can choose the names you want to use for your node pool, select the machine type and node plan you want to work with for your cluster capacity, you can create more add additional node pool.
- Name the cluster, add a tag if you want to, and select the project you want your cluster to belong to.
- Click Create Cluster and follow the Getting Started steps to Provision your cluster. Setting up the cluster takes some time.
- Download the cluster configuration file by clicking Actions, then Download Configuration from the cluster home page.
- Create a token, Click the API link in the main navigation, which takes you to the Applications & API page on the Tokens/Keys tab. In the Personal access tokens section, click the Generate New Token button.
- Authenticate your DigitalOcean Account to use the
doctlCLI by typing
doctl auth initin your terminal, then paste the token you generated from the control plane in step 7.
Get the Kubernetes cluster list by typing
doctl kubernetes cluster liston your terminal.
Save your cluster credentials using
doctl kubernetes cluster kubeconfig save <cluster name>
Confirm to be sure that your current context is the same as the cluster you created.
kubectl config get-contextsto see the clusters you set-up
kubectl config current-contextstells you which cluster is your active
kubectl config use-context [name]lets you switch to a different one.
- You can see the different nodes name and status on your terminal by running
kubectl get nodes
- Head over to Axiom, Head to Settings → Datasets and create a new dataset for your DKS logs.
- Create an ingest token on the Axiom UI, to create an ingest token.
- On the Axiom UI, Click on settings, select ingest token.
- Select Add ingest token.
- Enter a name and description and select ADD.
- Copy the generated token to your clipboard. Once you navigate from the page, the token can be seen again by selecting Ingest Tokens.
- Axiom uses Filebeats to forward logs from your Kubernetes cluster. Setting it up is as simple as copying the daemonset’s yaml file from the Axiom Docs and configuring it by adding the environment variable values for your Axiom host, dataset, and the ingest token.
env: - name: AXIOM_HOST value: http://cloud.axiom.co // replace with your self-host url if needed - name: AXIOM_DATASET_NAME value: aks-logs - name: AXIOM_INGEST_TOKEN value: xait-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
- Apply and run your
kubectl apply -f (NAME OF FILE).yaml
- The Filebeats nodes will be created and you should see them running on the DigitalOcean cloud portal.
- Back in Axiom, you will see your logs from DOKS been ingested live to the dataset you created earlier.
- You can now collect and store insights on your logs and run aggregations across all the matching data to generate visualization as well as a table of results.
- You can also stream your logs on Axiom, this allows you to inspect individual events from DigitalOcean Kubernetes Service and watch as they are ingested live.
You can now see how direct it is for you to ingress logs from DigitalOcean Kubernetes Service to Axiom. You can read more about Axiom Kubernetes configuration on our docs.
We do love discussing logging, observability, analytics if you have any thoughts, questions, or suggestions kindly contact us. You can also follow us on Twitter and on our blog. And if you’ve enjoyed this post, please, take a second to share it on Twitter.