Ingest using FluentD


FluentD is an open-source log collector that allows you to collect, aggregate, process, analyze and route log files.

With FluentD you can collect logs from multiple sources and ship it instantly into Axiom


Visit the FluentD download page to install FluentD on your system.


FluentD lifecycle consist of five different components which are:

  • Setup: Configure your fluent.conf file.
  • Inputs: Define your input listeners.
  • Filters: Create a rule to allow or disallow an event.
  • Matches: Send output to Axiom when input data match and pair specific data from your data input within your configuration.
  • Labels: Groups filters and simplifies tag handling.

When setting up fluentD, the configuration file .conf is used to connect its components.

The example below shows fluentD configuration that sends data to Axiom:

=== "Conf"

# FluentD is listening for forward, monitor agent and debug agent using the source element.
# built-in TCP input
# $ echo <json> | fluent-cat <tag>
@type forward
@id forward_input
@type monitor_agent
@id monitor_agent_input
port 24220
# Listen DRb for debug
@type debug_agent
@id debug_agent_input
port 24230
## matches the input and sends output to **Axiom host**
<match *.**>
@type elasticsearch
## host is the url of your Axiom deployment
hosts "<my-datasets>/elastic"
## api_key can be your ingest or personal token
api_key "user:token"
verify_es_version_at_startup false