Ingesting using Fluent Bit

Fluent Bit

Fluent Bit is an open source Log Processor and Forwarder which allows you to collect any data like metrics and logs from different sources, enrich them with filters and send them to multiple destinations like Axiom.

Installation

Visit the Fluent Bit download page to install Fluent Bit on your system.


You'd need to specify the org-id header if you are using personal token, it's best to use an ingest token to avoid the need to specify the org-id header.

Learn more about ingest and personal token


Configuration

Fluent Bit configuration file supports four types of sections:

  • Service: Defines global properties of your service using different keys available for a specific version.
  • Input: Defines the input plugin and base configuration of your file.
  • Filter: Defines the input plugin and configure the pattern tags for your configuration.
  • Output: Specify a destination that certain records should follow after a Tag match.

All sections will be configured in your .conf file.

example

The example below shows fluent Bit configuration that sends data to Axiom:

[SERVICE]
Flush 5
Daemon off
Log_Level debug
[INPUT]
Name cpu
Tag cpu
[OUTPUT]
Name http
Match *
Host cloud.axiom.co
Port 443
URI /api/v1/datasets/$DATASET_NAME/ingest
# Authorization Bearer should be an ingest token
Header Authorization Bearer xait-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx
compress On
format json
json_date_key _time
json_date_format iso8601
tls On