For Kubernetes, a DaemonSet ensures that some or all nodes run a copy of a pod. This implementation is uses a Fluentd DaemonSet to collect Kubernetes logs. Fluentd is flexible enough and has the proper plugins to distribute logs to different third parties such as Logz.io.
The logzio-k8s image comes pre-configured for Fluentd to gather all logs from the Kubernetes node environment and append the proper metadata to the logs.
You have two options for deployment:
For most environments, we recommend using the default configuration. However, you can deploy a custom configuration if your environment needs it.
Save your Logz.io shipping credentials as a Kubernetes secret.
Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.
Replace <<LISTENER-HOST>> with your region's listener host (for example, listener.logz.io).
For more information on finding your account's region,
see Account region.
kubectl create secret generic logzio-logs-secret \
--from-literal=logzio-log-shipping-token='<<SHIPPING-TOKEN>>' \
--from-literal=logzio-log-listener='https://<<LISTENER-HOST>>:8071' \
-n kube-systemFor an RBAC cluster:
kubectl apply -f https://raw.githubusercontent.com/logzio/logzio-k8s/master/logzio-daemonset-rbac.yamlOr for a non-RBAC cluster:
kubectl apply -f https://raw.githubusercontent.com/logzio/logzio-k8s/master/logzio-daemonset.yamlFor container runtime Containerd:
kubectl apply -f https://raw.githubusercontent.com/logzio/logzio-k8s/master/logzio-daemonset-containerd.yamlGive your logs some time to get from your system to ours, and then open Kibana.
If you still don't see your logs, see log shipping troubleshooting.
You can customize the configuration of the Fluentd container. This is done using a ConfigMap that overwrites the default DaemonSet.
Save your Logz.io shipping credentials as a Kubernetes secret.
Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.
Replace <<LISTENER-HOST>> with your region's listener host (for example, listener.logz.io).
For more information on finding your account's region,
see Account region.
kubectl create secret generic logzio-logs-secret \
--from-literal=logzio-log-shipping-token='<<SHIPPING-TOKEN>>' \
--from-literal=logzio-log-listener='https://<<LISTENER-HOST>>:8071' \
-n kube-systemDownload either the RBAC DaemonSet or the non-RBAC DaemonSet or the containerd Daemonset and open the file in your text editor.
Customize the integration environment variables configurations with the parameters shown below.
Parameters
| Parameter | Description |
|---|---|
| output_include_time | Default: true To append a timestamp to your logs when they're processed, true. Otherwise, false. |
| LOGZIO_BUFFER_TYPE | Default: file Specifies which plugin to use as the backend. |
| LOGZIO_BUFFER_PATH | Default: /var/log/Fluentd-buffers/stackdriver.buffer Path of the buffer. |
| LOGZIO_OVERFLOW_ACTION | Default: block Controls the behavior when the queue becomes full. |
| LOGZIO_CHUNK_LIMIT_SIZE | Default: 2M Maximum size of a chunk allowed |
| LOGZIO_QUEUE_LIMIT_LENGTH | Default: 6 Maximum length of the output queue. |
| LOGZIO_FLUSH_INTERVAL | Default: 5s Interval, in seconds, to wait before invoking the next buffer flush. |
| LOGZIO_RETRY_MAX_INTERVAL | Default: 30s Maximum interval, in seconds, to wait between retries. |
| LOGZIO_FLUSH_THREAD_COUNT | Default: 2 Number of threads to flush the buffer. |
| LOGZIO_LOG_LEVEL | Default: info The log level for this container. |
For the RBAC DaemonSet:
kubectl apply -f /path/to/logzio-daemonset-rbac.yamlFor the non-RBAC DaemonSet:
kubectl apply -f /path/to/logzio-daemonset.yamlFor container runtime Containerd:
kubectl apply -f https://raw.githubusercontent.com/logzio/logzio-k8s/master/logzio-daemonset-containerd.yamlGive your logs some time to get from your system to ours, and then open Kibana.
If you still don't see your logs, see log shipping troubleshooting.
To suppress Fluentd system messages, set the FLUENTD_SYSTEMD_CONF environment variable to disable in your Kubernetes environment.
By default, latest images launch prometheus plugins to monitor fluentd.
You can disable prometheus input plugin by setting disable to FLUENTD_PROMETHEUS_CONF environment variable in your kubernetes configuration.
- v1.1.3
- Support containerd.
- v1.1.2
- Fix token display issue.
- v1.1.1
- Upgrade fluentd base image to v.1.10.4
- v1.1.0
- Update deprecated conifg
- v1.0.9
- Update base image
- Update libjemalloc package
- v1.0.8
- Update deprecated APIs
- v1.0.7
- Update dependencies
- v1.0.6
- Use Kubernets secrets for Shipping Token and Listener URL.
- Fix log level