Fluentd plugin to read data from files and move it Swift object storage as is.
Add this line to your application's Gemfile:
gem 'fluent-plugin-swift-sweep'Or install it yourself as:
$ gem install fluent-plugin-swift-sweep
Assume your files are inside /tmp/test directory as
tmp/test
├── core.files1.log
├── core.files2.log
└── core.files2.log
This plugin watches the directory (file_path_with_glob tmp/test/*.log).
And reads the file and sends it to swift object storage. Finally removes the file, after sending it to swift.
The files after getting moved to swift container:
├── tmp/test/core.files1.log
├── tmp/test/core.files2.log
├── tmp/test/core.files3.log
<source>
type swift_swwep
# Required. process files that match this pattern using glob.
file_path_with_glob /tmp/imp/*.log
# Required. Authentication URL
auth_url <Authentication url>
# Required. Authenticated User Name
auth_user <User name>
# Required. Password
auth_api_key <Password>
# Required. The name of the openstack tenant
auth_tenant <Openstack Tenant>
# Required. The name of the swift container
swift_container <Container Name>
ssl_verify false
</source>
- This plugin supports fluentd from v0.12.33
- Fork it ( https://github.com/deepakarumugham/fluent-plugin-swift-sweep/fork )
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request