[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

bigobject-inc/fluent-plugin-bigobject

Repository files navigation

fluent-plugin-bigobject

Fluentd output plugin for inserting data to BigObject

Installation

Add this line to your application's Gemfile:

gem 'fluent-plugin-bigobject'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fluent-plugin-bigobject

Usage

Configure BigObject URL and the table/column to be mapped in BigObject

#sample source to read csv file
<source>
  type tail

  #path- where you placed your input data
  path ./input/Customer.csv

  # pos_file where you record file position
  pos_file ./log/customer.log.pos

  # for bigobject output plugin, use tag bigobject.${table_pattern}.${event}.${primary_key}
  # ${primary_key} is not needed for insert
  tag bigobject.cust.insert

  #input file format
  format csv

  # keys - columns in csv file
  keys id,name,language,state,company,gender,age

  #types - string/bool/integer/float/time/array
  types age1:integer

</source>

# Send data to BigObject using Restful API. Tables need to be created in advance in BigObject.
# depending on the event in tag received, will send data to BigObject for insert/update/delete. 
#
# Tag for each event - bigobject.${table_pattern}.${event}.${primary_key}.
# ${table_pattern} : will match to the <pattern> in <table> section of bigobject output plugin
# ${event} : valid event type by insert/update/delete.
# ${primary_key} : the primary key for table, optional for insert event.
# if primary_key is integer type in BigObject, set bo_primary_key_is_int to true 
#
# Eg:
# tag bigobject.cust.insert ==> INSERT INTO <table> VALUES ...
# tag bigobject.cust.delete.id ==> DELETE FROM <table> WHERE id=...
# tag bigobject.cust.update.id ==> UPDATE <table> SET ... WHERE id=...

<match bigobject.**>
  type bigobject

  log_level info

  # specify the bigobject host/port to connect to
  bigobject_hostname 192.168.59.103
  bigobject_port 9090

  remove_tag_prefix bigobject.
  flush_interval 60s

  <table>
      table Customer
      pattern cust

      #optional-
      #bo_primary_key_is_int true #defualts to false
      #column_mapping id,name,language,state,company,gender,age
      #bo_workspace
      #bo_opts
  </table>
</match>

License

The gem is available as open source under the terms of the MIT License.

About

Fluentd output plugin for BigObject

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages