##Toiler Toiler is a AWS SQS long-polling thread-based message processor. It's based on shoryuken but takes a different approach at loadbalancing and uses long-polling.
##Features ###Concurrency Toiler allows to specify the amount of processors (threads) that should be spawned for each queue. Instead of shoryuken's loadbalancing approach, Toiler delegates this work to the kernel scheduling threads.
###Long-Polling A Fetcher thread is spawned for each queue. Fetchers are resposible for polling SQS and retreiving messages. They are optimised to not bring more messages than the amount of processors avaiable for such queue. By long-polling fetchers wait for a configurable amount of time for messages to become available on a single request, this prevents unneccesarilly requesting messages when there are none.
###Message Parsing Workers can configure a parser Class or Proc to parse an SQS message body before being processed.
###Batches Toiler allows a Worker to be able to receive a batch of messages instead of a single one.
##Instalation
Add this line to your application's Gemfile:
gem 'toiler'And then execute:
$ bundle
Or install it yourself as:
$ gem install toiler
class MyWorker
include Toiler::Worker
toiler_options queue: 'default', concurrency: 5, auto_delete: true
toiler_options parser: :json
# toiler_options parser: ->(sqs_msg){ REXML::Document.new(sqs_msg.body) }
# toiler_options parser: MultiJson
# toiler_options auto_visibility_timeout: true
# toiler_options batch: true
def perform(sqs_msg, body)
puts body
end
endaws:
access_key_id: ... # or <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: ... # or <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
region: us-east-1 # or <%= ENV['AWS_REGION'] %>
wait: 20 # The time in seconds to wait for messages during long-pollingYou can tell Toiler to load your Rails application by passing the -R or --rails flag to the "toiler" command.
If you load Rails, and assuming your workers are located in the app/workers directory, they will be auto-loaded. This means you don't need to require them explicitly with -r.
bundle exec toiler -r worker.rb -C toiler.ymlOther options:
toiler --help
-d, --daemon Daemonize process
-r, --require [PATH|DIR] Location of the worker
-C, --config PATH Path to YAML config file
-R, --rails Load Rails
-L, --logfile PATH Path to writable logfile
-P, --pidfile PATH Path to pidfile
-v, --verbose Print more verbose output
-h, --help Show helpMuch of the credit goes to Pablo Cantero, creator of Shoryuken, and everybody who contributed to it.
- Fork it ( https://github.com/sschepens/toiler/fork )
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request