Model bindings to aggregate Rails Models to a daggregator server
Just add it to your Gemfile and bundle
gem "daggregator_client"
bundle install
Most of what Daggregator does can be accomplished using SQL joins and aggregation functions, so why would you bother with another service?
In cases where the relationships between object types are simple and you know before hand what types of data you need to aggregate, you probably shouldn't. Daggregator provides two benefits: performance with complicated relationship structures and flexibility in constructing queries.
Daggregator stores data in a graph database which is optimized for subgraph queries. What this means is that if you're dealing with a lot of data and your relationships are nested (user -> account -> posts -> comments -> likes), Daggregator may be better at handling aggregation than SQL. For instance if you want to find out how many likes a user's comments get, Daggregator's syntax may be easier than attempting to join through 4 associations.
The second strength of Daggregator is that it allows you to quickly explore what types of data may be useful. In the example above, defining the join model betwen a user and a group of likes requires you to know all the intermediate associations; with daggregator you simply query the user class for the average number of likes.
There are also cases where data needs to be aggregated across database instances (sharded databases, or heterogenous data stores (MySQL/Redis/Mongo). In this case, Daggregator provides a convenient way to combine important data.
daggregator_client provides a basic wrapper around the JSON REST calls.
The first thing you'll need to do is to set up some basic configuration
(if you're using Rails, this should go in an initializer):
Daggregator.configure do |config|
config.server = 'my.daggregator.com' # The hostname of your daggregator server
config.port = '3000' # Defaults to 80
endNow you're ready to go! The Daggregator module provides methods for all
the API endpoints:
Daggregator.get_node 'foo'
Daggregator.get_aggregates 'foo', 'sum', ['key1', 'key2'] # :sum / :count (more to come)
Daggregator.put_node 'foo', {'key1' => 1, 'key2' => 3}
Daggregator.put_flow 'source', ['target1', 'target2']
Daggregator.delete_key 'foo', 'key1'
Daggregator.delete_flow 'source', 'target'
Daggregator.delete_key 'foo', 'key1'The response from these functions will either be a hash (see the
Daggregator API for the structure
of the responses) or an exception. daggregator-client uses rest-client
for transport; see their documentation
for a list of the exceptions which will be raised on non-200 responses.
If you're using ActiveModel-esque models, daggregator-client provides a
DSL for mapping attributes on your models to keys on daggregator
nodes and has_many relationships between your models to daggregator flows.
All you need to do is specify which attributes and relationships
you want to track.
# app/models/user.rb
class User
include Daggregator::Model # Include the DSL
has_many :postings
has_many :conversations, :through => :postings
has_many :friendings
has_many :friends, :through => :friendings
aggregate_to do |node|
node.key :popularity # Maps user.popularity to nod[:popularity]
node.key :user_age, :from => :age # Maps user.age to node[:user_age]
node.key :friend_count do # Block syntax. Assigns the result of executing the block in
friends.count # instance's scope to the key specified (friend_count)
end
node.flow_to :conversations, :as => :thread # Map conversations relationship to flows
# Since converstions map to two nodes (see below), we must
# specify which node type we want to flow to
end
aggregate_to(:friend) do |node|
node.key :friends_age, :from => :age
node.key :friends_popularity, :from => :popularity
node.flow_to :friends do # If a block is specified, it will be executed
select(:id).where('popularity > 100') # in the association's scope and the returned value
end # will be used
end
end# app/models/conversation.rb
class Conversation
include Daggregate
has_many :postings
has_many :users, :through => :postings
aggregate_to(:contrib) do |node| # Passing an argument adds a key {:type => :contrib} to the node. Defaults to class name
node.identifier do |conversation| # Constructe node identifier a block. The model instance will be passed
"contribution_to_#{conversation.id}" # to the block.
end # Defaults to the node type and instance id (ie Conversation_3984)
node.key(:thread_count) { postings.count }
node.key :likes
node.flow_to :users # Simplified flow syntax, since users only map to a single node per instance
end
aggregate_to('Thread') do |node| # Multiple nodes can be created, but they must have different types (:contribution != 'Thread')
node.identifier lambda {|c| "thread_#{c.id}"} # Construct node id
7550
entifier using the passed lambda
end
endThis setup will will allow you to execute the following queries on your models:
@user.aggregate(:count, :type => :contrib) # Counts the number of conversations a user has contributed to
@user.aggregate(:average, 'likes') # The average number of 'likes' for conversations the user has contributed to
@user.aggregate(:sum, 'posting_count') # The total number of posts in conversations the user has contributed to
@user.aggregate(:average, 'friends_popularity') # The average popularity of a user's friends
@user.aggregate(:sum, 'friends_age') # The cumulative age of a user's friends
@conversation(:count, :type => 'User') # Counts the users who have contributed to the conversation
@conversation(:count, :type => 'Thread') # The sum of the number of conversations contributed to by all the conversation's contributors
@conversation(:average, 'user_age')
@conversation(:average, 'friend_count')
@conversation(:average, 'thread_count') # The average number of postings for conversations which share a user with this one
@conversation(:average, 'friends_age') # The average age of the friends of users contributing to the conversation- Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
- Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
- Fork the project
- Start a feature/bugfix branch
- Commit and push until you are happy with your contribution
- Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
- Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
Copyright (c) 2012 Ryan Michael. See LICENSE.txt for further details.