Moneta provides a standard interface for interacting with various kinds of key/value stores. Moneta supports the well-known NoSQL and document based stores.
A short overview of the features:
- Supports a lot of backends with consistent behaviour (See below)
- Allows a full configuration of the serialization -> compression -> adapter stack using proxies (Similar to Rack middlewares)
- Configurable serialization via
Moneta::Transformer
proxy (Marshal/JSON/YAML and many more) - Configurable value compression via
Moneta::Transformer
proxy (Zlib, Snappy, LZMA, ...) - Configurable key transformation via
Moneta::Transformer
proxy
- Configurable serialization via
- Expiration for all stores (Added via proxy
Moneta::Expires
if not supported natively) - Atomic operations
- Atomic incrementation and decrementation for most stores (Method
#increment
and#decrement
) - Atomic creation of entries (Method
#create
) - Shared/distributed database-wide synchronization primitives
Moneta::Mutex
andMoneta::Semaphore
- Atomic incrementation and decrementation for most stores (Method
- Includes a simple pure-ruby key/value server (
Moneta::Server
) and client (Moneta::Adapters::Client
) - Integration with Rails, Rack/Rack-Cache, Sinatra, Padrino and Ramaze.
If you are not yet convinced, you might ask why? What are the goals of the project?
- Get people started quickly with key/value stores! Therefore all the adapters are included in the gem and you are ready to go. Tilt does the same for template languages.
- Make it easy to compare different key/value stores and benchmark them
- To hide a lot of different and maybe complex APIs behind one well-designed and simple Moneta API
- Give people a starting point or example code to start working with their favourite key/value store. Feel free to copy code, please mention Moneta then :)
- Create a reusable piece of code, since similar things are solved over and over again (Rails brings its own cache stores, and many frameworks do the same...)
Moneta is tested thoroughly using Travis-CI.
Install Moneta via Rubygems
$ gem install moneta
or add it to your Gemfile
gem 'moneta'
Now you are ready to go:
require 'moneta'
# Create a simple file store
store = Moneta.new(:File, dir: 'moneta')
# Store some entries
store['key'] = 'value'
# Read entry
store.key?('key') # returns true
store['key'] # returns 'value'
store.close
- Source: http://github.com/minad/moneta
- Bugs: http://github.com/minad/moneta/issues
- Tests and benchmarks: http://travis-ci.org/minad/moneta
- API documentation:
- Latest Gem: http://rubydoc.info/gems/moneta/frames
- GitHub master: http://rubydoc.info/github/minad/moneta/master/frames
Out of the box, it supports the following backends. Use the backend name symbol in the Moneta constructor (e.g. Moneta.new(:Memory)
).
- Memory:
- In-memory store (
:Memory
) - LRU hash - prefer this over :Memory! (
:LRUHash
) - LocalMemCache (
:LocalMemCache
) - Memcached store (
:Memcached
,:MemcachedNative
and:MemcachedDalli
)
- In-memory store (
- Relational Databases:
- DataMapper (
:DataMapper
) - ActiveRecord (
:ActiveRecord
) - Sequel (
:Sequel
) - Sqlite3 (
:Sqlite
)
- DataMapper (
- Filesystem:
- Key/value databases:
- Berkeley DB using DBM interface or NDBM (Depends on Ruby environment) (
:DBM
) - Cassandra (
:Cassandra
) - Daybreak (
:Daybreak
) - GDBM (
:GDBM
) - HBase (
:HBase
) - LevelDB (
:LevelDB
) - LMDB (
:LMDB
) - Redis (
:Redis
) - Riak (
:Riak
) - SDBM (
:SDBM
) - KyotoCabinet (
:KyotoCabinet
) - TokyoCabinet (
:TokyoCabinet
) - TokyoTyrant (
:TokyoTyrant
) - Simple Samba database TDB (
:TDB
)
- Berkeley DB using DBM interface or NDBM (Depends on Ruby environment) (
- Document databases:
- Moneta network protocols:
- Moneta key/value client (
:Client
works withMoneta::Server
) - Moneta HTTP/REST client (
:RestClient
works withRack::MonetaRest
)
- Moneta key/value client (
- Other
- Fog cloud storage which supports Amazon S3, Rackspace, etc. (
:Fog
) - Storage which doesn't store anything (
:Null
)
- Fog cloud storage which supports Amazon S3, Rackspace, etc. (
Some of the backends are not exactly based on key/value stores, e.g. the relational ones. These are useful if you already use the corresponding backend in your application. You get a key/value store for free then without installing any additional services and you still have the possibility to upgrade to a real key/value store.
NOTE: The backend matrix is much more readable on rubydoc.info than on github. Go there!
Adapter | Required gems | Multi-thread safe[1] | Multi-process safe[2] | Atomic increment[8] | Atomic create[9] | Native expires[3] | Persistent | Description |
---|---|---|---|---|---|---|---|---|
Persistent stores | ||||||||
Mongo | mongo or moped | âś“ | âś“ | âś“ | âś“ | âś“ | âś“ | MongoDB database |
MongoOfficial | mongo | âś“ | âś“ | âś“ | âś“ | âś“ | âś“ | MongoDB database |
MongoMoped | moped | âś“ | âś“ | âś“ | âś“ | âś“ | âś“ | MongoDB database |
Redis | redis | âś“ | âś“ | âś“ | âś“ | âś“ | âś“ | Redis database |
ActiveRecord | activerecord | âś“ | âś“ | âś“ | âś“ | âś— | âś“ | ActiveRecord ORM |
File | - | âś“ | âś“ | âś“ | âś“ | âś— | âś“ | File store |
LMDB | lmdb | âś“ | âś“ | âś“ | âś“ | âś— | âś“ | Symas Lightning Memory-Mapped Database (LMDB) |
Sequel | sequel | âś“ | âś“ | âś“ | âś“ | âś— | âś“ | Sequel ORM |
TokyoTyrant | tokyotyrant or ruby-tokyotyrant | âś— | âś“ | âś“ | âś“ | âś— | âś“ | TokyoTyrant database |
PStore | - | âś— | âś“[10] | âś“ | âś“ | âś— | âś“ | PStore store |
YAML | - | âś— | âś“[10] | âś“ | âś“ | âś— | âś“ | YAML store |
Sqlite | sqlite3 | ? | âś“[10] | âś“ | âś“ | âś— | âś“ | Sqlite3 database |
Daybreak | daybreak | âś— | (âś“)[7] | âś“ | âś“ | âś— | âś“ | Incredibly fast pure-ruby key/value store Daybreak |
DBM | - | âś— | âś— | âś“ | âś“ | âś— | âś“ | Berkeley DB using DBM interface or NDBM (Depends on Ruby environment) |
GDBM | - | âś— | âś— | âś“ | âś“ | âś— | âś“ | GDBM database |
LevelDB | leveldb | âś— | âś— | âś“ | âś“ | âś— | âś“ | LevelDB database |
SDBM | - | âś— | âś— | âś“ | âś“ | âś— | âś“ | SDBM database |
TDB | tdb | âś— | âś— | âś“ | âś“ | âś— | âś“ | TDB database |
KyotoCabinet | kyotocabinet | âś— | âś— | âś“ | âś“ | âś— | âś“ | KyotoCabinet database |
TokyoCabinet | tokyocabinet | âś— | âś— | âś“ | âś“ | âś— | âś“ | TokyoCabinet database |
DataMapper | dm-core, dm-migrations | âś“ | âś“ | âś— | âś“ | âś— | âś“ | DataMapper ORM |
Couch | faraday, multi_json | âś— | âś“ | âś— | âś“ | âś— | âś“ | CouchDB database |
HBase | hbaserb | ? | âś“ | âś“ | âś— | âś— | âś“ | HBase database |
Cassandra | cassandra | ? | âś“ | âś— | âś— | âś“ | âś“ | Cassandra distributed database |
LocalMemCache | localmemcache | âś“ | âś“ | âś— | âś— | âś— | âś“ | LocalMemCache database |
Fog | fog | ? | âś“ | âś— | âś— | âś— | âś“ | Fog cloud store |
Riak | riak-client | âś— | âś“ | âś— | âś— | âś— | âś“ | Riak database |
Non persistent stores | ||||||||
MemcachedDalli | dalli | âś“ | âś“ | âś“ | âś“ | âś“ | âś—[4] | Memcached database with Dalli library |
Memca 6D47 ched | dalli or memcached | ? | âś“ | âś“ | âś“ | âś“ | âś—[4] | Memcached database |
MemcachedNative | memcached | âś— | âś“ | âś“ | âś“ | âś“ | âś—[4] | Memcached database with native library |
Cookie | - | âś— | (âś“)[6] | âś“ | âś“ | âś“ | âś— | Cookie in memory store |
LRUHash | - | âś— | (âś“)[6] | âś“ | âś“ | âś— | âś— | LRU memory store |
Memory | - | âś— | (âś“)[6] | âś“ | âś“ | âś— | âś— | Memory store |
Null | - | âś“ | âś“ | âś— | âś— | âś— | âś— | No database |
Client | - | âś— | âś“ | ?[5] | ?[5] | ?[5] | ?[5] | Moneta client adapter |
RestClient | faraday | âś— | âś“ | âś— | âś— | âś— | ?[5] | Moneta REST client adapter |
- [1]: Make adapters thread-safe by using
Moneta::Lock
or by passing the optionthreadsafe: true
toMoneta#new
. There is alsoMoneta::Pool
which can be used to share a store between multiple threads if the store is multi-process safe. I recommend to add the option:threadsafe
to ensure thread-safety since for example under JRuby and Rubinius even the basic datastructures are not thread safe due to the lack of a global interpreter lock (GIL). This differs from MRI where some adapters might appear thread safe already but only due to the GIL. - [2]: Share a Moneta store between multiple processes using
Moneta::Shared
(See below). - [3]: Add expiration support by using
Moneta::Expires
or by passing the optionexpires: true
toMoneta#new
. - [4]: There are some servers which use the memcached protocol but which are persistent (e.g. MemcacheDB, Kai, IronCache, Roma, Flare and Kumofs)
- [5]: Depends on server
- [6]: Store is multi-process safe because it is an in-memory store, values are not shared between multiple processes
- [7]: Store is multi-process safe, but not synchronized automatically between multiple processes
- [8]: If a store provides atomic increment it can be used with
Moneta::Semaphore
. You can add weak#increment
support using theMoneta::WeakIncrement
proxy. - [9]: If a store provides atomic creation it can be used with
Moneta::Mutex
. You can add weak#create
support using theMoneta::WeakCreate
proxy. - [10]: Sqlite/YAML/PStore are multiprocess safe, but the performance suffers badly since the whole database file must be locked for writing. Use a key/value server if you want multiprocess concurrency!
In addition it supports proxies (Similar to Rack middlewares) which add additional features to storage backends:
Moneta::Proxy
andMoneta::Wrapper
proxy base classesMoneta::Expires
to add expiration support to stores which don't support it natively. Add it in the builder usinguse :Expires
.Moneta::Stack
to stack multiple stores (Read returns result from first where the key is found, writes go to all stores). Add it in the builder usinguse(:Stack) {}
.Moneta::Transformer
transforms keys and values (Marshal, YAML, JSON, Base64, MD5, ...). Add it in the builder usinguse :Transformer
.Moneta::Cache
combine two stores, one as backend and one as cache (e.g.Moneta::Adapters::File
+Moneta::Adapters::LRUHash
). Add it in the builder usinguse(:Cache) {}
.Moneta::Lock
to make store thread safe. Add it in the builder usinguse :Lock
.Moneta::Pool
to create a pool of stores as a means of making the store thread safe. Add it in the builder usinguse(:Pool) {}
.Moneta::Logger
to log database accesses. Add it in the builder usinguse :Logger
.Moneta::Shared
to share a store between multiple processes. Add it in the builder usinguse(:Shared) {}
.Moneta::WeakIncrement
andMoneta::WeakCreate
to add#create
and#increment
support without atomicity (weak) to stores which don't support it.
Supported serializers:
- BEncode (
:bencode
) - BERT (
:bert
) - BSON (
:bson
) - JSON (
:json
) - Marshal (
:marshal
) - MessagePack (
:msgpack
) - Ox (
:ox
) - PHP (
:php
) - TNetStrings (
:tnet
) - YAML (
:yaml
)
Supported value compressors:
- Bzip2 (
:bzip2
) - LZ4 (
:lz4
) - LZMA (
:lzma
) - LZO (
:lzo
) - Snappy (
:snappy
) - QuickLZ (
:quicklz
) - Zlib (
:zlib
)
Supported encoders:
- Base64 (
:base64
) - Url escape (
:escape
) - Hexadecimal (
:hex
) - QP (
:qp
) - UUEncode (
:uuencode
)
Special transformers:
- Digests (MD5, Shas, CityHash, ...)
- Add prefix to keys (
:prefix
) - HMAC to verify values (
:hmac
, useful forRack::MonetaCookies
)
The Moneta API is purposely extremely similar to the Hash API with a few minor additions. Every method takes also a optional option hash. In order so support an identical API across stores, Moneta does not support iteration or partial matches.
#initialize(options) options differs per-store, and is used to set up the store.
#[](key) retrieve a key. If the key is not available, return nil.
#load(key, options = {}) retrieve a key. If the key is not available, return nil.
#fetch(key, options = {}, &block) retrieve a key. If the key is not available, execute the
block and return its return value.
#fetch(key, value, options = {}) retrieve a key. If the key is not available, return the value,
#[]=(key, value) set a value for a key. If the key is already used, clobber it.
keys set using []= will never expire.
#store(key, value, options = {}) same as []=, but you can supply options.
#delete(key, options = {}) delete the key from the store and return the current value.
#key?(key, options = {}) true if the key exists, false if it does not.
#increment(key, amount = 1, options = {}) increment numeric value. This is an atomic operation
which is not supported by all stores. Returns current value.
#decrement(key, amount = 1, options = {}) increment numeric value. This is an atomic operation
which is not supported by all stores. Returns current value.
This is just syntactic sugar for incrementing with a negative value.
#create(key, value, options = {}) create entry. This is an atomic operation which is not supported by all stores.
Returns true if the value was created.
#clear(options = {}) clear all keys in this store.
#close close database connection.
#features return array of features, e.g. [:create, :expires, :increment]
#supports?(feature) returns true if store supports a given feature
There is a simple interface to create a store using Moneta.new
. You will
get automatic key and value serialization which is provided by Moneta::Transformer
.
This allows you to store arbitrary Ruby objects. You can tune some options
when you call Moneta.new
. However for very fine tuning use Moneta.build
.
store = Moneta.new(:Memcached, server: 'localhost:11211')
store['key'] = 'value'
store['hash_key'] = {a: 1, b: 2}
store['object_key'] = MarshallableRubyObject.new
If you want to have control over the proxies, you have to use Moneta.build
:
store = Moneta.build do
# Adds expires proxy
use :Expires
# Transform key using Marshal and Base64 and value using Marshal
use :Transformer, key: [:marshal, :base64], value: :marshal
# IMPORTANT: adapter must be defined last for the builder to function properly.
# Memory backend
adapter :Memory
end
You can also directly access the underlying adapters if you don't want to use the Moneta stack.
db = Moneta::Adapters::File.new(dir: 'directory')
db['key'] = {a: 1, b: 2} # This will fail since you can only store Strings
# However for Mongo and Couch this works
# The hash will be mapped directly to a Mongo/Couch document.
db = Moneta::Adapters::Couch.new
db['key'] = {a: 1, b: 2}
db = Moneta::Adapters::Mongo.new
db['key'] = {a: 1, b: 2}
The Cassandra, Memcached, Redis and Mongo backends support expiration natively.
cache = Moneta::Adapters::Memcached.new
# Or using the builder...
cache = Moneta.build do
adapter :Memcached
end
# Expires in 60 seconds
cache.store(key, value, expires: 60)
# Never expire
cache.store(key, value, expires: 0)
cache.store(key, value, expires: false)
# Update expires time if value is found
cache.load(key, expires: 30)
cache.key?(key, expires: 30)
# Or remove the expiration if found
cache.load(key, expires: false)
cache.key?(key, expires: 0)
You can add the expires feature to other backends using the Moneta::Expires
proxy. But be aware
that expired values are not deleted automatically if they are not looked up.
# Using the :expires option
cache = Moneta.new(:File, dir: '...', expires: true)
# or manually by using the proxy...
cache = Moneta::Expires.new(Moneta::Adapters::File.new(dir: '...'))
# or using the builder...
cache = Moneta.build do
use :Expires
adapter :File, dir: '...'
end