Examples and documentation for storage and persistence #29

frgomes · 2016-01-05T14:28:42Z

Please provide an example (possibly under cache2k-benchmark) which employs a cache backed by a file storage.

At the moment, I'm doing the file storage thing by hand.
I suspect that PassingStorage is meant to do what I'm doing by hand.

This is my implementation:
https://gist.github.com/frgomes/f027f696299c624666f3

The text was updated successfully, but these errors were encountered:

cruftex · 2016-01-05T18:45:00Z

It actually works (better read: "should work") with:

private val images  =
  org.cache2k.CacheBuilder
    .newCache(classOf[String], classOf[ImageType])
    .name(dir.getName)
    .maxSize(cacheSize)
    .source(source)
    .persistence // <--- turn on persistence!
    .build

Ideally that is the minimum configuration to just store your objects in a file, if these are serializable.

However, there is a big BUT: The code is not yet production ready. I have some heavy duty tests with lots of concurrency and see some issues. Those are related with some tough functions: clear() and the iterator().... I'll just opened #30 to track how we move on with the persistence support now.

Some background for your understanding:

The storage is designed to be a path additional to the read through with the CacheSource (in the git master there is also write through with a CacheWriter). So you can have a CacheSource, e.g. fetching the weather for a location from an external service, make the data persistent and keep half of it in the heap.

I will keep this issue to provide some examples for the persistence configuration, either I paste some here directly or I do write up some unit tests.

frgomes · 2016-01-05T19:19:48Z

Thanks a lot for you prompt answer.

Do you consider CacheWriter (and the logic employing its methods) good enough?
By "good enough" I do not mean production quality, but something which passes a test suite is would be relatively ... good enough.

I will spend some time tomorrow with code from master and I can give you some feedback, if you wish.

cruftex · 2016-01-05T21:24:37Z

The funny thing is, after 16 years of doing caching, in our Java applications we never made use of a CacheWriter.

Question:

In case a fetch from the source is processing and an entry is written, what should happen:

Value is written immediately. When the fetch finishes, the value from the CacheSource is discarded
Value is written immediately, When the fetch finishes, the previously written value is replaced
Writing of the value stalls after the fetch finishes. When the fetch finishes this value is written to the cache, after that write proceeds.

Variant 1&2 implies that the CacheWriter may be called during an access of the cache source of the same key. Currently variant 3 should be how it works.

Do really need the CacheWriter? Do you get external calls storing data or can do requests or generate the value by the key? If the latter is the case, you can put all the logic in the CacheSource only.

Data get(K key) {
   Data d = storage.read(key);
   if (d == null) {
     d = realSource.get(key); 
     storage.write(key, d);
   }
   return d;
}

This is thread safe, no source call to the same key is made at one time.

frgomes · 2016-01-06T13:56:37Z

I'm really confused. I don't even understand the purpose of the concept of "storage" in the domain of cache2k

My understanding of storage is that there's a layer of software responsible for managing all reads/writes of data from/to a physical persistent device (not on-heap memory!). But I guess my [pre-]concept of storage is wrong in the domain of cache2k.

So, considering my understanding or more precisely: lack of understanding, I'd better stick to my current implementation, since there are not enough examples and documentation about how PassingStorage should be employed, managed and configured. I simply do not feel confident to use something I do not understand to a minimum level.

Besides, CacheWriter is available in master, but not in a tag, and not published to bintray. So, I'd better not waste time at this point creating a fork or something that I'm not really skilled enough to do anything with it other than publishing onto my local ivy repository.

Answering your question: "Do you really need a CacheWriter?" ... well... it's an interface which exposes methods which I'm already using anyway (conceptually... not really the exact method signatures!). So, why should I reinvent the wheel? Also, my supposition (possibly a wrong supposition) is that PassingStorage would manage the physical underlying device for me; so I'd like to use it, since I would not have to reinvent the wheel. But I guess that my supposition is simply wrong from what you've exposed.

cruftex · 2016-01-06T16:59:37Z

Sorry for the confusion, your feedback is much appreciated. As promised, I will put together some examples.

cruftex · 2016-01-08T09:49:30Z

Examples are here now:

https://github.com/headissue/cache2k/blob/master/core/src/test/java/org/cache2k/core/test/PersistenceExampleTest.java

I also went through the configuration and added some description of the options.

The example works with master, but should do the same with the released version.

That should shed some light on the whole thing and how everything is thought out.

Questions? Thoughts? Suggestions?

frgomes · 2016-01-08T13:38:20Z

Hi Jens, thanks a lot for that :-)
The examples are definitely clarifying, in particular the test case which employs passivation.

cruftex added the enhancement label Jan 5, 2016

cruftex changed the title ~~Provide example/documentation on how a storage can be defined/configured.~~ Examples and documentation for storage and persistence Jan 5, 2016

frgomes closed this as completed Jan 8, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Examples and documentation for storage and persistence #29

Examples and documentation for storage and persistence #29

Examples and documentation for storage and persistence #29

Examples and documentation for storage and persistence #29

Comments