An Index with Caching Mechanism for Real-time Processing System

Real-time processing mode is becoming a mainstream technology in many domains. For example, real-time monitoring system can effectively improve the power engineering or the efficiency of the transport sector. On behalf of a data structure to accelerate retrieval, indexing plays a vital role in data processing. That is because a well-designed index can reduce the time complexity of retrieval from O(n) to O(log n) or even O(1). Generally speaking, index, such as B-tree and R-tree, are designed for the speed of data retrieval rather than data update. To keep the real-time data, a large number of data updating tasks need to be performed in real-time processing system. However, the update speed of these indexes cannot meet the requirement of real-time processing system. In this paper, we present a caching mechanism before the updates of the index. In particular, we change the mode of index's update from one-by-one to batch operation by maintaining an inserting cache and a deleting cache. Our experiments show that this caching mechanism can effectively improve the index update speed by using the data of power grid system, while it has little impact on the speed of index retrieval.