Desearch is an experimental system towards a decentralized search engine. To achieve scalability and fault tolerance, desearch decouples computation and storage using a stateless trusted network and a stateful cloud store. The network is traced by witnesses and the store is regulated by blockchain. The current implementation of desearch uses Intel SGX as the trusted hardware and Redis as the store.
Warning: This repo hosts an academic proof-of-concept prototype and has not received a careful code review.
Please refer to our OSDI'21 paper for more details.
@inproceedings{li2021desearch,
author = {Mingyu Li and Jinhao Zhu and Tianxu Zhang and Cheng Tan and Yubin Xia and Sebastian Angel and Haibo Chen},
title = {Bringing Decentralized Search to Decentralized Services},
booktitle = {15th {USENIX} Symposium on Operating Systems Design and Implementation, {OSDI} 2021},
pages = {331--347},
publisher = {{USENIX} Association},
year = {2021},
}
Hardware Requirement: SGX-capable desktops or SGX-capable cloud machines. To check whether your machine supports SGX, please refer to Intel SGX Hardware.
Note that if you wish to run several SGX nodes on your local machine without support of scalable SGX, it might take longer to bootstrap the whole system because of the scarce encrypted memory (usually 128MB/256MB on SGXv1).
This repo provides a non-SGX version in the executor folder, and an SGX version in the sgx-executor folder. The non-SGX version helps you debug more easily if you want to build extensions to desearch; it uses the same folder structure as the SGX version. Note that only the SGX version contains ORAM-based queriers.
Under Ubuntu 20.04, building desearch executors requires the following depecencies installed:
You can refer to the deps folder for the correct versions of Redis, Hiredis, and Redis++.
apt install -y libboost-all-dev
cd deps
tar zxf redis-6.2.6.tar.gz
tar zxf redis-plus-plus-1.2.3.tar.gz
pushd redis-6.2.6/deps/hiredis/
make -j$(nproc) && make install
popd
pushd redis-plus-plus-1.2.3
mkdir build && cd build && cmake ..
make -j$(nproc) && make install
popdKanban is an unmodified Redis that leverages relatively cheap cloud storage.
Simply start the Redis server:
cd Kanban
cd deps && make hdr_histogram linenoise jemalloc
cd - && make all
./src/redis-server ./redis.confTo clear all states from Kanban, you can issue ./src/redis-cli FLUSHALL.
Manager is a special executor that makes Kanban resistant to tampering.
cd manager
make all
./managerExecutor consist of the whole search pipeline. This release consolidates all search roles within one executable. You are free to modify executor/start.sh to launch more executors as you like.
cd executor
make all
bash ./start.shClient is a Web server that serves as the entry for desearch.
cd client
make all
./clientThen double click the client/index.html, or use a fancy Web entry client/WebUI/index.html.
To extend to a WAN setup, you need to modify a few network configurations:
config.hpp: changeKANBAN_ADDRto a global public IPexecutor/start.sh: change querier IP address to a public one that clients can reach
- Mingyu Li: maxullee [at] sjtu [dot] edu [dot] cn
- Jinhao Zhu: zhujinhao [at] sjtu [dot] edu [dot] cn
- Tianxu Zhang: zhangtianxu [at] sjtu [dot] edu [dot] cn
- Sajin Sasy: ZeroTrace
MulanPSL-2.0 (see here)