Hypercore is a protocol and p2p network for distributing and replicating static feeds of binary data. It is the low level component that Hyperdrive is built on top off. For a detailed technical explanation on how the feed replication works see the Hyperdrive Specification
npm install hypercore
First lets add a stream of data to hypercore
var level = require('level')
var hypercore = require('hypercore')
var db = level('hypercore.db')
var core = hypercore(db) // db can be any levelup instance
var ws = core.createWriteStream() // lets add some data
ws.write('hello')
ws.write('world')
ws.end(function () {
// will print e1a31bb8716f0a0487377e22dbc7f0491fb47a712ac21519792a4e32cf56fb6f
console.log('data was stored as', ws.id.toString('hex'))
})Now to access it create a read stream with the same id
var rs = core.createReadStream('e1a31bb8716f0a0487377e22dbc7f0491fb47a712ac21519792a4e32cf56fb6f')
rs.on('data', function (data) {
console.log(data.toString()) // prints 'hello' and 'world'
})If we were only interested in the second block of data we can access the low-level feed instead of a read stream
var feed = core.get('e1a31bb8716f0a0487377e22dbc7f0491fb47a712ac21519792a4e32cf56fb6f')
// feeds give us easy random access
feed.get(1, function (err, block) {
console.log(block.toString()) // prints 'world'
})To start replicating this feed to other peers we need pipe a peer stream to them.
// create a peer stream to start replicating feeds to other peers
var stream = core.createPeerStream()
stream.pipe(anotherCore.createPeerStream()).pipe(stream)To find other hypercore peers on the internet sharing feeds we could use a peer discovery module such as discovery-channel which uses the BitTorrent dht and multicast-dns to find peers.
// lets find some hypercore peers on the internet sharing or interested in our feed
var disc = require('discovery-channel')() // npm install discovery-channel
var net = require('net')
var id = new Buffer('e1a31bb8716f0a0487377e22dbc7f0491fb47a712ac21519792a4e32cf56fb6f', 'hex')
var server = net.createServer(onsocket)
server.listen(0, function () {
announce()
setInterval(announce, 10000)
var lookup = disc.lookup(id.slice(0, 20))
lookup.on('peer', function (ip, port) {
onsocket(net.connect(port, ip))
})
})
function onsocket (socket) {
// connect the peers
socket.pipe(core.createPeerStream()).pipe(socket)
}
function announce () {
// discovery-channel currently only works with 20 bytes hashes
disc.announce(id.slice(0, 20), server.address().port)
}Create a new hypercore instance. db should be a levelup instance.
Create a new writable stream that adds a new feed and appends blocks to it.
After the stream has been ended (finish has been emitted) you can access stream.id and stream.blocks to get the feed metadata.
Create a readable stream that reads from a the feed specified by id. Optionally you can specify the following options:
{
start: 0, // which block index to start reading from
limit: Infinity // how many blocks to read
}Returns a readable stream that will emit the id of all feeds stored in the core.
Create a new peer replication duplex stream. This stream should be piped together with a remote peer's stream to the start replicating feeds.
When a feed is being used hypercore will emit interested with the corresponding feed id. You can use this to query for peers that shares this feed using an external peer discovery mechanism.
This is emitted when a feed is no longer being used.
When we receive extended handshake cb will be called with (id, peer). See he-chat
Everytime you write a stream of data to hypercore it gets added to an underlying binary feed. Feeds give you more low-level access to the data stored through the following api.
Create a new feed. Call feed.append to add blocks and feed.finalize when you're done and ready to share this feed.
Access a finalized feed by its id. By getting a feed you'll start replicating this from other peers you are connected too as well.
Get a block from the the feed. If you .get a block index that hasn't been downloaded yet this method will wait for that block be downloaded before calling the callback.
Append a block of data to a new feed. You can only append to a feed that hasn't been finalized. Optionally you can pass in an array of blocks instead of single one to add multiple blocks at the same time.
Finalize a feed. Will set feed.id when done. This is the id that identifies this feed.
Call this method to wait for the feed to have enough metadata to populate its internal state.
After the callback has been called feed.blocks is guaranteed to be populated. You do not have to call feed.ready before trying to .get a block. This method is just available for convenience.
Property containing the number of blocks this feed has. This is only known after at least one block has been fetched.
Boolean indicating wheather or not a block has been downloaded. Note that since this method is synchronous you have to wait for the feed to open before calling it.
MIT