Hello when insert an Item it uses the json file but when quering it loads the whole index in memory #57

hamidpro · 2024-09-23T00:13:10Z

Hello thank you for this project, I want to understand it correctly when adding new index with embedding it appends to json file but when queering it loads entire index into memory (first is that correct please?)

The questions is if I have thousand of embedding in an index it will overload the memory how to prevent that, is there a way to do similar search in-disk instead of in-memory please?
cc @Stevenic

hamidpro · 2024-09-25T20:51:54Z

what is the difference between vectra and HNSWLib please? cc @Stevenic

Stevenic · 2024-11-04T11:39:34Z

Hey there @hamidpro sorry for the delay... You are correct... Vectra loads the entire index into memory which is what makes it so fast. It's really geared for smaller indexes with maybe a few thousand entries. If you have hundreds of thousands of entries or your entries change a lot then you're probably better off using something else. I haven't looked too closely at HNSWLib but it looks pretty good from what I've seen. I built vectra because I wanted something insanely fast that would do everything against a small in-memory vector db. I also wanted something that was free. :)

i'm not sure you'll find anything faster then vectra but that speed comes with tradeoffs. There are lots of improvements that could be made to vectra but there are so many other choices, like HNSWLib, that have a ton of bells and whistles that I don't see the point in trying to compete with those libraries feature for feature. I'd love to add re-ranking for example but even that feature requires adding a bunch of complexity I'm not crazy about.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hello when insert an Item it uses the json file but when quering it loads the whole index in memory #57

Hello when insert an Item it uses the json file but when quering it loads the whole index in memory #57

Hello when insert an Item it uses the json file but when quering it loads the whole index in memory #57

Hello when insert an Item it uses the json file but when quering it loads the whole index in memory #57

Comments