8000 GitHub - zilliztech/pyglass: Graph Library for Approximate Similarity Search
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

zilliztech/pyglass

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Graph Library for Approximate Similarity Search

pyglass is a library for fast inference of graph index for approximate similarity search.

Features

  • Supports multiple graph algorithms, like HNSW and NSG.
  • Supports multiple hardware platforms, like X86 and ARM. Support for GPU is on the way
  • No third-party library dependencies, does not rely on OpenBLAS / MKL or any other computing framework.
  • Sophisticated memory management and data structure design, very low memory footprint.
  • It's high performant.

Installation

Installation from Source

sudo apt-get update && sudo apt-get install -y build-essential git python3 python3-distutils python3-venv
pip3 install numpy
pip3 install pybind11
bash build.sh

Quick Tour

A runnable demo is at examples/demo.ipynb. It's highly recommended to try it.

Usage

Import library

>>> import glass

Load Data

>>> import numpy as np
>>> n, d = 10000, 128
>>> X = np.random.randn(n, d)
>>> Y = np.random.randn(d)

Create Index pyglass supports HNSW and NSG index currently, with different quantization support

>>> index = glass.Index(index_type="HNSW", metric="L2", R=32, L=50)
>>> index = glass.Index(index_type="NSG", metric="L2", R=32, L=50, quant="SQ8U")

Build Graph

>>> graph = index.build(X)

Create Searcher

>>> searcher = glass.Searcher(graph=graph, data=X, metric="L2", quantizer="SQ4U")
>>> searcher.set_ef(32)

(Optional) Optimize Searcher

>>> searcher.optimize()

Searching

>>> ret = searcher.search(query=Y, k=10)
>>> print(ret)

Supported Quantization Methods

  • FP8_E5M2
  • PQ8
  • SQ8
  • SQ8U
  • SQ6
  • SQ4
  • SQ4U
  • SQ4UA
  • SQ2U
  • BinaryQuant

Rule of Thumb: Use SQ8U for indexing, and SQ4U for searching is almost always a good choice.

Performance

Glass is among one of the top performant ann algorithms on ann-benchmarks

fashion-mnist-784-euclidean

gist-960-euclidean

sift-128-euclidean

Quick Benchmark

  1. Change configuration file examples/config.json
  2. Run benchmark
python3 examples/main.py

Citation

You can cite the PyGlass repo as follows:

@misc{PyGlass,
    author = {Zihao Wang},
    title = {Graph Library for Approximate Similarity Search},
    url = {https://github.com/zilliztech/pyglass},
    year = {2025},
    month = {4},
}

About

Graph Library for Approximate Similarity Search

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

0