8000 Implement Palu: Compressing KV-Cache with Low-Rank Projection · Issue #195 · eole-nlp/eole · GitHub

More Web Proxy on the site http://driver.im/

Implement Palu: Compressing KV-Cache with Low-Rank Projection #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

vince62s opened this issue Jan 22, 2025 · 0 comments

Labels

enhancement help wanted

Contributor

vince62s commented

Would be super cool to integrate this.

Paper is here: https://arxiv.org/html/2407.21118v1

Some repo with code:
https://github.com/shadowpa0327/Palu/tree/master
https://github.com/kvcache-ai/ktransformers/blob/main/ktransformers/operators/attention.py

cc: @francoishernandez

vince62s added enhancement help wanted labels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

0