8000 Document level metrics · Issue #177 · RUC-NLPIR/FlashRAG · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Document level metrics #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
yemaedahrav opened this issue May 9, 2025 · 1 comment
Open

Document level metrics #177

yemaedahrav opened this issue May 9, 2025 · 1 comment

Comments

83DE
@yemaedahrav
Copy link

I notice that there are two types of metrics, one is token-level and other is retrieval level, where check for the golden answer snippets is looked for in retrieved documents. Are there plans for adding document level retrievals? We measure metrics on the retrieved documents against the ground truth source documents. If the dataset is such. This is useful in a lot of scenarios. I can implement this maybe.
@ignorejjj
Copy link
Member

This is a great idea, but due to the limitations of our own dataset (lack of golden documents), we do not have a similar plan. If you can implement one, we would be happy to add it to the library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yemaedahrav @ignorejjj
0