8000 Add support for metrics comparing two stages of a pipeline · Issue #767 · lenskit/lkpy · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add support for metrics comparing two stages of a pipeline #767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mdekstrand opened this issue May 23, 2025 · 0 comments
Open

Add support for metrics comparing two stages of a pipeline #767

mdekstrand opened this issue May 23, 2025 · 0 comments
Labels
evaluation

Comments

@mdekstrand
Copy link
Member

Right now, all recommendation lists only look at the final stage.

However, we also want to support things like rank-biased overlap, which measures the difference between two stages of the pipeline.

This may be distinct from #766, as that ticket is about supporting lists from two different parameters. However, it is likely that we want to use the same machinery.

Specifically, the following:

  • The run analyzer supports pairwise metrics.
  • For pairwise metrics, a reference run can be specified. If specified, all other runs are compared to that run.
  • To support RBO, a plain Top-N run can be used as the reference run.
  • If no reference is supplied, all pairs are computed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
evaluation
Projects
LensKit Development
Status: Ready
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mdekstrand
0