8000 Changes for 0.5 · Issue #486 · chalk-lab/Mooncake.jl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Changes for 0.5 #486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
willtebbutt opened this issue Feb 14, 2025 · 3 comments
Open

Changes for 0.5 #486

willtebbutt opened this issue Feb 14, 2025 · 3 comments
< 8000 span class="text-bold color-fg-muted col-3 col-sm-2 flex-shrink-0">Labels
question Further information is requested

Comments

@willtebbutt
Copy link
Collaborator
willtebbutt commented Feb 14, 2025 •
edited
Loading

There are a couple of changes that I want to make that are breaking. In particular,

  1. use Config directly in build_rrule, prepare_gradient_cache, and prepare_pullback_cache, etc, rather than passing in kwargs, and
  2. redesign prepare_pullback_cache and value_and_pullback!! so that they're possible to use without requiring a call to zero_tangent.

This will bring immediately advantages to DI's interface with Mooncake. For example, if we add fields to Config (in a non-breaking way), DI won't need to change to support them. It should also reduce the amount of kwargs... expressions floating around in the Mooncake codebase, which can only be a good thing. Moreover, the change to prepare_pullback_cache and value_and_pullback!! ought to ensure that DI only needs to use these two functions + Mooncake.Config (i.e. what we're calling the public interface), and not anything else.

Technically non-breaking changes I plan include:

  1. removing "convenience" methods of build_rrule. I'm anticipating that all users should generally be using prepare_{gradient,pullback}_cache, so the convenience methods are redundant. If people need to use this (internal) function, they will obviously still be able to.
  2. Separating uses of CoDual into distinct types #275 , which is very breaking.

I expect to add to this list before actually making the release.

If anyone happens to know of any unsatisfactory bits of Mooncake which, whether technically breaking or just "probably disruptive to someone", ought to go into 0.5, please feel free to mention them here!
@willtebbutt willtebbutt added the question Further information is requested label Feb 14, 2025
@aj-fleming
Copy link

Far beyond the scope of this issue, but what would happen for Mooncake to need to support the differentiation of vector-valued functions?

@willtebbutt
Copy link
Collaborator Author

If by support differentiating vector-valued functions you mean compute Jacobian-like things, then I would suggest taking a look at DifferentiationInterface -- It's default Jacobian computation functionality already works for Mooncake :)

@willtebbutt willtebbutt mentioned this issue Mar 10, 2025
Open
@willtebbutt
Copy link
Collaborator Author

As discussed in https://discourse.julialang.org/t/large-memory-consumption-when-using-mooncake-via-differentiationinterface-for-gaussian-process-optimisation/127289/8?u=willtebbutt , there's not currently a good way to clear the cached OpaqueClosures that Mooncake keeps around. We should probably add a function to the interface which promises to completely clear out any cached data that Mooncake currently holds on to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@willtebbutt @aj-fleming
0