8000 Add ska_helpers.utils module with LazyDict by taldcroft · Pull Request #16 · sot/ska_helpers · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add ska_helpers.utils module with LazyDict #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 12, 2020
Merged

Conversation

taldcroft
Copy link
Member
@taldcroft taldcroft commented Nov 2, 2020

Description

This adds a new module ska_helpers.utils as a generic drop box for Ska utilities that don't have a better home.

The first entry is the LazyDict class, which currently appears separately in Ska.engarchive and maude. This version is more complete/robust and somewhat tested.

This PR also converts the docstrings to numpydoc format.

Testing

  • Passes unit tests on MacOS, linux, Windows (at least one required)
  • [N/A] Functional testing

@javierggt
Copy link
Collaborator

This smells like collections.defaultdict.

In what way is it different?

@javierggt
Copy link
Collaborator

ah, never mind. I see the difference.

Copy link
Collaborator
@javierggt javierggt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of lazy-initialized variables.

I am not a fan of the get() syntax. Where do you intend to use LazyVal?

To some extent, this functionality can be provided by module attributes, providing a module's __getattr__ method (https://www.python.org/dev/peps/pep-0562/). Why would you prefer this?

I note that the documentation is not yet finished:

  • LazyVal is not documented
  • no code examples
  • ska theme is not used

I assume you are doing this, hence your comments in slack today.

self._loaded = True

# Delete these so pickling always works (pickling a func can fail)
del self._load_func
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought maybe there were some pickle-specific methods you could define in the class (short of a custom Pickler) that would be more general, but this is readable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I was thinking of __getstate__.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current method would help with other serializations like YAML as well.

@taldcroft
Copy link
Member Author

@javierggt - I'm also not a fan of the .get() syntax, but I don't see a way around it.

My first thought for this task was also the module level __getattr__ and indeed I was excited to use this new feature. However, __getattr__ only works for attribute access from outside the module itself. So my_module.lazy_attribute works, but within my_module trying to access lazy_attribute just gives a NameError.

If you can find a way to make this work then that would be great, though there are some other issues with __getattr__ in this context. The main is that in any module that wants to use lazy values then a __getattr__ function needs to be defined by the user module to use those lazy values. It isn't obvious there is a nice way to handle this, especially if the user module is independently defining __getattr__ for another reason.

I used LazyVal here:
https://github.com/sot/chandra_aca/pull/108/files#diff-21392387835d093a39ee20461a8c92f248b70ac2a69150f23ffbf4c7a7f15860R29

@taldcroft
Copy link
Member Author

@javierggt @jeanconn - I think I have addressed all the review comments.

@taldcroft taldcroft merged commit 95f25d0 into master Nov 12, 2020
@taldcroft taldcroft deleted the utls-lazy-dict branch November 12, 2020 14:39
@javierggt javierggt mentioned this pull request Dec 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

@jeanconn jeanconn jeanconn left review comments

@javierggt javierggt javierggt approved these changes

Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0