8000 Add config module as per #2349 by belluzj · Pull Request #2416 · fonttools/fonttools · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add config module as per #2349 #2416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 14, 2022
Merged

Add config module as per #2349 #2416

merged 4 commits into from
Apr 14, 2022

Conversation

belluzj
Copy link
Contributor
@belluzj belluzj commented Sep 24, 2021

See issue #2349

  • [misc] Add configTools module, a generic configuration system
  • [config] Add config module, a fontTools-specific configuration system using the above + tests + doc
  • [ttLib] Attach a Config object to TTFont
  • [otlLib] Replace environment variable for GPOS compression level with the config system

@belluzj
Copy link
Contributor Author
belluzj commented Sep 24, 2021

I tried to move the register_option(...) to be located in the module that is concerned with the option, but that creates an issue when parsing the configuration: if the module in question is not imported yet, its configuration options are not registered yet and so the contents of the configuration cannot be validated.

This could be solved by:

  1. listing all options in the config module instead of in the various modules
  2. deferring validation of config contents to the point where the option values are used, hoping that the code that uses the option is in the same module that registered the option, so the option will have been registered by then (I don't like this because I think it would be better for the user if they get a crash asap if the config contains errors)
  3. importing all modules that declare options when the config module is loaded (not great because why import code that you won't use?)
  4. having in each module a config file that registers options for the module, and import all of these when the top-level config module is loaded. Eg. otlLib.config, varLib.config etc

@anthrotype or @behdad would you be able to advise please?

image

@anthrotype
Copy link
Member

The first option -- "listing all options in the config module instead of in the various modules" -- is the easiest, though probably not the most elegant.
The fourth option (otlLib.config, varLib.config etc) may also work, however in fonttools the subpackages __init__.py are usually not empty, so importing varLib.config would also import varLib's __init__.

What about, instead of needing to have calls to register_option() from some python module, you define a text file or files (e.g. configparser's .ini files) that contain the names of the modules and the options and the respective default values, and upon importing the global fontTools.config we parse that file or files (fetching it via importlib.resources and embedded in the distribution as package_data) and populate the Config class (calling register_option in turn for each one defined in the default config)?

@belluzj
Copy link
Contributor Author
belluzj commented Sep 27, 2021

a text file or files (e.g. configparser's .ini files) that contain the names of the modules and the options and the respective default values

That sounds OK except it's not Python code, so one would not be able to implement parsing/validation logic (or one would have to express that declaratively, like with JSON Schema for example). For that reason I would say it would be better as Python code. Also, I was quite happy with the idea that you could import the config options into your code, to refer to them without having hardcoded strings in the code; that wouldn't be simple with the text-file-approach.

Maybe the first option can be made to look organized enough by having config.varLib, config.otlLib etc modules to store the options of each module? It's still the same as option 1 but maybe it would make immediate sense to contributors. What do you think about that?

@anthrotype
Copy link
Member

implement parsing/validation logic

can you make an example of what you have in mind? I thought the options values would be simple str, float, int, or stuff that you can store as json.

you could import the config options into your code, to refer to them without having hardcoded strings in the code

Not sure what you mean by hard coded string. You could still import the default config object and get the already-parsed values from it.

@anthrotype
Copy link
Member

config.varLib, config.otlLib etc modules to store the options of each module?

yeah, if you actually need to have code to register options, then that could be an option to keep it organized. I'm still not 100 % that a declarative approach wouldn't be sufficient for our use case.

@belluzj
Copy link
Contributor Author
belluzj commented Sep 27, 2021

can you make an example of what you have in mind? I thought the options values would be simple str, float, int, or stuff that you can store as json.

Yes it's indeed very simple stuff, but still, assuming people can give options by writing them in a config file, we should expect to get the wrong type or values out of range. JSON schema would handle these cases for sure. See the tests below: parsing an input string to int, checking that the value is in the expected range.

def test_value_gets_parsed():
# Note: value given as a string
ttFont = TTFont(config={"otlLib.optimize.gpos.mode": "8"})
assert 8 == ttFont.config.get("otlLib.optimize.gpos.mode")
def test_value_gets_parsed_but_can_error():
with pytest.raises(ConfigValueParsingError):
TTFont(config={"otlLib.optimize.gpos.mode": "not an int"})
def test_value_gets_validated():
8000 # Note: 12 is not a valid value for GPOS compaction mode (must be in 0-9)
with pytest.raises(ConfigValueValidationError):
TTFont(config={"otlLib.optimize.gpos.mode": 12})

Not sure what you mean by hard coded string. You could still import the default config object and get the already-parsed values from it.

See the two tests below: you have the config object, and you can index it either by string or by the Option object. I feel that indexing by option object should be the preferred way, it would help prevent typos; but I admit it's not a massive win at all, just a small thing.

def test_ttfont_has_config():
ttFont = TTFont(config={"otlLib.optimize.gpos.mode": 8})
assert 8 == ttFont.config.get("otlLib.optimize.gpos.mode")
def test_can_access_config_by_option_object():
from fontTools.otlLib.optimize.gpos import MODE_OPTION
ttFont = TTFont(config={"otlLib.optimize.gpos.mode": 8})
assert 8 == ttFont.config.get(MODE_OPTION)
assert 8 == ttFont.config.get(MODE_OPTION, 3)

@behdad
Copy link
Member
behdad commented Sep 28, 2021

Collecting options from source into a toplevel config.py module is another option. We can make that run in CI and fail CI if config.py changed.

@behdad
Copy link
Member
behdad commented Sep 28, 2021

I'll comment on the rest of the approach soon. Thanks!

@belluzj
Copy link
Contributor Author
belluzj commented Sep 29, 2021

Cosimo: don't perform conversions from string to int in the default constructor; have a special one for par 8000 sing from file that does the conversions. In general you wouldn't expect an object that accepts an int value to also accept a string value and convert it.

@belluzj
Copy link
Contributor Author
belluzj commented Sep 29, 2021

Instead of reading configuration from files, we could read it from a lib key in the designspace / UFO or a custom parameter in Glyphs.app source.

E.g. in designspace:

<lib>
  <key>org.fontTools.config</key>
  <dict>
    <key>fontTools.otlLib.optimize.gpos.mode</key>
    <int>8</int>
  </dict>
</lib>

in ufo2ft:

ttFont = TTFont(config=designspace.lib.get('org.fontTools.config', {}))

@belluzj
Copy link
Contributor Author
belluzj commented Sep 29, 2021

Cosimo: should this module be in misc? It's not font-related

Also maybe there's an off-the-shelf thing in general for all this?

@behdad
8000
Copy link
Member
behdad commented Sep 29, 2021

Instead of reading configuration from files, we could read it from a lib key in the designspace / UFO or a custom parameter in Glyphs.app source.

It can come from there as well, yes.

    <key>fontTools.otlLib.optimize.gpos.mode</key>

I find mode very confusing. Can we call this compressionLevel?

@behdad
Copy link
Member
behdad commented Sep 29, 2021

I feel like the module is fundamental enough that should be at toplevel fonttools.config. But I'm fine if it goes in misc.config, and for everyone to use. That's a pattern we've been following. And I don't see anything simple off the shelf that we can use.

As for the TTFont property, at the risk of bikeshedding, I suggest we call it font.cfg. :D.

@belluzj
8000 Copy link
Contributor Author
belluzj commented Oct 4, 2021
  • about where to define options: I implemented the simplest solution (all options to be listed in the config module's __init__.py)
  • the module stays top-level but it's split between fontTools.config.generic (not related to fonts) and fontTools.config (the one to use everywhere in fontTools and client libraries). I could move the generic part to misc if that makes sense? Or not.
  • I renamed TTFont.config to TTFont.cfg and mode to compressionLevel
  • I'm using a dataclass in the config module and rebased on top of drop-3.6 so that tests pass (sorry for the noise in the diffs, I'll rebase on top of main when possible)

@belluzj
Copy link
Contributor Author
belluzj commented Oct 4, 2021

@anthrotype @behdad if you agree with how it is at the moment I'll start adding other options. I would also be happy to know which options you have in mind to add; I'm think to start with the parameters to TTFont().

@behdad
Copy link
Member
behdad commented Oct 4, 2021

Thanks Jany. This is concise and good. I have some ideas forward. Let me sketch those.

@behdad
Copy link
Member
behdad commented Oct 6, 2021

Copy/pasting from chat to Jany/Cosimo:

Jany, Cosimo, I have some Evil config ideass 😀
gimme a week to cook them.
Here's the sneak peak:
based on your design so far.
In your getitem

def __getitem__(s):

  if hasattr(s, '__module__'):
    s = s.__module__ + s.__name__
  elif hasattr(s, '__name__'):
    s = s.__name__

then accept a tuple: (path,node):

  s = path+node

in short, you can then do:

  cfg[obj,"COMPRESSION_LEVEL"]

and avoid repeating "fontTools.ttLib....."

any classtype or module can be passed there.

better yet, instead of a tuple, we can accept a colon as Cosimo liked 😛

  cfg[obj:"COMPRESSION_LEVEL"]

Next. If only a path is passed and no node, we return the subconfig. That is, cfg[cls] will return a new cfg that has subconfig for the class. __getattr__ will also be overriden to do the right thing. So we can do: cfg[cls].COMPRESSION_LEVEL

Finally, __call__ will be used to set config on multiple objects as a context:

with cfg(obj1, obj2, obj3, mod1, cls1, ...):
  ...

and the main way to use the config is to use to populate key/value arguments of constructors, methods, and functions. The syntax for that I have not fully figured out yet.

  cfg.invoke(obj.method) (params)

That would pass all relevant cfg to method.

Anyway; this is what's been brewing in my head.

As for your Option type, I want use the typing module types directly.

No need for extra Option type.

Behdad Esfahbod,
Lemme copy/paste this on the issue.

Jany Belluz,
Ok, thanks

That sounds a bit complicated to be honest, but maybe it looks nice when using it in the code

Behdad Esfahbod,
Yeah I feel like ergonomically it works.

Jany Belluz,
Ok cool

(sounds like how a Ruby developer would think, not a Python one)
I was going for pedestrian and non-surprising

Behdad Esfahbod,
I understand 🙂

I'm not sure mine are good. Evil it might be.

But want to explore 😀

Jany Belluz,
Yes anyway on the issue would be good 🙂 I'm curious to see how the usage looks

@belluzj
Copy link
Contributor Author
belluzj commented Apr 6, 2022

@anthrotype I rebased on latest main

@belluzj
Copy link
Contributor Author
belluzj F438 commented Apr 14, 2022

@anthrotype I fixed all your comments and added the new module to the Sphinx documentation.

I had an idea: how neat would it be if the list of registered options was parsed by Sphinx and added to the documentation page as well? I had a quick look and it sounds possible by adding some Sphinx code with new nodes and stuff, but I didn't go down the rabbit hole. Just a thought for later :)

@belluzj belluzj marked this pull request as ready for review April 14, 2022 10:26
@anthrotype
Copy link
Member

would it be if the list of registered options was parsed by Sphinx

that would be nice, yeah. File an issue so we don't forget

@belluzj
Copy link
Contributor Author
belluzj commented Apr 14, 2022

Done for the classvar and the issue

@belluzj belluzj merged commit afa0998 into fonttools:main Apr 14, 2022
@belluzj
Copy link
Contributor Author
belluzj commented Apr 14, 2022

@anthrotype I wrote release notes in the description

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0