fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process #1584

paulinebanye · 2025-04-24T02:29:38Z

Description

This PR addresses the unpickable configuration value warning encountered during the Sphinx make html build process (Issue #1202).

The warning occurred because the matplotlib_skore_scraper function in sphinx_gallery_conf (defined in conf.py) cannot be serialized by Python’s pickle module.

# configuration cache warning
- pickling environment... WARNING: cannot cache unpickable configuration value: 'sphinx_gallery_conf' (because it contains a function, class, or module object) [config.cache]

Environment

Model: Dell Latitude E7440
RAM: 12GB
OS: Windows 10 pro
Python Version: Python 3.12
Terminal: Ubuntu 22.04 LTS

Steps to reproduce:

Clone the skore repository
Create and activate a virtual environment using the Ubuntu terminal
Run the make install-skore command.
Cd into the sphinx directory
Run make html.

Proposed Solutions

There are several ways this warning can be resolved.
1: Using the built-in Matplotlib scraper

Pros: It is simple to maintain and has no pickling issues.
Cons: Requires extra configuration to make it match the custom function defined in sphinxext/matplotlib_skore_scraper.py.

# in conf.py
import matplotlib
matplotlib.rcParams['savefig.bbox'] = 'tight' # extra configuration to match the custom scraper function

sphinx_gallery_conf = {
    "image_scrapers": ['matplotlib'], # use the built-in matplotlib scraper
}

Converting the custom matplotlib_skore_scraper function to a class.

Pros: This option is flexible, serializable and can be customized further, if required.
Cons: More complex than using the built-in scraper.

# in matplotlib_skore_scraper.py
class matplotlib_skore_scraper:
    def __call__(self, *args, **kwargs):
        kwargs.setdefault("bbox_inches", "tight")
        return matplotlib_scraper(*
8000
args, **kwargs)

# in conf.py
sphinx_gallery_conf = {
    "image_scrapers": [matplotlib_skore_scraper()],
}

Defining the Scraper directly in conf.py: This requires adding the custom matplotlib_skore_scraper function directly in the conf.py file.

Pros: No pickling issue or import errors when importing the function from a different file.
Cons: It would clutter the conf.py file.

# in conf.py
def __call__(self, *args, **kwargs):
   kwargs.setdefault("bbox_inches", "tight")
   return matplotlib_scraper(*args, **kwargs)

sphinx_gallery_conf = {
    "image_scrapers": [matplotlib_skore_scraper],
}

Supressing the warning.

Pros: No major code change required.
Cons: It does not address the root cause of this warning, and it would mask other caching issues because it is declared globally.

# in conf.py
suppress_warnings = [
   'config.cache'  # silence config warnings
]

I chose the class option, because it resolves the pickling issue, it is flexible and keeps the code organized. However, I'm happy to modify this if another solution would work better.

cc @sylvaincom @glemaitre @thomass-dev

thomass-dev · 2025-04-24T07:30:58Z

Thanks @paulinebanye for the detailed description and your complete investigation.

IMO, the solution 1 is better because it implies less maintenance (all in the conf.py; no custom) and it is the way to globally configure matplotlib. What's your opinion @glemaitre ?

thomass-dev

Waiting for the previous comment resolution.

github-actions · 2025-04-24T07:48:33Z

Documentation preview @ 12d514a

paulinebanye · 2025-04-24T17:46:23Z

Thanks @paulinebanye for the detailed description and your complete investigation.

IMO, the solution 1 is better because it implies less maintenance (all in the conf.py; no custom) and it is the way to globally configure matplotlib. What's your opinion @glemaitre ?

Hi @thomass-dev, you're welcome 😃.

Yeah, solution 1 works perfectly, I’ll just need to update the conf and remove the custom matplotlib_scraper file. Let me know what the final decision is.

thomass-dev · 2025-04-28T07:26:51Z

@paulinebanye we have enabled the GitHub merge queue last weekend: you no longer need to update your branch with main, unless there is an apparent conflict.

paulinebanye · 2025-04-28T15:05:39Z

@paulinebanye we have enabled the GitHub merge queue last weekend: you no longer need to update your branch with main, unless there is an apparent conflict.

Oh great! That's a relief.
Thanks for the information @thomass-dev

paulinebanye · 2025-04-28T15:07:46Z

@thomass-dev 8000 has there been any decision regarding using the built-in scraper? Should I update the PR?

thomass-dev · 2025-04-28T15:20:16Z

Thanks @paulinebanye for the detailed description and your complete investigation.

IMO, the solution 1 is better because it implies less maintenance (all in the conf.py; no custom) and it is the way to globally configure matplotlib. What's your opinion @glemaitre ?

ping @glemaitre .

thomass-dev · 2025-04-30T07:36:46Z

Please take a look at the diff, it includes bad changes as it was done in #1592. Thanks.

paulinebanye · 2025-04-30T08:05:59Z

Please take a look at the diff, it includes bad changes as it was done in #1592. Thanks.

Hi @thomass-dev, yeah I noticed that last night when I was trying to resolve #1592.

There are files here that have been modified by the pull. I'm currently working on fixing that as we speak. I have to repeat the process of recloning and pushing.

Can I tag you once it's resolved?

paulinebanye · 2025-04-30T12:01:04Z

Hi @thomass-dev I have integrated the current changes and resolved the errors. Please review.

…nx make html process (probabl-ai#1584)

@auguste-probabl

…les (probabl-ai#1623) Following probabl-ai#1512 With the help of @auguste-probabl: Co-authored-by: auguste-probabl <auguste@probabl.ai> fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process (probabl-ai#1584) chore(dependencies): GITHUB-ACTIONS: Bump MishaKav/pytest-coverage-comment from 1.1.53 to 1.1.54 (probabl-ai#1627) Bumps [MishaKav/pytest-coverage-comment](https://github.com/mishakav/pytest-coverage-comment) from 1.1.53 to 1.1.54. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/mishakav/pytest-coverage-comment/releases">MishaKav/pytest-coverage-comment's releases</a>.</em></p> <blockquote> <h2>v1.1.54</h2> <h2>What's Changed</h2> <ul> <li>Improvements by <a href="https://github.com/MishaKav"><code>@MishaKav</code></a> in <a href="https://redirect.github.com/MishaKav/pytest-coverage-comment/pull/206">MishaKav/pytest-coverage-comment#206</a></li> </ul> <ul> <li>add support for new format for <code>pytest-coverage-path</code>, basically it add support for output of <code>pytest-cov >= 6.0.0</code></li> <li>bump dev dependencies</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/MishaKav/pytest-coverage-comment/compare/v1.1.53...v1.1.54">https://github.com/MishaKav/pytest-coverage-comment/compare/v1.1.53...v1.1.54</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/MishaKav/pytest-coverage-comment/blob/main/CHANGELOG.md">MishaKav/pytest-coverage-comment's changelog</a>.</em></p> <blockquote> <h1>Changelog of the Pytest Coverage Comment</h1> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.54">Pytest Coverage Comment 1.1.54</a></h2> <p><strong>Release Date:</strong> 2025-04-30</p> <h4>Changes</h4> <ul> <li>add support for new format for <code>pytest-coverage-path</code>, basically it add support for output of <code>pytest-cov >= 6.0.0</code></li> <li>bump dev dependencies</li> </ul> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.53">Pytest Coverage Comment 1.1.53</a></h2> <p><strong>Release Date:</strong> 2024-10-10</p> <h4>Changes</h4> <ul> <li>add option <code>xml-skip-covered</code> to skip lines that covered for 100% based on xml report, thanks to <a href="https://github.com/NikTerentev"><code>@NikTerentev</code></a> for contribution</li> <li>bump dev dependencies and minor for <code>@actions/core</code></li> </ul> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.52">Pytest Coverage Comment 1.1.52</a></h2> <p><strong>Release Date:</strong> 2024-06-30</p> <h4>Changes</h4> <ul> <li>fix commit <code>sha</code> and <code>ref</code> for <code>workflow_run</code>, instead of from the default branch, thanks to <a href="https://github.com/cecheta"><code>@cecheta</code></a> for contribution</li> <li>use <code>label</code> instead of <code>ref</code> for <code>workflow_run</code> and <code>workflow_dispatch</code>, thanks to <a href="https://github.com/cecheta"><code>@cecheta</code></a> for contribution</li> <li>use data from all testsuites instead the first one, thanks to <a href="https://github.com/eltoder"><code>@eltoder</code></a> for contribution</li> </ul> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.51">Pytest Coverage Comment 1.1.51</a></h2> <p><strong>Release Date:</strong> 2024-01-13</p> <h4>Changes</h4> <ul> <li>add <code>workflow_run</code> to the events that can trigger this action, big thanks to <a href="https://github.com/Bouni"><code>@Bouni</code></a> for contribution</li> </ul> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.50">Pytest Coverage Comment 1.1.50</a></h2> <p><strong>Release Date:</strong> 2023-11-26</p> <h4>Changes</h4> <ul> <li>add support for updateing the comment in PR through <code>workflow_dispatch</code> event by passing manually issue number, thanks to <a href="https://github.com/alexjyong"><code>@alexjyong</code></a> for contribution</li> </ul> <h2><a href="https://github.com/MishaKav/pytest-coverage-comment/tree/v1.1.49">Pytest Coverage Comment 1.1.49</a></h2> <p><strong>Release Date:</strong> 2023-11-15</p>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/MishaKav/pytest-coverage-comment/commit/13d3c18e21895566c746187c9ea74736372e5e91"><code>13d3c18</code></a> Improvements (<a href="https://redirect.github.com/mishakav/pytest-coverage-comment/issues/206">#206</a>)</li> <li>See full diff in <a href="https://github.com/mishakav/pytest-coverage-comment/compare/81882822c5b22af01f91bd3eacb1cefb6ad73dc2...13d3c18e21895566c746187c9ea74736372e5e91">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=MishaKav/pytest-coverage-comment&package-manager=github_actions&previous-version=1.1.53&new-version=1.1.54)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> fix: Set size limit of `DiskCacheStorage` from 1go to unlimited (probabl-ai#1625) Fixes failing pipelines from probabl-ai#1617. --- https://grantjenks.com/docs/diskcache/tutorial.html#settings https://grantjenks.com/docs/diskcache/tutorial.html#eviction-policies ``` size_limit, default one gigabyte. The maximum on-disk size of the cache. cull_limit, default ten. The maximum number of keys to cull when adding a new item. Set to zero to disable automatic culling. eviction_policy, default “least-recently-stored”. The setting to determine [eviction policy](https://grantjenks.com/docs/diskcache/tutorial.html#tutorial-eviction-policies). "none" disables cache evictions. Caches will grow without bound. ``` docs: Avoid failure when pytorch load weights from TextEncoder with parallelism (probabl-ai#1617) It is a workaround on the failure that is sometimes happening when request `transform` from `TextEncoder` in parallel. In short, there is a failure in the weights loading from PyTorch where some tensors are loaded on a meta device (only metadata) and does not load the weight. I did not yet found the root cause of the failure why the weights are not loaded properly. In the meanwhile, we can avoid this issue by storing the weights. We should monitor that loading several time the language model does not make blow up the RAM on the GitHub actions. Othwerwise, we need to deactivate the parallelism most probably. Co-authored-by: Thomas S. <thomas@probabl.ai>

github-actions bot assigned paulinebanye Apr 24, 2025

thomass-dev requested review from thomass-dev and glemaitre April 24, 2025 07:29

thomass-dev requested changes Apr 24, 2025

View reviewed changes

paulinebanye changed the title ~~fix: Fix unpickable configuration value during Sphinx make html process~~ fix: Fix unpickable configuration value warning during Sphinx make html process Apr 25, 2025

paulinebanye changed the title ~~fix: Fix unpickable configuration value warning during Sphinx make html process~~ fix: Fix cannot cache unpickable configuration value warning during Sphinx make html process Apr 25, 2025

paulinebanye changed the title ~~fix: Fix cannot cache unpickable configuration value warning during Sphinx make html process~~ fix: Fix cannot cache unpickable configuration value warning - Sphinx make html process Apr 25, 2025

paulinebanye mentioned this pull request Apr 25, 2025

docs: Fix warnings during documentation building #1202

Closed

paulinebanye requested review from rouk1 and auguste-probabl as code owners April 29, 2025 22:55

paulinebanye added 9 commits April 30, 2025 09:48

new clone

5c750f6

Merge branch 'main' of https://github.com/probabl-ai/skore

d2d0490

new clone

cdefe65

Merge branch 'main' of https://github.com/probabl-ai/skore

173fef9

pull from upstream

2773b84

pull from upstream

1c819f3

pull upstream

444bc08

Merge branch 'main' of https://github.com/probabl-ai/skore

fe83179

resolve configuration warning

12d514a

paulinebanye force-pushed the warnings-config branch from cbeeb43 to 12d514a Compare April 30, 2025 11:13

pre-commit and minor edits

0387ac9

thomass-dev enabled auto-merge April 30, 2025 12:24

thomass-dev disabled auto-merge April 30, 2025 12:24

thomass-dev approved these changes Apr 30, 2025

View reviewed changes

thomass-dev enabled auto-merge April 30, 2025 12:24

thomass-dev disabled auto-merge April 30, 2025 12:25

thomass-dev merged commit 8b7a6b4 into probabl-ai:main Apr 30, 2025
18 checks passed

paulinebanye deleted the warnings-config branch May 2, 2025 06:22

paulinebanye restored the warnings-config branch May 2, 2025 06:23

Muhammad-Rebaal pushed a commit to Muhammad-Rebaal/skore that referenced this pull request May 5, 2025

fix: Fix cannot cache unpickable configuration value warning - Sphi…

31097e5

…nx make html process (probabl-ai#1584)

paulinebanye deleted the warnings-config branch May 8, 2025 16:39

paulinebanye restored the warnings-config branch May 8, 2025 16:39

paulinebanye deleted the warnings-config branch May 12, 2025 10:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process #1584

fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process #1584

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fix: Fix cannot cache unpickable configuration value warning - Sphinx make html process #1584

fix: Fix cannot cache unpickable configuration value warning - Sphinx make html process #1584

Uh oh!

Conversation

Uh oh!

Description

Environment

Steps to reproduce:

Proposed Solutions

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process #1584

fix: Fix `cannot cache unpickable configuration value` warning - Sphinx make html process #1584