8000 python with pyinstrument seems to show a memory leak · Issue #368 · LLNL/units · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
python with pyinstrument seems to show a memory leak #368
Open
@mattfil

Description

@mattfil

Hi,
I am using Units from python. One thing that I have noticed is that, when I set up a test with pyinstrument @Profile (https://github.com/joerick/pyinstrument), the inscances of Unit seems to be leaked. Here a minimal example:

import functools
from collections.abc import Callable
from typing import Any, TypeVar

from memory_profiler import profile
from pyinstrument import Profiler
from units_llnl import Unit

F = TypeVar("F", bound=Callable[..., Any])

PROFILED_FILES: list[str] = []


def profile_function(output_file: str) -> Callable[[F], F]:

    def decorator(func: F) -> F:
        @functools.wraps(func)
        def wrapper(*args: Any, **kwargs: Any) -> Any:
            profiler = Profiler(0.00001, async_mode="enabled")
            profiler.start()
            result = func(*args, **kwargs)  # Run the function
            profiler.stop()
            namefile = f"{output_file}.html"
            profiler.write_html(namefile)
            return result

        return wrapper  # type: ignore

    return decorator


@profile
@profile_function("eval")
def test_silly() -> None:
    class dummy:
        def __init__(self, theunit: str) -> None:
            self.unit = Unit(1, theunit)

    for ii in range(100):
        aa = dummy("m")
        bb = dummy("kg")

For such example i get in the terminal:

nanobind: leaked 2 instances!
nanobind: leaked 1 types!
  leaked type "units_llnl.units_llnl_ext.Unit"
nanobind: leaked 34 functions!
  leaked function "to_dict"
  leaked function "__mul__"
  leaked function "is_equation"
  leaked function "isinf"
  leaked function "__hash__"
  leaked function "__ne__"
  leaked function "__truediv__"
  leaked function "is_valid"
  leaked function "is_exactly_the_same"
  leaked function ""
  leaked function "is_convertible_to"
  ... skipped remainder
nanobind: this is likely caused by a reference counting issue in the binding code.

This actually seems a true leak, here the report from the memory_profiler:

46    263.5 MiB    263.5 MiB           1           @functools.wraps(func)
47                                                 def wrapper(*args: Any, **kwargs: Any) -> Any:
48    263.5 MiB      0.0 MiB           1               profiler = Profiler(0.00001, async_mode="enabled")
49    263.5 MiB      0.0 MiB           1               profiler.start()
50    266.6 MiB      3.1 MiB           1               result = func(*args, **kwargs)  # Run the function
51    266.6 MiB      0.0 MiB           1               profiler.stop()
52    266.6 MiB      0.0 MiB           1               namefile = f"{output_file}.html"
53    267.6 MiB      1.0 MiB           1               profiler.write_html(namefile)
54    267.6 MiB      0.0 MiB           1               return result

Thank you,
Mattia

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0