8000 Adding memory profiling ? · Issue #216 · pyutils/line_profiler · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Adding memory profiling ? #216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Abdelgha-4 opened this issue Jun 13, 2023 · 4 comments
Closed

Adding memory profiling ? #216

Abdelgha-4 opened this issue Jun 13, 2023 · 4 comments

Comments

@Abdelgha-4
Copy link

As memory_profiler is no longer maintained, I see this project as a perfect place to continue on it's work and provide a full powerful line_profiler that combine both information on time and memory consumption in one place, providing for instance some profiling results like this:

Line #    Mem usage    Increment     Hits     Time       Line Contents
======================================================================
     3                                                   @profile
     4                                                   def my_func():
     5    7.676 MiB    7.676 MiB        1    0.20 ms         a = [1] * (10 ** 6)
     6  160.301 MiB  152.625 MiB        1    0.25 ms         b = [2] * (2 * 10 ** 7)
     7    7.813 MiB -152.488 MiB        1    0.02 ms         del b
     8    7.813 MiB    0.000 MiB        1    1.200 s         sleep(1.2)
     9    7.813 MiB    0.000 MiB        1    0.01 ms         return a

I'd really like to know what do you think about this possibility ?
I've also an idea of letting the user add more columns to this results based on his use case, for example:

def energy_tracker(time_inc, memory_inc): #maybe other args
    return TIME_RATE * time_inc + MEM_RATE * memory_inc

profiler.register_trackers({"Energy": energy_tracker})

Which would add a new column called Energy with the given formula applied
@Erotemic
Copy link
Member

I think this is beyond the scope of this project. Tracking memory will impact the performance of tracking time.

For energy tracking, check out codecarbon.

@Abdelgha-4
Copy link
Author

Tracking memory will impact the performance of tracking time.

I guess if that's the main concern, we could give the user an option to choose what he wants to track.
This would allow both, having a powerful all-in-one profiler that can provide insights and diagnosis for the code from multiple angles, while also keeping the option of having a lightweight and fast time-only profiler.

For energy tracking, check out codecarbon.

Thank you for the reference, it's indeed a great package, but it lacks the line-by-line details and also the flexibility that "user specific columns" might add (for example a cost function, or a function to highlight only specific consumption patterns or some other custom logic ..)

@Erotemic
Copy link
Member

If you feel like you can write the logic to make it work, by all means go for it. I think it would be an interesting tool, and I may even use it. However, I don't have the bandwidth to entertain adding a new scope to this project. As such I'm going to close this issue as it pertains to this repository.

But don't let that stop you from implementing the feature. If you can get a working proof of concept, I'd love to hear about it and I'd link to it in the README here in related work. You're more than welcome to take this source c 8A2C ode as a starting point and redistribute it under a new name.

Also I think scalene does have some sort of memory profiling (but probably not line-by-line).

@Abdelgha-4
Copy link
Author

I perfectly understand, thank you for the encouragement!

Since I'm not that familiar with Cython (which is heavily used by line_profiler), I guess I'll start the other way around, forking memory_profiler (which is in pure python) and I'll see what I can get from there.

Anyway, it was nice hearing your opinion. Thank you for your time.

@tmm1 tmm1 mentioned this issue Aug 21, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Erotemic @Abdelgha-4
0