8000 update cypher query to use less memory by ajtmccarty · Pull Request #6580 · opsmill/infrahub · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

update cypher query to use less memory #6580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 2, 2025
Merged

update cypher query to use less memory #6580

merged 2 commits into from
Jun 2, 2025

Conversation

ajtmccarty
Copy link
Contributor
@ajtmccarty ajtmccarty commented May 30, 2025 •
edited
Loading

fixes #6568

updates the cypher query to save/update a diff to use less memory
my testing on a branch with ~9,000 nodes indicates that performance is basically the same (maybe 5% slower) and that memory usage is reduced by 95% (which seems too high). my local testing on a small branch showed a lower memory usage improvement of more like 70%, but I think it is safe to say that this should reduce the memory usage

@ajtmccarty ajtmccarty requested a review from a team as a code owner May 30, 2025 23:50
@github-actions github-actions bot added the group/backend Issue related to the backend (API Server, Git Agent) label May 30, 2025
@codspeed-hq CodSpeed HQ
Copy link
codspeed-hq bot commented May 30, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #6580 will not alter performance

Comparing ajtm-05302025-diff-update-memory (3d2568e) with develop (eabfeed)

Summary

✅ 10 untouched benchmarks

@ajtmccarty ajtmccarty marked this pull request as draft May 31, 2025 01:43
@ajtmccarty
Copy link
Contributor Author

after an update to fix handling for UNWIND on an empty list ending a query, my testing to update a large diff indicates that memory usage is reduced by over 90%, which is good

@ajtmccarty ajtmccarty marked this pull request as ready for review June 1, 2025 20:49
@ajtmccarty ajtmccarty merged commit ffb23f1 into develop Jun 2, 2025
35 checks passed
@ajtmccarty ajtmccarty deleted the ajtm-05302025-diff-update-memory branch June 2, 2025 01:58
ajtmccarty added a commit that referenced this pull request Jun 2, 2025
@ajtmccarty
update cypher query to use less memory (#6580)
1cb2124 
* update cypher query to use less memory

* update to handle UNWIND on empty list ending query
@ajtmccarty ajtmccarty mentioned this pull request Jun 2, 2025
Merged
@dgarros
Copy link
Collaborator
dgarros commented Jun 2, 2025

after an update to fix handling for UNWIND on an empty list ending a query, my testing to update a large diff indicates that memory usage is reduced by over 90%, which is good

This is fantastic,

  • @ajtmccarty could we extract a best practice here that can be applied to other queries ?
  • @fatih-acar how could we track the memory usage of our queries in the performance testing framework ?

@fatih-acar
Copy link
Contributor

It looks like @LucasG0 implemented part of the solution here: #4504

I think we could make that a runtime config, so we can enable profiling queries during a test run and expose the values as prometheus metrics (that would be then collected and returned by the testing framework).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dgarros dgarros dgarros approved these changes

Assignees
No one assigned
Labels
group/backend Issue related to the backend (API Server, Git Agent)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ajtmccarty @dgarros @fatih-acar
0