8000 feat: Implement SQL Runner Job by tanmoysrt · Pull Request #2461 · frappe/press · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: Implement SQL Runner Job #2461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

feat: Implement SQL Runner Job #2461

wants to merge 20 commits into from

Conversation

tanmoysrt
Copy link
Member
@tanmoysrt tanmoysrt commented Feb 5, 2025 •
edited
Loading

Context:

For building database related feature like SQL Playground or DB Analzyer, usually we keep adding new functions and api endpoints for that.

But for any fix or quicker iteration of these tools, we need to do modification on agent side and deploy those. So, planned to design a simple api interface to run short term or long lived SQL queries with some flexibility.

So, it will be just a abstraction and a wrapper around Agent and Agent Job to simplify the things.

Tasks:

  • Allow to run short term queries over a agent request (like SHOW TABLES, SHOW DATABASES)
  • Run long running queries (like Fetch Table Schema or Fetching bunch of perf schema response) with Agent Job
  • Support to run multiple queries together
  • Support to continue on error [If any query failed, dont stop execution there] [Configurable]
  • Add SQL Query Profile support.
  • Add callback support (same as process_job_updates of Agent Job)
  • Add utility functions to make it easier to use.
  • Add expiry fields to remove the records automatically.

Utility Function:

def run_queries(
	queries: str | list[str],
	target_type: Literal["Database Server", "Site"],
	target: str,
	mode: Literal["r", "rw"] = "r",
	user: Literal["Root User", "Site User"] = "Root User",
	type: str = "Generic",
	allow_ddl_query: bool = False,
	allow_any_query: bool = False,
	async_task: bool = False,
	continue_on_error: bool = False,
	wait_timeout: int = 30,
	lock_wait_timeout: int = 30,
	max_statement_time: int = 600,
	profiling: bool = False,
) -> SQLJob:

Doctype Preview:

image

Data

image

Error Details

image

Profile

image

Agent : frappe/agent#163

@codecov Codecov
Copy link
codecov bot commented Feb 5, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 17.87072% with 216 lines in your changes missing coverage. Please review.

Project coverage is 36.73%. Comparing base (d4bb4a0) to head (8720a5e).

Files with missing lines Patch % Lines
press/press/doctype/sql_job/sql_job.py 20.40% 156 Missing ⚠️
press/press/doctype/sql_job_query/sql_job_query.py 0.00% 32 Missing ⚠️
press/utils/__init__.py 19.04% 17 Missing ⚠️
press/agent.py 9.09% 10 Missing ⚠️
press/press/doctype/agent_job/agent_job.py 66.66% 1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (17.87%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #2461      +/-   ##
==========================================
- Coverage   36.90%   36.73%   -0.17%     
==========================================
  Files         403      405       +2     
  Lines       32073    32335     +262     
==========================================
+ Hits        11835    11877      +42     
- Misses      20238    20458     +220

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tanmoysrt tanmoysrt mentioned this pull request Feb 5, 2025
Open
@tanmoysrt tanmoysrt marked this pull request as ready for review February 7, 2025 13:02
@tanmoysrt tanmoysrt self-assigned this Feb 12, 2025
@tanmoysrt tanmoysrt moved this to Todo in Tanmoy's Roadmap Mar 25, 2025
@tanmoysrt tanmoysrt moved this from Todo to In Progress in Tanmoy's Roadmap Mar 25, 2025
@tanmoysrt tanmoysrt moved this from In Progress to Stalled in Tanmoy's Roadmap Mar 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@tanmoysrt tanmoysrt

Labels
None yet
Projects
Tanmoy's Roadmap
Status: Stalled
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@tanmoysrt
0