[Enhancement] Add more informations for information_schema.task_runs #60054
+509
−219
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // process finish time | ||
| addField(resultRow, TimeUtils.longToTimeString(refreshJobStatus.getMvRefreshEndTime())); | ||
| // process duration | ||
| addField(resultRow, formatDuration(refreshJobStatus.getTotalProcessDuration())); | ||
| // last refresh job id | ||
| addField(resultRow, refreshJobStatus.getJobId()); | ||
| // last refresh state | ||
| addField(resultRow, refreshJobStatus.getRefreshState()); | ||
| // whether it's force refresh |
There was a problem hiding this comment.
The most risky bug in this code is:
Assigning mvRefreshStartTime to mvRefreshProcessTime, potentially causing misleading timestamps.
You can modify the code like this:
// start time
long mvRefreshCreateTime = firstTaskRunStatus.getCreateTime();
status.setMvRefreshStartTime(mvRefreshCreateTime);
// process time - assumed it should be a different timestamp, correcting that
long mvRefreshProcessTime = firstTaskRunStatus.getProcessStartTime();
status.setMvRefreshProcessTime(mvRefreshProcessTime);Ensure that getCreateTime() and getProcessStartTime() return the expected timestamps for their respective usage.
fe/fe-core/src/main/java/com/starrocks/scheduler/persist/TaskRunStatus.java
Show resolved
Hide resolved
|
@LiShuMing, this will change user interface, please don't cherry-pick to v3.5 |
a643e7d to
91006b8
Compare
91006b8 to
b4cef8d
Compare
34f1fdb to
fdff3f7
Compare
|
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[FE Incremental Coverage Report]❌ fail : 121 / 155 (78.06%) file detail
|
[BE Incremental Coverage Report]❌ fail : 1 / 22 (04.55%) file detail
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
information_schema.task_runsorinformation_schema.materialized_views;job(the mv refresh task).so I added more informations about those two situations.
What I'm doing:
This pull request introduces enhancements to the schema scanner and materialized views system, focusing on adding new fields, improving data type consistency, and refining the handling of materialized view refresh statuses. The changes span multiple files and primarily affect the backend schema scanner, frontend catalog system, and materialized view status representation.
Backend Schema Scanner Enhancements:
LAST_REFRESH_PROCESS_TIMEandLAST_REFRESH_JOB_IDto theSchemaMaterializedViewsScannerand updated thefill_chunkmethod to include these fields. [1] [2]JOB_ID,JOB_STATE, andPROCESS_TIMEto theSchemaTaskRunsScanner, with corresponding updates to thefill_chunkmethod to handle these fields. [1] [2]Frontend Catalog System Updates:
MaterializedViewsSystemTableto include new columns (LAST_REFRESH_PROCESS_TIME,LAST_REFRESH_JOB_ID, etc.) and changed data types for several existing columns (e.g.,MATERIALIZED_VIEW_IDtoBIGINT,LAST_REFRESH_DURATIONtoDOUBLE). [1] [2]TaskRunsSystemTableby addingJOB_ID,JOB_STATE, andPROCESS_TIMEcolumns, along with corresponding data type adjustments. [1] [2]Materialized View Status Improvements:
jobIdandmvRefreshProcessTimetoShowMaterializedViewStatusto track job-specific and process-related information. [1] [2] [3]ShowMaterializedViewStatusto populate and display the new fields (jobId,mvRefreshProcessTime) in both thrift and result set representations. [1] [2] [3]Data Type Consistency:
ShowMaterializedViewsStmtandMaterializedViewsSystemTable, converting several fields (e.g.,id,task_id,rows) toBIGINTand others (e.g.,last_refresh_duration) to more appropriate types likeDOUBLEorDATETIME. [1] [2]TaskRun Status Refinement:
TaskRunStatusto better handle refresh states for materialized views, including distinguishing between running and finished states. [1] [2]Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: