8000 Update monitoring resource by arpitjasa-db · Pull Request #161 · databricks/mlops-stacks · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Update monitoring resource #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 6, 2024
+19 −18
Merged

Update monitoring resource #161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
10000
Diff view
2 changes: 2 additions & 0 deletions ...e_alphanumeric_underscore` .}}/monitoring/notebooks/MonitoredMetricViolationCheck.py.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ from metric_violation_check_query import sql_query
table_name_under_monitor = dbutils.widgets.get("table_name_under_monitor")
metric_to_monitor = dbutils.widgets.get("metric_to_monitor")
metric_violation_threshold = dbutils.widgets.get("metric_violation_threshold")
num_evaluation_windows = dbutils.widgets.get("num_evaluation_windows")
num_violation_windows = dbutils.widgets.get("num_violation_windows")

formatted_sql_query = sql_query.format(
table_name_under_monitor=table_name_under_monitor,
Expand Down
35 changes: 17 additions & 18 deletions ...emplate `project_name_alphanumeric_underscore` .}}/resources/monitoring-resource.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,39 +1,38 @@
# Please complete all the TODOs in this file.
# The regression monitor defined here works OOB with this example regression notebook: {{ template `generate_doc_link` (map (pair "cloud" .input_cloud) (pair "path" "_extras/notebooks/source/monitoring/regression-monitor.html")) }}
# NOTE: Monitoring only works on Unity Catalog tables.

new_cluster: &new_cluster
new_cluster:
num_workers: 3
spark_version: 13.3.x-cpu-ml-scala2.12
node_type_id: {{template `cloud_specific_node_type_id` .}}
custom_tags:
clusterSource: mlops-stacks_{{template `stacks_version` .}}

common_permissions: &permissions
permissions:
- level: CAN_VIEW
group_name: users

resources:
quality_monitors:
{{ .input_project_name }}_quality_monitor:
full_name: {{ .input_inference_table_name }}
table_name: {{ .input_inference_table_name }}
# TODO: Update the output schema name as per your requirements
output_schema_name: ${bundle.target}.{{ .input_project_name }}
# TODO: Update the below parameters as per your requirements
assets_dir: /Users/${workspace.current_user.userName}/databricks_lakehouse_monitoring
inference_log:
granularities: [1 day]
model_id_col: model_id
prediction_col: predictions
label_col: labels
prediction_col: prediction
label_col: price
problem_type: PROBLEM_TYPE_REGRESSION
timestamp_col: timestamp
schedule:
quartz_cron_expression: 0 0 8 * * ? # Run Every day at 8am
timezone_id: UTC

new_cluster: &new_cluster
new_cluster:
num_workers: 3
spark_version: 13.3.x-cpu-ml-scala2.12
node_type_id: {{template `cloud_specific_node_type_id` .}}
custom_tags:
clusterSource: mlops-stacks_{{template `stacks_version` .}}

common_permissions: &permissions
permissions:
- level: CAN_VIEW
group_name: users

resources:
jobs:
retraining_job:
name: ${bundle.target}-{{ .input_project_name }}-monitoring-retraining-job
Expand Down
Loading
0