update .mlops-setup-scripts/README.md #9
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cleaned up the instruction to run cicd/bootstrap.py
smurching
reviewed
Sep 27, 2022
arpitjasa-db
pushed a commit
that referenced
this pull request
May 19, 2023
* Remove experiment parent dir Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Update experiment to be in the current user's directory Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix test Signed-off-by: Mingyu Li <mingyu.li@databricks.com> --------- Signed-off-by: Mingyu Li <mingyu.li@databricks.com>
arpitjasa-db
added a commit
that referenced
this pull request
May 23, 2023
* Remove terraform, bootstrap script, terraform related github actions Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Remove project_name_alphanumeric. Add project_name_alphanumeric_underscore and update the project folder name to project_name_alphanumeric_underscore Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Move the requriements file into the project folder and update mlflow version to resolve a recipe issue Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Update the model name and experiment under different environments. The experiment folder will be removed until bundles supports managing it. Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add project name as prefix of github and ADO workflows Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Update notebooks to enter notebook folder and current folder. Update format. Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Directly do model deployment in the workflow run and avoid the callback to github. Add model name parameter to BatchInference notebook to avoid reading it from terraform output file Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * add bundle file and bundle resources. Some small fix Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Prepare the batch inference with FS input table in CUJ prod workspace, update bundle to use the new table and fix fs batch inference Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix tests Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Support shared cluster between tasks in one job. Fix a path. Remove explicit token. Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add CI CD support for mlops using bundle and update notebook path according latest bundle update Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix CI PR comment Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix integration tests Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add git source info to the notebook parameters Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Remove Bricks CLI repo token Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Support authentication for an Azure service principal in mlops-stack bundle Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Remove experiment parent dir (#9) * Remove experiment parent dir Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Update experiment to be in the current user's directory Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix test Signed-off-by: Mingyu Li <mingyu.li@databricks.com> --------- Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add new bricks bundle ML resource configuration to docs and remove terraform (#5) * update docs Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add bundle and remove terraform from gitignore Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Remove terraform from MLOps-stacks except ADO CICD. Fix tests and update more docs Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Add file names to gitignore Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * More doc changes. Move the CICD set up section from ML resource README to mlops-setup doc Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Address comments Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Do not use hard coded aws doc for bricks CLI bundles since azure doc is also available Signed-off-by: Mingyu Li <mingyu.li@databricks.com> --------- Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Use variables for bundles * Add example and fix tests * Fix link * Address bug bash issues and polish stacks for private preview * Add bundle run example to docs * Fix nits * Resolve merge conflicts and add TODOs * Fix broken link * Rename bricks CLI to databricks * Add doc for generating example batch inference input (#14) Signed-off-by: Mingyu Li <mingyu.li@databricks.com> * Fix Bug Bash Issues * Fixes * Azure DevOps integration using Databricks Asset Bundles (#12) * vscode gitignore * vscode gitignore * typo in test configs * update reqs path * update tests-ci to use bundles * add script to install bricks cli * update main -> {{cookiecutter.default_branch}} * add bundle cicd * update docs * typo * update docs to have ado * update pipelines to use cookiecutter URLs * doc updates * remove print statement * update setup script bricks -> databricks rename * update to use databricks cli and lowercase staging * lowercase standardize, databricks CLI * update conditions * update name * parameterize databricks cli version * bricks -> databricks * lowercase displayName * update pipelines * minor doc updates * revert condition * Update {{cookiecutter.root_dir__update_if_you_intend_to_use_monorepo}}/.azure/devops-pipelines/{{cookiecutter.project_name}}-tests-ci.yml Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> * Update {{cookiecutter.root_dir__update_if_you_intend_to_use_monorepo}}/{{cookiecutter.project_name_alphanumeric_underscore}}/databricks-resource/README.md Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> * Update tests/example-project-configs/aws/aws-github.yaml Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com> * Update {{cookiecutter.root_dir__update_if_you_intend_to_use_monorepo}}/.azure/devops-pipelines/{{cookiecutter.project_name}}-bundle-cicd.yml Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> * Update {{cookiecutter.root_dir__update_if_you_intend_to_use_monorepo}}/.azure/devops-pipelines/{{cookiecutter.project_name}}-bundle-cicd.yml Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> * update url * update docs to use SPs over PATs * use SP credentials * use SP credentials * update path * standardize on Azure Pipelines * minor doc updates * typos * Fix links (#16) Signed-off-by: Mingyu Li <mingyu.li@databricks.com> --------- Signed-off-by: Mingyu Li <mingyu.li@databricks.com> Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com> Co-authored-by: Mingyu Li <mingyu.li@databricks.com> * some docs fixes. * more changes * fix broken links. * fix links * add cleanup in hook * remove mlflow recipes from another place. * make changes * small fix * doc changes --------- Signed-off-by: Mingyu Li <mingyu.li@databricks.com> Co-authored-by: Mingyu Li <mingyu.li@databricks.com> Co-authored-by: Niall Turbitt <52167569+niall-turbitt@users.noreply.github.com> Co-authored-by: Vladimir Kolovski <vladimir.kolovski@databricks.com> Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com> Co-authored-by: Jerry Liang <jerry.liang@databricks.com> Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
remove the redundant comments in the section
Run the scripts, bootstrap cicd step to avoid confusion