From 7bec5ab4d38b94337dbbca79299de051603bbb83 Mon Sep 17 00:00:00 2001 From: jsburckhardt Date: Fri, 13 Sep 2024 09:19:17 +0000 Subject: [PATCH 1/2] feat(docs, config, devcontainer): enhance README and configuration for gic usage - Updated README with comprehensive installation and usage instructions for the `gic` tool. - Added configuration examples for `.gic` and devcontainer setup, including GPU support and Ollama feature. - Introduced a flow diagram to visually represent the usage process, improving clarity for users. Closes #12345 --- .devcontainer/devcontainer.json | 14 +- .vscode/launch.json | 3 +- README.md | 156 ++++++++++- docs/flow.excalidraw | 472 ++++++++++++++++++++++++++++++++ docs/flow.png | Bin 0 -> 13676 bytes 5 files changed, 627 insertions(+), 18 deletions(-) create mode 100644 docs/flow.excalidraw create mode 100644 docs/flow.png diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index fa23407..830ee64 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -1,12 +1,12 @@ { "name": "gic", "image": "mcr.microsoft.com/devcontainers/go:dev-1.23", - // "runArgs": [ - // "--gpus", "all" - // ], + "runArgs": [ + "--gpus", "all" + ], "hostRequirements": { "cpus": 4, - // "memory": "16gb" + "memory": "16gb" }, "features": { "ghcr.io/devcontainers/features/common-utils:2": { @@ -17,9 +17,9 @@ "ghcr.io/devcontainers/features/azure-cli:1": { "installBicep": true }, - // "ghcr.io/prulloac/devcontainer-features/ollama:1": { - // "pull": "phi3.5" - // } + "ghcr.io/prulloac/devcontainer-features/ollama:1": { + "pull": "phi3.5" + }, "ghcr.io/stuartleeks/dev-container-features/shell-history:0": {} }, diff --git a/.vscode/launch.json b/.vscode/launch.json index a999a91..680d451 100644 --- a/.vscode/launch.json +++ b/.vscode/launch.json @@ -10,7 +10,8 @@ "request": "launch", "mode": "auto", "program": "main.go", - "cwd": "/workspaces/gic" + "cwd": "/workspaces/gic", + "envFile": "${workspaceFolder}/.env", } ] } diff --git a/README.md b/README.md index 6a31317..5ae4521 100644 --- a/README.md +++ b/README.md @@ -2,24 +2,150 @@ Reducing cognitive load by automating commit message generation, allowing developers to focus on coding instead of crafting messages. A tool that helps developers generate git commit messages based on the `git diff` of staged files, following instructions. It's ideal for use alongside [Semantic Release](https://github.com/semantic-release/semantic-release). -## AzureAD +## How to use it -Remember to assign the `Azure Cognitive Openai User` role to any user that is going to consume the resource. +The flow is simple. Install the binary, create a config (`.gic` or `.gic.yaml`) file in your project root, and run `gic` in your terminal. The tool will generate a commit message based on the staged files and the instructions provided in the `.gic` file. -In your `.gic` config you would add: +![flow](docs/flow.png) +![alt text](image.png) + +### Install the binary + +Run the following command to install the latest version of `gic`: + +```bash +curl -sSfL https://raw.githubusercontent.com/jsburckhardt/gic/main/script/install-gic.sh | sh -s +``` + +or if you prefer, add the `devcontainer` feature to your `.devcontainer/devcontainer.json` file: + +```json + "features": { + "ghcr.io/jsburckhardt/devcontainer-features/gic:1": {} + } +``` + +### Create a `.gic` file + +There are different options to create the config file. + +- You can create it by using the [sample.gic.yaml](https://raw.githubusercontent.com/jsburckhardt/gic/main/sample.gic.yaml) file in the repository. + +- The second option is to ask gic to create a basic config file. Run the following command in your terminal: + + ```bash + gic --create-sample-config + ``` + +## Config file sample ```yaml +llm_instructions: | + You are a commit message generator that follows the semantic release format based on Angular commit guidelines. The user will provide a git diff, and your task is to analyze the changes and generate a SINGLE appropriate git commit message. The message should clearly indicate the type of changes (e.g., feat, fix, chore, docs, style, refactor, test, build, ci, perf, or revert), a brief summary of the change in imperative mood, and optionally include a scope in parentheses. If applicable, include a body with additional details and a footer with references to any related issues or breaking changes. + Commit message can have more than one scope and can be multiline. + + Use breaking change only and only if the change is a feature based on code changes. + + Example Format between ~~~: + + ~~~ + (): + + [optional body] + + [optional footer(s)] + ~~~ + + return ONLY and ONLY the commit message with no ~~~. + + Example Usage: + + Input: git commit diff: ... + Output: A commit message following the format based on the analysis of the diff. + + Example Commit Messages: + + feat(api): add new endpoint for user authentication + fix(ui): resolve button alignment issue on mobile devices + chore(deps): update dependencies to latest versions + docs(readme): add instructions for setting up the project locally + refactor(auth): simplify token validation logic + test(auth): add unit tests for login functionality + perf(core): improve rendering performance by optimizing the DOM updates + +############################################ +##### sample ollama +# connection_type: "ollama" +# azure_endpoint: "http://127.0.0.1:11434/" +# model_deployment_name: "phi3.5" + +############################################ +# sample azure +# connection_type: "azure" +# azure_endpoint: "https://generic-aoai-01.openai.azure.com/" +# model_deployment_name: "gpt-4o-mini" + +############################################ +##### sample azure_ad connection_type: "azure_ad" -azure_endpoint: "https://.openai.azure.com/" -model_deployment_name: "" +azure_endpoint: "https://generic-aoai-01.openai.azure.com/" +model_deployment_name: "gpt-4o-mini" +should_commit: true # this will not only print the commit message but also commit the changes +tokens: 4000 +api_version: "2024-02-15-preview" ``` -## Ollama Locally in your devcontainer (or any machine) +## Customizing the config + +### Using Azure OpenAI resources + +There are two common ways to authenticate with AOAI resources. The first one is by using a **api_key** and the second one is by using an **azure_ad** token. + +- **api_key**: For this flow. You'll need to configure the set `connection_type` to `azure`. Add the details for `azure_endpoint`, `tokens`, `model_deployment_name`, and `api_version`. Additonally you'll need to add the environment variable `API_KEY` with the value of the key. -Here is an example to run ollama in your devcontainer and pulling phi3.5 image. + ```yaml + # .gic + connection_type: "azure" + azure_endpoint: "https://.openai.azure.com/" + model_deployment_name: "" + api_version: "" + tokens: 4000 + commit: false + llm_instructions: | + - Generate a commit message based on the staged files + - follow semantic release guidelines + ``` + + ```bash + # In the terminal + export API_KEY= + ``` + +- **azure_ad**: For this flow. You'll need to configure the set `connection_type` to `azure_ad`. Add the details for `azure_endpoint`, `model_deployment_name`, and `api_version`. Additonally, you'll to assign the Azure Cognitive Openai User role to any user that is going to consume the resource. + + ```yaml + connection_type: "azure_ad" + azure_endpoint: "https://.openai.azure.com/" + model_deployment_name: "" + api_version: "" + tokens: 4000 + commit: false + llm_instructions: | + - Generate a commit message based on the staged files + - follow semantic release guidelines + ``` + +### Using LLMs hosted in Ollama Locally in your devcontainer (or any machine) + +Here is an example to run `ollama` in your `devcontainer` and pulling phi3.5 image. ```json ... +// add this args to load the GPUs to your devcontainer +"runArgs": [ + "--gpus", "all" +], +// add the ollama feature and pull a model e.g. phi3.5 "features": { "ghcr.io/prulloac/devcontainer-features/ollama:1": { "pull": "phi3.5" @@ -34,26 +160,36 @@ In your `.gic` config you would add: connection_type: "ollama" azure_endpoint: "http://127.0.0.1:11434/" model_deployment_name: "phi3.5" +tokens: 4000 +commit: false +llm_instructions: | + - Generate a commit message based on the staged files + - follow semantic release guidelines ``` +## Different outputs per model + >[!CAUTION] >When choosing a model validate the instructions and generation matches what you expect. As an example, noticed phi3.5 didn't really generated a commit message with the requested instructions. More details in [here](#different-outputs-per-model). -## Different outputs per model +For this sample we are doing some Documentation changes in the `README.md` file. The instructions are the same for all models. Instructions: ```yaml llm_instructions: | - + - Generate a commit message based on the staged files + - follow semantic release guidelines ``` ### phi3.5 ```bash +commit message: ```\nGenerate a commit message for staged files following semantic release guidelines using Ollama locally or Azure Cognitive OpenAI service. The tool, configured through `.gic` config file, auto-generates messages to streamline the process and maintain consistency in your project's history.\n``` ``` -### gpt-4o-mini +### gpt-4o ```bash +commit message: docs: update README with detailed usage instructions and config options\n\n- Reorganized sections for better clarity and usability\n- Added installation instructions for the `gic` binary\n- Provided steps for creating and customizing the `.gic` config file\n- Included guidance on using Azure OpenAI resources with different authentication methods\n- Detailed instructions on using LLMs hosted locally via Ollama\n- Enhanced documentation with images and sample commands" ``` diff --git a/docs/flow.excalidraw b/docs/flow.excalidraw new file mode 100644 index 0000000..ae64df6 --- /dev/null +++ b/docs/flow.excalidraw @@ -0,0 +1,472 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor", + "elements": [ + { + "id": "QV8uB_cf1BbP2MRSlP4su", + "type": "rectangle", + "x": 381.7599792480469, + "y": 257.82411193847656, + "width": 150.18441772460938, + "height": 79.4813232421875, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 279745161, + "version": 40, + "versionNonce": 509191879, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "jSmokzYcDwvmtDcZVRpGt" + }, + { + "id": "j_HJDocJNwqX_FY1sdzIl", + "type": "arrow" + } + ], + "updated": 1726206145791, + "link": null, + "locked": false + }, + { + "id": "jSmokzYcDwvmtDcZVRpGt", + "type": "text", + "x": 388.93223571777344, + "y": 285.0647735595703, + "width": 135.83990478515625, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 910357449, + "version": 16, + "versionNonce": 2080986439, + "isDeleted": false, + "boundElements": null, + "updated": 1726206132545, + "link": null, + "locked": false, + "text": "Install Binary", + "fontSize": 20, + "fontFamily": 1, + "textAlign": "center", + "verticalAlign": "middle", + "baseline": 17, + "containerId": "QV8uB_cf1BbP2MRSlP4su", + "originalText": "Install Binary", + "lineHeight": 1.25 + }, + { + "type": "rectangle", + "version": 74, + "versionNonce": 1247479209, + "isDeleted": false, + "id": "ShIM_C-jbJVToCVmltv7z", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 602.2503509521484, + "y": 257.82411193847656, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 150.18441772460938, + "height": 79.4813232421875, + "seed": 253602983, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "boundElements": [ + { + "type": "text", + "id": "1amWIp1Y5P0amLR2IsJoh" + }, + { + "id": "j_HJDocJNwqX_FY1sdzIl", + "type": "arrow" + }, + { + "id": "e64YUXlovto_y4qXGN9rw", + "type": "arrow" + } + ], + "updated": 1726206159847, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 75, + "versionNonce": 205844105, + "isDeleted": false, + "id": "1amWIp1Y5P0amLR2IsJoh", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 614.6426239013672, + "y": 272.5647735595703, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 125.39987182617188, + "height": 50, + "seed": 1345290183, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1726206474493, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "Create / \nconfigure .gic", + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "ShIM_C-jbJVToCVmltv7z", + "originalText": "Create / configure .gic", + "lineHeight": 1.25, + "baseline": 42 + }, + { + "type": "rectangle", + "version": 129, + "versionNonce": 1264674247, + "isDeleted": false, + "id": "vECBL-A5es3x6L0XcfNqQ", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 822.74072265625, + "y": 257.82411193847656, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 150.18441772460938, + "height": 79.4813232421875, + "seed": 384758953, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "boundElements": [ + { + "type": "text", + "id": "FZEOO-PxrEazbjcSDCZQe" + }, + { + "id": "e64YUXlovto_y4qXGN9rw", + "type": "arrow" + }, + { + "id": "1n56d3vNSA6GF3C9JgLaw", + "type": "arrow" + } + ], + "updated": 1726206162389, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 128, + "versionNonce": 1125678345, + "isDeleted": false, + "id": "FZEOO-PxrEazbjcSDCZQe", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 844.0229873657227, + "y": 285.0647735595703, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 107.61988830566406, + "height": 25, + "seed": 1952216969, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1726206138621, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "stage files", + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "vECBL-A5es3x6L0XcfNqQ", + "originalText": "stage files", + "lineHeight": 1.25, + "baseline": 17 + }, + { + "type": "rectangle", + "version": 164, + "versionNonce": 1729748999, + "isDeleted": false, + "id": "FWouZ1c7-aX6iKEGw1RHc", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1043.2310943603516, + "y": 257.82411193847656, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 150.18441772460938, + "height": 79.4813232421875, + "seed": 1130781449, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "boundElements": [ + { + "type": "text", + "id": "WFYaiHVvZEjbL760jNSo4" + }, + { + "id": "1n56d3vNSA6GF3C9JgLaw", + "type": "arrow" + } + ], + "updated": 1726206162389, + "link": null, + "locked": false + }, + { + "type": "text", + "version": 173, + "versionNonce": 1530029735, + "isDeleted": false, + "id": "WFYaiHVvZEjbL760jNSo4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1106.1033172607422, + "y": 285.0647735595703, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 24.439971923828125, + "height": 25, + "seed": 1037570537, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1726206132546, + "link": null, + "locked": false, + "fontSize": 20, + "fontFamily": 1, + "text": "gic", + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "FWouZ1c7-aX6iKEGw1RHc", + "originalText": "gic", + "lineHeight": 1.25, + "baseline": 17 + }, + { + "id": "j_HJDocJNwqX_FY1sdzIl", + "type": "arrow", + "x": 543.5654907226562, + "y": 297.4664406970717, + "width": 47.40673828125, + "height": 0.314666748046875, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 566333129, + "version": 40, + "versionNonce": 1550839175, + "isDeleted": false, + "boundElements": null, + "updated": 1726206167504, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 47.40673828125, + 0.314666748046875 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "QV8uB_cf1BbP2MRSlP4su", + "focus": 0.05565282860641122, + "gap": 11.62109375 + }, + "endBinding": { + "elementId": "ShIM_C-jbJVToCVmltv7z", + "focus": -0.09202357547573803, + "gap": 11.278121948242188 + }, + "startArrowhead": null, + "endArrowhead": "triangle" + }, + { + "id": "e64YUXlovto_y4qXGN9rw", + "type": "arrow", + "x": 762.3154907226562, + "y": 298.6245200497517, + "width": 48.28564453125, + "height": 0.62933349609375, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1979026023, + "version": 33, + "versionNonce": 527060391, + "isDeleted": false, + "boundElements": null, + "updated": 1726206173513, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 48.28564453125, + 0.62933349609375 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "ShIM_C-jbJVToCVmltv7z", + "focus": -0.0011727263301429115, + "gap": 9.880722045898438 + }, + "endBinding": { + "elementId": "vECBL-A5es3x6L0XcfNqQ", + "focus": -0.06940233898445695, + "gap": 12.13958740234375 + }, + "startArrowhead": null, + "endArrowhead": "triangle" + }, + { + "id": "1n56d3vNSA6GF3C9JgLaw", + "type": "arrow", + "x": 984.5377197265625, + "y": 297.3522776190496, + "width": 48.10107421875, + "height": 0.314697265625, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1508022857, + "version": 27, + "versionNonce": 1061594279, + "isDeleted": false, + "boundElements": null, + "updated": 1726206167504, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 48.10107421875, + 0.314697265625 + ] + ], + "lastCommittedPoint": null, + "startBinding": { + "elementId": "vECBL-A5es3x6L0XcfNqQ", + "focus": 0.0040859279345153605, + "gap": 11.612579345703125 + }, + "endBinding": { + "elementId": "FWouZ1c7-aX6iKEGw1RHc", + "focus": -0.039941510446498234, + "gap": 10.592300415039062 + }, + "startArrowhead": null, + "endArrowhead": "triangle" + } + ], + "appState": { + "gridSize": null, + "viewBackgroundColor": "#ffffff" + }, + "files": {} +} \ No newline at end of file diff --git a/docs/flow.png b/docs/flow.png new file mode 100644 index 0000000000000000000000000000000000000000..75144e3a5adb160f626539f488d4e95f246f230e GIT binary patch literal 13676 zcmZvDbySqm*Dl@N;7E6ubW68@ba#g!(%mT_Al)F+CEYDu(%mqGfCvb9kH7EU@7`~% zJAcf~nmF%y&)M-j&)z3mT~!VfjSLM21_o0>URnbN23{Pzw?Kgd&q66oh~Nd*T|-U^ zrg{=`3;IrqY+f^8|X(SvQ^DDwf$=<}EEnnZdymYF&r~3qd(S87I&0u8!;Z zL(xw6!wq4c57y_M+1K#dt?uW7Q}-fM+1KFZRvZo;12zKYbr4k#jwuQ!0twuZz8%>*6{!5KmYk6E)2%PMRt}bCH22AT`Jj={%A^8DNU##%-bKN zPEmfT{^_Yp#eEui`|Kn0RiSrybNNO3 z1_hY!hu6_Pu$EsdDrCf{q)7rVzQAdrNvrE|Uu!3QEz8RfwiA1DdCyJ^u!x`5QXMG=ZOc`0#t;x~y)f|UBu(p+Zw6E8BZr0WDCl0^< z(lf*rqo^nem)XwR7cqKxz3pO(t1XU*L?nk>NOeFB(U~qo;~;h9XU} zC8r`%YGlB&a+E3xyyVw)h0CeFW?IT5z9&|T-M+pbxvpKZt)RC~nOC;WiW`M3Nb5B7 zl}2%EUstHY7!wggsBMTFh1+3K!geW8sTb40KKb}5*!eS&Lc3&E2`xR;Yz4RH%_cRK ztAIm?d_brjf2{z5M3&ZOPPw@Yue*&fWLI7JyRjs8why{|hT{s+z=OQgPi1@OCC-el zhJn&#Z-Hd{H_x$PbQ;<|n}emvRV<1%)bpRPl-e+@!6(0QcJjeBUT}Y%FH(^&T!A%yYnhFJfC|c z1Xk`8sART~a`~I!RLs85%{6{5X8g+SYf+>igYzp5#(=H?&w0)Ay9mUmL6p>cdVwls zXxb!LFY_}RiN0`hUy({?s%wg?DIZh0zrWbPGLKA_mZ{5UT+ME=S1(nV_ts*FaM+O7 z#pJM&8x+=0j(e~(KtmVIH=Vpkk zgTmM~-#_3rD&SA`l8!CSE1_2)q2d*?FbHiRPI?!Lm zL*ws5U-1_I`F&BN-}JS(*wWfsw%?JiAtZO!5*p9OnLOY0hCH*rEVMLv;tk0lip;w{ z37uSjrUw4f0NhI1`=|gc&3b;Gx6Efm?+?aFrAEG|vdGEFrLbYt@~0%Z96hUOms=Wy z1Y8Z5xIEx8xhatNDna^h4kY>{OUqI$TuL2n<|-1k(wr48;%-rrQ7eGd)SSRj&65%m#G#Rzk#3UXTKPE+)Qr zG$U_lZPgRYpx>MvAOAN`9-fAp8nqM*LLRRDFf3H$!L|m3HkqDEl@jI68UrKFUKrTH zN!5%_F2?v+7>DHfQ)g@I()#-3U4bi+PwDA@?{*VU+^3o8YMUP(e5$xVjUM)eqt|p? zDZYOFniA)uLg}_n&xZ0h(Uh>z2*$cfjkus4*0(YH3LOHPLaq3bgA#&_SRSd%Ss5by zNVD1Wh1|ELctgv}%U4(Lq@u?5?HwE{zJA^5E2yYYvce*xhriX<((?GZGl|7E4c6?3maJyma~{uaY2ZsXK2&^jmxI$k z#90p$WxbXdjG?_-GevT!RA7};Qc^-F$0sJ7R=>sLXuNroL{>Jxwq|HzvKfufz|72? z#%`n-Kn|WCn!BIaNHFYedqk!v!~;ZVXm%g(ugR<{KBqD*zj&Npsz#JKJLQA5aA|ux zQAnr^e>;>(yQ`mgzm&BjH7W4v{q=!L#X@FYp16oeTumsv<9geBwLG!G$V3Wpb`Fka z{VJl^lKlLwIW=*kE$=%g>ZAsnspLH5vbp8?d0tM=%YJNe+GO{}50McO5G*-6JG<}S zy7#BJV(1&3|J|#>!WV*ZKPYCVqPxqbMmcaX3+P0C%$%%aLp?ucRgyM4Xee)@CZVudnax*RO%k*LjUPF0QVe3=Cz(#l=-sRdsbqIQ1683HSHjente| z4>xv3Mmg)whXRxP@77z~UVoB!wxGoMQd!xkUuEL=Gu%&cqWcw{Z)jIu&hOIJrefCD z*LEN4nm>`MC8H@ub!NB-B-4I+pH+3g^y21;tiJj9_!wL}I~zi4Hq(f@(en8C===K| zHDv4e9|L{;`uh4GH74RbJUm=nZ?v?eQR?)oB(?++PLGa)*5HCAUcZKSjMl9)mywk0 zrTBX{+}0+fuxMmrLNV;d4Rx}$Wn^H8#4&GPHOpMEZ^FXBkWLY~5^QD_PseI`~%M+(ggc(7AaLO&yj zMpd_@bCk43Aa^|l>lvGd|br)YOkt!V@KdT6kF_bcJ`ik zS5XoClTEP1*3nE6WY}rFjc`&cD?8h;)8`e{9(1`O$*MQon)jAZNa(~vh>b|e-~R~; zbz~&4x3#4V`?6Ui#Kl$TdA8Eg(NR!PFfyC+6js-4N#mIRC?_j>3thej!#0j#Kqr(R zU--Guudd~I3M-Vr#O8EL%0<1*+Pv7Tw~95_$x+0lYhoSRAlOFSG|<*2@|rI!K^_|& z89BPRATHVVeoj#nqY7iyt1m7nF!S?!JXxr4_)$p}1`!X)KD<9&tg*4Q+)6`{#Mwfn zDhB@Ptn0K+$wR830W)6-K)X{<;tQAkKAjl*QpZ7m#~So&T*APgdqptMMWwDf{X zT0+R{OsZDls^h-6s${ZXVfXlL>^sTpl z0CuKRZJ3ds9x3Pw(Rus!m9TKE;D~g!VQX?y(i>yrYVJ;D?VtXGO9p-|q(@bzoOh!P zdAe2Fz@LBopy!!^g<{Gr+Evn6xY!}&5wb-~NI}_1|NNmXQQqipSzb*Pf7UfN#!~7# zJvk8)7N!vK`uHFtg-YGOltl*fv$i=qJG-l^OP!J6o~X82$Jm?hU0}~bg$^neGB!B| z0$gZlC_~@&k1B%}rh@YFXdDe49b&X_Sy@>==e3qC@0D*(t2TN5)|kjMKZ0jwW>Q&n z3L{BIAj-whPZxmVJOvL$~+=EhL}+)|&o;DNC@#X*S9YzJXuG**7k$b7E>rM_XIW zCGh!g_qy{Ym_n!TZ%d?44_pK!>fD=Ig|d$Uf&TuTgXY0iF!%TOt*x!Kqxs*y@sVJ_ z`O<_*v%`n?i-|p6_S+uT1>-(wC4QpsA^8U|{g! z!v{ON@^xpEPM=?Wun1U@yXWienUshIk9ZwtxI&X4)Yw8z59asZjgBWphjsh>vX_+| zD$5I4Szh+u8%Y8|Hpz9V!~6R8W*~=RWpiR&=U{Qt=;&w)gIcv-1Cnj_+_A{Jy^*7% zFO(>*g)Y|C)_XrxYwP0T;%d4c6zc_Y4{Hot;nEuH7rq8QotiAAAi@O?4avq1*8AOG z_4Z17`8zv1gF6XB%i?p&%*@0?fUV>b7S0mORIQ&Li3ElWf@OIJ0gDEbB#!dY*nr}WZgLuQ7F0}0m{}(hx_#OL`q8fd33H!Z803)AEBb- z%a&fj;zB|~Lf7@AaEnIF;r-CiP+S~F>e$eJ?A)zE&VN|Wf9P?}oh2!Whd!BJ?qQy& zFVXRJj}bb0**NU+1dH!%ZPd17AajJo{f!-l9EyqVoaCHzMQ3t^*huu$23Lj-^$FX zjGEeeD?1XDu6LJac%Fj91O(IQ%B!oZ@0%Q`*f5o7Kl5cx9DB&N-P34(7vXBYI0HVX zr>8gL*T!8-a#Rtvl%=e;ytx^8ce$OGmS)uMi7Rz`u_^xQRk)^jCd|K8StXar_5PgF zM<`>zhrnffi^R{=3;EF-iVRQ+@Tl$lnk;^oucJIxb{>5j5*T<`s%1P$&IU6X+1!rD zA=BYV?~DJkn{++y490qxr~Z_n)fMgKVo!lnPcSZop7 z_u5M&`ZIOBM$GaHs^Zep(xReWfTtp{DS(y82B;~erg_o&;v>L%p0FCV@$&Gbu1Occ23B{{_WpFILBqX)zh)7ghqksxoQ*F9DwQ(0q3*1k^s?0j3x9xM z=JA`Hsf)l98Wi)xb$JrzO1k{!^Vw>1I;$QWLsNuQ0dU~d)YOHAh3ITX`xtp+vX}Mz zcvzhEas7m4YPU#aeqh;wkGZ+I1q1|mdwaXPH%uM&ccPQoXB^&x^p+_0FtYz{|L{;p zORMqQH!+_Z`PJ=|2)H@DHqr)`1i{Co-VU0H=&dX&C$%`&iyA1*dJBO)T+pZ@3-3g)qyw6wG|Z1Z5k zQf~C8TPD-uV4K(onB}5k(XNbIPz>(OfZCFi4S`s?va&Ket09_AERSCLR^%93RazP$ zhAG*(Ha9m1K+@pgplAEKeN#xF)b%QPaA(0G#*#s!yE3?#;&TOV$>bb+1RMHU#{GrUif8aWE^8g5}0hVGAV~(r;Kfc^ls&f`a-ZGkEQ|0419Wc zuWN(Z{IGQ9c^BR1lo)tFX3}$}&=(9)IY5%}adFZi zC4#=82fqMnA&Ny*U@&5zy8dv!)*AR%Z0fq9p#hK_u^(6|;Zg-0U3Y~Y9c%mV*eWm2 zpwKu=3$Wa^hg@;ff1m1fStlkYnw{6{^{YxsN(6NJt{+YQJ}d&r_M5{0CdqcSNOkGO zHyU-~EmvI*si@U6?a}R^&f|h4Lft5ISWB0aKLst{-*_q1C&p#<(@k#Vt1z;&mj?bK zirv4S5|8X0&B(|Ai*G_HBY;0_)K;4thSTS^w$bVP z=ivc-V|7l^E7atY9zySuxyGkZbE+;Nz{Op?IB%>WJ*^W^EIq9%sjDy@>~TU+ub z*^wC~4had#(b-ulgC)R+0HYseq(O5qwH(&A=;oG3&-8;lwXLZR0dx5N{W}^O+SW5g zn6;^Ck9`whPjQKfHAd}eVf_)9FF*#_Vr9{#^E{d^aOt|IUztySbV}^+?+1DJ;}if$ z*H>5Bstg-Z;mJ3*x3jt4mz}_E)k-CMKTyGSHM?Stu+(&fI8}g@>*$EE?VWppjx32o zRNZ0N9pJC3IvI+LZCb^i-52=h)@j7m)01eDdRu^`_(MW`yr}=f&04Gb5@w#DJJN?TsaEdai2ahZVL0hy@9(dyehE)e+T3>| znY+Lok8yOfPHh8!$hTl)WDH_9U?mc{HpB^&OC)b?vP(GuX9|kN0gNm6cn_u zu(`VJ*ijp~#!kVUd#Bp@VoTB-$x-n`*6w(Z&@_>1?$GD!Hl!-++`$QkmJ z@1(|dTm{$-TVNhhRk`9jJ{$q0e{+8a$h^rn#|&FC-b|RwY-^V=H1y%3Evkyz`TqTP z{(ql-kWRbZ&_~nzcy_JCXwIoDNCVw+DYd*T|BS5xSa%9T zY95mt#sT+M|H7%t(<`3nhGu|Hl9TaV>i|S1zE>%JS0IyY>B9U%)>3{fox{b^(b3*M zB*hJ24O4Q;q;`+j1=C41awt?F+7oja12BU!4u>zb;s1NRx|&@Ig`GwW05#EJAyH8_ zcazH|heZe$G8HWg%PAn)E7ez8qZ@u=c{EigI<3B`!wMafbaAJ zZrPv_A*r;afQ(4*JZlZW@cD9K70S0*8H zB;owrIrp%D_?v!J60*qg*MzG++JOy%MUr3|w%JrfXC6E(FkcX4YFl9g8Vl6FR8*ix z*g}R=)6x>+;-+&MzG(2?e(Py!c3PFC5JFSS7no_fyM*Q!iyCjaO85-|vR~LE|_{f$1=LjG^fasJX9!S3O3Ub_sFD38oeGeD_ z%KJ!E6NLjbbWFDrc}|kG_7629FOV}(Lh?s8YFQA2e_j3_BsTeE=S<@JdY_Cpgq}gr z9jzaA^%~|i`a{hgW9T_L&7YZ(jUR{CP2axBHr%Eg`!H!^r3?6m=0Mrl*gztf{-&j^ z4LDJAl77O_h8AC3uPIK)2au#1ZQwGGew{9gdtV{YBky!6@n?pGP)aNYvAL?KVAk`C zcr&c{_l8g^tEy^iYln<+vanQ?l*m;4aS7@iy%VG@TPey=r=FrAP8&J=257oGjoxQO zZXw^$$rbyiZ;g#x25Xf40p!g#Jh?(6FCoGx$GVN!$H_cXKM33uz3-PoN5coiN z5&lH$c8io8i$#hWE+r}|iWLc;m7AGKCMECU!f{7qhP1cRduqv4^CjB%opdyq=0l(?W+r_2>4i{}yas@YvzrXbQ-B z3YY9gwq4V`vgp`G+}}2MeKk8cNUnq7u_R4n2dOW<4m9u4#6TwwuO-Qn_b4qgl1!?M z+WGW5Yt7Q_REs(*A`hpp`%-x*)hKc9_EOYPs6e(8U(xw13Q_Wo@F-&;p-Z$W{&=?26z6ml~(C*-ICJj2fetc;8}D&|)r*m6X# z4{GU)3JPHDzq=V)SkSPlz)wLc!&Vz?u$3WUQUw6|0s<7V+X9b_l`Z1^&g_c7MqJX- zbuoI+(b+jdNL5Yk-z+BH> z*91aJ>egzgUD4Fh0dX!$d=)vEVVv99t8cXTgi|n#Azg&CjQiatj)p~{syzEciXSn> z!&87y>+0T~C6KZf?Dqu{#84rsErUs%A6sv4JxyKg$z+#s2i*+y^p>~#BbyzU;a(pB zTA-xeTyDRX#0f&$ z79fn#cHb~yB*@Ioz5VmlGnB7VTm}2Rj}2EBr`V+3^K4)Mp%mG9adj1a?wV0>*bS5b z1G*^^X_{Bh&<}C`5j9y75BiU*{gK!JUvMJdYlbY;@)K}-<3elg>bTf<$>InU0Cx4r zNbTS5F=9&MZeZYrv&G;#&Y28Okn>Eu5h=GNO3gJLT-TE3>$uyA<>ue-ARpX-m#&xsIVeK0frgdj==-54}G57v25;t06yAh62y#v9Jy zql;+ieE%F|xoB7F!s@<*=olLQlbzSs@JYeJuq&m#lzGbj%*+jMoidYkgEImk!D zZZoASZi`aN-QM?jyMH`^uZN*U8>W@u-y(@(o-ZC>TucGD_8N2dwOvw2iP|mwJC}in zl$n9wNd5}|Ep7Q2x>j`m9D0F?i7CI1AQ`l~NmQy?rF{yBECBrxG!owUg)d}<=vdEs zj>B6nW=pQMNU;&%fFJ|X%631j_N&oNVi95C)!A7%WM*6`Q1LkNYzIOI9daAqSu{W92`;<)DCN!z3P_? zzMqhueH*q!$3Y7x7xkG36nK5ze`g@Nf9f@$@{6Yn{kVrBQ;Q1=R@T8)cnF|n*0w3n`%sb1}@RZl<`6Y3H^m_#eTc5T-#llzA#Uuo(LG0}qH!1lYhkm#7 zESy5*TiM)5>9*0vT7FC?m~X=QG^FoTca7SZ+BrTC;b2H&cm3-NLFcvmxf7L@*vXbH zCtnkn2#;^k*|(HccS9K3SZj&EZzpM%5zsyGy%oxrDd>s^-X z7*IjNkkhK?&pqTYYC~ub-9CUG99a7xR!klG9_D;7RjZpjuB@nd9dY>kPmM`87BYSh zqI4}2tZEq5Ht61&$`Ss()eArPQbfXonzVTHq8r4+r$4{#%5B4uLVE#@m2$gVbZ$23 z@WM4AB_mtu@NO712a+ntiG9q|_g@mnHVqT*;}Q~{@23J~lK{`PvcCMu_B5V7OHD_P z42kR8I68X_4-c=cGPb^V+_^gvN1o0P?kJ1-nA!8&6O;~Zv#Ycn6ckX0-P%7~@J)n8 z6e5A5rt8Sp3=Dl-jw0G1wz`Owkz+A6>9PU= zt>vk;sVVjF9>nR21^qW~xWvQ~Y(fDyH8VGd<3OR(3KDjj$`c2VO=2utC7=iZ`}vr* z+Yib_sG(o+G0N(CU`^z6*oR2}6j z+>ER-_+SZ8dLVyD)G0VTH~=aHjJUI{t-2=K!v?P&a(U?hK{3_#$Bk4!uD3t|sV*r& zbj-ym%=H`AiV4PZXmZdMe>=o8ySwo_KeFx4uPgoqf3ho(uoTu?3nL$9xovwk;<)pR zE1b@~KxOgE7hM5ZdAE8Ct_J=+QCAi6Y^xZ9v@Kh8DS1grNl@yWx-L*Z0)&2iYAUkm zZ2K3e4q?9FbA|l=Vc;SN!Rm;!^70Z878Yh;IBb{_GZ^UxI8jvsM0p?+bpvp>^RBh) z@3SxPb>mJnf6F&-WcfSa~oLbsV3r=P~^x_!JNsTKi%>_fbAlBnS;nM z=J$MmNEB;zf3Q`WB3o769d2TD(B9 z#@HMbd|w&#Z~btZ38z@BF+nCnkPFz`-;d@*(AU=0T>@!sv7s+x{3F%cM%0HzS#k(U znN2TZ3rNE*9Y2vazQBGDL4<^T1iGAnu<<}2hha;P8$?~GrT_KpXiT%S<5KN5SVss2 zg*eEpgXYl|HjLzR6lVulb#--peNvaWebgTw_;3Z9Sgwq56m~74DtJV&#qNYK;2r~hb#VCUw_pT4DU5a zsvOZbU~6nl4vnFB`I3#9nR;7YPmdG>DYz3XfU?%Wf5~|gK|o>zg%SXso12@58?C5t zP?Xiw`YppAC-$$euZ2CKlwlqo9x!x9OSEp=eTZv_fm7gLe|40X}ku7=z>;Nv-baF=rHw_JZha39X z;`gyJGjV9W)mV$`Qm5|+kdnar^UZE-Xcf?zJs4JRZamD)3Nl~#7A>EEgtY3?t%AbD z!C~_GEZ%cmbj=WpIIREe=U&s=x;os_R2=}xmX-tr1W*>WWm#ea5}@$`wl@M`$Gq|5 ztCMv1`Auw)p(+#9DWKAS3>u<%Q@P8QMsi5Z#X?mvj~(@t3IL{p#qGeHfm8^5%OfJ4!zAE(f`52u2tg9m z!9l&RX)hcN1WDDh2A~aXgx7Bs?gJJNDzS4mst-@kK;P>Z{q5>x#Yy7m;9$`D&Pqo| z$Hk=v)F8dDibux6Sr+~Y0W%Ru`lcBI%0YNiroITpah~q2lqdjUu;^5G`2J=$99~{n zNX^Pp`eNF=3ix=nmop03Rz)+D5_H(>_ntjKxJf+@$w_4i1r_mlwu6toI)^41shxu! zF%y6kY;BnWvBcZEb=|pHmo+{<{&=MwVXmguTeIY6kOeC@SM^60c6M@7(qP{%aBm3| z;^!Vh?(g0OS;=;xsDXlpMm=X|XF&5dwF8QJjArvR zZN@mAL|^dz2JP7Z4mK!>lsCZN5@~AMp{THtQ-O5#uUpB|(A^BG;?i~s5bel_x;JlT z;U!qWmXGX@7Ej06!*0NigNh;ud6JZIyxm(Z>jlV}`4rstGH!L`1&WO@=q1)rJ|}ke zB5@3D?C?c5cB812#3Ailt11W-cVa0AwcrNPkAl_)(0p?qdJf30nk=GwPIKRQV#3Er4uE@fq;!BMiu(wd9iBEcy}h0cO3=-uUGeJNS3bo)+bax>ing3TJya$0*pv;h_s)z;4TUCtBdhc4dwGE) z#jgnAKYcpUo0+(8(n}@mMqKnh{{<_kID57hCKO21|Zdpc-_K zc|1nam(b=a*gLZW09mgsQYDWbyauk@v0DCNz z3mh6oS#yL%HKB7`)$5y!43gD)0OAS%=9k!43T>|hy?c8oRhGdRK^8{WPdj@H_D3#t zTwMEbRE-K*6Z8#!->1oVOD>PF(-t$~eESWTKF{iE!^QAa%n_`+RF?6^=$R`~i)ZpG zzNg(sXs}c%p%r-Wi%Q_vdt=-Pe26W;QN4m7-DAOt>58Gk-w@4>t1|^`&`v)i8 z+ahKj;#Uf>t}~6pyONunyROU=^yQoBK`^C&g|$wxMZNJNgOHDYlYNu5dws9Pr~Og+`c}jLxN#4~ zbO*Oq7Fvr!XnsPEpDryfI6bxnlUPi~P^FjZ_RE?kPs7UXGi$0)a^twJf)+*>f15LT zBEuB}tmWjfM_Ye^5QH)zFb5ZQBc$6@x%97+4yj~Ouw)A*ocxYS^C}zabD>k>G`GK4 zUYwO;Nc0zayqS*GP-o*VC&i6s?u8^trdj%BX1lzpNa5#Ny@qASCHb~>=P<3Cin137 zQL!tSsEcrK4bzr$uJr1#!hZgk)9;KJ^qB7|jomVya zpj1gE_N$?F3bn&P^t_A}QrlP7CEBb$*|AR^e~%QcVkJLIYvitz**@=1tKpS|FM~i}9yPyl?9tb+`?AU>g08&2mmnEzIOl>ij_YXD z-X%Ku_lCg21{@XrH;A|hWI}%OtF)(-MWj3{R6=~Ty$D)%k9IGQ$NlyO3I-ztjF_T| zP$k2VqzQ;$#T6zN4%~4qH!RF}IOQKMh0@p-M#;-@G82_^Yb?G?Gi$t{c0eZgz(wIz zPbQJ_xnur$ZUB%kC684hH$q&v1npwtadaK?SaCcdtc(RtJVw(4iJo(1a>Gq5POyAK zl6PThmFyUU{tVJAx`hjp1`+=MJ$(aC=LDTxZ;)vHk7GI{a6S@2_Zv6)i1hIP<2+Ck z3g>Mp5mQ>ie;?oZHvl<~DO3ep-<12`;iCT@NIe#u^|4PfKmLC{GetRqg&q3;4*bXg YlP~Gtk!e-{j!(fT$f!zJOPPlJFT{Ga)&Kwi literal 0 HcmV?d00001 From b51367e6ab292af8273d541fa20f7e9c2fa907f6 Mon Sep 17 00:00:00 2001 From: jsburckhardt Date: Fri, 13 Sep 2024 09:22:46 +0000 Subject: [PATCH 2/2] docs(readme): add notes on model memory handling for Ollama usage Added clarification on the memory mounting behavior of Ollama when running models, including the time required for the initial mount and offloading after inactivity. --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 5ae4521..0d8c072 100644 --- a/README.md +++ b/README.md @@ -137,6 +137,10 @@ There are two common ways to authenticate with AOAI resources. The first one is ### Using LLMs hosted in Ollama Locally in your devcontainer (or any machine) +>[!NOTE] +>When using Ollama, it mounts the model to memory the first time you run it, which can take some time. After 5 minutes of inactivity, Ollama offloads the model. + + Here is an example to run `ollama` in your `devcontainer` and pulling phi3.5 image. ```json