GitHub has launched GitHub Models, a free capability aimed at letting developers explore various AI models from within the GitHub tool ecosystem and make it easier to deploy AI-based services using Azure AI. GitHub Models includes both private and public models and is currently in closed preview.
GitHub Models provides a playground that lets developers test prompts and fine-tune model parameters. In order to use a model, you go to the GitHub Marketplace and select the one you want to work with, for example, Meta-Llama-3-70B-Instruct. From the model page, you can get further details about how to get started, as well as inspect rate limits associated with that model. The playground also makes it easy to switch from one model to another and compare their behavior. For example, GitHub describes a scenario where a developer defines their prompts using GPT4o, then switches to a lighter-weight model like Phi-3 mini to see how it performs with those same prompts. Finally, when they are satisfied with the results, the model can be deployed directly to Azure AI by replacing GitHub tokens with Azure credentials.
The GitHub Models playground is also integrated with GitHub Codespaces, allowing developers to spin a new pre-configured Codespace environment directly from the playground once they are satisfied with the results. The integration with Codespaces appears to rely on existing SDKs that are already available for Azure AI with support for several languages. The generated codespace uses the entitlements that come with a GitHub account, which means you do not need to add an API key or sign up for any other services, making it straightforward to start coding with any of the supported AI models.
The new capability is also integrated with the GitHub CLI, allowing users to run a given prompt against one of the supported models and see the results in the terminal. The run command can be combined with other git commands, for example, to summarize the last commits from a repo, you could use:
git log -n 10 | gh models run phi-3-mini-4k-instruct "Summarize this commit history. Use max 1 line per commit"
A few commenters on Hacker News pointed out that while closely related to Azure AI Studio, GitHub Models has the advantage of being free from the constraints deriving from Azure AI Studio enterprise-level security features. These features make the latter cumbersome to use when it comes to authentication and lag behind competitor playgrounds in terms of supported model versions. On the other hand, some criticize that the only paid tier available is through upgrading to Azure AI.
At the time of the announcements, GitHub Models supported Llama 3.1, GPT-4o and GPT-4o mini, Phi 3, Mistral Large 2, and others. However, users cannot upload and run their custom models. As a final note, according to GitHub, no prompts or generated outputs are shared with model providers or used to train or improve the models.