AI Toolkit for VS Code simplifies generative AI app development by bringing together cutting-edge AI development tools and models from Azure AI Studio Catalog and other catalogs like Hugging Face. You will be able browse the AI models catalog powered by Azure ML and Hugging Face, download them locally, fine-tune, test and use them in your application.

You can also fine-tune and deploy models to the cloud (preview).

In this section you will learn how to quickly start with AI Toolkit.

AI Toolkit Preview will run locally. Depends on the model you selected, some tasks have Windows and Linux support only, MacOS support is coming soon. For local inference or fine-tune, depends on the model you selected, you may need to have GPU such as NVIDIA GPUs for the preview. If you run remotely the cloud resource needs to have GPU, please make sure to check your environment. For local run on Windows + WSL, WSL Ubuntu distro 18.4 or greater should be installed and is set to default prior to using AI Toolkit. Learn more how to install Windows subsystem for Linux and changing default distribution.

AI Toolkit is shipped as a Visual Studio Code Extension, so you need to install VS Code first, and download AI Toolkit from the VS Marketplace.

Now, you are ready to use the extension! You will be prompted to sign in to GitHub, so please click "Allow" to continue. You will be redirected to GitHub signing page. Please sign in and follow the process steps. After successful completion, you will be redirected to VS Code.

Let's explore the available actions!

Upon launching AI Toolkit from VS Code side bar, you can select from the following options:

• Find a supported model from Model Catalog and download locally
• Test model inference in the Model Playground
• Fine-tune model locally or remotely in Model Fine-tuning
• Deploy fine-tuned models to cloud via command palette for AI Toolkit

To initiate the local fine-tuning session using QLoRA, select a model you want to fine-tune from our catalog.

Note You do not need an Azure Account to download the models

Start by selecting a project name and location. Next, select a model from the model catalog. You will be prompted to download the project template. You can then click "Configure Project" to adjust various settings. We use Olive to run QLoRA fine-tuning on a PyTorch model from our catalog. All of the settings are preset with the default values to optimize to run the fine-tuning process locally with optimized use of memory, but it can be adjusted for your scenario.

After all the parameters are set, click Generate Project. This will:

• Initiate the model download
• Install all prerequisites and dependencies
• Create VS Code workspace

When the model is downloaded, you can launch the project from AI Toolkit.

Note If you want to try preview feature to do inference or fine-tuning remotely, please follow this guide

AI Toolkit offers the collection of publicly available AI models already optimized for Windows. The models are stored in the different locations including Hugging Face, GitHub and others, but you can browse the models and find all of them in one place ready for downloading and using in your Windows application.

Please refer to our Q&A page for most common issues and resolutions

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.