A package with the logic for implementing Stable Diffusion Text-to-Image generative AI capabilities into your application leveraging OnnxRuntime.
$ dotnet add package StableDiffusion.ML.OnnxRuntimeThis package contains the logic to do inferencing for the popular Stable Diffusion deep learning model in C#. Stable Diffusion models take a text prompt and create an image that represents the text.
Download the ONNX Stable Diffusion models from Hugging Face
Once you have selected a model version repo, click Files and Versions, then select the ONNX branch. If there isn't an ONNX model branch available, use the main branch and convert it to ONNX. See the ONNX conversion tutorial for PyTorch for more information.
Clone the model repo:
git lfs install
git clone https://huggingface.co/CompVis/stable-diffusion-v1-4 -b onnx
Copy the folders with the ONNX files to the C# project folder models. The folders to copy are: unet, vae_decoder, text_encoder, safety_checker.
Install the following NuGets for DirectML
<PackageReference Include="Microsoft.ML" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.OnnxRuntime.DirectML" Version="1.14.1" />
Cuda support coming soon.
Sample logic for implementing in your project
//Default args
var prompt = "a fireplace in an old cabin in the woods";
Console.WriteLine(prompt);
var config = new StableDiffusionConfig
{
// Number of denoising steps
NumInferenceSteps = 15,
// Scale for classifier-free guidance
GuidanceScale = 7.5,
// Set your preferred Execution Provider. Currently DirectML and CPU are supported.
ExecutionProviderTarget = StableDiffusionConfig.ExecutionProvider.DirectML,
// Set GPU Device ID.
DeviceId = 1,
// Update paths to your models
TextEncoderOnnxPath = @".\models\text_encoder\model.onnx",
UnetOnnxPath = @".\models\unet\model.onnx",
VaeDecoderOnnxPath = @".\models\vae_decoder\model.onnx",
SafetyModelPath = @".\models\safety_checker\model.onnx",
};
// Inference Stable Diff
var image = UNet.Inference(prompt, config);
// If image failed or was unsafe it will return null.
if (image == null)
{
Console.WriteLine("Unable to create image, please try again.");
}
Set Build for x64
Hit F5 to run the project in Visual Studio or dotnet run in the terminal to run the project in VS Code.