Image to Image

Generate images based on image input

Input image obtained from the Unity camera or external images

Image to image allows you to add new elements for blending for original image

Currently (April 2023), Image to Image also recommends the use of WebUI, as it is possible to use the Inpaint function and other functions that allow partial modifications by using WebUI.

Since stable-diffusion-for-unity runs StableDiffusionWebUI, it is possible to use the WebUI functionality from the local server URL listed in the command prompt started by StartServer from Unity.

local server URL

WebUI Inpaint

For Image2Image on Unity

For generate image with image to Image, attach the Img2Img.cs component to the object

It is assumed that the local server is up and running.

Please refer to the following for StartServer before use.

🖥️About local server

A parameter summary is shown below. Parameters that are adjusted frequently are bolded.

Parameter	Summary
ImageSource	Whether the input element is a prepared image or a camera input in Unity
Image	If Texture is selected in ImageSource, select the image to be used as input Details are listed at the bottom of the table
Prompt	Enter the words you want to generate in natural language
NegativePrompt	Input elements in natural language that you do not want to include in the generated image
Width	Width of the generated image in pixels
Height	Height of the generated image in pixels
Steps	Increase this parameter if the generated image looks blurry
Cfg Scale	Strength to follow prompts Input larger value to strongly follow the prompt.
Denoising Strength	How strongly follow the input image If you want to change the input image only slightly, decrease the value. If you want to change the image significantly, increase the value.
Seed	Fixing the seed value to generate a similar image In case of -1, the image will be random
Batch Count	The number of images to be generated. Select 1 to 100 images. Generated images will be stored in StreamingAssets/StableDiffusion. Please check images by date sort in the explorer.
ExportType	Selectable output types: PNG, JPG, TGA Apply to Image, RawImage, Material, etc
Sampler	Changing the nuances of the generated image
Model	Base trained model See the "Switch Models/LoRA" to switch new model
Lora	Additional trained Lora to generate consitent specific pattern image See the "Switch Models/LoRA" to switch new LoRA If you select Lora, <lora:filename:1>will be added to the Prompt and LoRA will be available

Settings when Texture is selected as input

Check >Read/Write in ImportSettings of the image.