Image to Image

Generate images based on image input

Input image obtained from the Unity camera or external images

Image to image allows you to add new elements for blending for original image

Currently (April 2023), Image to Image also recommends the use of WebUI, as it is possible to use the Inpaint function and other functions that allow partial modifications by using WebUI.

Since stable-diffusion-for-unity runs StableDiffusionWebUI, it is possible to use the WebUI functionality from the local server URL listed in the command prompt started by StartServer from Unity.

local server URL
WebUI Inpaint

For Image2Image on Unity

For generate image with image to Image, attach the Img2Img.cs component to the object

It is assumed that the local server is up and running.

Please refer to the following for StartServer before use.

🖥️About local server

A parameter summary is shown below. Parameters that are adjusted frequently are bolded.

Parameter
Summary

ImageSource

Whether the input element is a prepared image or a camera input in Unity

Image

If Texture is selected in ImageSource, select the image to be used as input Details are listed at the bottom of the table

Prompt

Enter the words you want to generate in natural language

NegativePrompt

Input elements in natural language that you do not want to include in the generated image

Width

Width of the generated image in pixels

Height

Height of the generated image in pixels

Steps

Increase this parameter if the generated image looks blurry

Cfg Scale

Strength to follow prompts Input larger value to strongly follow the prompt.

Denoising Strength

How strongly follow the input image If you want to change the input image only slightly, decrease the value. If you want to change the image significantly, increase the value.

Seed

Fixing the seed value to generate a similar image

In case of -1, the image will be random

Batch Count

The number of images to be generated.

Select 1 to 100 images.

Generated images will be stored in StreamingAssets/StableDiffusion. Please check images by date sort in the explorer.

ExportType

Selectable output types: PNG, JPG, TGA

Apply to Image, RawImage, Material, etc

Sampler

Changing the nuances of the generated image

Model

Base trained model See the "Switch Models/LoRA" to switch new model

Lora

Additional trained Lora to generate consitent specific pattern image

See the "Switch Models/LoRA" to switch new LoRA

If you select Lora, <lora:filename:1>will be added to the Prompt and LoRA will be available

Settings when Texture is selected as input

Check >Read/Write in ImportSettings of the image.

Last updated