Sep 15, 2022 20:00:00

Summary of how to use model data 'Waifu-Diffusion' specialized for drawing illustrations with image generation AI 'Stable Diffusion'

' Waifu-Diffusion ' is an image generation AI that tuned the image generation AI '

Stable Diffusion ', which was released to the public in August 2022, with a data set of more than 4.9 million two-dimensional illustrations. In order to run this Waifu-Diffusion in a local Windows environment, I actually downloaded and installed the model data.

GitHub - harubaru/waifu-diffusion: stable diffusion finetuned on danbooru
https://github.com/harubaru/waifu-diffusion

◆Generation example with Waifu Diffusion
I actually set the prompt, sampling steps, sampling method, CFG Scale, and seed value exactly the same, and generated images with Stable Diffusion (left) and Waifu-Diffusion (right). The following images can be compared by moving the center slide bar left and right.

The prompt is 'Hatsune Miku in anime style cooking in the beach, high quality, pixiv', Sampling Steps is 120, Sampling method is Euler a, CFG Scale is 7, Seed value is 793512668. For some reason, the one generated by Stable Diffusion has a strange hand shape, and the structure of the clothes is also collapsing, but Waifu-Diffusion has a fairly decent balance of the face and body, According to the prompt 'cooking in the beach', something like cooking is drawn in front of you.

The prompt is 'girl with dark skin, summer blue sky, sharpness, makoto shinkai, mamoru hosoda, okama, ookami uo, 4k,8k, manga, moe, ponytail, pixiv ranking', Sampling Steps is 20, Sampling method is Euler a, CFG Scale is 7, seed value is 3081179311. The lower half of the face of the girl in the image generated by Stable Diffusion is no longer that of a human, but the one generated by Waifu-Diffusion is based on the profile, but the composition is greatly drawn. And you can see that the face is considerably better.

As you can see from the above example, Waifu-Diffusion is an image generation AI that specializes in 2D illustrations. It will finish the illustration at a level that can be seen properly.

◆ How to introduce Waifu-Diffusion
This time, we have introduced the Waifu-Diffusion model data to the multi-functional 'AUTOMATIC 1111 version Stable Diffusion web UI' that allows you to easily operate Stable Diffusion from the UI. You can understand how to install the AUTOMATIC 1111 version Diffusion web UI by reading the following article.

Image generation AI ``Stable Diffusion'' works even with 4 GB GPU & various functions such as learning your own pattern can be easily operated on Google Colabo or Windows Definitive edition ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' installation method summary - GIGAZINE

Also, the basic usage of the AUTOMATIC1111 version of Stable Diffusion web UI is explained in the following article.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' that can easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion''-GIGAZINE

◆ Download model data
Access the following page to download the Waifu-Diffusion model data.

waifu-diffusion/README.md at main harubaru/waifu-diffusion GitHub
https://github.com/harubaru/waifu-diffusion/blob/main/docs/en/weights/danbooru-7-09-2022/README.md

There are Google Drive, Torrent, HTTPS mirror, and HTTP mirror, but this time I downloaded from the fastest HTTPS mirror. Please note that the file size is about 7.2 GB, which is quite a lot.

Save the downloaded model data 'wd-v1-2-full-ema.ckpt' in the 'models\Stable-diffusion' folder in the AUTOMATIC1111 version Stable Diffusion web UI folder.

◆ Change model data
Next, start the AUTOMATIC 1111 version Stable Diffusion web UI and click the 'Settings' tab at the top.

You can change the usage model by changing the 'Stable Diffusion ceckpoint' in the 'Stable Diffusion' item. The default is models.ckpt (Stable Diffusion v1.4), but change it to 'wd-v1-2-full-ema.ckpt'.

Click 'Apply Settings' at the top of the settings to apply the settings. However, the model changes will be reflected after restarting, so shut down the AUTOMATIC1111 version Stable Diffusion web UI once.

Preparations for introducing Waifu-Diffusion are now OK. Execute 'webui-user.bat' to restart the AUTOMATIC1111 version Stable Diffusion web UI.

Access the URL displayed in the command prompt and display the UI. After that, just like Stable Diffusion, you can generate images with Waifu-Diffusion by entering prompts, setting various elements, and clicking 'Generate'.

According to Mr. Harubaru, the author of Waifu-Diffusion, Waifu-Diffusion is an overseas illustration community site Danbooru , from May 24, 2005 to December 31, 2021 4.9 million images collected Training is being done with a total of 4.9 TB dataset ' Danbooru2021 ' consisting of the above and its metadata.

waifu-diffusion/dataset.md at main harubaru/waifu-diffusion GitHub
https://github.com/harubaru/waifu-diffusion/blob/main/docs/en/training/dataset.md#organizing-the-dataset

The tags used in Danbooru are registered in the metadata included in Danbooru2021, and are associated with tags based on work names, character names, and propensity. In addition, content that suggests sexual content and pornographic content are not prohibited, and it seems that the evaluation value of 'explicit' is included in the metadata.

·postscript
Waifu-Diffusion v1.3, which was a beta version, has been officially released.

Official Release Notes for Waifu Diffusion 1.3 GitHub
https://gist.github.com/harubaru/f727cedacae336d1f7877c4bbe2196e1