Image generation AI 'Stable Diffusion' AUTOMATIC 1111 version of `` Aesthetic Gradients '' that can learn and apply the style and pattern of illustrations Summary of how to use



The image generation AI '

Stable Diffusion ' can be used easily on the browser.The most energetic development has been done among web UI, and it can be said that it is already the definitive GUI of Stable Diffusion ' AUTOMATIC 1111 version Stable Diffusion webUI ' AI image generation. The `` Aesthetic Gradients '' function has been added, which allows you to give a specific orientation, so I actually tried using it.

[2209.12330] Personalizing Text-to-Image Generation via Aesthetic Gradients
https://arxiv.org/abs/2209.12330

Extensions · AUTOMATIC1111/stable-diffusion-webui Wiki · GitHub
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Extensions#aesthetic-gradients

GitHub - AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients: Aesthetic gradients extension for web ui
https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients

The following article summarizes how to install the AUTOMATIC 1111 version of Stable Diffusion webUI.

Image generation AI ``Stable Diffusion'' works even with 4 GB GPU & various functions such as learning your own pattern can be easily operated on Google Colabo or Windows Definitive edition ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' installation method summary - GIGAZINE



Also, you can understand the basic usage of AUTOMATIC 1111 version Stable Diffusion webUI by reading the following article.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' that can easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion''-GIGAZINE



Aesthetic Gradients is a function that adds orientation with prepared image data to the work called 'Embedding' that converts the sentences input with Stable Diffusion into vector representation and orients the image generation. By performing Aesthetic Gradients with the data composed of the picture you drew, it is possible to bring the pattern of the generated image closer to your picture.

First, access the AUTOMATIC1111 version Stable Diffusion webUI directory with Git and execute the following code.
[code]git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients extensions/aesthetic-gradients[/code]



Running the code will clone the 'aesthetic-gradients' repository into the 'stable diffusion webui/extensions/' folder. Aesthetic Gradients are not implemented in the AUTOMATIC 1111 version of Stable Diffusion webUI itself, but added as extensions.



Embedding data used for Aesthetic Gradients is required to orient image generation. This embedding data is published on the following GitHub page by Víctor Gallego, the author of the Aesthetic Gradients paper.

stable-diffusion-aesthetic-gradients/aesthetic_embeddings at main vicgalle/stable-diffusion-aesthetic-gradients GitHub
https://github.com/vicgalle/stable-diffusion-aesthetic-gradients/tree/main/aesthetic_embeddings

Access the above site and click the embedding data you want to download.



Click 'Download'.



Save the embedding data in the 'stable diffusion webui/extensions/aesthetic-gradient/aesthetic_embeddings' folder. This time, I saved aivazovsky.pt, flower_plant.pt, laion_7plus.pt, and sac_8plus.pt.



Start AUTOMATIC1111 version Stable Diffusion webUI. There is a part called 'Open for Clip Aesthetic!' at the bottom of the setting panel, so click it.



This will expand the Aesthetic Gradients settings panel.



There are mainly four setting items for Aesthetic Gradients.

・Aesthetic Weight : Weight per step of Aesthetic Gradients. The higher the value, the stronger the embedding influence.
・Aesthetic Steps : Number of steps of Aesthetic Gradients. The more, the stronger the Embedding effect.
・Aesthetic learning rate : Learning rate of Aesthetic Gradients. According to Gallego, the default value of 0.0001 works well, and in most cases there is no need to change it.
・Aesthetic img embedding : Select data for embedding.



I actually checked how much the generated image will be affected by using Aesthetic Gradients. Below is 'Portrait of a full body of beautiful young female detective, d & d, sleeveless turtleneck, pleated skirt, fantasy, flat lighting, intricate, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, misa amane, art by simon bisley and greg rutkowski and alphonse mucha, natural tpose', set the resolution to 512x512 pixels, the number of generation steps to 73, the generation sampler to Euler a, the CFG scale to 7, and the seed value to 8923. This is an image generated by



Aesthetic Weight is 0.9, Aesthetic Steps is 3, Embedding data is aivazovsky.pt, and Aesthetic Gradients is executed with the settings at the time of generation as they are. aivazovsky.pt consists of works by

Ivan Aivazovsky, a painter who painted many landscape paintings with the sea as a motif, and if you look closely, something like a model of a ship is born on the left hand of a woman.



flower_plant.pt is data composed of more than 2000 images of plants and flowers. A meadow appears in the background of the woman, with wildflowers and trees.



laion_7plus.pt uses the embedding data used in

LAION-Aesthetics_Predictor V1 . For some reason the woman's hairstyle is swelling like a mushroom, but the clothes and background are more detailed.



sac_8plus.pt is based on the embedding data used in

Simulacra Aesthetic Captions . The pattern has changed completely, and the pattern has become quite like Japanese manga and anime.



Below is the prompt 'empty sidewalk of a cyberpunk megacity, dramatic lighting, detailed background, gorgeous view, realistic, high detail, depth of field, lightrays, atmospheric, digital art', the number of generation steps is 80, and the generation sampler is Euler a , generated with a CFG scale of 7, a seed value of 20221031, and a batch size of 4. I don't use Aesthetic Gradients. The background that seems to appear in science fiction movies and the people walking in it are drawn.



With the same settings, Aesthetic Weight is 0.9, Aesthetic Steps is 15, Embedding data is aivazovsky.pt, and Aesthetic Gradients looks like this. Perhaps because I made the Aesthetic Steps quite high, I was greatly influenced by Aivazovsky's work and changed to a picture depicting the sea. One of the four pieces is in the frame, so it looks like a painting that is completely displayed in an art museum.



With flower_plant.pt, plants are drawn in the picture, but the building is drawn in the foreground more than the original image, so it looks civilized.



With laion_7plus.pt, not only are buildings and plants added, but the style of painting has changed greatly. It differs greatly from the original image generated by specifying it as a cyberpunk style, and a somewhat fantasy-like atmosphere is also created.



On the other hand, sac_8plus.pt is a generated image that is very close to the original generated image, but the background is a fairly civilized and futuristic cityscape instead of a devastated land.



In addition, you can create the data for embedding with an image prepared by yourself. Click the 'Create aesthetic embedding' tab, enter the name of the data in 'Name', enter the directory containing the image in Source Directory, and click 'Create Images embedding'.



However, according to the following explanatory video by Mr. koiboi , an Australian engineer, it is necessary to prepare thousands of images for embedding data to obtain a sufficient effect.

How to use Aesthetic Gradients: Stable Diffusion Tutorial - YouTube

Related Posts:

in Review,   Software,   Web Service,   Web Application,   Video,   Art, Posted by log1i_yk