Collage training for Chroma and Illustrious

Consider that you can create AI images from existing training images

That means that the AI model learns patterns from the training image , and uses those different patterns it knows to make new images

If you zoom in on the pixel pattern in any AI image you can find jpeg artifacts within

the AI image , these normally only appear at the edges in normal images

Suffice to say this method works 😀

Practically , if you make a sword in illustrious , you see how the sword sometimes goes out both ends?

The CFG parameter in the AI model is a blend between what you prompt , and 'adjecent pixels in the image'

The patterns which the AI model learns are 'localized' within the training image

So the reason why the collage method works is thanks to unconditional prompting parameter

The prompt you actually feed the AI model isnt purely the text prompt

Ask GROK on Twitter

what is the relationship between cfg conditional prompting and unconditional prompting?

The relation is

 X = CFG  x_conditional + (1-CFG) x_unconditional

The x_unconditional uses the image created thus far as the input argument

So it 'fills in patterns' where it is likely for those patterns to be

But the gist is that this relationship is all about pixel to pixel adjecency

So location in the training image for specific pattern don't matter

The collages is like a difficult math problem you task the AI model to solve , so it can become more adept at solving easier math problems later on

So the AI model will usually never be able to recreate the collage training images 1:1 , but the AI model will become very adept at recreating the patterns within the training image in the attempt

The tool to build collages doesn't matter

The most important thing to know is that the trained pattern will always be relative to the image

So ideally one should have at least one pattern that goes end to end in the image

Example image here

The second thing to know is that AI images learn patterns that have good contrast to background

image

The 'shape layer' as I call it

Benefit of collage training that you can easily crop out bad patterns. Condensing the set to only good patterns.

One more thing; if the background of the collage is #181818 gray it will perfectly match the gray background on Tensor

This creates a cool 3D effect in the gallery 😀

Syntax has examples too in illustrious

I always link this video as a source if you wanna know absolutely all of the theory. Its from the SD1.5 days but still is true for present models https://youtu.be/sFztPP9qPRc?si=B_B353yktSpeKeic

This one has a lot of nonsense overly dense information but the gradient illustration is very cool https://youtu.be/NrO20Jb-hy0?si=6us5FRM7qhmD_auH

Cheers!