I had one vision when I started working with Illustrious - creating a realistic merge that did not ruin the base model's extensive danbooru knowledge. The model is not too realistic as it is now, but I plan to make it at least as real as Quintessence in the subsequent future versions.
This model responds to prompts really well, and in a similar fashion to base Illustrious/NoobAI models. This also implies that you can use artist styles in your prompts for uncertain ways in which the realistic part gets affected.
Now, here's some tips for squeezing the most amount of realism from this checkpoint (these are just my observations, feel free to experiment):
Make sure that you use lower values for your CFG (3-5).
Use DPM++ 2M with the "Beta" scheduler, and do a highres pass at 1.5x at 0.33 denoise and half the original steps.
Remove all instances of artist names and other "painted" styles from your prompts (unless they tend towards realism).
For the positive prompts, include this in the first part (Kinda IMPORTANT):
real, photoreal, DSLR photo, real skin,
For the negative prompts, maybe use this?
cropped, lowres, worst quality, sketch, unfinished, greyscale, faded, low saturation, jpeg artifacts, watermark, text, anime, CGI, 2D
Now, most of you might not call this model very "realistic", but it certainly does have that 3D style that I really love. The end goal is always to achieve more realism, while making sure that the model's base knowledge is not disturbed.
There are several other Illustrious based models that can provide you with more actual "realism", but all of them have to sacrifice some of the model's incredible compositional and conceptual abilities. Some models lose character knowledge, some lose the character clothing knowledge, the dynamic compositions, the multi character abilities and much much more in search of truly realistic outputs.
And that is exactly what I hope to avoid with my merge, keeping almost everything in the model, while bringing the knowledge of the 3D world into it, and slowly increasing the realism.
If you want to support the creation of more models like this, or you just like what you see, try out the buffet. It would really help me out.
Models Merged (Credits to the original authors of these models) -
ReallyIllustrious (base) - TrueToLife_Fauxto
Alchemist Mix (Creative Realism) - Urameshi
coco-Illustrious-NoobAI-XL-Style - COCO_OIOI01
Illustrious_Sky-(rim) 3D - stableskynet171