Multi-character Prompting & Inpainting Guide (Step by step)
Foxy Fluffs wants to hug Hatsune Miku! but how to gen two different characters together?We will follow these steps:Engineer an SD model prompt for our charactersRun our prompt and refine until we get a near-matchInpaint our chosen image section by section to correct errorsUpscale the final product.lets get into it!1) Engineer an SD model prompt for our charactersWhen prompting in SD (with either TAMS 2.0 or A1111 parsing method), the best practice I've found for images of multiple characters is to use the following prompt structure:quality tags, general prompts, BREAK, character1 description and lora keywords, BREAK character2 descripton and lora keywordsquality tags (like "best quality", "score_9", or "Newest") will differ depending on the base model or checkpoint used (SD1.5, Pony, Illustrious, etc)general prompts refers to all the prompts that apply to the scene and to both characters (eg. indoors, 2girls, hugging)BREAK tells the model to treat the text following it as a new block of instructions. It differs from a comma which tells the model to apply the following prompt to the previous one (up to 75 tokens)*character descriptions refers to the unique identifiers that will help to depict each character's unique qualities in the scene (eg. "Foxy Fluffs, fox girl, green top," and "Hatsune Miku, human, aqua hair"). Expect bleedthrough no matter how carefully you structure this, however we will choose the image with the least mistakes and edit out any remaining errors with inpainting.*BREAK doesn't function with all parsing methods, however as a best practice I like to keep it in between the unique character descriptions to act as a visual cue of where to place my prompts. I use full stops as well at the end of unique descriptors to further reinforce this (see below).Here's the prompt I engineered for my image of Foxy and Miku with Illustrious model quality tags:Positive Prompt: masterpiece, best quality, amazing quality,very aesthetic,high resolution,ultra-detailed,absurdres,newest,scenery,depth of field,volumetric lighting, 2girls, hugging, looking at another, indoors, BREAK Foxy Fluffs, 1girl, anthro, furry, foxgirl, orange fur, long brown hair, brown eyes, slit pupils, black choker with a silver heart-shaped pendant, green top, black bottoms, blushing, nervous, FoxyFluffs. BREAK hatsune miku, 1girl, human, absurdly long hair, aqua hair, twintails, hair ornament, sidelocks, hair between eyes, parted bangs, aqua eyes, (happy), smiling, white shirt, collared shirt, bare shoulders, sleeveless shirt, aqua necktie, detached sleeves, black sleeves, shoulder tattoo, fringe, black thighhighs, miniskirt, pleated skirt, zettai ryouiki, thigh boots.Negative Prompt: [blank]**I don't use negative prompts most of the time. Maybe it's a thing with images involving fox girls (because there's nothing negative about fox girls!), but I find I get better results without negatives, so i keep the negative blank except if obvious errors creep in that need to be prompted out. YMMV!2) Run our prompt and refine until we get a near-matchI ran the prompt using the following Illustrious Models with square aspect and 15 steps:Nova Anime XL (IL v2.5 Merry Christ)Foxy Fluffs OC Character Lora (0.8 weight)Hatsune Miku -Vocaloid (1.0 weight)After 12 gens, I lucked out and got the following image:Miku seems to have come out almost perfectly (besides her expression), while Foxy's outfit, hair and expression got corrupted a bit.So now that we have a usable image, we will move on to inpainting!3) Inpaint our chosen image section by section to correct errorsWe will inpaint this in stages to keep control of the overall image.For the first inpaint, we will fix Foxy's Hair and Face. We will mask the existing hair AND the area where her long brown hair should be, and we will also mask her face. (we could do these separately, but this gives us a free shot at fixing her face without spending an extra credit)Now we've masked the parts of the hair and face, we adjust the prompt by TEMPORARILY removing the Hatsune Miku prompts and lora to ensure they don't write over Foxy's appearance again (we will need to add these all back in again for the final upscale). Now our positive prompt looks like this:No need to adjust the base settings. We run like this and see what we get! If the hair isn't quite what we want, we can adjust the denoise setting up or down, and we can adjust the mask as well until we get the right result.Result of inpaint No 1:Foxy's hair is fixed, and her face now matches the intended prompt. As an unexpected bonus, the frills on her vest got painted out too! (this was a detail from Miku's costume). Lucky!Now we want to fix the rest of Foxy's costume. Her pendant is missing from her collar, and she prefers to wear shorts and no leggings. So we will mask out the area in front of her choker, as well as the miniskirt and her legs all the way down, like so:We could have probably tried this all in one go with the previous mask just to save credits, but this way gives us a little more control over individual elements of the image. The prompt remains the same (without Miku and her keywords).Result of inpaint No 2:So her shorts are fixed! But the pendant is looking a bit strange. We'll mask the pendant only on this image and try again! (I'll skip the mask image since I think you get the idea now). One thing we WILL do is add greater emphasis in the prompt, so we will add multiple brackets around (((silver heart-shaped pendant))) so the model knows to give it more importance.Result of inpaint No 3:Not perfect, but we will roll the dice on the upscaler fixing it at the end, and move on to Miku!So now we want to give Miku her correct expression, and add back in all of her prompts so that when we upscale in the next step, it will reinforce her attributes and not turn her into another fox girl (which is what will happen if I only leave foxy's prompts in and don't add in Miku's).I also remove the emphasis brackets around Foxy's pendant so that it doesn't wind up on Miku's neck in the final image too. Using the whole original prompt, I go ahead and mask up Miku's face ONLY and then run the inpaint tool one last time.Result of inpaint No 4:So now Miku has this lovely happy cute expression, and foxy has her nervous expression at meeting her idol. Their outfits are pretty much correct too (though foxy's pendant is fading, but TRUST IN THE UPSCALE!).Before we move on to the final step, a confession. I totally forgot to add Miku's lora back in with her prompts 🙈 I don't know how much of a difference that made, but the final image only has my Lora included. Fortunately, Miku is such a popular character, so her data must already exist in the checkpoint I used, otherwise the next step may not have worked...4) Upscale the final productUpscaling can be quite tricky, depending on the image, higher resolution and denoise can introduce a lot of hallucinations. To avoid this, I usually prefer to upscale by 1.5x and a low denoise like 0.1-0.2 to preserve the underlying image.The following Image was Upscaled at 1.5x, 0.2 denoise, and 35 steps (which is excessive, you can still get great results at the limit for free accounts, so don't worry if you don't have Pro!).Foxy meets a celebrity! (Link to original post. If you can't see it, it's still caught up in the NSFW bug, which hopefully will be fixed soon!)I was very happy with this image. There was a tiny bit of bleedthrough from miku's outfit again (note the aqua-coloured outline to foxy's shorts, and the green frills reintroduced on her vest), but overall the image is very sharp, pretty, and Foxy got her pendant in the upscale! (woohoo!)So, that's all there is to it! You can of course try this with more than 2 characters, though you'll need to do a lot more inpainting as more characters tend to introduce a lot more variables into the mix, but this technique should work for any number of subjects as long as you're patient enough to inpaint each of them.I hope this guide is a help to those who read this far! Please do leave your comments below if this was helpful, or if you have suggestions on how I can improve too!Lastly, a quick shoutout to Superpat50 and XMPL on Discord who inspired me to write this article in the first place! Thanks guys!