https://tensor.art/articles/868883505357024765 ( 1 )
https://tensor.art/articles/868883998204559176 ( 2 )
https://tensor.art/articles/868884792773445944 ( 3 )
https://tensor.art/articles/868885754846123117 ( 4 )
Extract the character from the image and place them onto a true white background. You might lose a bit of original coloring or brushstroke texture, but compared to the convenience it brings, that’s a minor issue.
But don’t be too naive—things like expression, pose, clothing, and camera angle still need to be described properly. Doing so helps the AI learn accurate depth of field, which in turn helps it learn the correct body proportions. After that, even if you don’t include camera-related prompts when using the LoRA, it’ll still consistently output correct body shapes.
A lot of people use cutout characters for their training data, but their tags miss things like camera info. So you might get a buff adult under the “upper body” prompt, and a toddler under “full body.”
By now, you should have a solid understanding of how to prepare your training dataset.
5. Parameter Settings
This part is quite abstract and highly variable. There are already many tutorials, articles, and videos online that go into detail about training parameters and their effects. Based on those resources, I arranged several parameters and ran exhaustive tests on them. Since I’m not particularly bright and tend to approach things a bit clumsily, brute-force testing has always been the most effective method for me.
However, given the limits of my personal time and energy, my sample size is still too small to really compare the pros and cons of different parameter sets. That said, one thing is certain: do not use any derivative checkpoints as your base model. Stick with foundational models like Illustrious or Noobai. Using a derived checkpoint will make Lora work only in this one checkpoint.
Another helpful trick for learning from others is that when you deploy LoRA locally, you can directly view the training metadata within SD or WebUI. I’ll also include the main training parameters in the descriptions of any LoRAs I upload in the future for reference.
In the following section, I’ll use the long and drawn-out process of creating the LoRA for Ragun Kyoudai as an example, and give a simple explanation of what I learned through it.
But before we get into that case study, let’s quickly summarize the basic LoRA training workflow:
The real first is always your passion.
2. Prepare your dataset.
3. Tag your dataset thoroughly and accurately.
4. Set your training parameters and begin training.
Wait—and enjoy the surprise you’ll get at the end.
As mentioned earlier, the first LoRA I made for Ragun Kyoudai was somewhat disappointing. The generated images had blurry eyes and distorted bodies. I chalked it up to poor dataset quality—after all, the anatomy and details in the original artwork weren’t realistic to begin with. I thought it was a lost cause. I searched through all kinds of LoRA training tutorials, tips, articles, and videos in hopes of salvaging it. And surprisingly, I stumbled upon something that felt like a breakthrough: it turns out you can train a LoRA using just a single image of the character.
The method is pretty simple. Use one clear image of the character’s face, and then include several images of unrelated full body figures. Use the same trigger words across all of them, describing shared features like hair color, eye color, and so on. Then adjust the repeat values so that the face image and the body images get the same total weight during training. When you use this LoRA, you just need to trigger it with the facial feature tags from the original image, and it swaps in a consistent body from the other images. The resemblance outside the face isn’t great, but it dramatically reduces distortion.
This inspired me—what if the “swapped-in” body could actually come from the original character, especially when working with manga? That way, I could use this method to supplement missing information. I went back through the manga and pulled references of side profiles, back views, full body shots, and various camera angles that weren’t available in the color illustrations. I tagged these grayscale images carefully using tags like greyscale, monochrome, comic, halftone, etc., to make sure the AI learned only the body shape, hairstyle, and other physical features, without picking up unwanted stylistic elements.
This approach did help. But problems still lingered—blurry eyes, malformed hands, and so on. So I pushed the idea further: I used the trained LoRA to generate high-quality character portraits using detail-focused LoRAs, specific checkpoints, and adetailer. These results then became new training data. In parallel, I used other checkpoints to generate bodies alone, adjusting prompt weights like shota:0.8, toned:0.5, to guide the results closer to the target physique or my own expectations. The idea was that the AI could “fit” these new generated samples to the rest of the dataset during training. And it worked. This is how the Lagoon Engine Beta version came to be.
At this point, I could completely ditch the low-resolution color and manga images from the training dataset and just use AI-generated images. I used prompts like simple background + white background to create portrait and upper body images with only the character. To avoid having blurry eyes and inconsistent facial features in full body shots, I used the faceless tag or even manually painted over the heads to prevent the AI from learning them—allowing it to focus solely on body proportions.
That said, white background tends to be too bright and can wash out details, while darker backgrounds can cause excessive contrast or artifacts around the character edges. The most effective backgrounds, in my experience, are grey or pink.
During this time, I also experimented with making a LoRA using just one single character portrait—again from Lagoon Engine. It was just one full color image with a clear, unobstructed view. And when I applied the same method and added new characters to create a LoRA with four characters, I hit a wall. The characters started blending together—something I’d never encountered before. With En & Jin, mixing was incredibly rare and negligible, but with four characters, it became a real problem.I adjusted parameters based on references from other multi-character LoRAs, but nothing worked. I’m still testing—trying to find out if the problem is with parameters, the need for group images, or specific prompt settings. Although the four-character LoRA was a failure, one great takeaway was this: black-and-white manga can be used to make LoRAs. With current AI redrawing tools, you can generate training data using AI itself.
Example：
Compared to LoRAs based on rich animation materials, using black-and-white manga is much more difficult and time-consuming. But since it’s viable, even the most obscure series have a shot at making a comeback.
To summarize, creating multiple LoRAs for the same target is a process of progressive refinement, like crafting a drink from a shot of espresso. The first dataset with detailed tags is your espresso—it can be consumed as-is or mixed however you like. This method also works surprisingly well when creating LoRAs for original characters (OCs). Since OCs can have more complex features, you can start by generating a base image using a fixed seed with just hair/eye color and style. Train a first LoRA on that, then gradually add more features like dyed highlights or complex hairstyles during image generation. If the added features aren’t stable, remove some, and train another LoRA. Repeat this layering process until your OC’s full complexity is captured. This approach is far more stable than trying to generate all features in one go, even with a fixed seed, due to randomness, it’s hard to maintain complex character traits across different angles without breaking consistency.
One more note: regarding character blending when using multiple LoRAs—there seems to be no foolproof way to prevent it. Even adding regularization sets during training doesn’t completely avoid it. As of now, the lowest error rate I’ve seen is when using characters from the same series, trained with the same parameters, by the same author, and ideally in the same training batch.
And with that, we’ve reached the end—for now. I’ll continue to share new insights as I gain more experience. See you next time~
