Tensor.Art
Create

Model Training - Illustrious NoobAI LoRA Discussion

Updated:

Let's talk about Illustrious and NoobAI LoRA's

Preface

I am currently using tensor.art with Professional Mode to train my Lora, this article will mainly discuss what I've tried and I welcome others to discuss too as there's no official finetune guide.

Guidelines

  • Higher rates = stronger character features but potential loss in image quality

  • Lower rates = better image quality but weaker character features

  • Most character Loras work well with UNET around 0.0003 and TE around 0.00003

  • Lower learning rates will adapt the features better but can also take longer. As for the dataset lets say i have 40 images , 5-10 repeats, 10 epochs, 4 batch size, this usually adds up to the total steps and then hopefully a model is trained well enough

The ideal ratio is typically UNET:TE = 10:1

  • UNET Rates (0.0005 - 0.0001):

    • 0.0005: Very strong influence, can overpower the base model. Good for exact character matching but may reduce image quality

    • 0.0003: Balanced influence, commonly used for character Loras

    • 0.0001: Subtle influence, maintains high image quality but character features may be less pronounced

  • Text Encoder (TE) Rates (0.00005 - 0.00001):

    • 0.00005: Strong text conditioning, helps with character recognition

    • 0.00003: Moderate text influence, good balance for most character Loras

    • 0.00001: Light text conditioning, useful when you want minimal style transfer

Dimension Ranks (DR) - Network Dim

  • 32: Standard/Default rank, good balance of detail and file size

  • 64: Higher detail capture, larger file size

  • 128: Very high detail, much larger file size

  • 256: Maximum detail, extremely large file size

Network Alpha (AR) - Network Alpha

Alpha is typically set to match or be slightly lower & higher than the rank.

Common ratios:

  • AR may be half the rank or even a quarter less than the DR

  • AR: Standard training stability (1:1 ratio), same as the DR

  • AR× 1.5: Increased stability, a quarter more than the DR

  • AR× 2: Maximum stability, double the DR

The values below are not 100% but they are being figured out still.

Basic Character Lora (Base Model's preference)

DR 64, AR 32
- Best for: Simple anime/cartoon characters
- File size: ~70MB
- Good balance of detail and stability

Complex Character Lora

DR 64-48, AR 32-24
- Best for: Most character types
- File size: ~100MB
- Excellent for anime/game characters

Style Lora

example : https://tensor.art/models/806682226684073145/NAI3-Kawaii-Style-Illustrious-NoobAI-nai-IL-V0.1

nai3_kawaii, 1girl, arona \(blue archive\), solo, bubble blowing, pink hair, closed eyes, chewing gum, halo, blue halo, multicolored hair, white choker, choker, colored inner hair, blue hair, upper body, white hairband, cloud, blue shirt, hairband, sky, blurry, from side, bow, short hair, outdoors, sailor collar, braid, blush, shirt, white sailor collar, blue sky, school uniform, bowtie, bubble, white bowtie, white bow, day, profile, serafuku, depth of field, two-tone hair, blurry foreground, side braid, white ribbon, single braid, leaf, blurry background, bow hairband, aqua hair, ribbon, hair ribbon , masterpiece, best quality, absurdres, highres

example : https://tensor.art/models/806356844256811271/Anima-Crayon-Sketch-Illustrious-IL-V0.1

crayon_sketch 1girl solo looking at viewer short hair blue eyes brown hair green eyes parted lips lips portrait smile
original article says :

DR 128, AR 64 to 32 - seems to be the best for a combination of complex features etc 
if the style is very detailed. otherwise lower ranks work too.Learning rates can vary:
CAME and RAWR = 0.0002 UNET and 0.00002 TE will need about 2500 to 3000 steps
ADAMW8BIT & ADAFACTOR between 0.0003-0.0005 UNET and 0.00003-0.00005 at 1000 steps

but what i use instead :

Parameter Settings

Network Module
LoRA
Use Base Model
rMix NNNoobAI - V1.1
Trigger words
nai3_kawaii
Image Processing Parameters

Repeat
10
Epoch
10
Save Every N Epochs
1
Training Parameters

Seed
-
Clip Skip
-
Text Encoder learning rate
0.00004
Unet learning rate
0.00035
LR Scheduler
cosine_with_restarts
Optimizer
AdamW8bit
Network Dim
32
Network Alpha
16
Gradient Accumulation Steps
-
Label Parameters

Shuffle caption
true
Keep n tokens
1
Advanced Parameters

Noise offset
0.0357
Multires noise discount
0.15
Multires noise iterations
8
conv_dim
-
conv_alpha
-
Batch Size
2
Sample Image Settings

Prompt
nai3_kawaii 1girl solo long hair looking at viewer blush bangs blue eyes hair ornament dress ribbon sitting closed mouth pink hair sleeveless hairclip sailor collar two side up book blue dress sailor dress . masterpiece, best quality, amazing quality, very aesthetic, absurdres
Sampler
euler

What works?

I'd like to hear what works and doesn't work for illustrious:

  • Optimizer

    • Learning Rates could change dependent on the optimizer chosen.

    • Scheduler

  • Network Settings

    • (DR) Dimension rank 128, 96, 64, 32, 16, 4

    • (AR) Alpha rank 128, 96, 64, 32, 16, 4

Don't use:

  • Prodigy

Can use:

  • AdamW8Bit

    • Constant

      • 0.0003 LR (TE & UNET) - Aggressive Learning for characters

      • 0.0002 LR - Medium learning for characters (DR 128 AR 64)

  • AdaFactor

    • Scheduler

      • Cosine with restart

        • 0.0005-0.0003 LR (UNET)

        • 0.00005-0.00003 LR (TE)

        • DR 128-32, AR 64-16 - usually i go half the Network Dimension Rank

plagiarized and inspired from : https://civitai.com/articles/9148/illustrious-lora-training-discussion
model used for my training : rMix NNNoobAI v1.1 - https://tensor.art/models/805164110363975687

8
0