Model Training - Illustrious NoobAI LoRA Discussion
Let's talk about Illustrious and NoobAI LoRA'sPrefaceI am currently using tensor.art with Professional Mode to train my Lora, this article will mainly discuss what I've tried and I welcome others to discuss too as there's no official finetune guide.GuidelinesHigher rates = stronger character features but potential loss in image qualityLower rates = better image quality but weaker character featuresMost character Loras work well with UNET around 0.0003 and TE around 0.00003Lower learning rates will adapt the features better but can also take longer. As for the dataset lets say i have 40 images , 5-10 repeats, 10 epochs, 4 batch size, this usually adds up to the total steps and then hopefully a model is trained well enoughThe ideal ratio is typically UNET:TE = 10:1UNET Rates (0.0005 - 0.0001):0.0005: Very strong influence, can overpower the base model. Good for exact character matching but may reduce image quality0.0003: Balanced influence, commonly used for character Loras0.0001: Subtle influence, maintains high image quality but character features may be less pronouncedText Encoder (TE) Rates (0.00005 - 0.00001):0.00005: Strong text conditioning, helps with character recognition0.00003: Moderate text influence, good balance for most character Loras0.00001: Light text conditioning, useful when you want minimal style transferDimension Ranks (DR) - Network Dim32: Standard/Default rank, good balance of detail and file size64: Higher detail capture, larger file size128: Very high detail, much larger file size256: Maximum detail, extremely large file sizeNetwork Alpha (AR) - Network AlphaAlpha is typically set to match or be slightly lower & higher than the rank.Common ratios:AR may be half the rank or even a quarter less than the DRAR: Standard training stability (1:1 ratio), same as the DRAR× 1.5: Increased stability, a quarter more than the DRAR× 2: Maximum stability, double the DRThe values below are not 100% but they are being figured out still.Basic Character Lora (Base Model's preference)DR 64, AR 32
- Best for: Simple anime/cartoon characters
- File size: ~70MB
- Good balance of detail and stabilityComplex Character LoraDR 64-48, AR 32-24
- Best for: Most character types
- File size: ~100MB
- Excellent for anime/game charactersStyle Loraexample : https://tensor.art/models/806682226684073145/NAI3-Kawaii-Style-Illustrious-NoobAI-nai-IL-V0.1example : https://tensor.art/models/806356844256811271/Anima-Crayon-Sketch-Illustrious-IL-V0.1original article says :
DR 128, AR 64 to 32 - seems to be the best for a combination of complex features etc
if the style is very detailed. otherwise lower ranks work too.Learning rates can vary:
CAME and RAWR = 0.0002 UNET and 0.00002 TE will need about 2500 to 3000 steps
ADAMW8BIT & ADAFACTOR between 0.0003-0.0005 UNET and 0.00003-0.00005 at 1000 steps
but what i use instead :Parameter Settings
Network Module
LoRA
Use Base Model
rMix NNNoobAI - V1.1
Trigger words
nai3_kawaii
Image Processing Parameters
Repeat
10
Epoch
10
Save Every N Epochs
1
Training Parameters
Seed
-
Clip Skip
-
Text Encoder learning rate
0.00004
Unet learning rate
0.00035
LR Scheduler
cosine_with_restarts
Optimizer
AdamW8bit
Network Dim
32
Network Alpha
16
Gradient Accumulation Steps
-
Label Parameters
Shuffle caption
true
Keep n tokens
1
Advanced Parameters
Noise offset
0.0357
Multires noise discount
0.15
Multires noise iterations
8
conv_dim
-
conv_alpha
-
Batch Size
2
Sample Image Settings
Prompt
nai3_kawaii 1girl solo long hair looking at viewer blush bangs blue eyes hair ornament dress ribbon sitting closed mouth pink hair sleeveless hairclip sailor collar two side up book blue dress sailor dress . masterpiece, best quality, amazing quality, very aesthetic, absurdres
Sampler
eulerWhat works?I'd like to hear what works and doesn't work for illustrious:OptimizerLearning Rates could change dependent on the optimizer chosen.SchedulerNetwork Settings(DR) Dimension rank 128, 96, 64, 32, 16, 4(AR) Alpha rank 128, 96, 64, 32, 16, 4Don't use:ProdigyCan use:AdamW8BitConstant0.0003 LR (TE & UNET) - Aggressive Learning for characters0.0002 LR - Medium learning for characters (DR 128 AR 64)AdaFactorSchedulerCosine with restart0.0005-0.0003 LR (UNET)0.00005-0.00003 LR (TE)DR 128-32, AR 64-16 - usually i go half the Network Dimension Rankplagiarized and inspired from : https://civitai.com/articles/9148/illustrious-lora-training-discussionmodel used for my training : rMix NNNoobAI v1.1 - https://tensor.art/models/805164110363975687
