Samplers, Schedulers, and Sigmas, Oh My! Cutting Through Denoise
Samplers, Schedulers, and Sigmas, Oh My!Duplicated from my original article here: https://civitai.com/articles/9191NEW! Companion workflow with some advanced techniques and more implementation examples: https://civitai.com/models/987817Samplers and schedulers are an intrinsic part of AI image generation, but confusing vocabulary and lack of information can shroud these concepts in mystery. The reality is actually quite boring - they dictate the formula used for the 'noise' added to and taken from images during training and image creation, and they specify and sometimes guide the strength of image creation in each step, typically by dividing the steps evenly along a xy curve (see above graphic depicting the curves of some popular schedulers).Sigmas: The Step WeightsSigmas are simply a list of numbers that control how strong each denoising step is:Example: [17, 15.2, 12.0, 10.3, 7.3, 3.0, 2.1, 3.2, 0, 0]Some schedulers add redundant zeros at the end, others don'tHigher numbers = stronger denoising (image generation) stepsSchedulers: The Step ManagersSchedulers are rule sets that:Generate sigma weights for your requested stepsOnly need to know the model's proper range (starting number)Create steps using specific curve formulasIn ComfyUI, this is all that "samplers" actually doMore advanced schedulers can:Evaluate predicted noise shiftsAdjust weights based on actual vs. predicted noise shiftsPerform other optimizationsSamplers: The Noise ControllersThese define the mathematical patterns for:Adding noise during trainingDriving the AI's noise prediction algorithms during image creationThe Denoising Parameter: Just the final % of steps1.0 for txt2img0.01-0.8 for img2imgSpecifies where in the sigma schedule to start/stopExample: 0.2 denoise = using only the final 20% of sigma stepsHow Steps and Denoise Interact in Most ComfyUI NodesWhen you set:0.2 denoise at 10 stepsSystem calculates 10/0.2 = 50 total stepsGets all 50 steps from schedulerUses only the final 10 steps (last 20%)Alternative approach:Some nodes allow you to specify total steps and start/end points directly50 total steps, starting at step 40 = identical to 0.2 denoise/10 stepsMore transparent for what's actually happeningEasier to automateComfyUI's 'split sigmas' nodes can divide steps between different operations(see example images below)Key TakeawaysUnderstanding that denoise is simply the percentage of steps where you start during img2img, and that sigmas are just a list of step strengths (high to low), you can more confidently use different interfaces and also set up more complex workflows. This knowledge enables you to 'split sigmas' between different schedulers, changing settings, prompt or lora weights, upscaling, or more.A final note to leave with - not all platforms implement schedulers the same way, and you can get more out of them if you use software that supports a more advanced scheduler. The diffusers back end implements flowmatch schedulers that are capable of adjusting sigma weights during generation for better results (and in fewer steps). That being said, you can still get similar quality with simpler schedulers, it will usually just take more steps and more tweaking.Here's a link to some good information on schedulers: https://huggingface.co/docs/diffusers/api/schedulers/overviewEXAMPLES:Link to example workflow showing many examples and techniques: https://civitai.com/models/987817Additional wiring examples for reference (these do no suggest specific settings, only how to wire split sigmas node):
