Skip to content

[UniPCMultistepScheduler] Fix final-step NaN/inf at sigma=0 and rks dtype mismatch - #15079

Open
deep1blue wants to merge 1 commit into
huggingface:mainfrom
deep1blue:fix/unipc-zero-final-sigma-and-rks-dtype
Open

deep1blue wants to merge 1 commit into
huggingface:mainfrom
deep1blue:fix/unipc-zero-final-sigma-and-rks-dtype

Conversation

@deep1blue

@deep1blue deep1blue commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Fixes #14887
Fixes #14888

Root Cause & Fixes

  1. Final-step NaN / inf / _LinAlgError when final_sigmas_type="zero" (UniPCMultistepScheduler returns nan with solver_type="bh1", and inf or nan with lower_order_final=False or predict_x0=False, on the final step to sigma = 0 #14887):
    Since final_sigmas_type defaults to "zero", the final step reaches sigma_t = 0, making lambda_t = inf and h = lambda_t - lambda_s0 = inf:

    • solver_type="bh1": On a 1st-order final step (D1s is None), multistep_uni_p_bh_update set pred_res = 0 and evaluated x_t_ - alpha_t * B_h * pred_res. For bh1, B_h = hh = -inf, so -inf * 0 = nan corrupted the output even though no multistep residual exists when D1s is None. Guard the B_h * pred_res subtraction behind if D1s is not None: and set x_t = x_t_ otherwise.
    • lower_order_final=False: Attempting an order >= 2 predictor update into sigma_t = 0 gives rk = (lambda_si - lambda_s0) / inf = 0 and D1s = (mi - m0) / 0 = inf (and a singular R matrix for order >= 3). Force this_order = 1 on the last step when self.config.final_sigmas_type == "zero", matching DPMSolverMultistepScheduler.
    • predict_x0=False: With sigma_t = 0 and hh = +inf, the noise-prediction branch multiplies 0 * inf = nan. Raise a ValueError in __init__ when not predict_x0 and final_sigmas_type == "zero", matching DPMSolverMultistepScheduler.
  2. torch.linalg.solve dtype mismatch under torch.set_default_dtype(torch.float64) (UniPCMultistepScheduler fails in torch.linalg.solve under a float64 default dtype: the unit entry of rks takes the default dtype #14888):
    In both multistep_uni_p_bh_update and multistep_uni_c_bh_update, the first order - 1 entries of rks inherit h.dtype (float32), whereas rks.append(torch.ones((), device=device)) used the global default dtype. Under torch.set_default_dtype(torch.float64), torch.stack(rks) promoted R to float64 while b remained float32, raising RuntimeError: linalg.solve: Expected A and B to have the same dtype. Replace torch.ones((), device=device) with torch.ones_like(h).

Validation

  • Added test_last_step_to_sigma_zero_is_the_data_prediction, test_predict_x0_false_requires_final_sigmas_type_sigma_min, and test_default_dtype_float64 to tests/schedulers/test_scheduler_unipc.py.
  • Verified all 68 tests in tests/schedulers/test_scheduler_unipc.py pass.

Credits & Acknowledgments

…type mismatch

Fixes huggingface#14887
Fixes huggingface#14888

1. On the final denoising step under `final_sigmas_type="zero"` (the default), `sigma_t = 0` and `h = lambda_t - lambda_s0 = inf`:
   - In `multistep_uni_p_bh_update`, when `D1s is None` (first-order update), `pred_res = 0` was multiplied by `B_h = hh = -inf` under `solver_type="bh1"`, producing `-inf * 0 = nan`. Only subtract the `B_h * pred_res` residual term when `D1s is not None`.
   - In `step`, force `this_order = 1` when `self.config.final_sigmas_type == "zero"` on the final step (`self.step_index == len(self.timesteps) - 1`), matching `DPMSolverMultistepScheduler` so `lower_order_final=False` does not divide by `rk = (lambda_si - lambda_s0) / inf = 0` or build a singular `R` matrix.
   - In `__init__`, raise a `ValueError` when `not predict_x0 and final_sigmas_type == "zero"`, matching `DPMSolverMultistepScheduler`.
2. In `multistep_uni_p_bh_update` and `multistep_uni_c_bh_update`, replace `rks.append(torch.ones((), device=device))` with `rks.append(torch.ones_like(h))` so `R` preserves `h.dtype` instead of promoting to `float64` when `torch.set_default_dtype(torch.float64)` is active.

Signed-off-by: Parth Vadera <parthvadera@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant