[Bug] Batched CFG produces white images because GROUP_NORM leaves batch items after the first unwritten #2149

Git commit

8c442fb

release master-969-8c442fb (commit).

I’m using the official prebuilt macOS release, not compiling stable-diffusion.cpp myself: sd-master-8c442fb-bin-Darwin-macOS-26.6.2-arm64.zip. Only the standalone C reproduction was compiled locally and linked against that release’s dylib.

Operating System & Version

macOS 27.0.1, arm64 - Apple M5 Max, 128 GiB

GGML backends

Metal

Command-line arguments used

./group_norm

Steps to reproduce

1. On macOS with Metal, download and extract the official master-969-8c442fb library:

   curl -fL -o release.zip https://github.com/leejet/stable-diffusion.cpp/releases/download/master-969-8c442fb/sd-master-8c442fb-bin-Darwin-macOS-26.6.2-arm64.zip
   unzip -j release.zip '*.dylib' -d lib

2. Obtain the matching GGML headers:

   git clone https://github.com/leejet/ggml.git
   git -C ggml checkout d25b121ce1d6ae0c7e62ad080460d9fc1e46dd93

3. Save the complete model-free C reproduction included in this report as group_norm.c.

4. Compile and run it against the official dylib:

   clang -O2 -I ggml/include group_norm.c -L "$PWD/lib" \
     -lstable-diffusion -Wl,-rpath,"$PWD/lib" -o group_norm
   ./group_norm

5. Observe that CPU correctly computes both batch items, and Metal correctly computes batch item zero. However, Metal leaves all 4,096 output values in batch item one unchanged at the prefilled sentinel value 123456:

   GROUP_NORM backend=Metal batch=2 item=1 unchanged=4096/4096 max_error=123458
   

What you expected to happen

Metal group normalization should compute and write normalized output for every batch item, matching the CPU reference within floating-point tolerance. For a [8,8,64,2] tensor, both items should be processed; the second item must not retain the prefilled sentinel values.

Consequently, enabling batched CFG should produce valid generated images rather than all-white output caused by an unwritten CFG branch.

What actually happened

Metal group normalization processed only the first batch item. For a [8,8,64,2] tensor, all 4,096 output values in the second item remained unchanged at the prefilled sentinel value 123456. CPU correctly processed both items.

Actual reproduction output:
GROUP_NORM backend=Metal batch=2 item=1 unchanged=4096/4096 max_error=123458

In SD 1.5 img2img generation, enabling batched CFG produced all-white output. Disabling batched CFG avoided the failure. The isolated normalization defect reproduces against the official master-969-8c442fb macOS dylib without Go bindings or a custom library build.

Logs / error messages / stack trace

Model-free GROUP_NORM reproduction against the official master-969-8c442fb macOS dylib:

GROUP_NORM backend=CPU batch=1 item=0 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=CPU batch=2 item=0 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=CPU batch=2 item=1 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=Metal batch=1 item=0 unchanged=0/4096 max_error=1.01877e-07
GROUP_NORM backend=Metal batch=2 item=0 unchanged=0/4096 max_error=1.01877e-07
GROUP_NORM backend=Metal batch=2 item=1 unchanged=4096/4096 max_error=123458

Native SD 1.5 img2img reproduction against the same release:

RESULT batch=1 size=64 steps=4 call=0 range=255..255 mean=255.0000 hash=fe4ec469d4f39325

No crash or stack trace occurs in these reproductions. The operation reports success but leaves the second batch item unwritten. The image-generation call also succeeds, but every returned pixel byte is 255 (all white).

Additional context / environment details

High-Level Summary

Tested on Apple M5 Max with 128 GiB unified memory, macOS 27.0.1, arm64.

I used the official prebuilt sd-master-8c442fb-bin-Darwin-macOS-26.6.2-arm64.zip from release master-969-8c442fb, not a custom stable-diffusion.cpp build. The standalone reproduction was compiled locally and linked directly against that release’s dylib, using headers from its pinned GGML revision, d25b121ce1d6ae0c7e62ad080460d9fc1e46dd93. No Go bindings are involved in the minimal reproduction.

The same isolated defect reproduces on release master-968-8cd731e. On 968, batched-CFG image generation failed at 64×64, 128×128, 256×256, and 512×512; disabling batching avoided the failure in those controls. A fresh context fails on its first generation, so repeated calls are not required.

Disabling conditioning caching, explicitly disabling flash attention and direct convolution, or setting GGML_METAL_TENSOR_DISABLE=1 did not resolve the 968 reproduction.

As a diagnostic on 968, supplying only the missing group-normalization outputs for batch items after the first through the backend evaluation callback removed the traced NaNs and restored nonconstant images. No other operations were replaced.

Our current application workaround is to disable batched CFG on Metal. Other Apple GPU generations have not been tested.

Summary

The official macOS binaries for master-969-8c442fb and
master-968-8cd731e produce all-white output in the SD 1.5 img2img workload
below with batched CFG enabled. Disabling batched CFG avoids the failure.

An isolated, model-free GGML reproduction identifies the underlying problem:
Metal GROUP_NORM writes only batch item zero. For a [8,8,64,2] tensor,
all 4,096 output floats in item one retain their prefilled sentinel value.
CPU computes both items correctly. This reproduces directly against the
official stable-diffusion dylib, without Go bindings or a custom build.

Environment

• Apple M5 Max, 128 GiB RAM, macOS 27.0.1, arm64, Metal.
• Release: master-969-8c442fb, commit
  8c442fb.
• Official asset: sd-master-8c442fb-bin-Darwin-macOS-26.6.2-arm64.zip.
• Archive SHA-256: 129b62de61772f368fa9f9ee8130d62a4dadd6ad99e7243b9734cff97f539191.
• Extracted dylib SHA-256: b8d76b876ac598881b1fbeeb1ae5a63893176ee870b5f971e5f474c3723f9783.
• GGML submodule: d25b121.

Minimal reproduction (no model required)

Save this as group_norm.c. It tests both backends with batch sizes one
and two. Expected values are calculated independently from the input,
group means, and variances. The second batch item has an asymmetric offset.

#include "ggml.h"
#include "ggml-backend.h"
#include "ggml-alloc.h"
#include "ggml-cpu.h"
#include "ggml-metal.h"
#include <math.h>
#include <stdio.h>
#include <stdlib.h>

int main(void) {
    for (int metal = 0; metal <= 1; metal++) {
        for (int batch = 1; batch <= 2; batch++) {
            ggml_backend_t backend = metal ? ggml_backend_metal_init() : ggml_backend_cpu_init();
            if (!backend) return 2;
            struct ggml_context *ctx = ggml_init((struct ggml_init_params){16*1024*1024, NULL, true});
            struct ggml_tensor *x = ggml_new_tensor_4d(ctx, GGML_TYPE_F32, 8, 8, 64, batch);
            struct ggml_tensor *out = ggml_group_norm(ctx, x, 32, 1e-5f);
            struct ggml_cgraph *graph = ggml_new_graph(ctx);
            ggml_build_forward_expand(graph, out);
            ggml_backend_buffer_t buf = ggml_backend_alloc_ctx_tensors(ctx, backend);
            if (!buf) return 3;
            int n = 4096 * batch;
            float *input = malloc(n*sizeof(float)), *output = malloc(n*sizeof(float));
            for (int i = 0; i < n; i++) {
                input[i] = (i%29)*0.125f + (i/4096)*7.0f;
                output[i] = 123456.0f;
            }
            ggml_backend_tensor_set(x, input, 0, n*sizeof(float));
            ggml_backend_tensor_set(out, output, 0, n*sizeof(float));
            if (ggml_backend_graph_compute(backend, graph) != GGML_STATUS_SUCCESS) return 4;
            ggml_backend_tensor_get(out, output, 0, n*sizeof(float));
            for (int b = 0; b < batch; b++) {
                int unchanged = 0;
                double max_error = 0;
                for (int g = 0; g < 32; g++) {
                    int start = b*4096 + g*128;
                    double mean = 0, var = 0;
                    for (int i = 0; i < 128; i++) mean += input[start+i];
                    mean /= 128;
                    for (int i = 0; i < 128; i++) var += (input[start+i]-mean)*(input[start+i]-mean);
                    var /= 128;
                    for (int i = 0; i < 128; i++) {
                        double expected = (input[start+i]-mean)/sqrt(var+1e-5);
                        double error = fabs(output[start+i]-expected);
                        if (error > max_error) max_error = error;
                        if (output[start+i] == 123456.0f) unchanged++;
                    }
                }
                printf("GROUP_NORM backend=%s batch=%d item=%d unchanged=%d/4096 max_error=%g\n",
                       metal ? "Metal" : "CPU", batch, b, unchanged, max_error);
            }
            free(input);
            free(output);
            ggml_backend_buffer_free(buf);
            ggml_free(ctx);
            ggml_backend_free(backend);
        }
    }
}

Build against the official release and the matching GGML headers:

git clone https://github.com/leejet/ggml.git
git -C ggml checkout d25b121ce1d6ae0c7e62ad080460d9fc1e46dd93
curl -fL -o release.zip https://github.com/leejet/stable-diffusion.cpp/releases/download/master-969-8c442fb/sd-master-8c442fb-bin-Darwin-macOS-26.6.2-arm64.zip
unzip -j release.zip '*.dylib' -d lib
clang -O2 -I ggml/include group_norm.c -L "$PWD/lib" \
  -lstable-diffusion -Wl,-rpath,"$PWD/lib" -o group_norm
./group_norm

Actual output on 969

GROUP_NORM backend=CPU batch=1 item=0 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=CPU batch=2 item=0 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=CPU batch=2 item=1 unchanged=0/4096 max_error=1.92534e-07
GROUP_NORM backend=Metal batch=1 item=0 unchanged=0/4096 max_error=1.01877e-07
GROUP_NORM backend=Metal batch=2 item=0 unchanged=0/4096 max_error=1.01877e-07
GROUP_NORM backend=Metal batch=2 item=1 unchanged=4096/4096 max_error=123458

Expected: every item is written, with numerical errors comparable to item
zero and the CPU controls, rather than retaining the sentinel.

Source-level explanation

In the pinned GGML revision:

• ggml_metal_op_group_norm
  passes only dimensions 0–2 and dispatches ngrp threadgroups, with no
  fourth-dimension iteration.
• kernel_group_norm_f32
  bounds processing by ne00 * ne01 * ne02; it has no per-batch offset.
• Batched CFG packs positive/unconditional branches into batch dimension 3.
  Shape checks succeed even though the second branch's values are unwritten.

The parent of the batched-CFG introduction, the introduction
(PR #2085), 968,
and 969 all pin the same GGML revision. Batching exposes a pre-existing
Metal limitation rather than a failure of CFG batching on every backend.

Image-level confirmation and controls

Using initialized native C context/generation parameters, SD 1.5
v1-5-pruned-emaonly.safetensors, prompt a watercolor landscape, seed 91,
strength 0.5, four configured steps (three actual denoising iterations), and
an RGB-gradient init image:

• Batched CFG enabled: all-white output on 968 at 64, 128, 256, and 512 pixels.
  A fresh process/context fails on its first generation. 969 also reproduces
  the 64x64 failure directly against its official dylib.
• Batched CFG disabled: nonconstant output in the tested 968 size controls.
• CPU with batching enabled: nonconstant output in the tested 64x64 case.
• Disabling conditioning caching, explicitly disabling flash attention and
  direct convolution, or setting GGML_METAL_TENSOR_DISABLE=1 does not fix
  the 968 failure. Canny preprocessing and earlier calls are not required.

For a causal control on 968, the backend-evaluation callback supplied only
the missing group-normalization outputs for batch indices >=1. No library
rebuild, model changes, or other operation replacements were made:

• Without that intervention: traced activations become extreme and then NaN;
  final pixels are all 255.
• With it: zero traced NaNs/positive infinities in the F32 outputs, and
  nonconstant output at both 64x64 and 512x512. Repeated 64x64 calls agree.
• The 512x512 mean pixel value is 128.3913 versus 128.3934 for the unbatched
  control; these are not claimed to be bit-identical images.

Workaround and requested fix

Disable batched CFG on Metal until group normalization processes every
batch item. CPU/non-Metal batching need not be disabled on this evidence.

Please fix batch-dimension handling in Metal GROUP_NORM, or temporarily
exclude Metal from batched CFG eligibility. I have not tested other Apple
GPU generations or established the absence of other batch-sensitive kernels.