Uncompressed 16-bit, half, float, RGB9e5 and signed texture formats fail to decode #386

parse_image_data fails for most uncompressed formats with more than 8 bits per channel and for all _SIGNED formats. The script below decodes one texel of each format as a 2x2 texture on 417d595 (1.25.4); every format in the list either raises or returns wrong values. The output is the same with Pillow 9.5.0, 10.4.0, 11.3.0 and 12.3.0, so the failures come from the arguments UnityPy passes, not from a Pillow change.

import struct

import PIL

import UnityPy
from UnityPy.enums import TextureFormat as TF
from UnityPy.export.Texture2DConverter import parse_image_data

print("UnityPy", UnityPy.__version__, "Pillow", PIL.__version__)

# format: (one texel, the texel as 8-bit RGBA with values clamped to [0, 1])
TEXELS = {
    "R16": (struct.pack("<H", 0x8000), (128, 0, 0)),
    "RG16": (bytes([0x80, 0x40]), (128, 64, 0)),
    "RG32": (struct.pack("<2H", 0x8000, 0x4000), (128, 64, 0)),
    "RGB48": (struct.pack("<3H", 0x8000, 0x4000, 0), (128, 64, 0)),
    "RGBA64": (struct.pack("<4H", 0x8000, 0x4000, 0, 0xFFFF), (128, 64, 0, 255)),
    "RHalf": (struct.pack("<e", 0.5), (128, 0, 0)),
    "RGHalf": (struct.pack("<2e", 0.5, 0.25), (128, 64, 0)),
    "RGBAHalf": (struct.pack("<4e", 0.5, 0.25, 0.0, 1.0), (128, 64, 0, 255)),
    "RFloat": (struct.pack("<f", 0.5), (128, 0, 0)),
    "RGFloat": (struct.pack("<2f", 0.5, 0.25), (128, 64, 0)),
    "RGBAFloat": (struct.pack("<4f", 0.5, 0.25, 0.0, 1.0), (128, 64, 0, 255)),
    "ARGBFloat": (struct.pack("<4f", 1.0, 0.5, 0.25, 0.0), (128, 64, 0, 255)),
    # R = G = 256 * 2**(15 - 24) = 0.5, B = 128 * 2**(15 - 24) = 0.25
    "RGB9e5Float": (struct.pack("<I", (15 << 27) | (128 << 18) | (256 << 9) | 256), (128, 128, 64)),
    "R8_SIGNED": (struct.pack("<b", 64), "signed"),
    "RG16_SIGNED": (struct.pack("<2b", 64, -64), "signed"),
    "RGB24_SIGNED": (struct.pack("<3b", 64, -64, 0), "signed"),
    "RGBA32_SIGNED": (struct.pack("<4b", 64, -64, 0, 127), "signed"),
    "R16_SIGNED": (struct.pack("<h", 16384), "signed"),
    "RG32_SIGNED": (struct.pack("<2h", 16384, -16384), "signed"),
    "RGB48_SIGNED": (struct.pack("<3h", 16384, -16384, 0), "signed"),
    "RGBA64_SIGNED": (struct.pack("<4h", 16384, -16384, 0, 32767), "signed"),
}

for name, (texel, expected) in TEXELS.items():
    try:
        image = parse_image_data(texel * 4, 2, 2, getattr(TF, name), (2022, 3, 0, 0), 0)
        result = f"{image.mode} {image.getpixel((0, 0))}"
    except Exception as exc:
        result = f"{type(exc).__name__}: {exc}"
    print(f"{name:14} {result:55} expected {expected}")

UnityPy 1.25.4 Pillow 12.3.0
R16            ValueError: unknown raw mode for given image mode       expected (128, 0, 0)
RG16           KeyError: 'raw'                                         expected (128, 64, 0)
RG32           KeyError: 'raw'                                         expected (128, 64, 0)
RGB48          RGB (0, 0, 0)                                           expected (128, 64, 0)
RGBA64         ValueError: unknown raw mode for given image mode       expected (128, 64, 0, 255)
RHalf          error: unpack requires a buffer of 2 bytes              expected (128, 0, 0)
RGHalf         KeyError: 'raw'                                         expected (128, 64, 0)
RGBAHalf       ValueError: bytes must be in range(0, 256)              expected (128, 64, 0, 255)
RFloat         ValueError: unknown raw mode for given image mode       expected (128, 0, 0)
RGFloat        KeyError: 'raw'                                         expected (128, 64, 0)
RGBAFloat      ValueError: unknown raw mode for given image mode       expected (128, 64, 0, 255)
ARGBFloat      ValueError: unknown raw mode for given image mode       expected (128, 64, 0, 255)
RGB9e5Float    TypeError: 'float' object cannot be interpreted as an integer expected (128, 128, 64)
R8_SIGNED      ValueError: unrecognized image mode                     expected signed
RG16_SIGNED    KeyError: 'raw'                                         expected signed
RGB24_SIGNED   ValueError: unknown raw mode for given image mode       expected signed
RGBA32_SIGNED  ValueError: unknown raw mode for given image mode       expected signed
R16_SIGNED     ValueError: unrecognized image mode                     expected signed
RG32_SIGNED    KeyError: 'raw'                                         expected signed
RGB48_SIGNED   ValueError: unknown raw mode for given image mode       expected signed
RGBA64_SIGNED  ValueError: unknown raw mode for given image mode       expected signed

With the 0.5 in the alpha channel of RGBAHalf instead of 1.0, it returns RGB (0, 128, 128): no alpha, and the channels are shifted.

Causes

parse_image_data calls conv_func(image_data, width, height, *conv_args), and the CONV_TABLE entries are (image mode, "raw", raw mode) triples. Five things go wrong for these formats:

1. rg() looks up RG_PADDING_MAP[codec], but codec receives the decoder name "raw"; the raw mode ("RG", "RGE", "RG;16", …) arrives as args. Every format routed through rg() raises KeyError: 'raw'.
2. half() reads width * height * len(codec) halves, which is always 3 (len("raw")): RHalf runs out of data, RGBAHalf reads 3 of its 4 channels per texel. It also scales by 256, so 1.0 becomes 256 and does not fit in a byte.
3. RHalf, R8_SIGNED and R16_SIGNED use the image mode "R", which Pillow does not have.
4. Several raw modes do not exist for the image mode they are used with: "R;16", "RF", "RGBAF", "RGBA;16" and the ;8s / ;16s modes. "RGB;16" exists, but it is Pillow's packed 16 bits per pixel (5-6-5) mode, so RGB48 reads 2 bytes per texel and decodes to wrong colors; the little-endian 16 bits per channel mode is "RGB;16L".
5. rgb9e5float() assigns floats into a bytearray, and the values are not clamped to 0–255.

GRAPHICS_TO_TEXTURE_MAP maps several GraphicsFormats to these texture formats, so Texture2DArrays of those formats are affected as well.

Open questions

A fix needs two conventions that are the maintainers' choice:

• Values outside [0, 1] in the half, float and RGB9e5 formats: clamp to [0, 1] in an 8-bit image, as the ASTC HDR decoder does, or return a float image where Pillow supports one (F for single-channel formats).
• Signed formats: map [-1, 1] to [0, 255] ((v + 1) / 2, as for normal maps), clamp negative values to 0, or keep the raw bytes.

I can send a PR with the decoders and a test per format once these are decided.

Tested with UnityPy 417d595 (1.25.4) and CPython 3.11.13 on Windows x86_64.