Phi-Bench (Φ-Bench): 85 open-source LLM-infrastructure engineering tasks for frontier LLMs & coding agents (KFC/LH/E2E). Self-contained public Dockerfile + offline scoring + reference solution per task. Website: llminfrabench.com
Reward-hardened evaluation for LLM-generated GPU kernels, built on KernelBench.
A RAG-augmented pipeline for generating, validating, and optimizing HIP GPU kernels using Qwen, with compiler-in-the-loop repair and performance evaluation through KernelBench on NVIDIA GPUs
Self-improvement research testbed for LLMs: Dream-RSI-style policy evolution (arXiv:2609.14858) + STaR-style weight upgrades, with a Rust rootless sandbox. Apache-2.0.
KernelBench for Huawei Ascend NPU: Torch-NPU reference auditing and isolated Ascend C kernel evaluation.
Profile-guided CUDA kernel optimization agent with pluggable OpenAI-compatible LLM providers and reproducible experiment telemetry.
Rule-evolving GPU kernel optimization loop — measurement feedback updates the rule table (matmul 6.4x, batched GEMM 4.5x on A100, KernelBench-validated)
Contract-aware, repeatability-focused evaluation for LLM-generated GPU kernels.
Plain English + a PyTorch reference → a verified, benchmarked CUDA kernel you can pip install and torch.compile.
To associate your repository with the kernelbench topic, visit your repo's landing page and select "manage topics."