NVIDIA (CUDA)

The CUDA backend enables Tribunus Compute on NVIDIA GPUs. It targets CUDA 12 with cuBLASLt for optimized matrix operations and Triton for custom kernel generation. The backend supports NVIDIA Ampere (SM 80), Ada Lovelace (SM 89), and Blackwell (SM 100) architectures, with architecture-specific codegen for tensor core utilization, shared memory tiling, and warp-level reductions.

Type: Primary · Status: In progress · Key Library: CUDA 12, cuBLASLt, Triton

cuBLASLt is used for all standard GEMM operations, providing hardware-tuned kernels for each supported GPU generation. For operations that do not map to cuBLASLt (attention, layer normalization, activation fusions), Triton generates custom kernels that are compiled ahead-of-time during the admission pipeline and cached in the compute image. The runtime does no JIT compilation — every Triton kernel was compiled, verified, and measured before deployment.

Accuracy gates from the numerical oracle are enforced for every kernel, including Triton-generated ones. The oracle runs on the CUDA backend’s native FP16 and BF16 paths, comparing against the Apple Silicon FP32 golden reference. Performance gates are calibrated to each GPU generation’s theoretical peak FLOP/s and memory bandwidth, so a kernel that passes on a Blackwell B200 must also meet the bar on an Ampere A100 with appropriate tile-size adjustments.

The compiler pass lowers each PhaseIR operation to CUDA Graphs where possible, capturing the decode loop’s kernel launch DAG as a single executable unit. This eliminates driver overhead in the autoregressive generation loop.