Backends
CPU execution is always complete. The CLI and dashboard default to auto, which selects a native GPU only when this binary can drive a real device, and every host fallback is reported.
| Backend | Default build | With vendor SDK and a device |
|---|---|---|
| CPU | Complete f32 kernel set with tiled reference GEMM | Dispatches to oneDNN or OpenBLAS when linked |
| CUDA | Runs, with all math on the host | Device memory, cuBLASLt, and Add, Mul, RMSNorm, Softmax, and Silu on device. Decode attention and RoPE run on GPU once native initialisation succeeds; prefill attention may still run on the host |
| Metal | Runs, with all math on the host | Shared buffers, with MatMul, Add, and RMSNorm compiled as runtime MSL |
| Vulkan | Runs, with all math on the host | Device buffers, with Add as a compute shader. Remaining operators are listed in the reported capabilities |
| WebGPU | Runs, with all math on the host | Buffer path when headers are present. Compute coverage is limited |
| ROCm | Runs, with all math on the host | HIP memory, MatMul through rocBLAS, and HIP kernels for Add and RMSNorm |
Device selection
uaii generate, chat, run, and the dashboard all default to --backend auto, which selects the first native GPU that reports a real device, probing CUDA, then Metal, ROCm, and Vulkan. A backend compiled without its vendor SDK does not qualify, so auto stays on CPU rather than claiming a device it cannot use. Pass --backend cpu, or select CPU in Settings, to pin the host. The C API and Python Session default to "cpu"; pass "auto" to run the same probe.
Scheduling
DeviceScheduler assigns each operator to the preferred device. When attention_host_fallback is set, attention and RoPE are scheduled onto the CPU, and the session run loop honours those assignments.
Verifying what will run
uaii doctor
Doctor reports the active GEMM provider, the per-backend host_fallback and attention_host_fallback flags, and a recommended: line matching what auto-select will choose. A backend that reports a GPU name while executing on the host without setting those flags is a bug worth reporting.