Configuration

Control logging, plugins, GEMM providers, backends, and session policy through CMake options, a TOML file, environment variables, and API options.

Configuration file

The runtime loads configs/uaii.toml, or the file you pass to --config. Environment variables override file values.

Environment variables

Variable	Purpose
UAII_LOG_LEVEL	Verbosity: `trace`, `debug`, `info`, `warn`, or `error`
UAII_LOG_COLOR	Enable or disable ANSI colour in log output
UAII_PLUGIN__DIRS	Comma-separated plugin search directories
UAII_NUM_THREADS	CPU kernel parallelism
UAII_GEMM	Preferred GEMM provider when more than one is linked
UAII_MAX_LAYERS	Cap the number of GGUF transformer layers loaded. `0` means unlimited, up to the hard maximum
UAII_BACKEND	Device override for the dashboard and CLI jobs: `auto` by default, or `cpu`, `cuda`, and so on
UAII_CAPI_PATH	Path to the `uaii_capi` shared library, used by the Python ctypes loader

CMake options

Option	Default	Meaning
`UAII_BUILD_TESTS`	ON	Unit and smoke tests
`UAII_BUILD_PLUGINS`	ON	Example plugins
`UAII_BUILD_PYTHON`	OFF	Native pybind11 extension for the Python SDK
`UAII_WITH_ONEDNN` / `OPENBLAS`	OFF	Link a vendor BLAS for faster CPU GEMM
`UAII_WITH_CUDA`	OFF	CUDA device path with cuBLASLt
`UAII_WITH_METAL` / `VULKAN` / `WEBGPU` / `ROCM`	OFF	Optional GPU backends
`UAII_WITH_SENTENCEPIECE`	OFF	SentencePiece tokenizer support

Session options

The CLI generate and chat commands apply the laptop preset and --backend auto. The C API 0.3.0 struct exposes a smaller surface: it defaults to backend = "cpu" and has no kv_window field yet.

backend_name: Named backend in C. Pass auto to probe for a native GPU, or name a device such as cpu, cuda, or metal
compute_dtype: F32 or F16 compute policy
keep_quantized_weights: Retain GGUF blocks so quantized GEMM can run directly
max_context: Generation length bound, where 0 reads graph metadata. With a sliding KV cache this bounds sequence length, not the memory allocation
kv_window, kv_sink: C++ and CLI only. The resident KV window, 2048 tokens with 4 sinks on the laptop preset
pin_layers, expert_cache_bytes: C++ and CLI only. The pinned trunk and the MoE expert LRU budget
enable_fusion and related: Also enable_memory_reuse, enable_streaming, and enable_profiler
weight_init: Defaults to none, so missing weights raise an error
weights_dir, weights_sandbox: Weight resolution and the path sandbox

C API callers must set struct_size = sizeof(uaii_session_options) immediately after uaii_session_options_init.

Global CLI flags

uaii --config configs/uaii.toml --log-level info --load-plugins doctor

Dashboard configuration

The operator UI reads dashboard/uaii-dash.json, modelled on uaii-dash.example.json, plus the environment variables below. Full guide: Operator UI.

Variable	Purpose
UAII_DASH_BIND	`127.0.0.1` for local use, or `0.0.0.0` to self-host
UAII_DASH_PORT	HTTP port (default `8787`)
UAII_DASH_TOKEN	Bearer token, required for any non-loopback bind
UAII_BIN / UAII_BENCH_BIN	Override the auto-detected CLI binaries
UAII_MODEL_DIR	Model library directory
UAII_USE_WSL / UAII_WSL_BIN	Use the WSL `uaii` binary when Windows blocks unsigned executables
UAII_BACKEND	Device for chat and run jobs: `auto` by default, or `cpu`, `cuda`, and so on

# Local, on loopback
cd dashboard && npm start

# Self-hosted, which requires a token
UAII_DASH_BIND=0.0.0.0 UAII_DASH_TOKEN=secret npm start