CLI reference

One binary to diagnose the install, pull Hugging Face models, convert weights, and run generation.

Global flags

--config <toml>: Load a configuration file
--log-level <level>: One of trace, debug, info, warn, or error
--no-color: Disable ANSI colour output
--load-plugins: Discover and load plugins, used with doctor

Commands

Command	Description
`doctor`	Reports loaded modules, the active GEMM provider, backend capabilities, and the `recommended:` device
`validate` / `inspect` / `graph`	Validate an IR graph and dump it as `text`, `dot`, `json`, or `plan`
`pull`	Download a Hugging Face repository into a local directory for the decoder-only path
`convert`	Compile a Hugging Face folder, GGUF, Safetensors, ONNX, MLX, or PyTorch sidecar into UAII IR
`tokenize`	Encode and decode with simple, BPE, SentencePiece, GGUF, or Hugging Face tokenizers
`generate`	Generate from a Hugging Face CausalLM folder, a GGUF `blk.*` model, or `--demo`. Accepts `--preset`, `--backend`, `--kv-window`, `--json`, and `--stream`
`chat`	Keep a warm session as a JSONL worker (`--jsonl`), which the operator UI drives
`run`	Execute an IR graph, or a built-in graph with `--demo`
`profile` / `benchmark` / `cache`	Chrome-trace capture, operator timings, and plan cache management
`help` / `version`	Usage text and the version string

Generate and chat

The device defaults to --backend auto, which uses a native GPU when this binary can drive one and CPU otherwise. Pass --backend cpu to pin the host. A physical GPU does not qualify if the binary was configured with UAII_WITH_CUDA=OFF, since only host math is compiled in; auto stays on CPU in that case.

The memory budget defaults to --preset laptop: roughly 8 GiB, a trunk that is auto-shrunk until it fits (layers, then lm_head, then embed), packed GGUF staging, an expert LRU filled from leftover RAM, and a 2048-token KV window with 4 sinks. GGUF Mixtral and Qwen packed banks are split per expert, and only the routed top-k are paged in. --preset none disables streaming entirely.

# One-shot generation from any GGUF with blk.* tensors
uaii generate --model path/to/model.gguf \
  --prompt "Hello" --max-new-tokens 64 --stream

# Built-in tiny model, no download required
uaii generate --demo --prompt "hi" --max-new-tokens 8 --json

# Longer conversations, with the sliding KV window kept at its default
uaii generate --model path/to/model.gguf --kv-window 2048 --prompt "Hello"

# Warm worker for the dashboard or automation
uaii chat --model path/to/model.gguf --jsonl

Cheatsheet

uaii doctor --load-plugins
uaii validate <ir>
uaii inspect <ir>
uaii graph <ir> --format plan
uaii convert model.gguf -o model.uaii.json
uaii tokenize encode hello world
uaii generate --model model.gguf --prompt "hi" --json
uaii run --demo toy_mlp
uaii run <ir> --input x=1,2,3,4 --backend auto
uaii profile --demo --output uaii_profile.json
uaii benchmark --demo
uaii cache status