Docs

CLI reference

One binary to diagnose the install, pull Hugging Face models, convert weights, and run generation.

Global flags

--config <toml>: Load a configuration file
--log-level <level>: One of trace, debug, info, warn, or error
--no-color: Disable ANSI colour output
--load-plugins: Discover and load plugins, used with doctor

Commands

Command	Description
`doctor`	Reports loaded modules, the active GEMM provider, backend capabilities, and the `recommended:` device
`validate` / `inspect` / `graph`	Validate an IR graph and dump it as `text`, `dot`, `json`, or `plan`
`pull`	Download a Hugging Face repository into a local directory for the decoder-only path
`convert`	Compile a Hugging Face folder, GGUF, Safetensors, ONNX, MLX, or PyTorch sidecar into UAII IR
`tokenize`	Encode and decode with simple, BPE, SentencePiece, GGUF, or Hugging Face tokenizers
`generate`	Generate from a Hugging Face CausalLM folder, a GGUF `blk.*` model, or `--demo`. Accepts `--preset`, `--backend`, `--kv-window`, `--json`, and `--stream`
`chat`	Keep a warm session as a JSONL worker (`--jsonl`), which the operator UI drives
`run`	Execute an IR graph, or a built-in graph with `--demo`
`profile` / `benchmark` / `cache`	Chrome-trace capture, operator timings, and plan cache management
`help` / `version`	Usage text and the version string

Generate and chat

The device defaults to --backend auto, which uses a native GPU when this binary can drive one and CPU otherwise. Pass --backend cpu to pin the host. A physical GPU does not qualify if the binary was configured with UAII_WITH_CUDA=OFF, since only host math is compiled in; auto stays on CPU in that case.

The memory budget defaults to --preset laptop (about 8 GiB, KV 2048, two staging buffers). --preset phone is about 2 GiB, KV 512, one buffer, and 4 threads, then half of free RAM. Desktop and server keep their targets when free RAM covers them. --preset none turns streaming off. The full table is on the memory page.

# One-shot generation from any GGUF with blk.* tensors
uaii generate --model path/to/model.gguf \
  --prompt "Hello" --max-new-tokens 64 --stream

# Built-in tiny model, no download required
uaii generate --demo --prompt "hi" --max-new-tokens 8 --json

# Phone budget. Chat help lists laptop|phone.
uaii generate --model path/to/model.q4_k.gguf --preset phone --prompt "Hello"
uaii chat --model path/to/model.q4_k.gguf --jsonl --preset phone

# Warm worker for the dashboard or automation
uaii chat --model path/to/model.gguf --jsonl

Cheatsheet

uaii doctor --load-plugins
uaii validate <ir>
uaii inspect <ir>
uaii graph <ir> --format plan
uaii convert model.gguf -o model.uaii.json
uaii tokenize encode hello world
uaii generate --model model.gguf --prompt "hi" --json
uaii run --demo toy_mlp
uaii run <ir> --input x=1,2,3,4 --backend auto
uaii profile --demo --output uaii_profile.json
uaii benchmark --demo
uaii cache status