Docs
Quick start
Build the runtime, then generate from a Hugging Face decoder folder or a GGUF checkpoint.
Defaults are --preset laptop and --backend auto. Phone builds use --preset phone. The dial is on Memory. Hub models are on Models.
Prerequisites
- C++17 compiler (MSVC 2019+, Clang 10+, or GCC 9+)
- CMake 3.20+
- Ninja (recommended) or your platform generator
- Optional: CUDA Toolkit, Vulkan SDK, ROCm, oneDNN, OpenBLAS, SentencePiece, Python 3.10+
Build
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \ -DUAII_BUILD_TESTS=ON -DUAII_BUILD_PLUGINS=ON cmake --build build --config Release --parallel
Windows multi-config generators place binaries under build/Release/. Single-config Ninja builds produce build/libs/uaii-cli/uaii, or uaii.exe on Windows.
First commands
uaii version uaii doctor # Hugging Face decoder-only uaii pull org/model --outdir ./models/hf/org__model uaii generate --model ./models/hf/org__model --prompt "hi" --max-new-tokens 64 --preset laptop # GGUF, laptop default. Phone preset pages into about 2 GiB. uaii generate --model path/to/model.q4_k.gguf --prompt "hi" --preset phone # Smoke test the IR stack with no model weights uaii generate --demo --prompt "hi" --max-new-tokens 4 --json
Hugging Face support requires a current build, so rebuild uaii after pulling. Point the dashboard at that binary with UAII_BIN.
Dashboard (optional UI)
cd dashboard npm run install:all npm run build && npm start # → http://127.0.0.1:8787 # Point at the rebuilt CLI, required for Hugging Face models UAII_BIN=/path/to/uaii npm start # Self-host on a LAN or server UAII_DASH_BIND=0.0.0.0 UAII_DASH_TOKEN=secret npm start
The console provides Hub pulls, streaming chat over HF and GGUF models, health checks, benchmarks, and an OpenAI-compatible /v1 endpoint. The default backend setting is automatic GPU detection. See the operator UI guide.
Run a graph
uaii run examples/ir/toy_mlp.uaii.json \ --weight-init ones \ --input x=1,2,3,4 \ --output y_prob
Python (optional)
python bindings/python/scripts/bundle_native.py --build-dir build pip install -e bindings/python python examples/python/load_run_profile.py
If the loader cannot locate the shared library, set UAII_CAPI_PATH to the full path of uaii_capi.dll, .so, or .dylib.
Tests
ctest --test-dir build -C Release --output-on-failure