Docs

Python SDK

Load a model or an IR graph, run inference, and capture profiles without touching the C++ internals.

Install

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --parallel

python bindings/python/scripts/bundle_native.py --build-dir build
pip install -e bindings/python

For the native extension, configure with -DUAII_BUILD_PYTHON=ON, which fetches pybind11. Without it, the package calls uaii_capi through ctypes.

If the shared library cannot be found, set UAII_CAPI_PATH to the full path of uaii_capi.

Load, run, profile

import uaii

session = uaii.Session.from_path(
    "examples/ir/toy_mlp.uaii.json",
    weight_init="ones",
    profile=True,
    trace_path="uaii_py_profile.json",
)
session.set_tensor("x", [1.0, 2.0, 3.0, 4.0])
session.run()
print(session.get_tensor("y_prob"))
print(session.profile_summary())

Convert models

uaii.convert_model("model.gguf", "model.uaii.json")

Defaults worth knowing

weight_init="none"

A model with missing weights raises an error rather than filling tensors with placeholder values

backend="cpu"

The default for both Python and the C API. Pass backend="auto" to run the same probe as the CLI, which selects a native GPU when one is usable. uaii doctor reports which backends fall back to the host

Memory preset

Session.from_path does not call uaii_resolve_memory_preset. Use the CLI, or the C API, for laptop and phone paging. The ctypes struct includes the 0.4.0 fields so a later binding can set them

session = uaii.Session.from_path("model.gguf", backend="auto")