ailia LLM Flutter Package

!! CAUTION !! “ailia” IS NOT OPEN SOURCE SOFTWARE (OSS). As long as user complies with the conditions stated in License Document, user may use the Software for free of charge, but the Software is basically paid software.

About ailia LLM

ailia LLM is a library for running local LLMs. It can load GGUF and easily implement chat functionality.

macOS Download Attribute

If a security error occurs with a downloaded file on macOS, please execute the following command to remove the download attribute.

xattr -d com.apple.quarantine macos/libailia_llm.dylib

API specification

https://github.com/ailia-ai/ailia-sdk

Regenerating FFI bindings and documentation

Run from the flutter directory after updating native/ailia_llm.h:

fvm dart run ffigen --config ffigen_ailia_llm.yaml
python3 tool/normalize_dartdoc.py
fvm dart doc

The normalization step converts the native Doxygen comments to English Dartdoc Markdown. Release packaging runs this step automatically.

MTP decoding

Open the target first, then its matching assistant before setting a prompt:

final model = AiliaLLMModel();
model.open('path/to/gemma4.gguf', 8192);
model.openMtpModel('path/to/mtp-gemma4.gguf', 3);
// Set the prompt and generate as usual.

For QNN, use an MTP target .qnn containing an AR-4 verification graph and its matching shared-native KV mtp-assistant.qnn. Generation still returns one token per call. Requires the ailia LLM 1.5.1 native library or later.