High Level API

This is an example of using ailia LLM.

AiliaLLMModel llm = new AiliaLLMModel();
private List<AiliaLLMChatMessage> messages = new List<AiliaLLMChatMessage>();
 
string asset_path = Application.streamingAssetsPath;
string model_path = "gemma-2-2b-it-Q4_K_M.gguf";
 
llm.Create();
llm.Open(asset_path + "/" + model_path, 8192);
 
AiliaLLMChatMessage message = new AiliaLLMChatMessage();
message.role = "system";
message.content = "語尾に「だわん」をつけてください。";
messages.Add(message);
 
AiliaLLMChatMessage message2 = new AiliaLLMChatMessage();
message2.role = "user";
message2.content = "こんにちは。";
messages.Add(message2);
inputFiled.text = "";
 
llm.SetPrompt(messages);
bool done = false;
string text = "";
while (true){
    bool status = llm.Generate(ref done);
    if (done == true || status == false){
        break;
    }
    string deltaText = llm.GetDeltaText();
    text = text + deltaText;
}
Debug.Log(text);
 
message = new AiliaLLMChatMessage();
message.role = "assistant";
message.content = text;
messages.Add(message);

Image (VLM) and audio (ALM) input

Open a compatible text model and matching projector with OpenMultimodalProjector. Check GetMultimodalCapabilities before submitting media. Match every <__media__> marker to one media_data entry in order.

using ailiaLLM;
using System.Collections.Generic;
 
var llm = new AiliaLLMModel();
if (!llm.Create() || !llm.Open(textModelPath, 8192) ||
    !llm.OpenMultimodalProjector(projectorPath)) return;
bool vision = false, audio = false;
if (!llm.GetMultimodalCapabilities(ref vision, ref audio)) return;
 
if (vision) {
    var image = new AiliaLLMMediaData { media_type = "image", file_path = imagePath };
    var message = new AiliaLLMMultimodalChatMessage {
        role = "user", content = "Describe this image: <__media__>",
        media_data = new List<AiliaLLMMediaData> { image }
    };
    if (!llm.SetMultimodalPrompt(new List<AiliaLLMMultimodalChatMessage> { message })) return;
    bool done = false;
    string text = "";
    while (!done) {
        if (!llm.Generate(ref done)) return;
        if (!done) text += llm.GetDeltaText();
    }
    Debug.Log(text);
}

For ALM, check audio and use media_type = "audio", a WAV/MP3/FLAC file_path, and an audio question. Encoded byte[] data is also supported; leave width and height zero for audio. Raw PCM and audio output are not supported. Submit image and audio as separate prompts.

Android QNN

With a QNN-enabled build, AiliaLLMModel.GetQNNModelName() gives the device model stem (for example, sm7635). Select the precompiled text .qnn and matching mmproj .qnn for the same SoC and model. Open them with Open and OpenMultimodalProjector; a text-only session needs no projector. Match the QNN runtime and plug-in versions as well. Without an explicit backend selection, .qnn selects HTP (QNN) automatically. The available-backend order is GPU, CPU, then HTP (QNN). Explicit CPU/GPU selection rejects .qnn, and explicit HTP selection rejects GGUF. HTP is listed only when a compatible QNN runtime is available.

string soc = AiliaLLMModel.GetQNNModelName();
// Select distributed text/projector paths matching soc.
var llm = new AiliaLLMModel();
if (!llm.Create() || !llm.Open(textQnnPath, 0) ||
    !llm.OpenMultimodalProjector(mmprojQnnPath)) return;

The model-name lookup fails for builds without QNN, missing plug-ins or unsupported SoCs. Inspect load/generation failures with GetErrorDetail().

Tool Use (JSON API)

Use ailiaLLMSetPromptJson from the first user message while tools are configured. The legacy ailiaLLMSetPrompt and ailiaLLMSetMultimodalPrompt return INVALID_STATE until tools are cleared. Define tools with ailiaLLMSetTools, set the JSON messages array, and call ailiaLLMGenerate until done. Deltas remain available through ailiaLLMGetDeltaText for previews; retrieving or concatenating them is optional. Retrieve assistant JSON with ailiaLLMGetResponseJsonSize / ailiaLLMGetResponseJson, append the object to history, execute complete calls, and append tool results with matching tool_call_id. Set the next JSON prompt to continue.

The SDK accumulates generated output. Reads do not consume it; a new prompt resets it.

string toolsJson = "[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"description\":\"Get weather for a city\",\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"]}}}]";
if (!llm.SetTools(toolsJson)) return;
if (!llm.SetPromptJson("[{\"role\":\"user\",\"content\":\"Weather in Tokyo?\"}]")) return;
bool done = false;
while (!done) { if (!llm.Generate(ref done)) return; }
string assistantJson = llm.GetResponseJson();
// Parse JSON, validate and execute complete tool_calls, then append tool results.

Structured history

[
 {"role":"user","content":"Weather in Tokyo?"},
 {"role":"assistant","content":"","tool_calls":[
  {"id":"call_0","type":"function","function":{"name":"get_weather","arguments":"{\"city\":\"Tokyo\"}"}}
 ]},
 {"role":"tool","tool_call_id":"call_0","content":"Snow, -3 C"}
]

Content is not reparsed as tool syntax. Supply assistant thinking in reasoning_content. Arguments must be a string containing a JSON object. Call IDs/names must be nonempty; IDs are unique per assistant, but may be reused in later turns. Consecutive tool results match the preceding assistant by ID, not order. Unknown IDs, duplicate results and conflicting names are rejected. Tool content must be a string; serialize objects first.

Images and audio

Load a compatible projector with ailiaLLMOpenMultimodalProjectorFileA and use ordered user content parts:

[{"role":"user","content":[
 {"type":"text","text":"Describe this image and audio."},
 {"type":"image","file_path":"/path/to/image.jpg"},
 {"type":"audio","file_path":"/path/to/audio.wav"}
]}]

Each image/audio part specifies exactly one of file_path or data. Data is standard padded Base64 containing encoded file bytes (JPEG/PNG, WAV/MP3/FLAC, etc.). No URL fetching, video or raw RGB/PCM is supported. Media works with or without tools and is re-evaluated each prompt. An unloaded/incompatible projector returns INVALID_STATE. Query image/audio support with ailiaLLMGetMultimodalCapabilities.

Incomplete output and errors

GetResponseJson rejects unfinished calls/thinking with PARSE_ERROR. It never repairs arguments or returns executable partial results. SetPromptJson rejects incomplete arguments, invalid JSON/types/ID associations and empty arrays with INVALID_ARGUMENT. If GetResponseJson returns PARSE_ERROR, do not append the failed assistant. Append a retry user message to the existing history; no partial content/reasoning retrieval is needed. Do not invent tool results.

After clearing tools with SetTools(NULL), SetPromptJson still accepts historical tool calls and results for summaries or final answers. Retrieve response JSON before clearing tools, which invalidates the output parser. Use finally to clear tools even when an exception occurs. An unset prompt or changed tools/thinking returns INVALID_STATE; before generation an initialized prompt returns an empty assistant. File failures return ERROR_FILE_API, invalid Base64 returns INVALID_ARGUMENT, and undecodable media buffers return ERROR_BUFFER_API.

if (!llm.SetPromptJson(messagesJson)) return;
bool done = false;
while (!done) { if (!llm.Generate(ref done)) return; }
string responseJson = llm.GetResponseJson();
// Empty string signals error. Otherwise append the object and ID-linked tool results.