AiliaLLM

Kotlin binding for ailia LLM library. Provides Large Language Model inference capabilities.

Calls JNI directly from Kotlin without going through Java wrappers.

When using QNN, call destroy or close on the same thread that constructed this instance. For QNN, release resources explicitly on that thread; do not rely on garbage collection.

Constructors

AiliaLLM

constructor()

Types

Companion

object Companion

Functions

open override fun close()

Implements Closeable for use with use { } blocks. When using QNN, this method must be called on the same thread that constructed this instance, as with destroy.

destroy

fun destroy()

Destroys the AiliaLLM instance and releases resources.

generate

fun generate(): Boolean

Generates one token.

getContextSize

fun getContextSize(): Int

Gets the context size of the model.

getDeltaText

fun getDeltaText(): String

Gets the generated text (delta).

getErrorDetail

fun getErrorDetail(): String

Detail of the last native error; valid until the next native operation.

getMultimodalCapabilities

fun getMultimodalCapabilities(): AiliaLLMMultimodalCapabilities

Gets the multimodal capabilities of the loaded model.

getPromptTokenCount

fun getPromptTokenCount(): Int

Gets the prompt token count.

getResponseJson

fun getResponseJson(): String

Gets buffered assistant JSON. Deltas need not be accumulated by the caller.

getTokenCount

fun getTokenCount(text: String): Int

Gets the token count for a text.

openModelFile

fun openModelFile(path: String, nCtx: Int)

Opens a model file.

openMtpModel

fun openMtpModel(path: String, nDraft: Int)

Open an MTP assistant and enable draft/target verification decoding. Call once after openModelFile and before setting a prompt. Supports text generation with CPU/GPU Gemma 4 GGUF and QNN models.

openMultimodalProjectorFile

fun openMultimodalProjectorFile(path: String)

Opens a multimodal projector file for vision/audio support. Must be called after openModelFile() to enable multimodal capabilities.

setBackend

fun setBackend(index: Int)

Selects a backend returned by getBackendCount and getBackendName. Must be called before openModelFile. HTP (QNN) accepts only .qnn; explicitly selected CPU/GPU backends reject .qnn. Without a selection, the model format determines the backend.

setNumThreads

fun setNumThreads(nThreads: Int)

Sets the CPU thread count before openModelFile. The default is 0 when this method is not called. On Android, automatic selection uses performance (big/prime) cores, falling back to the existing core-count heuristic when topology is unavailable. Target decode/prefill, assistant and VLM/ALM CPU inference threads share this value.

setPrompt

fun setPrompt(messages: Array<AiliaLLMChatMessage>)

Sets the prompt for generation.

fun setPrompt(messages: Array<AiliaLLMMultimodalChatMessage>)

Sets the prompt for generation with multimodal support. Automatically detects whether to use text-only or multimodal processing based on the presence of media data.

setPromptJson

fun setPromptJson(messagesJson: String)

Sets structured history; required with tools. User content arrays accept text and image or audio parts with file_path or base64 data. Load a matching projector for media. Submit image and audio in separate prompts.

setSamplingParams

fun setSamplingParams(topK: Int, topP: Float, temp: Float, seed: Int)

Sets sampling parameters.

setThinking

fun setThinking(enable: Boolean)

Enable or disable thinking (reasoning output). Controls whether thinking models (e.g. Gemma4) output their reasoning process. Must be called before setPrompt.

setTools

fun setTools(toolsJson: String?)

Sets the tool (function) definitions for tool use (function calling). The tools are rendered into the prompt through the chat template on the next setPromptJson call, the output is constrained to the tool call syntax, and the buffered output can be retrieved with getResponseJson().