ailia LLM MTP Benchmark

For how MTP works and how to use it, see the ailia LLM MTP Guide.

Android
- Snapdragon 8+ Gen 1 (SM8475): E2B
- Snapdragon 7s Gen 3 (SM7635): E2B / E4B
Windows
- Snapdragon X Plus (SC8380XP): E2B / E4B
- QCS6490: E2B

Android

Test Conditions

NPU: v1.5.1 MTP models (W4), QAIRT 2.47, context length 8192, AR-256 (prefill) / AR-4 (verification) / AR-1 configuration.
CPU: Runs with llama.cpp (4 threads). The main model is a GGUF (Q4_K_M for E2B, QAT Q4_0 for E4B) and the assistant is a Q4_0 GGUF.
MTP: "With MTP" opens the assistant (draft length 3). The acceptance rate is the fraction of assistant candidates accepted by the main model's verification.
Method: Greedy decoding, median of 2 runs per condition. Decode uses 81 input tokens and 1,000 generated tokens; prefill uses 64 to 2,048 input tokens and generates 1 token.
Image / audio input: One 768x768 image (100 output tokens) and a transcription of one English recording (32 output tokens). TTFT includes the encoder and excludes model opening.
CPU usage: Process CPU usage during decode, with all 8 cores fully used as 100%. The CPU runs 4 threads, so it stays around 50%.

Snapdragon 8+ Gen 1 (SM8475 / Hexagon v69)

E2B / FP16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	9.189 tokens/s	8.658 tokens/s	0.94x	31.1%
NPU	6.074 tokens/s	9.751 tokens/s	1.61x	31.6%

SM8475 E2B per-token decode throughput over 1000 generated tokens (CPU / NPU, without / with MTP) — SM8475 (E2B / FP16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	35.337	33.201	32.409	31.383	29.327	27.196
CPU (with MTP)	31.128	31.366	31.310	30.978	29.101	27.040
NPU (without MTP)	76.766	149.788	294.427	306.972	304.128	299.831
NPU (with MTP)	77.393	155.049	306.953	307.530	304.491	300.197

SM8475 E2B prefill throughput by input length (CPU / NPU, without / with MTP) — SM8475 (E2B / FP16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	33.46 s	8.784 tokens/s	—
Image	CPU (with MTP)	34.00 s	7.701 tokens/s	28.9%
Image	NPU (without MTP)	7.06 s	5.382 tokens/s	—
Image	NPU (with MTP)	6.92 s	10.476 tokens/s	41.7%
Audio	CPU (without MTP)	21.79 s	9.138 tokens/s	—
Audio	CPU (with MTP)	21.65 s	12.712 tokens/s	80.0%
Audio	NPU (without MTP)	4.20 s	5.344 tokens/s	—
Audio	NPU (with MTP)	4.53 s	18.054 tokens/s	100.0%

NPU (with MTP) TTFT is 1/4.8 (image) and 1/4.8 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.95x for image input and 3.38x for audio input.

SM8475 E2B VLM / audio input TTFT breakdown (CPU / NPU, without / with MTP) — SM8475 (E2B / FP16) VLM (left) / audio input (right) TTFT breakdown: prompt setup including the encoder, and decoder prefill + first token

SM8475 E2B VLM / audio input decode throughput (CPU / NPU, without / with MTP) — SM8475 (E2B / FP16) VLM (left) / audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	48.9%	48.3%	2.6%	4.4%
Image	48.7%	48.1%	3.7%	4.8%
Audio	48.6%	47.3%	3.6%	4.8%

SM8475 E2B average CPU usage during prefill (mean of 64 to 2,048-token inputs) — SM8475 (E2B / FP16) average process CPU usage during prefill (mean of 64 to 2,048-token inputs)

SM8475 E2B CPU usage during image / audio input TTFT — SM8475 (E2B / FP16) process CPU usage during image (left) / audio input (right) TTFT

Snapdragon 7s Gen 3 (SM7635 / Hexagon v73)

E2B / Int16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	8.423 tokens/s	11.336 tokens/s	1.35x	31.1%
NPU	6.680 tokens/s	10.579 tokens/s	1.58x	34.8%

SM7635 E2B per-token decode throughput over 1000 generated tokens (CPU / NPU, without / with MTP) — SM7635 (E2B / Int16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	57.466	54.395	51.547	47.661	37.408	33.600
CPU (with MTP)	50.451	49.910	49.452	46.387	34.608	31.654
NPU (without MTP)	32.911	64.958	129.053	130.981	128.974	127.976
NPU (with MTP)	33.344	66.363	132.603	132.104	130.779	129.264

SM7635 E2B prefill throughput by input length (CPU / NPU, without / with MTP) — SM7635 (E2B / Int16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	17.99 s	8.035 tokens/s	—
Image	CPU (with MTP)	17.74 s	11.046 tokens/s	28.9%
Image	NPU (without MTP)	8.41 s	6.715 tokens/s	—
Image	NPU (with MTP)	7.85 s	9.713 tokens/s	30.1%
Audio	CPU (without MTP)	14.14 s	7.972 tokens/s	—
Audio	CPU (with MTP)	14.10 s	16.313 tokens/s	80.0%
Audio	NPU (without MTP)	7.48 s	6.712 tokens/s	—
Audio	NPU (with MTP)	7.56 s	17.268 tokens/s	92.6%

NPU (with MTP) TTFT is 1/2.3 (image) and 1/1.9 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.45x for image input and 2.57x for audio input.

SM7635 E2B VLM / audio input TTFT breakdown (CPU / NPU, without / with MTP) — SM7635 (E2B / Int16) VLM (left) / audio input (right) TTFT breakdown: prompt setup including the encoder, and decoder prefill + first token

SM7635 E2B VLM / audio input decode throughput (CPU / NPU, without / with MTP) — SM7635 (E2B / Int16) VLM (left) / audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	49.0%	48.0%	2.4%	4.1%
Image	48.7%	47.7%	2.4%	4.2%
Audio	48.7%	47.0%	2.4%	4.3%

SM7635 E2B average CPU usage during prefill (mean of 64 to 2,048-token inputs) — SM7635 (E2B / Int16) average process CPU usage during prefill (mean of 64 to 2,048-token inputs)

SM7635 E2B CPU usage during image / audio input TTFT — SM7635 (E2B / Int16) process CPU usage during image (left) / audio input (right) TTFT

E4B / Int16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	4.217 tokens/s	6.944 tokens/s	1.65x	37.1%
NPU	3.191 tokens/s	3.163 tokens/s	0.99x	38.6%

SM7635 E4B per-token decode throughput over 1000 generated tokens (CPU / NPU, without / with MTP) — SM7635 (E4B / Int16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	34.773	35.415	38.182	36.076	29.317	27.488
CPU (with MTP)	33.002	34.064	32.739	32.094	29.620	26.869
NPU (without MTP)	20.380	40.495	79.377	82.226	82.662	82.534
NPU (with MTP)	20.623	40.630	80.440	83.119	82.887	82.257

SM7635 E4B prefill throughput by input length (CPU / NPU, without / with MTP) — SM7635 (E4B / Int16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	99.00 s	4.401 tokens/s	—
Image	CPU (with MTP)	103.83 s	6.446 tokens/s	37.6%
Image	NPU (without MTP)	15.15 s	3.140 tokens/s	—
Image	NPU (with MTP)	14.89 s	3.220 tokens/s	40.0%
Audio	CPU (without MTP)	52.73 s	4.055 tokens/s	—
Audio	CPU (with MTP)	50.40 s	7.507 tokens/s	100.0%
Audio	NPU (without MTP)	13.01 s	3.043 tokens/s	—
Audio	NPU (with MTP)	13.19 s	5.503 tokens/s	100.0%

NPU (with MTP) TTFT is 1/6.6 (image) and 1/4.0 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.03x for image input and 1.81x for audio input.

SM7635 E4B VLM / audio input TTFT breakdown (CPU / NPU, without / with MTP) — SM7635 (E4B / Int16) VLM (left) / audio input (right) TTFT breakdown: prompt setup including the encoder, and decoder prefill + first token

SM7635 E4B VLM / audio input decode throughput (CPU / NPU, without / with MTP) — SM7635 (E4B / Int16) VLM (left) / audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	48.3%	46.5%	1.4%	2.0%
Image	48.5%	46.8%	1.7%	2.0%
Audio	48.1%	42.5%	2.0%	2.1%

SM7635 E4B average CPU usage during prefill (mean of 64 to 2,048-token inputs) — SM7635 (E4B / Int16) average process CPU usage during prefill (mean of 64 to 2,048-token inputs)

SM7635 E4B CPU usage during image / audio input TTFT — SM7635 (E4B / Int16) process CPU usage during image (left) / audio input (right) TTFT

Windows

Test Conditions

OS: Windows 11 Pro on SC8380XP, and Windows 11 IoT Enterprise LTSC on QCS6490.
NPU: v1.5.1 MTP models (FP16 / W4 on SC8380XP, Int16 on QCS6490), QAIRT 2.47, context length 8192.
CPU / GPU: The CPU runs with llama.cpp (OpenMP disabled, on the 4 performance cores), with a QAT Q4_0 main model and a Q4_0 GGUF assistant. The GPU uses the llama.cpp OpenCL backend on the SC8380XP Adreno X1-45 (not measured on QCS6490).
Method: Greedy decoding, one run per configuration. Decode, prefill and acceptance rate are defined as on Android.
Image / audio input: One 768x768 image (100 output tokens) and a transcription of one English recording (33 output tokens). The encoders run on the CPU in the CPU and GPU configurations, and on the NPU in the NPU configurations.
CPU usage: Process CPU usage during decode, with all 8 cores fully used as 100%. The CPU uses only the 4 performance cores, so it stays around 50%.

Snapdragon X Plus (SC8380XP / Hexagon v73)

E2B / FP16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	36.097 tokens/s	45.952 tokens/s	1.27x	31.1%
GPU	15.789 tokens/s	7.956 tokens/s	0.50x	39.0%
NPU	7.396 tokens/s	10.725 tokens/s	1.45x	31.2%

Decode is fastest on the CPU. On the GPU, decode became slower with MTP.

Snapdragon X Plus E2B decode throughput (1000 generated tokens, CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) decode throughput (1000 generated tokens)

Snapdragon X Plus E2B per-token decode throughput over 1000 generated tokens (CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	261.792	243.510	228.444	203.791	180.420	162.898
CPU (with MTP)	226.239	209.196	212.324	198.671	179.041	159.733
GPU (without MTP)	191.567	211.200	226.293	210.324	185.268	155.789
GPU (with MTP)	110.794	150.182	181.306	187.165	173.085	149.692
NPU (without MTP)	142.326	268.719	550.238	558.241	556.910	535.660
NPU (with MTP)	146.658	291.834	572.231	550.662	545.243	549.844

For inputs of 128 tokens or more the NPU is the fastest; for a 2,048-token input it is 3.29x the CPU (without MTP). NPU prefill is almost the same with and without MTP.

Snapdragon X Plus E2B prefill throughput (2,048-token input, CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) prefill throughput (2,048-token input)

Snapdragon X Plus E2B prefill throughput by input length (CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	77.44 s	38.097 tokens/s	—
Image	CPU (with MTP)	77.44 s	61.421 tokens/s	43.2%
Image	GPU (without MTP)	77.83 s	17.383 tokens/s	—
Image	GPU (with MTP)	77.82 s	8.451 tokens/s	41.7%
Image	NPU (without MTP)	6.39 s	7.132 tokens/s	—
Image	NPU (with MTP)	5.95 s	11.525 tokens/s	39.9%
Audio	CPU (without MTP)	30.93 s	39.247 tokens/s	—
Audio	CPU (with MTP)	30.75 s	103.169 tokens/s	92.6%
Audio	GPU (without MTP)	30.77 s	16.935 tokens/s	—
Audio	GPU (with MTP)	30.80 s	15.584 tokens/s	92.6%
Audio	NPU (without MTP)	4.21 s	7.341 tokens/s	—
Audio	NPU (with MTP)	4.46 s	19.628 tokens/s	92.6%

NPU TTFT is 1/13.0 (image) and 1/6.9 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.62x for image input and 2.67x for audio input.

Snapdragon X Plus E2B VLM / audio input TTFT breakdown (CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) VLM (left) and audio input (right) TTFT breakdown (prompt setup including the encoder, and decoder prefill plus the first token)

Snapdragon X Plus E2B VLM / audio input decode throughput (CPU / GPU / NPU, without / with MTP) — SC8380XP (E2B / FP16) VLM (left) and audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	GPU (without MTP)	GPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	48.0%	46.1%	6.6%	1.4%	2.8%	4.9%
Image	48.5%	47.7%	6.2%	2.8%	2.9%	5.4%
Audio	49.1%	46.6%	7.4%	1.2%	2.7%	4.9%

The CPU runs on the 4 performance cores, so CPU usage stays around 50%.

Snapdragon X Plus E2B CPU usage during text generation decode — SC8380XP (E2B / FP16) process CPU usage during text generation decode

Snapdragon X Plus E2B average CPU usage during prefill (average over 64 to 2,048-token inputs) — SC8380XP (E2B / FP16) average process CPU usage during prefill (average over 64 to 2,048-token inputs)

Snapdragon X Plus E2B CPU usage during image / audio input TTFT — SC8380XP (E2B / FP16) process CPU usage during image (left) and audio input (right) TTFT

E4B / FP16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	20.209 tokens/s	34.010 tokens/s	1.68x	43.3%
GPU	8.434 tokens/s	4.962 tokens/s	0.59x	38.2%
NPU	4.346 tokens/s	7.172 tokens/s	1.65x	33.4%

Decode is fastest on the CPU. On the GPU, decode became slower with MTP.

Snapdragon X Plus E4B decode throughput (1000 generated tokens, CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) decode throughput (1000 generated tokens)

Snapdragon X Plus E4B per-token decode throughput over 1000 generated tokens (CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	129.117	124.315	119.669	112.318	106.239	99.274
CPU (with MTP)	114.588	115.837	117.360	112.383	104.566	97.730
GPU (without MTP)	101.494	115.771	122.735	118.920	108.919	96.327
GPU (with MTP)	64.261	87.397	102.919	107.840	102.897	92.966
NPU (without MTP)	81.311	161.784	312.190	325.714	318.495	317.629
NPU (with MTP)	81.512	164.540	324.876	326.486	321.507	319.063

For inputs of 128 tokens or more the NPU is the fastest; for a 2,048-token input it is 3.20x the CPU (without MTP). NPU prefill is almost the same with and without MTP.

Snapdragon X Plus E4B prefill throughput (2,048-token input, CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) prefill throughput (2,048-token input)

Snapdragon X Plus E4B prefill throughput by input length (CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	79.04 s	21.437 tokens/s	—
Image	CPU (with MTP)	78.75 s	33.824 tokens/s	37.6%
Image	GPU (without MTP)	79.38 s	9.423 tokens/s	—
Image	GPU (with MTP)	80.72 s	5.596 tokens/s	41.7%
Image	NPU (without MTP)	9.46 s	4.153 tokens/s	—
Image	NPU (with MTP)	8.14 s	7.744 tokens/s	43.4%
Audio	CPU (without MTP)	33.37 s	21.364 tokens/s	—
Audio	CPU (with MTP)	33.17 s	58.900 tokens/s	92.6%
Audio	GPU (without MTP)	33.04 s	9.406 tokens/s	—
Audio	GPU (with MTP)	33.07 s	9.795 tokens/s	92.6%
Audio	NPU (without MTP)	5.83 s	4.183 tokens/s	—
Audio	NPU (with MTP)	5.96 s	12.772 tokens/s	92.6%

NPU TTFT is 1/9.7 (image) and 1/5.6 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.86x for image input and 3.05x for audio input.

Snapdragon X Plus E4B VLM / audio input TTFT breakdown (CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) VLM (left) and audio input (right) TTFT breakdown (prompt setup including the encoder, and decoder prefill plus the first token)

Snapdragon X Plus E4B VLM / audio input decode throughput (CPU / GPU / NPU, without / with MTP) — SC8380XP (E4B / FP16) VLM (left) and audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	GPU (without MTP)	GPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	48.7%	48.1%	4.6%	2.1%	1.8%	3.0%
Image	49.0%	47.9%	5.4%	2.6%	1.8%	3.2%
Audio	48.9%	46.0%	5.6%	2.3%	1.7%	3.2%

The CPU runs on the 4 performance cores, so CPU usage stays around 50%.

Snapdragon X Plus E4B CPU usage during text generation decode — SC8380XP (E4B / FP16) process CPU usage during text generation decode

Snapdragon X Plus E4B average CPU usage during prefill (average over 64 to 2,048-token inputs) — SC8380XP (E4B / FP16) average process CPU usage during prefill (average over 64 to 2,048-token inputs)

Snapdragon X Plus E4B CPU usage during image / audio input TTFT — SC8380XP (E4B / FP16) process CPU usage during image (left) and audio input (right) TTFT

QCS6490 (Hexagon v68)

E2B / Int16

Decode throughput

Path	Without MTP	With MTP	MTP speedup	Acceptance
CPU	8.268 tokens/s	12.551 tokens/s	1.52x	32.4%
NPU	4.326 tokens/s	7.131 tokens/s	1.65x	36.0%

Decode is fastest on the CPU. MTP made decode 1.52x on the CPU and 1.65x on the NPU.

QCS6490 E2B decode throughput (1000 generated tokens, CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) decode throughput (1000 generated tokens)

QCS6490 E2B per-token decode throughput over 1000 generated tokens (CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) per-token decode throughput over 1000 generated tokens

Prefill throughput

Path	64	128	256	512	1024	2048
CPU (without MTP)	69.899	72.120	68.605	64.913	59.277	53.487
CPU (with MTP)	64.226	66.437	59.173	63.920	54.278	54.221
NPU (without MTP)	54.430	101.921	211.349	212.480	214.039	211.759
NPU (with MTP)	54.091	109.916	215.555	214.996	214.028	213.825

For inputs of 128 tokens or more the NPU is the fastest; for a 2,048-token input it is 3.96x the CPU (without MTP). NPU prefill is almost the same with and without MTP.

QCS6490 E2B prefill throughput (2,048-token input, CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) prefill throughput (2,048-token input)

QCS6490 E2B prefill throughput by input length (CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) prefill throughput by input length

VLM (image input) and audio input latency

Input	Path	TTFT	Decode	Acceptance
Image	CPU (without MTP)	84.75 s	8.417 tokens/s	—
Image	CPU (with MTP)	85.63 s	15.303 tokens/s	41.7%
Image	NPU (without MTP)	8.27 s	4.364 tokens/s	—
Image	NPU (with MTP)	7.27 s	7.735 tokens/s	41.7%
Audio	CPU (without MTP)	36.40 s	8.753 tokens/s	—
Audio	CPU (with MTP)	36.81 s	27.060 tokens/s	92.6%
Audio	NPU (without MTP)	6.25 s	4.325 tokens/s	—
Audio	NPU (with MTP)	6.19 s	12.199 tokens/s	92.6%

NPU TTFT is 1/10.2 (image) and 1/5.8 (audio) of the CPU (without MTP). With MTP, NPU decode became 1.77x for image input and 2.82x for audio input, and CPU decode became 1.82x and 3.09x.

QCS6490 E2B VLM / audio input TTFT breakdown (CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) VLM (left) and audio input (right) TTFT breakdown (prompt setup including the encoder, and decoder prefill plus the first token)

QCS6490 E2B VLM / audio input decode throughput (CPU / NPU, without / with MTP) — QCS6490 (E2B / Int16) VLM (left) and audio input (right) decode throughput

CPU usage

Input	CPU (without MTP)	CPU (with MTP)	NPU (without MTP)	NPU (with MTP)
Text	47.9%	47.9%	0.9%	1.4%
Image	47.9%	47.7%	0.9%	1.4%
Audio	47.9%	47.1%	1.0%	1.3%

The CPU runs on the 4 performance cores, so CPU usage stays around 50%.

QCS6490 E2B CPU usage during text generation decode — QCS6490 (E2B / Int16) process CPU usage during text generation decode

QCS6490 E2B average CPU usage during prefill (average over 64 to 2,048-token inputs) — QCS6490 (E2B / Int16) average process CPU usage during prefill (average over 64 to 2,048-token inputs)

QCS6490 E2B CPU usage during image / audio input TTFT — QCS6490 (E2B / Int16) process CPU usage during image (left) and audio input (right) TTFT