toggle menu
ailia_tokenizer
1.6.0.0
JVM
switch theme
search in API
ailia_tokenizer
/
axip.ailia_tokenizer
/
AiliaTokenizer
/
Companion
/
AILIA_TOKENIZER_TYPE_XLM_ROBERTA
AILIA_TOKENIZER_TYPE_XLM_ROBERTA
const
val
AILIA_TOKENIZER_TYPE_XLM_ROBERTA
:
Int
=
2
Tokenizer for XLM-RoBERTa models. Requires a SentencePiece model file.