ailiaTokenizerOpenAddedTokensFileA method

int ailiaTokenizerOpenAddedTokensFileA(
  1. Pointer<AILIATokenizer> net,
  2. Pointer<Char> path
)

Open added tokens file.

Parameters:

  • net: A pointer to the tokenizer instance pointer
  • path: Path for special token file

Returns:

If this function is successful, it returns AILIA_STATUS_SUCCESS , or an error code otherwise.

Open a added tokens file (json). This API only requires for AILIA_TOKENIZER_TYPE_WHISPER or AILIA_TOKENIZER_TYPE_XLM_ROBERTA.

Implementation

int ailiaTokenizerOpenAddedTokensFileA(
  ffi.Pointer<AILIATokenizer> net,
  ffi.Pointer<ffi.Char> path,
) {
  return _ailiaTokenizerOpenAddedTokensFileA(
    net,
    path,
  );
}