A Tantivy tokenizer using Vibrato.
let tokenizer = VibratoTokenizer::new("/path/to/dictionary")?;
let analyzer = TextAnalyzer::from(tokenizer).filter(LowerCaser);
index.tokenizers().register("lang_ja", analyzer);You need to specify a path to the Vibrato's dictionary file.