daac-tools/vibrato

This is stars and forks stats for /daac-tools/vibrato repository. As of 06 May, 2024 this repository has 262 stars and 12 forks.

🎤 vibrato: VIterbi-Based acceleRAted TOkenizer Vibrato is a fast implementation of tokenization (or morphological analysis) based on the Viterbi algorithm. A Python wrapper is also available here. Wasm Demo (takes a little time to load the model.) Features Fast tokenization Vibrato is a Rust reimplementation of the fast tokenizer MeCab, although its implementation has been simplified and optimized for even faster tokenization. Especially for language resources with a large matrix (e.g., unidic-cwj-3.1.1...