FICHA · AUR
ucto
An advanced rule-based (regular-expression) and unicode-aware tokenizer for various languages. Tokenization is an essential first step in any NLP pipeline.
- nlp-tool
- TERMINAL
- NLP
- LIBRARY
- Launchable
- Runs in terminal
· official+codex · reviewed · Jun 5, 2026 description in en
Natural-language text can be split into tokens with rule-based, Unicode-aware processing for multiple languages. Ucto is useful as an early step in NLP pipelines and corpus processing.
Run it from a shell or call the library tools on text you are allowed to process. Input corpora can contain personal data, so review outputs before sharing them.
How to run
ucto
Commands: ucto
Permissions
Permissions not analysed for this source yet.