Description
spaCy tokenization can be aligned with Rust-backed tokenization results for NLP workflows. Developers and researchers can map token boundaries between systems when preparing training data or model integrations. It is a helper library for text pipelines.