commit | d6a6eb848214135095e9034c24de748c4dfcbe53 | [log] [tgz] |
---|---|---|
author | Jonas Heinrich <Jonas.Heinrich@sap.com> | Tue Apr 16 09:50:57 2024 |
committer | Jonas Heinrich <Jonas.Heinrich@sap.com> | Tue Apr 16 10:59:44 2024 |
tree | 9e7617d2559e7fefae21aaed7765e786bad7f5dd | |
parent | baab923fa5d1bfbe24d3ea825d625300f700f7a0 [diff] |
Modify benchmarks to compare against stdlib functions This commit refactors and expands the microbenchmarks in order to evaluate the performance hit of handling full unicode. It is expected that `unicode-segmentation`'s functions are slower since they consider graphemes, the question is just how much. - bump criterion dependency - rename benchmarks to remove unicode/grapheme relationship - move benchmarks into benchmark group - add scalar versions with stdlib "equivalents" (scalars)
Iterators which split strings on Grapheme Cluster or Word boundaries, according to the Unicode Standard Annex #29 rules.
use unicode_segmentation::UnicodeSegmentation; fn main() { let s = "a̐éö̲\r\n"; let g = s.graphemes(true).collect::<Vec<&str>>(); let b: &[_] = &["a̐", "é", "ö̲", "\r\n"]; assert_eq!(g, b); let s = "The quick (\"brown\") fox can't jump 32.3 feet, right?"; let w = s.unicode_words().collect::<Vec<&str>>(); let b: &[_] = &["The", "quick", "brown", "fox", "can't", "jump", "32.3", "feet", "right"]; assert_eq!(w, b); let s = "The quick (\"brown\") fox"; let w = s.split_word_bounds().collect::<Vec<&str>>(); let b: &[_] = &["The", " ", "quick", " ", "(", "\"", "brown", "\"", ")", " ", "fox"]; assert_eq!(w, b); }
unicode-segmentation does not depend on libstd, so it can be used in crates with the #![no_std]
attribute.
You can use this package in your project by adding the following to your Cargo.toml
:
[dependencies] unicode-segmentation = "1.10.1"
#[inline]
opportunities, resulting in 15-40% performance improvement.GraphemeCursor
API allows random access and bidirectional iteration.as_str
methods to the iterator types.