mirror of
https://github.com/junegunn/fzf.git
synced 2026-10-28 08:05:53 +00:00
Junegunn Choi
a650900eda
asciiFuzzyIndex gave up on non-ASCII lines, so every item ran the full score matrix. A []rune is a fixed 4-byte stride, so the SIMD byte scanners can run over it directly: find the low byte, then confirm 4-byte alignment and three zero bytes. Case folding and normalization can turn a non-ASCII rune into the ASCII char being searched, which the scan cannot see. ToChars now flags lines holding such a rune and those keep the old path. Normalization is Latin-only, so Hangul, CJK, Cyrillic, Greek, Hebrew, Arabic, Thai, kana and emoji never set the flag. Chars had no spare padding, so inBytes moves into a flags byte. Measured on 1.4M-line corpora: - Mostly-ASCII paths behind a Hangul prefix: 'conf' 1.8x, 'binutils' 2.9x, 'ltversion' 4.0x, no-match 8.4x - Every line CJK: 17x on both matching and non-matching queries - ASCII input unchanged, non-ASCII patterns not covered yet