Skip to content

Fastvalidate integration #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

Fastvalidate integration #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bfffb43
Add SIMD based validate_ascii_fast and validate_utf8_fast
byroot Dec 11, 2019
629653a
Use validate_ascii_fast in coderange_scan
byroot Dec 11, 2019
7056c04
Use validate_utf8_fast in coderange_scan
byroot Dec 11, 2019
b583663
Enable -msse4.1
byroot Dec 11, 2019
64aed65
use AVX functions and add a benchmark
byroot Dec 12, 2019
fb12cbd
Refactor fast encoding check to automatically select the fastest avai…
byroot Dec 12, 2019
37182a0
Use faster is_valid_ascii in rb_enc_cr_str_copy_for_substr and rb_ext…
byroot Dec 12, 2019
f6a57ce
Fix simd_encoding_check.h when only SSE4.1 is available
byroot Dec 12, 2019
30f17d1
More coderange benchmarks
byroot Dec 12, 2019
0ba7d62
Fix fallback version of is_valid_utf8
byroot Dec 12, 2019
a410ce5
More benchmarks and add shortcircuiting logic
byroot Dec 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions benchmark/string_coderange_scan.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
prelude: |
ascii1 = [*"a".."m",*"N".."Z",*"0".."9"].join("")
ascii10 = ascii1 * 10
ascii100 = ascii10 * 10
ascii1000 = ascii100 * 10

utf81 = [*"a".."m",*"N".."Z", "éà€‹›Ç☃"].join("")
utf810 = utf81 * 10
utf8100 = utf810 * 10
utf81000 = utf8100 * 10

invalid_ascii_last = ascii1000 + "☃"
invalid_ascii_first = "☃" + ascii1000
benchmark:
ascii-coderange-scan-1: ascii1.force_encoding(Encoding::UTF_8).valid_encoding?
ascii-coderange-scan-10: ascii10.force_encoding(Encoding::UTF_8).valid_encoding?
ascii-coderange-scan-100: ascii100.force_encoding(Encoding::UTF_8).valid_encoding?
ascii-coderange-scan-1000: ascii1000.force_encoding(Encoding::UTF_8).valid_encoding?

utf8-coderange-scan-1: utf81.force_encoding(Encoding::UTF_8).valid_encoding?
utf8-coderange-scan-10: utf810.force_encoding(Encoding::UTF_8).valid_encoding?
utf8-coderange-scan-100: utf8100.force_encoding(Encoding::UTF_8).valid_encoding?
utf8-coderange-scan-1000: utf81000.force_encoding(Encoding::UTF_8).valid_encoding?

ascii-coderange-scan-first-1: invalid_ascii_first.force_encoding(Encoding::ASCII).valid_encoding?
ascii-coderange-scan-last-1: invalid_ascii_last.force_encoding(Encoding::ASCII).valid_encoding?
1 change: 1 addition & 0 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1513,6 +1513,7 @@ RUBY_DECL_ATTRIBUTE([__nonnull__(n)], [RUBY_FUNC_NONNULL(n,x)], [rb_cv_func_nonn
@%:@define n 1
])

RUBY_APPEND_OPTION(XCFLAGS, -march=native)
RUBY_APPEND_OPTION(XCFLAGS, -DRUBY_EXPORT)

AC_ARG_ENABLE(mathn,
Expand Down
Loading