这是indexloc提供的服务，不要输入任何密码
Skip to content

Add isValidASCII function #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add isValidASCII function #1

wants to merge 7 commits into from

Conversation

@hm-5el
Copy link
Owner

@hm-5el hm-5el commented Aug 15, 2025 •
edited
Loading

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

isValidASCII function added for validating if the set of bytes is correctly ASCII encoded. Returns 1, if the set of bytes is correctly encoded, otherwise 0.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Description:

The idea was inspired by https://github.com/cyb70289/utf8/blob/master/ascii.cpp (same repo as the UTF8 implementation), and extended for more SIMD instruction sets.

Performance Test Results:

  • MacBook Air M4 with 10 cores (ARM_NEON): About 4.225 GB/s
prewarm	0	is_valid_ascii.query0.prewarm0	0	0.11467170715332031
query	0	is_valid_ascii.query0.run0	0	0.11280584335327148
query	0	is_valid_ascii.query0.run1	0	0.11351609230041504
query	0	is_valid_ascii.query0.run2	0	0.11448001861572266
query	0	is_valid_ascii.query0.run3	0	0.11554813385009766
query	0	is_valid_ascii.query0.run4	0	0.11580014228820801
query	0	is_valid_ascii.query0.run5	0	0.11534285545349121
query	0	is_valid_ascii.query0.run6	0	0.1161658763885498
query	0	is_valid_ascii.query0.run7	0	0.11569809913635254
query	0	is_valid_ascii.query0.run8	0	0.1164560317993164
median	0	0.11554813385009766
stage	run	1.151	1.842
  • MacBook Air M4 with 10 cores (without SIMD): About 4.088 GB/s
prewarm	0	is_valid_ascii.query0.prewarm0	0	0.1201939582824707
query	0	is_valid_ascii.query0.run0	0	0.11874604225158691
query	0	is_valid_ascii.query0.run1	0	0.11927914619445801
query	0	is_valid_ascii.query0.run2	0	0.11931800842285156
query	0	is_valid_ascii.query0.run3	0	0.12025189399719238
query	0	is_valid_ascii.query0.run4	0	0.11960005760192871
query	0	is_valid_ascii.query0.run5	0	0.11968708038330078
query	0	is_valid_ascii.query0.run6	0	0.11976122856140137
query	0	is_valid_ascii.query0.run7	0	0.11947393417358398
query	0	is_valid_ascii.query0.run8	0	0.11939716339111328
median	0	0.11947393417358398
stage	run	1.197	1.853
  • MacBook Air M4 with 10 cores (without SIMD and optimization to handle 16 bytes at a time): About 3.877 GB/s
prewarm	0	is_valid_ascii.query0.prewarm0	0	0.12699389457702637
query	0	is_valid_ascii.query0.run0	0	0.1258530616760254
query	0	is_valid_ascii.query0.run1	0	0.12607502937316895
query	0	is_valid_ascii.query0.run2	0	0.12557315826416016
query	0	is_valid_ascii.query0.run3	0	0.12723684310913086
query	0	is_valid_ascii.query0.run4	0	0.12565183639526367
query	0	is_valid_ascii.query0.run5	0	0.1260087490081787
query	0	is_valid_ascii.query0.run6	0	0.12587499618530273
query	0	is_valid_ascii.query0.run7	0	0.12467789649963379
median	0	0.12586402893066406
stage	run	1.135	1.800
  • Intel i5-7200U machine (quite old) with 4 cores (AVX2): About 1.063 GB/s
prewarm 0 is_valid_ascii.query0.prewarm0 0 0.4746079444885254
query 0 is_valid_ascii.query0.run0 0 0.5055644512176514
query 0 is_valid_ascii.query0.run1 0 0.4705343246459961
query 0 is_valid_ascii.query0.run2 0 0.2454357147216797
median 0 0.4705343246459961
stage run 1.775 5.142

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hm-5el