Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ZTS: Replace MD5 and SHA256 wit XXH128 #16577

Merged
merged 1 commit into from
Sep 28, 2024
Merged

ZTS: Replace MD5 and SHA256 wit XXH128 #16577

merged 1 commit into from
Sep 28, 2024

Conversation

mcmilk
Copy link
Contributor

@mcmilk mcmilk commented Sep 27, 2024

For data integrity checks as done in ZTS, the verification for unintended data corruption with xxhash128 should be a lot faster and perfectly usable.

I removed the sysutils/coreutils dependency from FreeBSD and added xxhash everywhere.
The binaries md5, md5sum, sha256 and sha256sum are also not needed anymore.

How faster a whole functional testrun will be isn't known. But we can compare this runs of this PR with older ones.

Motivation and Context

Speedup ZTS a bit.

Description

How Has This Been Tested?

Timings for 2x zfs-tests.sh -vK -s 3GB -T rsend are like this:

system xxhash128 md5 / sha256
qemu-x86 (almalinux8) 26m 47s 27m 50s
qemu-x86 (almalinux9) 28m 47s 29m 55s
qemu-x86 (centos-stream9) 29m 32s 29m 53s
qemu-x86 (debian11) 22m 56s 25m 1s
qemu-x86 (debian12) 24m 59s 27m 44s
qemu-x86 (fedora39) 29m 26s 30m 50s
qemu-x86 (fedora40) 27m 19s 29m 52s
qemu-x86 (freebsd13) 22m 43s 21m 56s
qemu-x86 (freebsd13r) 21m 33s 21m 59s
qemu-x86 (freebsd14) 19m 55s 21m 12s
qemu-x86 (freebsd14r) 20m 45s 23m 16s
qemu-x86 (ubuntu20) 22m 50s 25m 5s
qemu-x86 (ubuntu22) 27m 38s 29m 8s
qemu-x86 (ubuntu24) 30m 51s 31m 52s
Setup + Cleanup 32s 31s
Total 5h 56m 33s 6h 16m 4s

Links:

It's not much, but it is just the rsend test, which uses checksums a lot.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@mcmilk
ZTS: Replace MD5 and SHA256 wit XXH128
9a160e5 
For data integrity checks as done in ZTS, the verification for
unintended data corruption with xxhash128 should be a lot faster
and perfectly usable.

Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
@behlendorf behlendorf added Component: Test Suite Indicates an issue with the test framework or a test case Status: Code Review Needed Ready for review and testing labels Sep 27, 2024
@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Sep 28, 2024
@behlendorf behlendorf merged commit b1958b5 into openzfs:master Sep 28, 2024
21 checks passed
@mcmilk mcmilk deleted the xxhash branch September 28, 2024 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@behlendorf behlendorf behlendorf approved these changes

Assignees
No one assigned
Labels
Component: Test Suite Indicates an issue with the test framework or a test case Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mcmilk @behlendorf