feat: rebuild tree_r_last trees from a replica file #1170
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a tool for rebuilding tree_r_last trees from a replica file. The intended usage is for replacing existing trees with an updated
rows_to_discard
setting, as trees with different values for that setting are incompatible.OPTIONAL (one time sanity check, but can be skipped entirely): To verify that the tool works as expected, let's use the 'old' default of rows_to_discard as 3 for 32GiB sectors and simply rebuild them to make sure the data produced exactly matches what's already present:
OPTIONAL (continued): After this completes, use md5sum to match the tree_r_last files produced in
/some/new/output/dir
against the ones already present in the path containing the existing tree_r_last files. If these match exactly, the real conversion step is run as follows:Example usage:
Note that in this case,
FIL_PROOFS_ROWS_TO_DISCARD
is changed from 3 to 2. Alternatively, you could unset that env var, as 2 is the new default value.The output trees produced in this step should be bigger, as less data is discarded. While it's recommended that you preserve the old tree_r_last files (just until we're sure everything starts up properly with the new code), the newly produced files should be placed where the 'old' ones were present.