feat: RocksDB storage and self-contained RevIndex with internal storage #3250
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## latest #3250 +/- ##
==========================================
+ Coverage 86.74% 86.75% +0.01%
==========================================
Files 136 137 +1
Lines 15873 15920 +47
Branches 2728 2728
==========================================
+ Hits 13769 13812 +43
- Misses 1795 1799 +4
Partials 309 309
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
(may I suggest renaming this to something like "support more flexible RocksDB sketch storage, including internal stores?" :) |
luizirber
force-pushed
the
lirber/rocksdb_storage
branch
from
c4da16d
to
6684126
Compare
luizirber
force-pushed
the
lirber/rocksdb_storage
branch
from
e57927d
to
dc2dc3c
Compare
luizirber
changed the title
luizirber
force-pushed
the
lirber/rocksdb_storage
branch
from
2a760b5
to
600915a
Compare
luizirber
force-pushed
the
lirber/rocksdb_storage
branch
from
347029e
to
cb14921
Compare
luizirber
added a commit
that referenced
this pull request
Jul 31, 2024
Minor new features: - RocksDB storage and self-contained RevIndex with internal storage (#3250) Bug fixes: Cleanup and documentation updates: - Update dev env instructions (#2990) Developer updates: - Enable codspeed for Rust perf tracking (#3231) - bump cibuildwheel for linux fixes, move wasm to use cibuildwheel too (#3241) Dependabot updates: - Break ahash dep cycle by bumping to 0.7.8, nix flake update (#3243) - pre-commit autoupdate (#3270) (#3260) (#3255) (#3238) - Bump DeterminateSystems/nix-installer-action from 12 to 13 (#3259) - Update pytest requirement from <8.3.0,>=6.2.4 to >=6.2.4,<8.4.0 (#3258) - Bump thiserror from 1.0.62 to 1.0.63 (#3257) - Bump thiserror from 1.0.61 to 1.0.62 (#3254) - Bump roaring from 0.10.5 to 0.10.6 (#3245) - Bump serde from 1.0.203 to 1.0.204 (#3244) - Bump counter from 0.5.7 to 0.6.0 (#3235) - Bump log from 0.4.21 to 0.4.22 (#3236) - Bump serde_json from 1.0.117 to 1.0.120 (#3234)
Merged
luizirber
added a commit
that referenced
this pull request
Jul 31, 2024
Minor new features: - RocksDB storage and self-contained RevIndex with internal storage (#3250) Bug fixes: - Break ahash dep cycle by bumping to 0.7.8, nix flake update (#3243) Cleanup and documentation updates: - Update dev env instructions (#2990) Developer updates: - Enable codspeed for Rust perf tracking (#3231) - bump cibuildwheel for linux fixes, move wasm to use cibuildwheel too (#3241) Dependabot updates: - Bump actions/checkout from 3 to 4 (#3265) - Bump moonrepo/setup-rust from 0 to 1 (#3266) - Bump CodSpeedHQ/action from 2 to 3 (#3264) - pre-commit autoupdate (#3270) (#3260) (#3255) (#3238) - Bump DeterminateSystems/nix-installer-action from 12 to 13 (#3259) - Update pytest requirement from <8.3.0,>=6.2.4 to >=6.2.4,<8.4.0 (#3258) - Bump thiserror from 1.0.62 to 1.0.63 (#3257) - Bump thiserror from 1.0.61 to 1.0.62 (#3254) - Bump roaring from 0.10.5 to 0.10.6 (#3245) - Bump serde from 1.0.203 to 1.0.204 (#3244) - Bump counter from 0.5.7 to 0.6.0 (#3235) - Bump log from 0.4.21 to 0.4.22 (#3236) - Bump serde_json from 1.0.117 to 1.0.120 (#3234)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement a RocksDB storage for making a self-contained RevIndex (containing both the revindex and the sigs needed for gather) and support more flexible RocksDB sketch storage.
rkyv, make it optionalstorage::rocksdbmoduleprepare_for_bulk_load, it is a footgun for large index constructionsourmash::storage::StorageErrornon-exhaustiveStorageInfo(rust lints beta CI check is failing onlatest#3261)