Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimal type encoding for DA layer storage #1605

Closed
4 tasks
Dentosal opened this issue Jan 18, 2024 · 1 comment
Closed
4 tasks

Optimal type encoding for DA layer storage #1605

Dentosal opened this issue Jan 18, 2024 · 1 comment
Assignees
@Dentosal @xgreenx
Labels
epic An epic is a high-level master issue for large pieces of work.

Comments

@Dentosal
Copy link
Member

Dentosal commented Jan 18, 2024 •
edited
Loading

Steps to take:

  • Reference-based compaction
  • TxId to TxPointer conversions

Possible improvments:

  • Template-based tx storage
  • Standard compression algorithms
@Dentosal Dentosal self-assigned this Jan 18, 2024
@xgreenx xgreenx mentioned this issue Jan 22, 2024
Merged
6 tasks
@xgreenx xgreenx added the epic An epic is a high-level master issue for large pieces of work. label Feb 4, 2024
@Dentosal Dentosal mentioned this issue Aug 27, 2024
Merged
@xgreenx xgreenx self-assigned this Sep 4, 2024
@xgreenx
Copy link
Collaborator

xgreenx commented Sep 11, 2024

We can consider it solved for now=) FuelLabs/fuel-vm#670

@xgreenx xgreenx closed this as completed Sep 11, 2024
xgreenx added a commit that referenced this issue Oct 5, 2024
@Dentosal @rymnc
@xgreenx @rafal-ch
DA compression (#1609)
1ec05a9 
Related #1605. VM PR
FuelLabs/fuel-vm#670.

This PR adds DA compression crate for Fuel blocks, performed upon block
creation. The compressed blocks are stored into the offchain database
and can be fetched using the GraphQL API.

## Note for reviewers

To keep this reasonably compact, decompression support is not included
in this PR, and will be done as a follow-up. As a result, the full data
roundtrip testing is not part of this PR. There's no proof here that
compression of full blocks is reversible.

## TODO
#### Features
- [x] Temporal registry db support
- [x] Optimize temporal registry eviction implementation
- [x] Implement TxId ↔ TxPointer lookups
- [x] Integrate with the block committer (GraphQL interface, probably)
#### Tests
- [x] compressed blocks are available from non-block-producer nodes
- [ ] e2e test for the full decompression cycle (moved to a follow-up)


## Follow-up issues
- Sync the node from L1:
#2208
- Decompression roudntrip tests
#2238
- Figure out which cache eviction algorithm/behavior is wanted:
#2231
- Figure out if we need to remove the compressed blocks from the db
after a while
- Merkle roots for fraud proofs
#2232

---------

Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>
Co-authored-by: Green Baneling <XgreenX9999@gmail.com>
Co-authored-by: Rafał Chabowski <88321181+rafal-ch@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Dentosal Dentosal

@xgreenx xgreenx

Labels
epic An epic is a high-level master issue for large pieces of work.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Dentosal @xgreenx