Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Empty Sector Update proofs (for SnapDeals) #1519

Merged
merged 74 commits into from
Dec 13, 2021
Merged

Add support for Empty Sector Update proofs (for SnapDeals) #1519

merged 74 commits into from
Dec 13, 2021

Commits on Dec 9, 2021

  1. feat: encode/decode/remove_data API with tests

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    46d3676 View commit details
    Browse the repository at this point in the history

  2. feat: add Jake's latest circuit code 

    feat: added some filecoin-proofs types for updated porep_config
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    868cf21 View commit details
    Browse the repository at this point in the history

  3. style: cargo fmt

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    7f4a2e8 View commit details
    Browse the repository at this point in the history

  4. feat: add a generate update proof stub API call

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    aa4353f View commit details
    Browse the repository at this point in the history

  5. feat: start hooking up vanilla merkle proof generation for challenges

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    988e1a3 View commit details
    Browse the repository at this point in the history

  6. feat: add gathering of apex leaves 

    fix: updates required after rebasing against master
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    4eb3bda View commit details
    Browse the repository at this point in the history

  7. feat: formalize the vanilla update and vanilla proof types

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    eb0eb39 View commit details
    Browse the repository at this point in the history

  8. fix: minor updates

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a42e643 View commit details
    Browse the repository at this point in the history

  9. feat: empty-sector-update ProofScheme and CompoundProof

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    659e1d4 View commit details
    Browse the repository at this point in the history

  10. feat: update filecoin-proofs with latest changes

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    8fc78f7 View commit details
    Browse the repository at this point in the history

  11. fix: properly calculate phi using comm_r instead of comm_r_last 

    feat: minor updates/cleanups
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    002c96d View commit details
    Browse the repository at this point in the history

  12. refactor: remove lifetime parameter from EmptySectorUpdate struct

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    13ee866 View commit details
    Browse the repository at this point in the history

  13. feat: complete vanilla proving and verify through tests

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    099c27e View commit details
    Browse the repository at this point in the history

  14. feat: remove t_aux from being required in vanilla encode

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    2aa4e47 View commit details
    Browse the repository at this point in the history

  15. feat: remove t_aux from vanilla PrivateInputs

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    81fa35c View commit details
    Browse the repository at this point in the history

  16. feat EmptySectorUpdateCompound tests

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    bba60d0 View commit details
    Browse the repository at this point in the history

  17. style: clippy

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    3d06fca View commit details
    Browse the repository at this point in the history

  18. style: clippy

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    73ebc64 View commit details
    Browse the repository at this point in the history

  19. fix: CompoundProof::circuit partition-index

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    ee7edb5 View commit details
    Browse the repository at this point in the history

  20. feat: expose some required data through filecoin-proofs 

    feat: minor updates
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    6465426 View commit details
    Browse the repository at this point in the history

  21. feat: add prove and verify API interfaces

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    093de8b View commit details
    Browse the repository at this point in the history

  22. feat: update benches with new porep_config fields

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    06e9ddf View commit details
    Browse the repository at this point in the history

  23. feat: generate empty sector update parameters for testing

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    df9d0b8 View commit details
    Browse the repository at this point in the history

  24. feat: update paramcache for generating empty sector update params

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    7263743 View commit details
    Browse the repository at this point in the history

  25. style: clippy

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    c521ab6 View commit details
    Browse the repository at this point in the history

  26. feat: bump parameter cache version to generate new params

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    ac05dbb View commit details
    Browse the repository at this point in the history

  27. feat: properly wire in prove/verify for empty sector updates

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    cd7bbfc View commit details
    Browse the repository at this point in the history

  28. style: typo update

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    60aa57e View commit details
    Browse the repository at this point in the history

  29. feat: add apex-por gadget tests

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    7ad5030 View commit details
    Browse the repository at this point in the history

  30. fix: high bits should include partition-index

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    9c9b371 View commit details
    Browse the repository at this point in the history

  31. style: rename 'blank' to 'empty'

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    4b893b5 View commit details
    Browse the repository at this point in the history

  32. style: move por_no_challenge_input gadget to storage_proofs_core

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    5605e96 View commit details
    Browse the repository at this point in the history

  33. fix: clippy and rustfmt

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    474ea89 View commit details
    Browse the repository at this point in the history

  34. feat: precompute rhos when encoding/decoding

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    47db059 View commit details
    Browse the repository at this point in the history

  35. feat: pack k and h_select into one circuit pub-input

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    f03d395 View commit details
    Browse the repository at this point in the history

  36. fix: apply some review feedback 

    docs: add comments and update h_select defaults
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    9d6e2a2 View commit details
    Browse the repository at this point in the history

  37. fix: apply more feedback

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a47a50d View commit details
    Browse the repository at this point in the history

  38. feat: bump CI parameter cache version

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    e340d89 View commit details
    Browse the repository at this point in the history

  39. fix: clippy

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    68e7550 View commit details
    Browse the repository at this point in the history

  40. feat: clean-up p_aux/t_aux usage

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    7efb706 View commit details
    Browse the repository at this point in the history

  41. feat: output converted fr bytes directly

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    0f9e3d9 View commit details
    Browse the repository at this point in the history

  42. docs: update incorrect comment

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    00474cf View commit details
    Browse the repository at this point in the history

  43. refactor: add structs for EmptySectorUpdateCircuit's public and priva… 

    …te inputs
    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    3300950 View commit details
    Browse the repository at this point in the history

  44. fix: address review comments

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    77d1927 View commit details
    Browse the repository at this point in the history

  45. fix: bump CI parameter cache version

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    3a0d99c View commit details
    Browse the repository at this point in the history

  46. feat: test challenges against hardcoded vectors

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    85cdaab View commit details
    Browse the repository at this point in the history

  47. feat: update parameter cache version for empty sector updates

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    66dc01f View commit details
    Browse the repository at this point in the history

  48. feat: isolate testing of empty circuit tests to reduce RAM usage

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    743c66a View commit details
    Browse the repository at this point in the history

  49. fix: update clippy settings due to isolated testing

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    63d8afd View commit details
    Browse the repository at this point in the history

  50. fix: updates required after rebase to master 

    style: clippy updates
style: rust fmt
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    d502987 View commit details
    Browse the repository at this point in the history

  51. fix: move comm_c from being a public input and into the proof

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    0e606dd View commit details
    Browse the repository at this point in the history

  52. feat: use poseidon dst when generating randomness

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    9ac1d45 View commit details
    Browse the repository at this point in the history

  53. fix: update challenge test vectors

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    f857f3b View commit details
    Browse the repository at this point in the history

  54. feat: bump ci param-cache version

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    b6ed117 View commit details
    Browse the repository at this point in the history

  55. feat: parallelize vanilla challenge-proof validation

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a826d59 View commit details
    Browse the repository at this point in the history

  56. refactor: improve naming and comments

    @DrPeterVanNostrand @cryptonemo
    DrPeterVanNostrand authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    1266749 View commit details
    Browse the repository at this point in the history

  57. Empty sector update tree r gpu (#1528) 

    * feat: re-factor tree_r_last building for external re-use
* reduce conversions in encode (@dignifiedquire)
* feat: use an enum return type to avoid transmute (unsafe)
* fix: keep new crate version consistent with latest release
* fix: add doc and additional type check
* feat: wrap settings for tree builders in a method
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    71e02fb View commit details
    Browse the repository at this point in the history

  58. fix: ensure data file is >= sector key file 

    This check was loosened since lotus may pad the data file
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    ae8ef3c View commit details
    Browse the repository at this point in the history

  59. feat: add larger commented ignored tests for sector updates

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    0ce29f6 View commit details
    Browse the repository at this point in the history

  60. fix: add partition arg in non-exposed single vanilla partition proof API 

    feat: add and expose an API to prove sector updates with vanilla proofs
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    1ee5d93 View commit details
    Browse the repository at this point in the history

  61. fix: apply missed doc and error handling feedback

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    6d602ae View commit details
    Browse the repository at this point in the history

  62. feat: expose sector update partition proof type

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    04aa4ba View commit details
    Browse the repository at this point in the history

  63. fix: add opencl/cuda feature flags to more storage update spots

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    f201536 View commit details
    Browse the repository at this point in the history

  64. feat: consolidate CPU tree building method

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    879259b View commit details
    Browse the repository at this point in the history

  65. fix: use different method to build binary tree 

    From the lotus side, additional data may be stored after the nodes we
want in the tree, so using a different builder method will allow that
to happen without a panic
    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a9ceb33 View commit details
    Browse the repository at this point in the history

  66. style: fix clippy warning

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a9f5f78 View commit details
    Browse the repository at this point in the history

  67. feat: add optional data mapping of a specified length

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    f3628dd View commit details
    Browse the repository at this point in the history

  68. fix: use proper node count

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    b080f18 View commit details
    Browse the repository at this point in the history

  69. feat: persist p_aux and t_aux into cache dir after encoding/remove

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    105e0ed View commit details
    Browse the repository at this point in the history

  70. fix: ensure data is >= replica in remove data call

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    335f513 View commit details
    Browse the repository at this point in the history

  71. Change H to 1024 

    Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
    @cryptonemo
    Jakub Sztandera authored and cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    4c23b61 View commit details
    Browse the repository at this point in the history

  72. feat: try to split lifecycle upgrade tests from others on OpenCL GPU

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    22f0921 View commit details
    Browse the repository at this point in the history

  73. fix: remove typo

    @cryptonemo
    cryptonemo committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    9e55fa4 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2021

  1. docs: add official external audit result to the repo

    @cryptonemo
    cryptonemo committed Dec 13, 2021
    Configuration menu
    Copy the full SHA
    06d2078 View commit details
    Browse the repository at this point in the history