feat: add shares inclusion proofs #1233

rach-id · 2023-01-13T13:04:34Z

Overview

This PR contains the changes proposed by #1042 and #1072.

The reason for creating a new PR is to target main branch instead of qgb-integration without having to cherry pick all the changes.

Closes: #1071 and #1044.

The other PRs will be closed.

Checklist

New and updated code has appropriate documentation
New and updated code has new and/or updated testing
Required CI checks are passing
Visual proof for any user facing features like CLI or documentation updates
Linked issues closed with keywords

rach-id · 2023-01-13T13:06:13Z

pkg/prove/proof.go

+		return beginShare, endShare, err
+	}
+
+	if len(decodedTx.GetMsgs()) == 0 {


[note to reviewers]
This helper method only proves one message per blob. We add multiple messages later

[blocking question]

This helper method only proves one message per blob. We add multiple messages later

should we check for or something similar then?

Suggested change

if len(decodedTx.GetMsgs()) == 0 {

if len(decodedTx.GetMsgs()) != 1 {

note, preview does not also change the error message below which we'll also have to change should this be accepted

There is a subsequent check:

if len(decodedTx.GetMsgs()) > 1 { return beginShare, endShare, fmt.Errorf("PayForBlob contains multiple messages and this is not currently supported") }

that should catch that

app/test/integration_test.go

rootulp · 2023-01-13T20:21:09Z

app/test/integration_test.go

+		// get the message shares
+		beginMsgShare, endMsgShare, err := prove.MsgSharesPosition(blockRes.Block.Txs[txResp.Index])


[nit] rename message to blob so something like:

// get the blob shares beginBlobShare, endBlobShare, err := prove.BlobSharePositions(blockRes.Block.Txs[txResp.Index])

also consider renaming MsgSharesPosition => BlobSharePositions because the function appears to return two positions. Alternative naming proposal: BlobShareRange

rootulp · 2023-01-13T20:23:08Z

app/test/integration_test.go

+		// verify the message shares proof
+		msgProof, err := node.ProveShares(


[nit] rename message to blob

// verify the blob shares proof blobProof, err := node.ProveShares(

rootulp · 2023-01-13T20:30:17Z

pkg/prove/proof.go

 // include a given txIndex. Returns an error if index is greater than the length
 // of txs.
-func txSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos uint64, err error) {
+func TxSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos uint64, err error) {


[optional][can be a FLUP issue] we may consider renaming to TxSharePositions because this function returns multiple positions. Alternative naming proposal: TxShareRange

What does FLUP mean? :D

FLUP: Follow-Up

Ah, sure #1241, thanks

this func was original done in core before our rather massive refactors to the shares pkg, so just noting mentally that we really need to refactor this one as well. Surprised this works tbh #703

rootulp · 2023-01-13T20:32:46Z

pkg/prove/proof.go

+// where a given message, referenced by its wrapped pfb transaction, was published at.
+// Note: only supports transactions containing a single message
+func MsgSharesPosition(tx types.Tx) (beginShare uint64, endShare uint64, err error) {
+	unwrappedTx, isMalleated := types.UnmarshalIndexWrapper(tx)


[nit] consider avoiding the malleated terminology given there is no more malleation

Suggested change

unwrappedTx, isMalleated := types.UnmarshalIndexWrapper(tx)

indexWrappedTx, isIndexWrapped := types.UnmarshalIndexWrapper(tx)

rootulp · 2023-01-13T20:54:18Z

pkg/prove/querier.go

+// example path for proving the set of shares [3, 5]:
+// custom/shareInclusionProof/3/5
+func QueryShareInclusionProof(_ sdk.Context, path []string, req abci.RequestQuery) ([]byte, error) {
+	// parse the shares range from the path


Suggested change

// parse the shares range from the path

// parse the share range from the path

rootulp · 2023-01-13T20:55:27Z

pkg/prove/querier.go

+// QueryShareInclusionProof defines the logic performed when querying for the inclusion
+// proofs of a set of shares to the data root.
+// the shares range should be appended to the path.


[nit] sentence case godoc comments

Suggested change

// QueryShareInclusionProof defines the logic performed when querying for the inclusion

// proofs of a set of shares to the data root.

// the shares range should be appended to the path.

// QueryShareInclusionProof defines the logic performed when querying for the

// inclusion proofs of a set of shares to the data root. The share range should

// be appended to the path. Example path for proving the set of shares [3, 5]:

// custom/shareInclusionProof/3/5

rootulp · 2023-01-13T20:57:14Z

pkg/prove/querier.go

+	}
+
+	if endingShare < startShare {
+		return nil, fmt.Errorf("ending share %d should be higher than starting share %d", endingShare, startShare)


[nit] the endingShare could be the same as the startingShare so the error message is incorrect

Suggested change

return nil, fmt.Errorf("ending share %d should be higher than starting share %d", endingShare, startShare)

return nil, fmt.Errorf("ending share %d cannot be lower than starting share %d", endingShare, startShare)

rootulp · 2023-01-13T20:57:46Z

pkg/prove/querier.go

+
+	for i, n := range rawShares[startShare:endingShare] {
+		if !bytes.Equal(nID, n.NamespaceID()) {
+			return nil, fmt.Errorf("shares range contain different namespaces: %d, %d %d", nID, n.NamespaceID(), i)


[question] is the index helpful in this error message? If not, proposal to remove. If yes, proposal to prefix it so their isn't a number in the error message without additional context.

Suggested change

return nil, fmt.Errorf("shares range contain different namespaces: %d, %d %d", nID, n.NamespaceID(), i)

return nil, fmt.Errorf("share range contains different namespaces: %d and %d", nID, n.NamespaceID())

rootulp · 2023-01-13T20:59:35Z

testutil/testfactory/blob.go

@@ -26,6 +28,25 @@ func GenerateRandomlySizedBlobs(count, maxBlobSize int) []types.Blob {
 	return blobs
 }

+// generateBlobsWithNamespace generates randomly sized blobs with
+// namespace ID `nID`.
+func GenerateBlobsWithNamespace(count, msgSize int, nID nmtnamespace.ID) types.BlobsByNamespace {


[nit] msg => blob

Suggested change

func GenerateBlobsWithNamespace(count, msgSize int, nID nmtnamespace.ID) types.BlobsByNamespace {

func GenerateBlobsWithNamespace(count, blobSize int, nID nmtnamespace.ID) types.BlobsByNamespace {

MSevey · 2023-01-13T21:29:25Z

app/test/integration_test.go

+	}
+
+	// wait a few blocks to clear the txs
+	for i := 0; i < 20; i++ {


[non-blocking] probably a worthwhile testing util helper

func WaitForNBlocks(num int) { for i := 0; i < n; i++ { require.NoError(s.network.WaitForNextBlock()) } }

Created an issue #1238 since we're using this in many tests. So, it makes more sense to have a separate PR making the changes throughout the whole repo. Thanks a lot for pointing it out 👍

rootulp · 2023-01-16T15:42:16Z

app/test/integration_test.go

@@ -284,3 +284,73 @@ func queryTx(clientCtx client.Context, hashHexStr string, prove bool) (*rpctypes

 	return node.Tx(context.Background(), hash, prove)
 }
+
+func (s *IntegrationTestSuite) TestSharesInclusionProof() {


Suggested change

func (s *IntegrationTestSuite) TestSharesInclusionProof() {

func (s *IntegrationTestSuite) TestShareInclusionProof() {

rootulp · 2023-01-16T15:43:41Z

app/test/integration_test.go

+		require.NoError(err)
+
+		// verify the blob shares proof
+		msgProof, err := node.ProveShares(


Suggested change

msgProof, err := node.ProveShares(

blobProof, err := node.ProveShares(

rootulp · 2023-01-16T15:44:03Z

app/test/integration_test.go

+		require.NoError(txProof.Validate(blockRes.Block.DataHash))
+
+		// get the blob shares
+		beginMsgShare, endMsgShare, err := prove.BlobShareRange(blockRes.Block.Txs[txResp.Index])


Suggested change

beginMsgShare, endMsgShare, err := prove.BlobShareRange(blockRes.Block.Txs[txResp.Index])

beginBlobShare, endBlobShare, err := prove.BlobShareRange(blockRes.Block.Txs[txResp.Index])

rootulp · 2023-01-16T15:44:59Z

pkg/prove/proof.go

@@ -107,6 +117,49 @@ func txSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos
 	return startSharePos, endSharePos, nil
 }

+// BlobShareRange returns the start and end positions for the shares
+// where a given message, referenced by its wrapped pfb transaction, was published at.


Suggested change

// where a given message, referenced by its wrapped pfb transaction, was published at.

// where a given blob, referenced by its wrapped PFB transaction, was published at.

rootulp · 2023-01-16T15:45:07Z

pkg/prove/proof.go

@@ -107,6 +117,49 @@ func txSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos
 	return startSharePos, endSharePos, nil
 }

+// BlobShareRange returns the start and end positions for the shares
+// where a given message, referenced by its wrapped pfb transaction, was published at.
+// Note: only supports transactions containing a single message


Suggested change

// Note: only supports transactions containing a single message

// Note: only supports transactions containing a single blob

rootulp · 2023-01-16T15:48:12Z

pkg/prove/proof.go

+// SharesInclusion generates an nmt inclusion proof for a set of shares to the data root.
+// Expects the share range to be pre-validated.
+// Note: only supports inclusion proofs for shares belonging to the same namespace.
+func SharesInclusion(


[optional] this function name seems a bit ambiguous. Do these naming proposals convey it's behavior any clearer:

GenerateShareInclusionProof

NewShareInclusionProof

rootulp · 2023-01-16T15:50:56Z

pkg/prove/proof_test.go

+	// not setting useShareIndexes because the transactions indexes do not refer
+	// to the messages because the square and transactions were created manually.


[nit] the following line sets useShareIndexes=false so minor improvement

Suggested change

// not setting useShareIndexes because the transactions indexes do not refer

// to the messages because the square and transactions were created manually.

// Explicitly set useShareIndexes=false because the transaction indexes do not refer

// to the blobs because the data square and transactions were created manually.

On a related note, do we want a test that uses useShareIndexes=true considering that will be the default case soon?

rootulp · 2023-01-16T15:58:47Z

pkg/prove/querier.go

+	return rawTxProof, nil
+}
+
+// ParseNamespaceID validates the shares range and returns their namespace ID.


Suggested change

// ParseNamespaceID validates the shares range and returns their namespace ID.

// ParseNamespaceID validates the share range contains only one namespace and returns that namespace ID.

rootulp · 2023-01-16T16:00:04Z

pkg/prove/querier.go

+}
+
+// ParseNamespaceID validates the shares range and returns their namespace ID.
+func ParseNamespaceID(rawShares []shares.Share, startShare int64, endingShare int64) (namespace.ID, error) {


[nit] for consistency, can both params please use the same state/tense?

Suggested change

func ParseNamespaceID(rawShares []shares.Share, startShare int64, endingShare int64) (namespace.ID, error) {

func ParseNamespaceID(rawShares []shares.Share, startShare int64, endShare int64) (namespace.ID, error) {

alternatively: startingShare and endingShare

Accepting this proposal implies correcting the error messages below

rootulp · 2023-01-16T16:01:28Z

testutil/testfactory/blob.go

@@ -26,6 +28,25 @@ func GenerateRandomlySizedBlobs(count, maxBlobSize int) []types.Blob {
 	return blobs
 }

+// generateBlobsWithNamespace generates randomly sized blobs with


Suggested change

// generateBlobsWithNamespace generates randomly sized blobs with

// GenerateBlobsWithNamespace generates blobs with

evan-forbes

nice, thanks @sweexordious! only left a single blocking question, let's try to merge this ASAP

evan-forbes · 2023-01-17T01:00:52Z

app/test/integration_test.go

-		s.T().Skip("skipping block size integration test in short mode.")
+		s.T().Skip("skipping app/test/integration_test in short mode.")


[optional nit]
this feels out of scope of this PR, also what is the ".../integration_test" supposed to refer to? isn't the file named something else?

It was named block_size_test or similar, but since I added the proofs test there too, I changed it to integration_test

evan-forbes · 2023-01-17T01:06:23Z

pkg/prove/proof.go

 // include a given txIndex. Returns an error if index is greater than the length
 // of txs.
-func txSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos uint64, err error) {
+func TxSharePosition(txs types.Txs, txIndex uint64) (startSharePos, endSharePos uint64, err error) {


this func was original done in core before our rather massive refactors to the shares pkg, so just noting mentally that we really need to refactor this one as well. Surprised this works tbh #703

evan-forbes · 2023-01-17T01:09:13Z

pkg/prove/proof.go

+	decoder := encCfg.TxConfig.TxDecoder()
+
+	decodedTx, err := decoder(indexWrappedTx.Tx)


[very optional]
feel free to ignore, but this is slightly more like the other times we use this function

Suggested change

decoder := encCfg.TxConfig.TxDecoder()

decodedTx, err := decoder(indexWrappedTx.Tx)

decodedTx, err := encCfg.TxConfig.TxDecoder()(indexWrappedTx.Tx)

evan-forbes · 2023-01-17T01:12:04Z

pkg/prove/proof.go

+		return beginShare, endShare, err
+	}
+
+	if len(decodedTx.GetMsgs()) == 0 {


[blocking question]

This helper method only proves one message per blob. We add multiple messages later

should we check for or something similar then?

Suggested change

if len(decodedTx.GetMsgs()) == 0 {

if len(decodedTx.GetMsgs()) != 1 {

note, preview does not also change the error message below which we'll also have to change should this be accepted

# Conflicts: # app/test/integration_test.go

rach-id · 2023-01-17T15:17:31Z

Merging 🎉 Thanks a lot everyone for the reviews

## Overview This command verifies that some shares were attested to in the QGB smart contract. Blocked by #1233 Closes #1184 ## Checklist  - [ ] New and updated code has appropriate documentation - [ ] New and updated code has new and/or updated testing - [ ] Required CI checks are passing - [ ] Visual proof for any user facing features like CLI or documentation updates - [ ] Linked issues closed with keywords

@distractedm1nd

refactor: rename stripCompactShares to stripPrefix chore: rename to NewShareInclusionProof feat!: keep track of share indexes when splitting txs fix lint wip: test write tx revert!: multi share commitment (celestiaorg#1275) Closes celestiaorg#1231 feat: Add QGB verification command (celestiaorg#1236) This command verifies that some shares were attested to in the QGB smart contract. Blocked by celestiaorg#1233 Closes celestiaorg#1184  - [ ] New and updated code has appropriate documentation - [ ] New and updated code has new and/or updated testing - [ ] Required CI checks are passing - [ ] Visual proof for any user facing features like CLI or documentation updates - [ ] Linked issues closed with keywords feat: check that `MsgPayForBlob` components are non-zero (celestiaorg#1279) feat: compact share splitter tracks tx share ranges test: TestExport_txKeyToShareIndex extract mergeMaps fix lint test: TxSharePosition test NewTxInclusionProof fix comment delete unused file wip: TestSplitTxs improve largeTx test case fix: shift pfbTxShare range by len of txShares refactor: use css.Count() for end share index refactor: rename to shareRanges feat: enforce that PFBs are always ordered after normal transactions (celestiaorg#1273) Closes: celestiaorg#1230 chore(deps): Bump github.com/celestiaorg/nmt from 0.12.0 to 0.13.0 (celestiaorg#1284) fix: IsCompactShare for PFB tx shares (celestiaorg#1281) Closes celestiaorg#1280 chore: fix naming from `client/cli/wirepayfordata.go` to blob (celestiaorg#1288)  After updating my forks, I've seen that `wirepayfordata` haven't received renaming treatment. Hence this PR solves this little thingy. We can also have naming like `wirepayfordatablob`. Still LMK if you prefer the later   - [x] New and updated code has appropriate documentation - [x] New and updated code has new and/or updated testing - [x] Required CI checks are passing - [x] Visual proof for any user facing features like CLI or documentation updates - [x] Linked issues closed with keywords chore: rename message to blob in QGB verify command (celestiaorg#1289)    - [ ] New and updated code has appropriate documentation - [ ] New and updated code has new and/or updated testing - [ ] Required CI checks are passing - [ ] Visual proof for any user facing features like CLI or documentation updates - [ ] Linked issues closed with keywords sanity check that raw data is in shares fix lint NewTxInclusionProof returns a TxProof fix: lint Update pkg/prove/proof.go Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com> Update pkg/shares/split_compact_shares.go Co-authored-by: CHAMI Rachid <chamirachid1@gmail.com> refactor: remove err return param chore(deps): Bump golangci/golangci-lint-action from 3.3.1 to 3.4.0 (celestiaorg#1283) Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.3.1 to 3.4.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/golangci/golangci-lint-action/releases">golangci/golangci-lint-action's releases</a>.</em></p> <blockquote> <h2>v3.4.0</h2> <h2>What's Changed</h2> <ul> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.42.0 to 5.42.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/601">golangci/golangci-lint-action#601</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.42.0 to 5.42.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/602">golangci/golangci-lint-action#602</a></li> <li>build(deps-dev): bump eslint from 8.27.0 to 8.28.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/604">golangci/golangci-lint-action#604</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.42.1 to 5.43.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/605">golangci/golangci-lint-action#605</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.42.1 to 5.43.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/607">golangci/golangci-lint-action#607</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.43.0 to 5.44.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/609">golangci/golangci-lint-action#609</a></li> <li>build(deps-dev): bump prettier from 2.7.1 to 2.8.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/611">golangci/golangci-lint-action#611</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.43.0 to 5.44.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/610">golangci/golangci-lint-action#610</a></li> <li>build(deps-dev): bump typescript from 4.8.4 to 4.9.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/606">golangci/golangci-lint-action#606</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.9 to 18.11.10 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/614">golangci/golangci-lint-action#614</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.44.0 to 5.45.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/615">golangci/golangci-lint-action#615</a></li> <li>build(deps-dev): bump eslint from 8.28.0 to 8.29.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/617">golangci/golangci-lint-action#617</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.44.0 to 5.45.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/616">golangci/golangci-lint-action#616</a></li> <li>build(deps-dev): bump typescript from 4.9.3 to 4.9.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/619">golangci/golangci-lint-action#619</a></li> <li>build(deps-dev): bump <code>@vercel/ncc</code> from 0.34.0 to 0.36.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/620">golangci/golangci-lint-action#620</a></li> <li>build(deps-dev): bump prettier from 2.8.0 to 2.8.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/622">golangci/golangci-lint-action#622</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.45.0 to 5.46.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/621">golangci/golangci-lint-action#621</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.45.0 to 5.46.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/623">golangci/golangci-lint-action#623</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.46.0 to 5.46.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/625">golangci/golangci-lint-action#625</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.10 to 18.11.17 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/628">golangci/golangci-lint-action#628</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.46.0 to 5.46.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/626">golangci/golangci-lint-action#626</a></li> <li>build(deps-dev): bump eslint from 8.29.0 to 8.30.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/627">golangci/golangci-lint-action#627</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.46.1 to 5.47.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/633">golangci/golangci-lint-action#633</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.46.1 to 5.47.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/634">golangci/golangci-lint-action#634</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.0.6 to 3.1.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/632">golangci/golangci-lint-action#632</a></li> <li>build(deps-dev): bump eslint from 8.30.0 to 8.31.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/635">golangci/golangci-lint-action#635</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.17 to 18.11.18 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/636">golangci/golangci-lint-action#636</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.47.0 to 5.47.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/637">golangci/golangci-lint-action#637</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.47.0 to 5.47.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/639">golangci/golangci-lint-action#639</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.0 to 3.1.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/638">golangci/golangci-lint-action#638</a></li> <li>build(deps): bump json5 from 1.0.1 to 1.0.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/642">golangci/golangci-lint-action#642</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.47.1 to 5.48.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/643">golangci/golangci-lint-action#643</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.47.1 to 5.48.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/644">golangci/golangci-lint-action#644</a></li> <li>build(deps-dev): bump prettier from 2.8.1 to 2.8.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/645">golangci/golangci-lint-action#645</a></li> <li>build(deps-dev): bump eslint-config-prettier from 8.5.0 to 8.6.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/646">golangci/golangci-lint-action#646</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.1 to 3.1.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/647">golangci/golangci-lint-action#647</a></li> <li>Support Caching in Mono Repo by <a href="https://github.com/bbernays"><code>@bbernays</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/629">golangci/golangci-lint-action#629</a></li> <li>build(deps-dev): bump eslint-plugin-import from 2.26.0 to 2.27.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/650">golangci/golangci-lint-action#650</a></li> <li>build(deps-dev): bump prettier from 2.8.2 to 2.8.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/651">golangci/golangci-lint-action#651</a></li> <li>build(deps-dev): bump eslint from 8.31.0 to 8.32.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/652">golangci/golangci-lint-action#652</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.0 to 5.48.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/653">golangci/golangci-lint-action#653</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.0 to 5.48.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/654">golangci/golangci-lint-action#654</a></li> <li>build(deps-dev): bump eslint-plugin-import from 2.27.4 to 2.27.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/655">golangci/golangci-lint-action#655</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.1 to 5.48.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/656">golangci/golangci-lint-action#656</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.1 to 5.48.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/657">golangci/golangci-lint-action#657</a></li> <li>build(deps-dev): bump eslint-plugin-simple-import-sort from 8.0.0 to 9.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/pull/658">golangci/golangci-lint-action#658</a></li> </ul> <h2>New Contributors</h2>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/golangci/golangci-lint-action/commit/08e2f20817b15149a52b5b3ebe7de50aff2ba8c5"><code>08e2f20</code></a> build(deps-dev): bump eslint-plugin-simple-import-sort from 8.0.0 to 9.0.0 (#...</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/8d110786c7428017295d1011939074e4880134c5"><code>8d11078</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.1 to 5.48.2 ...</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/724a5425dbb3605c8193e1236b39684908fe1faf"><code>724a542</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.1 to 5.48.2 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/656">#656</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/ac0edcd804b1b5a5a373d15812e3ca8e3009aaff"><code>ac0edcd</code></a> build(deps-dev): bump eslint-plugin-import from 2.27.4 to 2.27.5 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/655">#655</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/d6404ce2933838656ca2d23ed1d7c9bf7177e38b"><code>d6404ce</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.0 to 5.48.1 ...</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/cb88bde406513ba2b8f95895720d89bf588e2908"><code>cb88bde</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.0 to 5.48.1 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/653">#653</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/f26018a9c0d2934ae3bdaadb0d6bbbbe168f34ac"><code>f26018a</code></a> build(deps-dev): bump eslint from 8.31.0 to 8.32.0 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/652">#652</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/78451d099ca9433b4b6a0b5d9c7a2d7720928976"><code>78451d0</code></a> build(deps-dev): bump prettier from 2.8.2 to 2.8.3 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/651">#651</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/5570e667056be8be4c7e99ba31339d64a2c7ab29"><code>5570e66</code></a> build(deps-dev): bump eslint-plugin-import from 2.26.0 to 2.27.4 (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/650">#650</a>)</li> <li><a href="https://github.com/golangci/golangci-lint-action/commit/1626f2bd943cd202b1338d967340eb6c987e5067"><code>1626f2b</code></a> Support Caching in Mono Repo (<a href="https://github-redirect.dependabot.com/golangci/golangci-lint-action/issues/629">#629</a>)</li> <li>Additional commits viewable in <a href="https://github.com/golangci/golangci-lint-action/compare/v3.3.1...v3.4.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golangci/golangci-lint-action&package-manager=github_actions&previous-version=3.3.1&new-version=3.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> chore!: rename package prove to proof (celestiaorg#1291) Closes celestiaorg#1290 refactor: use WriteTx refactor: remove named return params refactor: use shareRangeOffset update to local celestia-core test: integ tests to use prove=true fix godoc chore: rename sharesProofs to shareProofs Update pkg/shares/share_splitting.go Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com> docs: describe mergeMaps chore: add more test cases for the compact share splitter (celestiaorg#1295) feat: Prevent any errors when creating the data folder (celestiaorg#1303) Hello team, This is an small feature/fix that prevents any error when the container starts and try to create the folder in `${CELESTIA_HOME}/data` using the option `-p` in the `mkdir` command. Thank you!  - [x] New and updated code has appropriate documentation - [x] New and updated code has new and/or updated testing - [x] Required CI checks are passing - [x] Visual proof for any user facing features like CLI or documentation updates - [x] Linked issues closed with keywords chore!: rename MsgPayForBlob to MsgPayForBlobs (celestiaorg#1304) Closes: celestiaorg#1221 Co-authored-by: Rootul P <rootulp@gmail.com> chore: downgrade nmt to 0.12.0 (celestiaorg#1307) Downgrade to NMT 0.12.0 while @distractedm1nd @liamsi investigate an issue in 0.13.0 chore: rename variable to shareProof lint with --verbose flag update to 1.50.1

## Overview This command verifies that some shares were attested to in the QGB smart contract. Blocked by #1233 Closes #1184 ## Checklist  - [ ] New and updated code has appropriate documentation - [ ] New and updated code has new and/or updated testing - [ ] Required CI checks are passing - [ ] Visual proof for any user facing features like CLI or documentation updates - [ ] Linked issues closed with keywords

## Overview This command verifies that some shares were attested to in the QGB smart contract. Blocked by celestiaorg/celestia-app#1233 Closes celestiaorg/celestia-app#1184 ## Checklist  - [ ] New and updated code has appropriate documentation - [ ] New and updated code has new and/or updated testing - [ ] Required CI checks are passing - [ ] Visual proof for any user facing features like CLI or documentation updates - [ ] Linked issues closed with keywords

rach-id added 5 commits January 13, 2023 12:48

feat: add shares inclusion proof support as a custom query

6d70792

test: add shares inclusion proof test

99204a3

test: add shares inclusion proof integration test

6a128f5

fix: use unmarshallIndexWrapper

68cbba3

fix: use new methods after depending on new version + fix tests

b565252

rach-id added enhancement New feature or request x/qgb testing items that are strictly related to adding or extending test coverage labels Jan 13, 2023

rach-id requested a review from rahulghangas January 13, 2023 13:04

rach-id requested a review from evan-forbes as a code owner January 13, 2023 13:04

rach-id self-assigned this Jan 13, 2023

rach-id linked an issue Jan 13, 2023 that may be closed by this pull request

QGB custom query to generate inclusion proofs #1044

Closed

MSevey requested a review from a team January 13, 2023 13:04

rach-id commented Jan 13, 2023

View reviewed changes

This was referenced Jan 13, 2023

Implement shares inclusion proofs e2e #1072

Closed

feat: add shares inclusion proofs #1042

Closed

fix: use positive value for size when generating random transactions

b885e29

rach-id mentioned this pull request Jan 13, 2023

feat: Add QGB verification command #1236

Merged

5 tasks

rootulp reviewed Jan 13, 2023

View reviewed changes

MSevey reviewed Jan 13, 2023

View reviewed changes

chore: apply suggestions from @rootulp

4595dd2

MSevey requested a review from a team January 14, 2023 00:39

rach-id requested a review from rootulp January 14, 2023 00:40

rach-id mentioned this pull request Jan 14, 2023

Create a WaitForBlocks util helper function #1238

Closed

rach-id requested a review from MSevey January 14, 2023 00:47

rach-id mentioned this pull request Jan 16, 2023

Rename TxSharePosition to TxShareRange #1241

Closed

rootulp reviewed Jan 16, 2023

View reviewed changes

rootulp mentioned this pull request Jan 16, 2023

Fix share inclusion proofs for PFB txs #1243

Closed

evan-forbes reviewed Jan 17, 2023

View reviewed changes

rach-id mentioned this pull request Jan 17, 2023

Add a new test for shares proofs that uses share indexes #1250

Open

chore: apply suggestions from @rootulp

508e24f

MSevey requested a review from a team January 17, 2023 10:40

chore: apply suggestions from @evan-forbes

b12d99e

rach-id requested review from rootulp and evan-forbes January 17, 2023 10:47

rach-id added 2 commits January 17, 2023 12:02

Merge branch 'main' into inclusion_proofs_main_copy

9d102da

# Conflicts: # app/test/integration_test.go

fix: update the implementation after merging main

be97907

evan-forbes approved these changes Jan 17, 2023

View reviewed changes

rach-id merged commit 007b350 into celestiaorg:main Jan 17, 2023

	if len(decodedTx.GetMsgs()) == 0 {
	if len(decodedTx.GetMsgs()) != 1 {

		// get the message shares
		beginMsgShare, endMsgShare, err := prove.MsgSharesPosition(blockRes.Block.Txs[txResp.Index])

		// verify the message shares proof
		msgProof, err := node.ProveShares(

	unwrappedTx, isMalleated := types.UnmarshalIndexWrapper(tx)
	indexWrappedTx, isIndexWrapped := types.UnmarshalIndexWrapper(tx)

	// parse the shares range from the path
	// parse the share range from the path

	return nil, fmt.Errorf("ending share %d should be higher than starting share %d", endingShare, startShare)
	return nil, fmt.Errorf("ending share %d cannot be lower than starting share %d", endingShare, startShare)

	return nil, fmt.Errorf("shares range contain different namespaces: %d, %d %d", nID, n.NamespaceID(), i)
	return nil, fmt.Errorf("share range contains different namespaces: %d and %d", nID, n.NamespaceID())

	func GenerateBlobsWithNamespace(count, msgSize int, nID nmtnamespace.ID) types.BlobsByNamespace {
	func GenerateBlobsWithNamespace(count, blobSize int, nID nmtnamespace.ID) types.BlobsByNamespace {

	func (s *IntegrationTestSuite) TestSharesInclusionProof() {
	func (s *IntegrationTestSuite) TestShareInclusionProof() {

	msgProof, err := node.ProveShares(
	blobProof, err := node.ProveShares(

	beginMsgShare, endMsgShare, err := prove.BlobShareRange(blockRes.Block.Txs[txResp.Index])
	beginBlobShare, endBlobShare, err := prove.BlobShareRange(blockRes.Block.Txs[txResp.Index])

	// where a given message, referenced by its wrapped pfb transaction, was published at.
	// where a given blob, referenced by its wrapped PFB transaction, was published at.

	// Note: only supports transactions containing a single message
	// Note: only supports transactions containing a single blob

		// not setting useShareIndexes because the transactions indexes do not refer
		// to the messages because the square and transactions were created manually.

	// ParseNamespaceID validates the shares range and returns their namespace ID.
	// ParseNamespaceID validates the share range contains only one namespace and returns that namespace ID.

	func ParseNamespaceID(rawShares []shares.Share, startShare int64, endingShare int64) (namespace.ID, error) {
	func ParseNamespaceID(rawShares []shares.Share, startShare int64, endShare int64) (namespace.ID, error) {

	// generateBlobsWithNamespace generates randomly sized blobs with
	// GenerateBlobsWithNamespace generates blobs with

		s.T().Skip("skipping block size integration test in short mode.")
		s.T().Skip("skipping app/test/integration_test in short mode.")

		decoder := encCfg.TxConfig.TxDecoder()

		decodedTx, err := decoder(indexWrappedTx.Tx)

feat: add shares inclusion proofs #1233

feat: add shares inclusion proofs #1233

Conversation

rach-id commented Jan 13, 2023

Overview

Checklist

rach-id Jan 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rach-id Jan 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

evan-forbes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rach-id commented Jan 17, 2023

rach-id Jan 13, 2023 •

edited

Loading

rach-id Jan 14, 2023 •

edited

Loading