Microsoft CCF based PDO transaction processor #259

prakashngit · 2020-04-07T04:39:56Z

This PR introduces two things:

An abstraction for ledger APIs so that we can use more than just Sawtooth as the ledger choice
Support for Microsoft CCF as a supported ledger for PDO. The PDO/CCF combo must be treated as an experimental effort at present. There is no docker support yet for PDO/CCF & we also haven't added any automated tests for PDO/CCF. The integration currently supports PDO in SIMULATE mode & CCF in virtual mode (meaning virtual enclaves for both PDO & CCF). Support for HW mode will be added soon. This PR integrates PDO with CCF tag 0.7.1. There are several enhancements that will be rolled out in subsequent PRs. These will be tracked via git issues.

Signed-off-by: prakashngit prakash.narayana.moorthy@intel.com

g2flyer · 2020-04-08T00:16:38Z

fyi: at least as far as SGX HW-mode in docker is concerned, this PR still does run nicely with the standard tests ...

masomel

Looks good so far. I do think the documentation needs some polishing. I'll be reviewing the files in batches.

README.md

build/__tools__/run-tests.sh

ccf_transaction_processor/Readme.md

bvavala

Huge PR (but nice!)

some naming convention issues
missing comments
and some I believe unnecessary message fields (or perhaps I just need a clarification)

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

python/pdo/contract/state.py

python/pdo/submitter/ccf/ccf_submitter.py

python/pdo/submitter/sawtooth/sawtooth_submitter.py

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

python/pdo/submitter/ccf/ccf_submitter.py

masomel

Mostly requesting naming changes and additional in-code documentation for the CCF TP. I do have a couple of questions about the JsonRPC fields for certain operations.

ccf_transaction_processor/transaction_processor/contract_registry.h

ccf_transaction_processor/transaction_processor/enclave_registry.h

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

ccf_transaction_processor/transaction_processor/pdo_tp.h

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

ccf_transaction_processor/transaction_processor/verify_signatures.cpp

docs/host_install.md

docs/install.md

python/pdo/submitter/ccf/ccf_submitter.py

python/pdo/submitter/sawtooth/sawtooth_submitter.py

python/pdo/submitter/submitter.py

…awtooth Submitter. Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com> Microsfot CCF based transaction processor for PDO, in addition to HL Sawtooth based TP. Commit includes client pieces as well as C++ code for the CCF-based PDO-TP. The transaction processor was desgined for CCF release 0.7.1. CCF 0.7.1 is included as a submodule. Certain python modules from this submodule are used as part of the client construction. CF based PDO-TP includes support for proof of commit - meaining, registry reads from ledger will be signed by CCF service which can then be used to construct offline-verifiable proof of txn commits. Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

lgtm-com · 2020-04-17T17:17:09Z

This pull request introduces 16 alerts and fixes 13 when merging 4ba4cc1 into c58d420 - view on LGTM.com

new alerts:

7 for Unused import
5 for Explicit export is not defined
2 for Unused local variable
1 for Use of the return value of a procedure
1 for Illegal raise

fixed alerts:

9 for Unused import
4 for Explicit export is not defined

masomel

Still some concerns about handling erroneous requests in the TP, and keeping the JsonRPC interface more consistent with Sawtooth protos.

EDIT: Not sure why some of my comments are appearing as resolved. Please do take a look. Also, I'm wondering if this has to do with comments being marked as resolved previously? Not sure why github doesn't automatically re-open those threads.

ccf_transaction_processor/Readme.md

ccf_transaction_processor/transaction_processor/contract_registry.h

ccf_transaction_processor/transaction_processor/enclave_registry.h

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

python/pdo/submitter/ccf/ccf_submitter.py

ccf_transaction_processor/transaction_processor/pdo_tp.cpp

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

lgtm-com · 2020-04-18T08:25:00Z

This pull request introduces 17 alerts and fixes 13 when merging 4cac5c5 into c58d420 - view on LGTM.com

new alerts:

8 for Unused import
5 for Explicit export is not defined
2 for Unused local variable
1 for Use of the return value of a procedure
1 for Illegal raise

fixed alerts:

9 for Unused import
4 for Explicit export is not defined

cmickeyb · 2020-04-20T17:19:08Z

couple requests... first @g2flyer @bvavala and @masomel please mark issues resolved when appropriate & then approve when all of your questions are satisfied.

@prakashngit please review the LGTM analysis. Several of the issues can/should be addressed.

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

lgtm-com · 2020-04-21T05:45:52Z

This pull request introduces 5 alerts and fixes 13 when merging d84e949 into c58d420 - view on LGTM.com

new alerts:

5 for Explicit export is not defined

fixed alerts:

9 for Unused import
4 for Explicit export is not defined

cmickeyb · 2020-04-22T16:26:10Z

There are still a number of outstanding issues here. I believe they have been (or will be) captured as issues where we can continue the discussion on a case by case basis.

prakashngit requested a review from cmickeyb April 7, 2020 04:43

prakashngit assigned cmickeyb Apr 7, 2020

prakashngit requested a review from g2flyer April 7, 2020 04:43

prakashngit changed the title ~~Ledger Submitter Abstractions to Integrate CCF submitter along with S…~~ Microsoft CCF based PDO transaction processor Apr 7, 2020

prakashngit requested review from bvavala and masomel April 7, 2020 04:44

cmickeyb added the enhancement New feature or request label Apr 7, 2020

masomel suggested changes Apr 8, 2020

View reviewed changes

bvavala requested changes Apr 10, 2020

View reviewed changes

masomel suggested changes Apr 13, 2020

View reviewed changes

masomel reviewed Apr 14, 2020

View reviewed changes

python/pdo/submitter/ccf/ccf_submitter.py Show resolved Hide resolved

masomel reviewed Apr 14, 2020

View reviewed changes

python/pdo/submitter/ccf/ccf_submitter.py Show resolved Hide resolved

masomel reviewed Apr 14, 2020

View reviewed changes

python/pdo/submitter/sawtooth/sawtooth_submitter.py Show resolved Hide resolved

masomel reviewed Apr 14, 2020

View reviewed changes

python/pdo/submitter/submitter.py Show resolved Hide resolved

prakashngit added 2 commits April 16, 2020 09:56

Rebasing on top of master, addressing review comments

4ba4cc1

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

prakashngit force-pushed the ccf_pdo_transaction_processor_simulate_mode branch from 1a48b35 to 4ba4cc1 Compare April 17, 2020 17:09

prakashngit requested review from masomel and bvavala April 17, 2020 17:13

masomel reviewed Apr 17, 2020

View reviewed changes

Fix for bug in git issue 260 & addressing some revie comments

4cac5c5

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

bvavala approved these changes Apr 20, 2020

View reviewed changes

masomel approved these changes Apr 21, 2020

View reviewed changes

fixes for lgtm alerts

d84e949

Signed-off-by: prakashngit <prakash.narayana.moorthy@intel.com>

cmickeyb approved these changes Apr 22, 2020

View reviewed changes

cmickeyb merged commit 1d73f8e into hyperledger-labs:master Apr 22, 2020

This was referenced Apr 24, 2020

Clarify behavior of add_enclave transaction #270

Open

Include interpreter identity in enclave registration #271

Open

Clarify behavior of get_*_info RPCs #272

Open

prakashngit deleted the ccf_pdo_transaction_processor_simulate_mode branch May 20, 2020 21:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Microsoft CCF based PDO transaction processor #259

Microsoft CCF based PDO transaction processor #259

prakashngit commented Apr 7, 2020 •

edited

Loading

g2flyer commented Apr 8, 2020

masomel left a comment

bvavala left a comment

masomel left a comment

lgtm-com bot commented Apr 17, 2020

masomel left a comment •

edited

Loading

lgtm-com bot commented Apr 18, 2020

cmickeyb commented Apr 20, 2020

lgtm-com bot commented Apr 21, 2020

cmickeyb commented Apr 22, 2020

Microsoft CCF based PDO transaction processor #259

Microsoft CCF based PDO transaction processor #259

Conversation

prakashngit commented Apr 7, 2020 • edited Loading

g2flyer commented Apr 8, 2020

masomel left a comment

Choose a reason for hiding this comment

bvavala left a comment

Choose a reason for hiding this comment

masomel left a comment

Choose a reason for hiding this comment

lgtm-com bot commented Apr 17, 2020

masomel left a comment • edited Loading

Choose a reason for hiding this comment

lgtm-com bot commented Apr 18, 2020

cmickeyb commented Apr 20, 2020

lgtm-com bot commented Apr 21, 2020

cmickeyb commented Apr 22, 2020

prakashngit commented Apr 7, 2020 •

edited

Loading

masomel left a comment •

edited

Loading