Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R4R: Enforce block maximum gas limit in DeliverTx #2795

Merged
merged 33 commits into from
Nov 26, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2f73cf4
block gas meter working
rigelrozanski Nov 13, 2018
956d351
basic structure in place
rigelrozanski Nov 13, 2018
ebaa394
modified app provider to pass genesis
rigelrozanski Nov 13, 2018
3bf67b6
compiling
rigelrozanski Nov 13, 2018
8069b2b
default infinite block gas meter
rigelrozanski Nov 13, 2018
2446830
examples installing
rigelrozanski Nov 13, 2018
eead278
initial test case
rigelrozanski Nov 13, 2018
7e6fcc0
passing test
rigelrozanski Nov 13, 2018
68e3b9a
only use block gas for deliver
rigelrozanski Nov 14, 2018
0d4dd87
fix baseapp tests
rigelrozanski Nov 14, 2018
5249064
add init chain block gas for gen-txs (all unit tests fixed)
rigelrozanski Nov 14, 2018
7be5179
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 14, 2018
90217e2
docs and pending
rigelrozanski Nov 14, 2018
2a594fe
basic cwgoes comments
rigelrozanski Nov 15, 2018
4818e67
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 16, 2018
56dc236
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 20, 2018
d911565
Fix compile
jaekwon Nov 20, 2018
10bdf8f
Store ConsensusParams to main store
jaekwon Nov 21, 2018
4afd53d
Consume block gas to tx gas limit even upon overconsumption
jaekwon Nov 21, 2018
bd982b1
Merge reference/baseapp and spec/baseapp/WIP_abci_application.md
jaekwon Nov 21, 2018
70e60c2
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 21, 2018
6fd3132
lint fix, merge fix
rigelrozanski Nov 21, 2018
972377c
lint
rigelrozanski Nov 22, 2018
b4b61b8
address some comments while reviewing Jaes work
rigelrozanski Nov 22, 2018
abed373
extra max block gas test at limit
rigelrozanski Nov 22, 2018
2d3e1af
Add demonstrative failing testcase
cwgoes Nov 22, 2018
56fa7dc
fix BlockGasRecovery
rigelrozanski Nov 22, 2018
0861112
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 22, 2018
ce10ef2
replaced proto with codec in baseapp
rigelrozanski Nov 22, 2018
5792e1d
Apply suggestions from code review
alexanderbez Nov 25, 2018
819af35
Final fixes from review
jaekwon Nov 25, 2018
0da2472
Merge remote-tracking branch 'origin/develop' into rigel/deliver-max-gas
rigelrozanski Nov 26, 2018
f12ac43
dep
rigelrozanski Nov 26, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion PENDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ FEATURES
* [app] \#2791 Support export at a specific height, with `gaiad export --height=HEIGHT`.

* SDK
* [simulator] \#2682 MsgEditValidator now looks at the validator's max rate, thus it now succeeds a significant portion of the time
* [simulator] \#2682 MsgEditValidator now looks at the validator's max rate, thus it now succeeds a significant portion of the time
* [core] \#2775 Add deliverTx maximum block gas limit

* Tendermint

Expand Down
141 changes: 110 additions & 31 deletions baseapp/baseapp.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"runtime/debug"
"strings"

"github.com/gogo/protobuf/proto"
"github.com/pkg/errors"

abci "github.com/tendermint/tendermint/abci/types"
Expand All @@ -19,11 +20,8 @@ import (
"github.com/cosmos/cosmos-sdk/version"
)

// Key to store the header in the DB itself.
// Use the db directly instead of a store to avoid
// conflicts with handlers writing to the store
// and to avoid affecting the Merkle root.
var dbHeaderKey = []byte("header")
// Key to store the consensus params in the main store.
var mainConsensusParamsKey = []byte("consensus_params")

// Enum mode for app.runTx
type runTxMode uint8
Expand All @@ -48,9 +46,11 @@ type BaseApp struct {
queryRouter QueryRouter // router for redirecting query calls
txDecoder sdk.TxDecoder // unmarshal []byte into sdk.Tx

anteHandler sdk.AnteHandler // ante handler for fee and auth
// set upon LoadVersion or LoadLatestVersion.
mainKey *sdk.KVStoreKey // Main KVStore in cms

// may be nil
anteHandler sdk.AnteHandler // ante handler for fee and auth
initChainer sdk.InitChainer // initialize state with validators and state blob
beginBlocker sdk.BeginBlocker // logic to run before any txs
endBlocker sdk.EndBlocker // logic to run after all txs, and to determine valset changes
Expand All @@ -66,7 +66,11 @@ type BaseApp struct {
deliverState *state // for DeliverTx
voteInfos []abci.VoteInfo // absent validators from begin block

// minimum fees for spam prevention
// consensus params
// TODO move this in the future to baseapp param store on main store.
consensusParams *abci.ConsensusParams
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved

// spam prevention
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably good to indicate that its local spam prevention, not consensus

minimumFees sdk.Coins

// flag for sealing
Expand All @@ -77,10 +81,6 @@ var _ abci.Application = (*BaseApp)(nil)

// NewBaseApp returns a reference to an initialized BaseApp.
//
// TODO: Determine how to use a flexible and robust configuration paradigm that
// allows for sensible defaults while being highly configurable
// (e.g. functional options).
//
// NOTE: The db is used to store the version number for now.
// Accepts a user-defined txDecoder
// Accepts variable number of option functions, which act on the BaseApp to set configuration choices
Expand All @@ -94,7 +94,6 @@ func NewBaseApp(name string, logger log.Logger, db dbm.DB, txDecoder sdk.TxDecod
queryRouter: NewQueryRouter(),
txDecoder: txDecoder,
}

for _, option := range options {
option(app)
}
Expand Down Expand Up @@ -137,21 +136,23 @@ func (app *BaseApp) MountStore(key sdk.StoreKey, typ sdk.StoreType) {
}

// load latest application version
func (app *BaseApp) LoadLatestVersion(mainKey sdk.StoreKey) error {
// panics if called more than once on a running baseapp
func (app *BaseApp) LoadLatestVersion(mainKey *sdk.KVStoreKey) error {
err := app.cms.LoadLatestVersion()
if err != nil {
return err
}
return app.initFromStore(mainKey)
return app.initFromMainStore(mainKey)
}

// load application version
func (app *BaseApp) LoadVersion(version int64, mainKey sdk.StoreKey) error {
// panics if called more than once on a running baseapp
func (app *BaseApp) LoadVersion(version int64, mainKey *sdk.KVStoreKey) error {
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved
err := app.cms.LoadVersion(version)
if err != nil {
return err
}
return app.initFromStore(mainKey)
return app.initFromMainStore(mainKey)
}

// the last CommitID of the multistore
Expand All @@ -165,13 +166,34 @@ func (app *BaseApp) LastBlockHeight() int64 {
}

// initializes the remaining logic from app.cms
func (app *BaseApp) initFromStore(mainKey sdk.StoreKey) error {
func (app *BaseApp) initFromMainStore(mainKey *sdk.KVStoreKey) error {

rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved
// main store should exist.
// TODO: we don't actually need the main store here
main := app.cms.GetKVStore(mainKey)
if main == nil {
mainStore := app.cms.GetKVStore(mainKey)
if mainStore == nil {
return errors.New("baseapp expects MultiStore with 'main' KVStore")
}

// memoize mainKey
if app.mainKey != nil {
panic("app.mainKey expected to be nil; duplicate init?")
}
app.mainKey = mainKey

// load consensus params from the main store
consensusParamsBz := mainStore.Get(mainConsensusParamsKey)
if consensusParamsBz != nil {
var consensusParams = &abci.ConsensusParams{}
err := proto.Unmarshal(consensusParamsBz, consensusParams)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to use a codec here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure @jaekwon

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switched to codec - Jae can switch back if there is a good reason. Otherwise yeah staying consistent makes sense

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be proto because abci.ConsensusParams are protobuf messages.

if err != nil {
panic(err)
}
app.setConsensusParams(consensusParams)
} else {
// It will get saved later during InitChain.
// TODO assert that InitChain hasn't yet been called.
}

// Needed for `gaiad export`, which inits from store but never calls initchain
app.setCheckState(abci.Header{})

Expand Down Expand Up @@ -220,6 +242,29 @@ func (app *BaseApp) setDeliverState(header abci.Header) {
}
}

// setConsensusParams memoizes the consensus params.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this further alludes to the statement above. I don't see the direct relationship between these params and the ones used via the context. I think at the very least the godoc should allude to what these consensus params are for.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. The types/Context consensus params were never used. We don't need them in the context for now, and even if we need them we'd want to modify them via a keeper, so for now will delete types/Context.ConsensusParams.

func (app *BaseApp) setConsensusParams(consensusParams *abci.ConsensusParams) {
app.consensusParams = consensusParams
}

// setConsensusParams stores the consensus params to the main store.
func (app *BaseApp) storeConsensusParams(consensusParams *abci.ConsensusParams) {
consensusParamsBz, err := proto.Marshal(consensusParams)
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
panic(err)
}
mainStore := app.cms.GetKVStore(app.mainKey)
mainStore.Set(mainConsensusParamsKey, consensusParamsBz)
}

// getMaximumBlockGas gets the maximum gas from the consensus params.
func (app *BaseApp) getMaximumBlockGas() (maxGas uint64) {
if app.consensusParams == nil || app.consensusParams.BlockSize == nil {
return 0
}
return uint64(app.consensusParams.BlockSize.MaxGas)
}

//______________________________________________________________________________

// ABCI
Expand All @@ -242,15 +287,27 @@ func (app *BaseApp) SetOption(req abci.RequestSetOption) (res abci.ResponseSetOp
}

// Implements ABCI
// InitChain runs the initialization logic directly on the CommitMultiStore and commits it.
// InitChain runs the initialization logic directly on the CommitMultiStore.
func (app *BaseApp) InitChain(req abci.RequestInitChain) (res abci.ResponseInitChain) {

// Stash the consensus params in the cms main store and memoize.
if req.ConsensusParams != nil {
app.setConsensusParams(req.ConsensusParams)
app.storeConsensusParams(req.ConsensusParams)
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved
}

// Initialize the deliver state and check state with ChainID and run initChain
app.setDeliverState(abci.Header{ChainID: req.ChainId})
app.setCheckState(abci.Header{ChainID: req.ChainId})

if app.initChainer == nil {
return
}

// add block gas meter for any genesis transactions (allow infinite gas)
app.deliverState.ctx = app.deliverState.ctx.
WithBlockGasMeter(sdk.NewInfiniteGasMeter())
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved

res = app.initChainer(app.deliverState.ctx, req)

// NOTE: we don't commit, but BeginBlock for block 1
Expand Down Expand Up @@ -424,9 +481,20 @@ func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeg
} else {
// In the first block, app.deliverState.ctx will already be initialized
// by InitChain. Context is now updated with Header information.
app.deliverState.ctx = app.deliverState.ctx.WithBlockHeader(req.Header).WithBlockHeight(req.Header.Height)
app.deliverState.ctx = app.deliverState.ctx.
WithBlockHeader(req.Header).
WithBlockHeight(req.Header.Height)
alexanderbez marked this conversation as resolved.
Show resolved Hide resolved
}

// add block gas meter
var gasMeter sdk.GasMeter
if maxGas := app.getMaximumBlockGas(); maxGas > 0 {
gasMeter = sdk.NewGasMeter(maxGas)
} else {
gasMeter = sdk.NewInfiniteGasMeter()
}
app.deliverState.ctx = app.deliverState.ctx.WithBlockGasMeter(gasMeter)

if app.beginBlocker != nil {
res = app.beginBlocker(app.deliverState.ctx, req)
}
Expand Down Expand Up @@ -464,9 +532,10 @@ func (app *BaseApp) CheckTx(txBytes []byte) (res abci.ResponseCheckTx) {

// Implements ABCI
func (app *BaseApp) DeliverTx(txBytes []byte) (res abci.ResponseDeliverTx) {

// Decode the Tx.
var result sdk.Result
var tx, err = app.txDecoder(txBytes)
var result sdk.Result
if err != nil {
result = err.Result()
} else {
Expand Down Expand Up @@ -617,6 +686,12 @@ func (app *BaseApp) runTx(mode runTxMode, txBytes []byte, tx sdk.Tx) (result sdk
ctx := app.getContextForTx(mode, txBytes)
ms := ctx.MultiStore()

// only run the tx if there is block gas remaining
if mode == runTxModeDeliver && ctx.BlockGasMeter().IsOutOfGas() {
result = sdk.ErrOutOfGas("no block gas left to run tx").Result()
rigelrozanski marked this conversation as resolved.
Show resolved Hide resolved
return
}

defer func() {
if r := recover(); r != nil {
switch rType := r.(type) {
Expand All @@ -633,6 +708,18 @@ func (app *BaseApp) runTx(mode runTxMode, txBytes []byte, tx sdk.Tx) (result sdk
result.GasUsed = ctx.GasMeter().GasConsumed()
}()

// If BlockGasMeter() panics it will be caught by the above recover and
// return an error - in any case BlockGasMeter will consume gas past
// the limit.
// NOTE: this must exist in a separate defer function for the
// above recovery to recover from this one
defer func() {
if mode == runTxModeDeliver {
ctx.BlockGasMeter().ConsumeGas(
ctx.GasMeter().GasConsumedToLimit(), "block gas meter")
}
}()

var msgs = tx.GetMsgs()
if err := validateBasicTxMsgs(msgs); err != nil {
return err.Result()
Expand Down Expand Up @@ -704,14 +791,6 @@ func (app *BaseApp) EndBlock(req abci.RequestEndBlock) (res abci.ResponseEndBloc
// Implements ABCI
func (app *BaseApp) Commit() (res abci.ResponseCommit) {
header := app.deliverState.ctx.BlockHeader()
/*
// Write the latest Header to the store
headerBytes, err := proto.Marshal(&header)
if err != nil {
panic(err)
}
app.db.SetSync(dbHeaderKey, headerBytes)
*/

// Write the Deliver state and commit the MultiStore
app.deliverState.ms.Write()
Expand Down
Loading