Tolerance should be required when verifying output

[bmaclach]

NoPCM and SWHS have a module for verifying that the output has the properties listed in "Properties of a Correct Solution", which in this case is that the output follows the law of conservation of energy (The module is not currently generated in Drasil but it exists in the manual versions). They define a cons_tol quantity for the tolerance to be used with this verification. This is an input to the program but Drasil does not know that it is required, and so will not throw an error if it is forgotten from the input list. Drasil should be updated to recognize this tolerance as required.

The way to do this might be based on requirements. If there is a requirement related to output verification, then a tolerance quantity should be required (whether as an input or a constant).

[smiths]

If there isn't a requirement for output verification, we should definitely add one.

[bmaclach]

@smiths Do you mean for all examples, or just for SWHS/NoPCM?

[smiths]

The requirement should appear in any example where we have properties of a correct solution. However, I don't think we have something filled in for all of the examples. The design pattern all of the examples essentially share is:

• read inputs
• verify inputs
• calculate values
• verify values
• output values

When we have our family approach working, we could certainly have family members that skip the verification. For performance reasons a user might explicitly decide to trust the input.

Something that we have to manually check at the moment, but that I would eventually like us to automatically check - all the "chunks" of knowledge in the document should be used in some way. If we state something, but never use it, then why did we state it. The properties of the correct solution fit in this category. If we state them, but then don't verify that they are achieved, or at least provide the option of verifying that they are achieved, why did we state them?

We haven't been rigorous in the above "test" of whether something should be in the SRS, but we are moving in that way. For instance, we are starting to catch if an assumption is never "invoked" in the SRS document. We shouldn't state an assumption and then never use it. That is why I think we should start having assumptions reference other assumptions.

[bmaclach]

The current state of this issue hasn't changed much from the above discussion. We still don't do anything with the properties of a correct solution during code generation.

A good way to start working towards this could be to start generating an "output verification" function/module that checks the constraints on the output variables. We could re-use a lot of the generator's functions for generating the input verification function.

Some properties of a correct solution can't be captured by our current constraint language, like the one discussed in this issue about conforming to the law of conservation of energy, so some issues related to expanding the constraint language (#1801 seems particularly relevant here) likely need to be tackled before we can really fix this issue.