Documenting Test cases and Test values for noPCM

[elwazana]

This will involve the writing of a PCM_VVPlan like file for the noPCM example.

[smiths]

Yes, this is a good document to add to the noPCM case study.

[elwazana]

@smiths Just for clarification will I be creating these:

or these:

Also with regards to the P01, P02, etc. I've looked through the tex file for the VVPlan for swhs as a reference, they don't seem to be called.
So is there a format you want me to follow? Should I just do the P0* input files and ignore the temperature1.m, energy1.m, etc.?

[smiths]

@elwazana, it looks to me like you should do both. 😄 We want test cases that test that the expected error messages are generated for abnormal inputs and we want tests cases that verify the functioning of the system for the normal inputs.

As far as the format goes, you are free to make your own choice. Think about writing something in the way that you would find most useful if you were writing (or reviewing) test cases. The rationale for the test case is important information.

I'm not sure what you are asking about the temperature1.m, etc files. I don't like these file names, since they don't have any information in them, but they aren't related to testing. We can just keep this issue focused on testing and worry about the other file names as a separate issue.

[elwazana]

@smiths This is what I've done so far I just wanted to pass it by you to get a check before continuing?
Test_Documentation.pdf

[smiths]

@elwazana, I don't really like the use of multiple fonts. It also appears that the page is landscape; I would prefer portrait. I know you want to fit as much as possible on one row, but I think it would be better if the tables used the paragraph mode for the cells with longer content, so that it just wraps around.

What distinguishes energyWater1 and energyWater2? There isn't enough information here to judge coverage. Where do the correct values come from? Again, there isn't enough information here to verify the test case.

For noPCM there is a closed form solution for everything. The solution can be solved analytically since there is only water. You can delete the FORTRAN tests. There is no point comparing to FORTRAN when we know the exact answer. You should give the closed form answer for T and energy. You could call it something like T_closed, or T_analytical, or T_true. Actually, I like T_true best. We know the true answer, so let us use it. You can then just invoke these closed form solutions in your documentation. If you test at t = 0, you can compare T_true(0) to the calculated value. You don't need to expand out the decimal versions of T_true(t), since your formula provides much clearer documentation.

[elwazana]

@smiths What do you mean where do the correct values come from?

caseStudies/CaseStudies/SolarHeatingWaterOnly/Python/Testing/energyTest.py

Lines 14 to 23 in 80250c2

	def test_E1W(self):
	temp = [40, 41, 42, 43, 44]
	expE = [0, 837095.09369793726918140921263406, 1674190.1873958745383628184252681,
	2511285.2810938118075442276379022, 3348380.3747917490767256368505363]
	eWat = energy.energyWat(temp, self.params)
	self.assertAlmostEqual(eWat[0], expE[0], places=None, msg='energy1Wat: eWat[0]', delta=1e-9)
	self.assertAlmostEqual(eWat[1], expE[1], places=None, msg='energy1Wat: eWat[1]', delta=1e-9)
	self.assertAlmostEqual(eWat[2], expE[2], places=None, msg='energy1Wat: eWat[2]', delta=1e-9)
	self.assertAlmostEqual(eWat[3], expE[3], places=None, msg='energy1Wat: eWat[3]', delta=1e-9)
	self.assertAlmostEqual(eWat[4], expE[4], places=None, msg='energy1Wat: eWat[4]', delta=1e-9)

The inputs are temp (used by eWat to calculate the energy in water) and expE (the correct values that get compared to the eWat values)

Also, for clarification can you please explain what you intend by verifying the test case?

[smiths]

Your two questions are related. It isn't that I doubt the veracity of your expected solutions; it is that I (as the document reviewer) have no way to verify your values. I have to take your word for it that 837095.09369793726918140921263406 is the expected value for the energy for a temperature of 41. I do not know, from reading the documentation, why this is the correct expected value.

All software engineering artifacts need to be verified. @niazim3 is familiar with that idea from taking SFWR ENG 2AA4/COMP SCI 2ME3. We often focus on verifying the code, but requirements need to be verified, design needs to be verified and test plans need to be verified. If the value of 837095.09369793726918140921263406 is given as the expected value, the verifier will have to reproduce all of the work done to originally find this value. It is better to give the information on how to find this value in the documentation.

In your case, you have a formula for calculating energy given the temperature. The documentation should give this formula and then the test case can invoke this formula.

[elwazana]

@smiths I've worked on this format, I'm still trying to find a better purpose description for the energyWater tests along with differentiating their inputs (which seem to just be different inputted initial temperatures).
TestDoc1.pdf

[smiths]

Yes, describing the purpose of the test case is not always easy. However, if there is no reason for the test case, then maybe we don't need it. You are in the difficult situation of "reverse engineering" the rational for the test case. I actually take this as support for my position that the rational needs to be recorded. The verifier of the document shouldn't have to guess why a test case was chosen. The goal of testing is for coverage of the different equivalence classes that cover the input domain.

It might help to distinguish "normal" test cases and "abnormal" test cases. The normal cases are when the inputs are in the usual range. The abnormal cases are extreme cases and error cases. The abnormal cases are partially covered with the test cases for bad input.

For the normal cases you want to include cases that are at the boundaries. For scientific computing, inputs of 0 are often on a boundary. If you have a range of potential inputs, you can test at the extremes of the range and somewhere in the middle.

For the normal case, where the typical values are input, you probably only need one test case.