Smoke test should abort early if the run fails

[phil-blain]

The smoke test does not exit with failure when the CICE run fails, it simply write the FAIL result to the test_output file:

CICE/configuration/scripts/tests/test_smoke.script

Lines 6 to 7 in 1c6c225

	./cice.run
	set res="$status"

[...]

CICE/configuration/scripts/tests/test_smoke.script

Lines 23 to 29 in 1c6c225

	set grade = FAIL
	if ( $res == 0 ) then
	set grade = PASS
	endif
	echo "$grade ${ICE_TESTNAME} run ${ttimeloop} ${tdynamics} ${tcolumn}" >> ${ICE_CASEDIR}/test_output
	echo "$grade ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output

In contrast, the restart test does exit 99:

CICE/configuration/scripts/tests/test_restart.script

Lines 10 to 22 in 1c6c225

	./cice.run
	set res="$status"
	if ( $res != 0 ) then
	mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev
	cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} run" >! ${ICE_CASEDIR}/test_output
	mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev
	cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} test " >! ${ICE_CASEDIR}/test_output
	rm -f ${ICE_CASEDIR}/test_output.prev
	echo "FAIL ${ICE_TESTNAME} run" >> ${ICE_CASEDIR}/test_output
	echo "FAIL ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output
	exit 99
	endif

This makes more sense in my opinion. If the initial run fails it makes no sense to go on and try to do the baseline generation, baseline comparing and BFB compare step, no ?

This can lead to misleading "missing data" results when in fact it is the data for the current test that is missing (because the run failed) and not the data for the run which we are comparing against.

[apcraig]

Thanks for catching this @phil-blain. What you propose makes sense. I can include this fix in my next PR unless you create a PR first.

[phil-blain]

You can go ahead. Might be worth it to check the other tests at the same time. From a quick look the decomp, logbfb and unittest tests have the same behaviour.