Adding Matt's documentation.

_sources/cice_3_user_guide.rst.txt

@@ -2300,6 +2300,55 @@ Implementation notes: 1) Provide a pass/fail on each of the confidence
 intervals, 2) Facilitate output of a bitmap for each test so that
 locations of failures can be identified.
 
+~~~~~~~~~~~~~~~~~~~~
+CICE Test Reporting
+~~~~~~~~~~~~~~~~~~~~
+
+The CICE testing scripts have the capability of posting the test results 
+to an online dashboard, located `on CDash <http://my.cdash.org/index.php?project=myCICE>`_.  
+There are 2 options for posting CICE results to CDash: 1) The automated 
+script, 2) The manual method.
+
+*****************
+Automatic Script
+*****************
+
+To automatically run the CICE tests, and post the results to the CICE Cdash dashboard,
+users need to copy and run the ``run.suite`` script:
+
+.. code-block:: bash
+
+  cp configuration/scripts/run.suite .
+  ./run.suite -m <machine> -testid <test_id> -bc <baseline_to_compare> -bg <baseline_to_generate>
+
+The run.suite script does the following:
+
+- Creates a fresh clone of the CICE-Consortium repository
+- ``cd`` to cloned repo
+- run ``create.case`` to generate the base_suite directories.  The output 
+  is piped to ``log.suite``
+- Running ``create.case`` submits each individual job to the queue.  
+- ``run.suite`` monitors the queue manager to determine when all jobs have 
+  finished (pings the queue manager once every 5 minutes).
+- Once all jobs complete, cd to base_suite directory and run ``./results.csh``
+- Run ``ctest -S steer.cmake`` in order to post the test results to the CDash dashboard
+
+*****************
+Manual Method
+*****************
+
+To manually run the CICE tests and post the results to the CICE CDash dashboard,
+users essentially just need to perform all steps available in run.suite, detailed below:
+
+- Pass the ``-report`` flag to create.case when running the ``base_suite`` test suite.
+  The ``-report`` flag copies the required CTest / CDash scripts to the suite 
+  directory.
+- ``create.case`` compiles the CICE code, and submits all of the jobs to the 
+  queue manager.  
+- After every job has been submitted and completed, ``cd`` to the suite directory.
+- Parse the results, by running ``./results.csh``.
+- Run the CTest / CDash script ``ctest -S steer.cmake``.
+
 .. _tabnamelist:
 
 -------------------------

cice_3_user_guide.html

@@ -2169,6 +2169,49 @@ <h4>3.6.5.3. Practical Testing Procedure<a class="headerlink" href="#practical-t
 locations of failures can be identified.</p>
 </div>
 </div>
+<div class="section" id="cice-test-reporting">
+<h3>3.6.6. CICE Test Reporting<a class="headerlink" href="#cice-test-reporting" title="Permalink to this headline">¶</a></h3>
+<p>The CICE testing scripts have the capability of posting the test results
+to an online dashboard, located <a class="reference external" href="http://my.cdash.org/index.php?project=myCICE">on CDash</a>.
+There are 2 options for posting CICE results to CDash: 1) The automated
+script, 2) The manual method.</p>
+<div class="section" id="automatic-script">
+<h4>3.6.6.1. Automatic Script<a class="headerlink" href="#automatic-script" title="Permalink to this headline">¶</a></h4>
+<p>To automatically run the CICE tests, and post the results to the CICE Cdash dashboard,
+users need to copy and run the <code class="docutils literal"><span class="pre">run.suite</span></code> script:</p>
+<div class="highlight-bash"><div class="highlight"><pre><span></span>cp configuration/scripts/run.suite .
+./run.suite -m &lt;machine&gt; -testid &lt;test_id&gt; -bc &lt;baseline_to_compare&gt; -bg &lt;baseline_to_generate&gt;
+</pre></div>
+</div>
+<p>The run.suite script does the following:</p>
+<ul class="simple">
+<li>Creates a fresh clone of the CICE-Consortium repository</li>
+<li><code class="docutils literal"><span class="pre">cd</span></code> to cloned repo</li>
+<li>run <code class="docutils literal"><span class="pre">create.case</span></code> to generate the base_suite directories.  The output
+is piped to <code class="docutils literal"><span class="pre">log.suite</span></code></li>
+<li>Running <code class="docutils literal"><span class="pre">create.case</span></code> submits each individual job to the queue.</li>
+<li><code class="docutils literal"><span class="pre">run.suite</span></code> monitors the queue manager to determine when all jobs have
+finished (pings the queue manager once every 5 minutes).</li>
+<li>Once all jobs complete, cd to base_suite directory and run <code class="docutils literal"><span class="pre">./results.csh</span></code></li>
+<li>Run <code class="docutils literal"><span class="pre">ctest</span> <span class="pre">-S</span> <span class="pre">steer.cmake</span></code> in order to post the test results to the CDash dashboard</li>
+</ul>
+</div>
+<div class="section" id="manual-method">
+<h4>3.6.6.2. Manual Method<a class="headerlink" href="#manual-method" title="Permalink to this headline">¶</a></h4>
+<p>To manually run the CICE tests and post the results to the CICE CDash dashboard,
+users essentially just need to perform all steps available in run.suite, detailed below:</p>
+<ul class="simple">
+<li>Pass the <code class="docutils literal"><span class="pre">-report</span></code> flag to create.case when running the <code class="docutils literal"><span class="pre">base_suite</span></code> test suite.
+The <code class="docutils literal"><span class="pre">-report</span></code> flag copies the required CTest / CDash scripts to the suite
+directory.</li>
+<li><code class="docutils literal"><span class="pre">create.case</span></code> compiles the CICE code, and submits all of the jobs to the
+queue manager.</li>
+<li>After every job has been submitted and completed, <code class="docutils literal"><span class="pre">cd</span></code> to the suite directory.</li>
+<li>Parse the results, by running <code class="docutils literal"><span class="pre">./results.csh</span></code>.</li>
+<li>Run the CTest / CDash script <code class="docutils literal"><span class="pre">ctest</span> <span class="pre">-S</span> <span class="pre">steer.cmake</span></code>.</li>
+</ul>
+</div>
+</div>
 </div>
 <div class="section" id="table-of-namelist-options">
 <span id="tabnamelist"></span><h2>3.7. Table of namelist options<a class="headerlink" href="#table-of-namelist-options" title="Permalink to this headline">¶</a></h2>
@@ -3582,6 +3625,7 @@ <h3><a href="index.html">Table Of Contents</a></h3>
 <li><a class="reference internal" href="#test-suite-format">3.6.3. Test suite format</a></li>
 <li><a class="reference internal" href="#example-tests-quickstart">3.6.4. Example Tests (Quickstart)</a></li>
 <li><a class="reference internal" href="#code-compliance-test">3.6.5. Code Compliance Test</a></li>
+<li><a class="reference internal" href="#cice-test-reporting">3.6.6. CICE Test Reporting</a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#table-of-namelist-options">3.7. Table of namelist options</a></li>

index.html

@@ -121,6 +121,7 @@ <h2>Table of Contents:<a class="headerlink" href="#table-of-contents" title="Per
 <li class="toctree-l3"><a class="reference internal" href="cice_3_user_guide.html#test-suite-format">3.6.3. Test suite format</a></li>
 <li class="toctree-l3"><a class="reference internal" href="cice_3_user_guide.html#example-tests-quickstart">3.6.4. Example Tests (Quickstart)</a></li>
 <li class="toctree-l3"><a class="reference internal" href="cice_3_user_guide.html#code-compliance-test">3.6.5. Code Compliance Test</a></li>
+<li class="toctree-l3"><a class="reference internal" href="cice_3_user_guide.html#cice-test-reporting">3.6.6. CICE Test Reporting</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="cice_3_user_guide.html#table-of-namelist-options">3.7. Table of namelist options</a></li>