Replace cice5 with cice6 in ufs-s2s-model

[DeniseWorthen]

Replaces CICE5 with CICE6: Issue #159

• adds required items to nems.configure. Removes non-cmeps nems.configure files since NEMS mediator is not compatible w/ CICE6

• updates ice_in_template for CICE6

• updates default_vars and edit_inputs to allow CICE6 resolution and processor count to be specified at run time

• updates to esmf8.1.0bs27 to allow ICE and WAV on separate PEs (Issue ice and waves need same pes  #172)

• add cpl_mode (Issue Add nems.configure variable for CMEPS fractional grid use #138) and use_coldstart as new configurable variables (Issues ocean import fields are zero when initialized from coldstart run #86, Revisit Cold Starts #53). Note MOM6 requires a commit to make use of this variable, MOM6 Issue #32.

• updates NEMS submodule for CICE6 compile

• fixes coldstart run sequence by calling the restart write phase before the accum avg phase (Issue current cold1 run sequence produces zero for ocean imports on first coupling timestep #167)

Associated PR: CICE #13
Associated PR: CMEPS #7
Associated PR: NEMS #75

[junwang-noaa]

I am running s2s RT on dell, it is really slow, the first executable has not been built yet for 2hours. But there is no error message either.

…

On Fri, Sep 4, 2020 at 3:35 PM Denise Worthen ***@***.***> wrote: The cpld_fv3_ccpp_384_mom6_cice_cmeps_1d_bmark_rt test did not complete on dell-p3. The out file shows many many print statements of [140] fix: decrease snow depth to: 0.200000000000000 [109] warning: ice fraction is low: 0.000000000000000E+000 [109] fix ice fraction: reset it to: 0.150000000000000 The model then stops. I do not see any error message. I was creating the baseline using sequential. I have created baselines for this same code successfully on hera and orion. I am temporarily pointing to Jessica's WW3 branch and have used her new WW3 inputs. The run directory is here: /gpfs/dell2/ptmp/Denise.Worthen/S2S_RT/rt_89374/cpld_fv3_ccpp_384_mom6_cice_cmeps_1d_bmark_rt — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TKYHOE4AEWXR5GRDG3SEE6PPANCNFSM4PZEZWEA> .

[DeniseWorthen]

I actually deleted the comment. I saw I had no jobs running and thought something had died. So I looked at that job and it looked like it died but I think it was just that the log output hadn't completed. It looked like it just died mid-step. But that job did complete after all.

The reason I had no current jobs is because it was in the compile step for the next job.

[junwang-noaa]

Maybe you can increase the wall clock time for that test?

…

On Fri, Sep 4, 2020 at 3:49 PM Denise Worthen ***@***.***> wrote: I actually deleted the comment. I saw I had no jobs running and thought something had died. So I looked at that job and it looked like it died but I think it was just that the log output hadn't completed. It looked like it just died mid-step. But that job did complete after all. The reason I had no current jobs is because it was in the compile step for the next job. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TN4ZDL62IKM7DI2XULSEFAETANCNFSM4PZEZWEA> .

[DeniseWorthen]

All jobs that have run have completed. It is now compiling the ww3 test.

[DeniseWorthen]

This PR is ready for final baseline verification and merging; this can be done once NEMS is updated (and after the W3 merge).

[MinsukJi-NOAA]

NOAA-EMC/NEMS#75 has been merged into develop

[DeniseWorthen]

@JessicaMeixner-NOAA I just ran my baseline on dell. All tests pass except one. It failed on one file on one test:

cpld_fv3_ccpp_384_mom6_cice_cmeps_ww3_1d_bmark_rt

Comparing 20130402.000000.out_grd.gwes_30m .........OK
Comparing 20130402.000000.out_pnt.points ............NOT OK
Comparing 20130402.000000.restart.gwes_30m .........OK

All remaining files in the test show OK. Does this make sense to you?

[DeniseWorthen]

The same single file fails comparison on both hera and orion also.

I ran a test baseline creation and verification on orion on 9/2. The comparison between that commit and the current commit is here. The changes were:

a) update WW3
b) update to esmfbs27
c) ice and waves were tested on separate PEs on 9/2
d) cold start fixed

[junwang-noaa]

I got the same when running RT with the PR171. But I see the /gpfs/dell2/emc/modeling/noscrub/emc.nemspara/RT/FV3-MOM6-CICE5/develop-20200903/RT-Baselines_1d_bmwav_ccpp_cmeps/20130402.000000.out_pnt.points was updated in baseline, and I got identical results now. You may need to rerun the test.

…

On Sun, Sep 6, 2020 at 7:35 AM Denise Worthen ***@***.***> wrote: The same single file fails comparison on both hera and orion also. I ran a test baseline creation and verification on orion on 9/2. The comparison between that commit and the current commit is here <https://github.com/DeniseWorthen/ufs-s2s-model/compare/20f3878565..DeniseWorthen:feature/cice6>. The changes were: a) update WW3 b) update to esmfbs27 c) ice and waves were tested on separate PEs d) cold start fixed — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TJO3XSHS3666ASDGFDSENXXFANCNFSM4PZEZWEA> .

[DeniseWorthen]

I will have a new baseline for cice6. I copied in the 20200903 baseline WW3_input_data files since they had been updated.

I created the 20200904 baseline for CICE6; I was running the regression test against that baseline when I got the single file failure. It doesn't make sense to me though---how can a single file no reproduce when all other files do?

In my 0902 test, I had created a baseline for testing purposes and ran against that test baseline just to confirm that I the restart test passed and my changes to the rt.sh etc were good.

[JessicaMeixner-NOAA]

@DeniseWorthen I'm not sure what happened, but the same thing happened to me and I copied that one file that was wrong into the baseline and now all passed? I don't know if it was just that one test or what? But I only had that happen on dell, not the other platforms.

[DeniseWorthen]

So after you created the baseline, you had to manually copy that one file. Then you ran against the baseline and all passed?

[JessicaMeixner-NOAA]

Yes. I’m not sure why. Because I had tested that test before and it was fine?

…

On Sunday, September 6, 2020, Denise Worthen ***@***.***> wrote: So after you created the baseline, you had to manually copy that one file. Then you ran against the baseline and all passed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFMBF3264NDHJQNMAXL3H6TSEQGAXANCNFSM4PZEZWEA> .

-- Jessica Meixner NCEP/EMC NOAA Center for Weather and Climate Prediction

[junwang-noaa]

I assume with mkmf and MOM6 src directly checked out in the app level, the MOM6-interface repo is not needed.

[DeniseWorthen]

Both MOM-interface and CICE-interface are just subdirectories now. But don't we need to retain MOM6-interface repo in order to maintain the ability to checkout older commits of s2s?

[junwang-noaa]

We can keep the MOM6-interface until everybody is switched to the new version. On the other side, from model structure level, I'd think it's good to keep MOM6 related code in MOM6-interface repo and to use git submodule to check them out, this will be consistent with other components (NEMS, FV3). We can discuss this at code manager meeting, the code changes are OK for this commit.