Daily CICE output : can it be combined into one monthly/annual file?

[DirkOlivie]

@JensBDebernard @YanchunHe @monsieuralok @matsbn @TomasTorsvik @tylov

(I start here an issue that came up during the preparation of the noresm2.0.6 release, but it was decided to go into the release without this feature)

Problem
In the current version of NorESM2, the daily output of CICE ends up in individual daily files, which creates a considerable number of files. Would it be possible to reduce the number of files? Can all the daily output files be put in one file per year during model execution?

Possible solution
During the post-processing step of the model execution (where output files are moved from folder /noresm to folder /archive and converted from netcdf3 to netcdf4), possibly the daily files can be concatenated to one file.
Maybe the script can be called in a similar way the script noresm2netcdf4.sh is called (which does the netcdf3 to netcdf4 conversion).
Maybe it can be based on Tyge's script that exists for concatenating daily output from CICE (and runs in an offline way).
Is it best to create 1 file per year, or is it better to have 1 file per month?

Drawbacks
This change might have implications for the cmorization package and the diagnostics package. Will this cause big problems?
How should the script work if simulations run for less than a year?

[blcc]

I'm not sure this name list set can be apply to the CICE version in NorESM. Need test.
https://cesmcice.readthedocs.io/en/latest/users_guide/ice_nml_var.html

[YanchunHe]

This can be quickly adpated in the cmor tool, if the daily output are written in the same file as monthly file (*.h.) or a year file (say, *.h0).

I can update the cmor tool when the model configuration for this change is made.

[gold2718]

This can be quickly adpated in the cmor tool, if the daily output are written in the same file as monthly file (*.h.) or a year file (say, *.h0).

How does this work? There is only one (unlimited) time dimension in the file so if you have daily output, you will have a daily frame for each monthly variable even though no data was written to it. Does the postprocessing code handle cases like this (where the time axis does not match the data)?

[YanchunHe]

This can be quickly adpated in the cmor tool, if the daily output are written in the same file as monthly file (*.h.) or a year file (say, *.h0).

How does this work? There is only one (unlimited) time dimension in the file so if you have daily output, you will have a daily frame for each monthly variable even though no data was written to it. Does the postprocessing code handle cases like this (where the time axis does not match the data)?

Sorry that I did not state clearly (not in the proper way). I meant that we can put the daily output to a single file for each month or year. These daily output will not be in the same file as monthly output or yearly output.

Now the file name tag *.cice.h.* is used for monthly output. *.cice.h1.* is used for daily output as individual files for each day.

By combing the daily files to monthly file or yearly file for daily output, for example, we can use 'cice.h1' for monthly file or 'cice.h0` for yearly file for daily output. These name convections are not critical for cmorization.

[gold2718]

Sorry that I did not state clearly (not in the proper way). I meant that we can put the daily output to a single file for each month or year. These daily output will not be in the same file as monthly output or yearly output.

Thank you for the clarification. Unfortunately, I do not see any way to do that with CICE.

You might try asking on the CICE Consortium Forum.

[YanchunHe]

OK, I thought @DirkOlivie already have solution to this, and was asking if it is possible for the cmorisation tool to adapt to it.

I don't know much about the CICE model, so I will not doing this by myself.

@DirkOlivie will decide if you will close this issue now or later.

[monsieuralok]

@YanchunHe We have discussed in a meeting that first we should confirm (with you) that there is not any issue with CMOR and then, we will develop a test case and then, I will implement merging of CICE daily files to monthly files as suggested by you with different name in post-processing. We do not want to implement it in CICE code as @JensBDebernard suggested probably this feature might be appearing in later CICE version code.

[YanchunHe]

Thanks for the explanation, @monsieuralok

Then I can test this once you have a test case with output.