refactor: run short simulation for reference SCM

[haakon-e]

Explicitly fetch/generate namelist from TC, then modify the time step in the namelist so that reference SCM simulation is only run 1 step forward. This generates the necessary z-levels, and cell/face variable info for calibration.

Depends on CliMA/TurbulenceConvection.jl#525.

TODO: Check that TC doesn't write namelist to output directory, otherwise all calibration studies will have incorrect t_max.

[ilopezgp]

The scm_pycles_pipeline experiment fails for me on central using this branch,

┌ Warning: No unique stats netCDF file found in │ scm_init/Output.Bomex.000000/stats. Extending search to other files. └ @ CalibrateEDMF.TurbulenceConvectionUtils /central/home/ilopezgo/CalibrateEDMF.jl/src/helper_funcs.jl:218 ┌ Warning: Extended search errored with: │ Base.IOError("readdir(\"scm_init/Output.Bomex.000000/stats\"): no such │ file or directory (ENOENT)", -2)

[haakon-e]

The scm_pycles_pipeline experiment fails for me on central using this branch,

┌ Warning: No unique stats netCDF file found in │ scm_init/Output.Bomex.000000/stats. Extending search to other files. └ @ CalibrateEDMF.TurbulenceConvectionUtils /central/home/ilopezgo/CalibrateEDMF.jl/src/helper_funcs.jl:218 ┌ Warning: Extended search errored with: │ Base.IOError("readdir(\"scm_init/Output.Bomex.000000/stats\"): no such │ file or directory (ENOENT)", -2)

Yeah, I'll have to look into this a bit closer. The warnings themselves are fine -- I added them because readdir errors if the directory doesn't exist (at least on HPC) -- but I think readdir is a fix for Windows systems, so I don't know what the subtleties are here...

Anyways, there's also a test that's failing which relates to les driven cases, which is not surprising that it fails -- but it's surprising because I didn't change that code... I'll investigate and see if I understand what's going on this afternoon.

[ilopezgp]

The scm_pycles_pipeline experiment fails for me on central using this branch,
┌ Warning: No unique stats netCDF file found in │ scm_init/Output.Bomex.000000/stats. Extending search to other files. └ @ CalibrateEDMF.TurbulenceConvectionUtils /central/home/ilopezgo/CalibrateEDMF.jl/src/helper_funcs.jl:218 ┌ Warning: Extended search errored with: │ Base.IOError("readdir(\"scm_init/Output.Bomex.000000/stats\"): no such │ file or directory (ENOENT)", -2)

Yeah, I'll have to look into this a bit closer. The warnings themselves are fine -- I added them because readdir errors if the directory doesn't exist (at least on HPC) -- but I think readdir is a fix for Windows systems, so I don't know what the subtleties are here...

Anyways, there's also a test that's failing which relates to les driven cases, which is not surprising that it fails -- but it's surprising because I didn't change that code... I'll investigate and see if I understand what's going on this afternoon.

The LesDrivenSCM error might be due to the construction of the dirname and namelist paths, which is not the same as for any other case (there are functions that add a suffix). Maybe @costachris can see if there is an issue.

[ilopezgp]

bors r+

[bors]

Build succeeded:

• ci 1.6.0 - macOS-latest
• ci 1.6.0 - ubuntu-latest
• ci 1.6.0 - windows-latest