Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fv3_esg_HAFS_v0_hwrf_thompson_debug fails when compiling with -init=snan,arrays on Cheyenne.intel #958

Closed
DeniseWorthen opened this issue Dec 14, 2021 · 7 comments
Labels
bug Something isn't working

Comments

@DeniseWorthen
Copy link
Collaborator

Description

In PR #935, the compile flag -init=snan,arrays was added for Intel debug. On Cheyenne, the
fv3_esg_HAFS_v0_hwrf_thompson_debug test failed with

MPT: shepherd terminated: r2i3n34.ib0.cheyenne.ucar.edu - job aborting

which usually indicates an issue w/ memory.

To Reproduce:

Run fv3_esg_HAFS_v0_hwrf_thompson_debug test on Cheyenne.intel. The job fails as described. The dynf000.nc is produced but not the phyf000.nc file.

Adding additional resources for the job (e.g. #PBS -l select=6:ncpus=18:mpiprocs=18) allows the job to complete.

@DeniseWorthen DeniseWorthen added the bug Something isn't working label Dec 14, 2021
@climbfuji
Copy link
Collaborator

We should consider removing the fv3_*_HAFS_v0_hwrf_thompson* tests entirely (including test setup files and input data), they are superseded by the new *hafs* (lowercase) tests.

@DeniseWorthen
Copy link
Collaborator Author

So that looks like these 4 tests both Intel and GNU:

fv3_HAFS_v0_hwrf_thompson
fv3_esg_HAFS_v0_hwrf_thompson
fv3_HAFS_v0_hwrf_thompson_debug
fv3_esg_HAFS_v0_hwrf_thompson_debug

@BinLiu-NOAA
Copy link
Contributor

@climbfuji, Agreed! We can add a hafs_regional_atm_hwrf_thompson test (or something similar) in the HAFS RT section, and retire these following four RTs:
fv3_HAFS_v0_hwrf_thompson
fv3_esg_HAFS_v0_hwrf_thompson
fv3_HAFS_v0_hwrf_thompson_debug
fv3_esg_HAFS_v0_hwrf_thompson_debug

@climbfuji
Copy link
Collaborator

So that looks like these 4 tests both Intel and GNU:

fv3_HAFS_v0_hwrf_thompson
fv3_esg_HAFS_v0_hwrf_thompson
fv3_HAFS_v0_hwrf_thompson_debug
fv3_esg_HAFS_v0_hwrf_thompson_debug

Yes, I think these can go. Note that this will mean that some "flavors" of the PBL scheme (triggered by the HWRF_PBL switch) will no longger be tested.

@junwang-noaa
Copy link
Collaborator

@BinLiu-NOAA Can we remove those 4 tests and you can create a hafs_regional_atm_hwrf_thompson test in HAFS RT section later? Thanks.

@BinLiu-NOAA
Copy link
Contributor

@junwang-noaa, That sounds good to me. I can work with @ChunxiZhang-NOAA to add a hafs_regional_atm_hwrf_thompson test in the HAFS RT section later. Thanks!

@climbfuji
Copy link
Collaborator

The fv3_*_HAFS_* tests were removed in a recent PR (#963).

JianpingHuang-NOAA added a commit to JianpingHuang-NOAA/ufs-weather-model that referenced this issue Apr 21, 2024
…munity#958)

* update setup.sh to use static fix dir

* change setup.py back to use statis fix files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants