Refactor vertical interpolation in EAMxx #2821
Conversation
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5398 FAILED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5695 FAILED (click to see last 100 lines of console output) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5400 FAILED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5697 FAILED (click to see last 100 lines of console output) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5402 FAILED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5699 FAILED (click to see last 100 lines of console output) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5421 PASSED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5701 FAILED (click to see last 100 lines of console output) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5423 PASSED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5703 FAILED (click to see last 100 lines of console output) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5430 PASSED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5708 FAILED (click to see last 100 lines of console output) |
Allow to strip any tag from a list, without erroring out if not found
Do away with vert interp utility. In VerticalRemapper, just use ekat::LinInterp manually, and in FieldAtPressureLevel, we don't even need LinInterp.
bartgol
force-pushed
the
bartgol/eamxx/vert-interp-recycle-lin-interp-object
branch
from
4a6272a
to
5d0cee7
Compare
|
Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
|
E3SM-Autotester
removed
the
AT: RETEST
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, target_sha=a15130cfca7eebaaca5d632a1e65163e14fe1808, However Inspection must be performed before merge can occur... |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, target_sha=2783f009006e5a54da94d7409c913782559c8a93, However Inspection must be performed before merge can occur... |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
|
|
All Jobs Finished; status = PASSED, target_sha=b7f853cf108dc55a33a69ae5986711f54cf3fe9b, However Inspection must be performed before merge can occur... |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
Using Repos:
Pull Request Author: bartgol |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: SCREAM_PullRequest_Autotester_Mappy
Jenkins Parameters
Build InformationTest Name: SCREAM_PullRequest_Autotester_Weaver
Jenkins Parameters
SCREAM_PullRequest_Autotester_Mappy # 5540 FAILED (click to see last 100 lines of console output)SCREAM_PullRequest_Autotester_Weaver # 5808 PASSED (click to see last 100 lines of console output) |
| std::shared_ptr<ekat::LinInterp<Real,SCREAM_PACK_SIZE>> m_lin_interp_mid_packed; | ||
| std::shared_ptr<ekat::LinInterp<Real,SCREAM_PACK_SIZE>> m_lin_interp_int_packed; | ||
| std::shared_ptr<ekat::LinInterp<Real,1>> m_lin_interp_mid_scalar; | ||
| std::shared_ptr<ekat::LinInterp<Real,1>> m_lin_interp_int_scalar; |
There was a problem hiding this comment.
It may not be too important, but this is another time where it would be nice to be able to have a scalar version (and not just pack size 1). Should be a quick task, may look into this over leave if you haven't already.
There was a problem hiding this comment.
Yes, it would be nice. But then you'd have 2 impls for LinInterp. I think at some point we need to a) think about switching to Kokkos::simd, and b) go through our basic functions, and make sure we can call them with either packs or scalars. E.g., index_and_shift (whose name I would probably change) only works for packed views, forcing different impls. If we had something like
template<typename ScalarT, typename IndexT>
ScalarT eval_at_index (const view_1d<ScalarT>& v, const IndexT& i, const int shift = 0);
with specializations for Packs1, then we may be able to have a single impl for LinInterp, which works for both packs and scalars.
Footnotes
-
since partial specialization is not possible for function, we'd need a helper struct, but that's a minor detail. ↩
|
The diff is b/c we forgot to bless baselines after merging the mam4xx non-bfb pr. Merging. |
|
Would it be surprising to see 2.8x speedup in vertical remap timer with cess-style ne1024 case? (on pm-gpu with 512 nodes) Trying to explain a large speedup between a run with checkout before this change and one with early July checkout. where horiz is about the same: |
|
@ndkeen It's great to see that the speedup is quite noticeable. The feature was not heavily used (if used at all), so I didn't have much data. But yes, I was expecting some speedup in vertical remap (it was the whole point of this PR, basically). 2.8x in that timer is a yummy treat. I'd have to check how many streams have vremap enabled, to realize how important it is in absolute terms. But given that vremap is 100% rank-local, it was disappointing to see it in the same ball park as hremap. 70s vs 240s is more satisfying. |
Before, the way we did vertical interpolation, for every field we were redoing the setup of LinInterp, as well as reallocating some views (not really good).
This PR changes a few things:
scream_vertical_interpolation*.hppfiles altogether.VerticalRemapper, callekat::LinInterpmanually. We create up to 4ekat::LinInterpobjects, depending on fields to remap: midpoints-vs-interfaces based and packed-nonpacked. At run time, we setup the interpolation objects and reuse them for all the fields of the same type. Also, no runtime device views allocations.FieldAtPressureLevel, we don't even needekat::LinInterp, since the tgt grid has a single entry, and we don't need to reuse the interpolation for multiple evaluations. Simply callekat::upper_boundand manually interpolate.I still need to chek the performance, but I'm fairly sure that, for a moderately large number of fields to remap (when using the vertical remap feature in IO), we would see a noticeable improvement.