Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADIOS2 variable shape LocalValue #1601

Open
wants to merge 13 commits into
base: dev
Choose a base branch
from

Conversation

franzpoeschel
Copy link
Contributor

@franzpoeschel franzpoeschel commented Mar 11, 2024

Ref https://adios2.readthedocs.io/en/v2.9.2/components/components.html#shapes

This PR considers LocalValues in ADIOS2 merely an implementation detail, without actually exposing an API analogous to to how LocalValues work in ADIOS2.

The reasoning is:

  1. LocalValues are useful not only for their API, but also since they give a hint to ADIOS2 that the variable should be handled as metadata and take part e.g. in metadata aggregation. With this PR, datasets such as the particlePatches in PIConGPU (one single value per MPI rank) can be given to the metadata system without changing a single line of code in PIConGPU. This can be crucial in streaming setups: When each reading rank loads the entire dataset naively (without this setting), the data plane of SST will open a connection between every writer and reader for just a single value each. Better to aggregate.
  2. If needed at some point, a LocalValue-like API can still be exposed on top of this.

TODO

  • Documentation

@franzpoeschel franzpoeschel force-pushed the adios2-local-values branch 3 times, most recently from a342b5d to 88b01e8 Compare March 12, 2024 10:50
examples/10_streaming_write.cpp Show resolved Hide resolved
src/IO/ADIOS/ADIOS2File.cpp Show resolved Hide resolved
src/IO/ADIOS/ADIOS2File.cpp Show resolved Hide resolved
examples/10_streaming_read.cpp Outdated Show resolved Hide resolved
test/ParallelIOTest.cpp Show resolved Hide resolved
include/openPMD/auxiliary/StringManip.hpp Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants