Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for multiple result nodes #391

Open
soxofaan opened this issue Mar 14, 2023 · 3 comments
Open

Support for multiple result nodes #391

soxofaan opened this issue Mar 14, 2023 · 3 comments

Comments

@soxofaan
Copy link
Member

follow up of Open-EO/openeo-processes#401

python client needs some kind of support for building process graphs with multiple result nodes

@jdries
Copy link
Collaborator

jdries commented Aug 14, 2024

It looks like we can create a valid process graph from multiple datacubes, as long as we use the same flattener.

    week = s2cube.aggregate_temporal_period(period="week",reducer="median")
    month = s2cube.aggregate_temporal_period(period="month",reducer="median")

    cube1=week.save_result("netCDF")
    cube2 = month.save_result("netCDF")

    from openeo.internal.graph_building import GraphFlattener
    flattener = GraphFlattener()
    dict1 = flattener.flatten(cube1.result_node())

    #only one node can be marked as 'result':True, so set all others to False
    for v in dict1.values():
        v["result"] = False

    secondary_result = cube2.result_node()

    #by reusing the flattener, the previously flattened nodes are also included
    dict2 = flattener.flatten(secondary_result)

    final_graph = {"process_graph":dict2}
    import json
    merged = json.dumps(final_graph, indent=2)
    print(merged)

    connection = openeo.connect("openeo.dataspace.copernicus.eu")
    connection.create_job(final_graph,title="job with 2 save results")

@jdries jdries removed their assignment Aug 14, 2024
@jdries
Copy link
Collaborator

jdries commented Aug 14, 2024

so way forward is either updating create_job or a variant of that method that accepts multiple datacubes?

or
cube.execute_batch(args..., extra_cubes=[extra_cube1])
or
connection.execute_batch([cube1,cube2]), other args...)

@soxofaan
Copy link
Member Author

soxofaan commented Aug 19, 2024

In the python client we already have a base class _ProcessGraphAbstraction, which is further implemented by DataCube, VectorCube, MlModel. So I was thinking along the lines of creating another subclass of that, say ProcessGraph or MultiResult, which you can pass one or more cubes and work with that. E.g.

results = MultiResult([cube_as_tiff, cube_as_netcdf, vector_cube])

results.execute_batch(...)

(all naming up for discussion of course)

soxofaan added a commit that referenced this issue Sep 24, 2024
- Check whole process graph for pre-existing `save_result` nodes, not just final node
- Disallow ambiguity of combining explicit `save_result` and download/create_job with format
soxofaan added a commit that referenced this issue Sep 25, 2024
…583, #391)

Make it more consistent with `download`
soxofaan added a commit that referenced this issue Sep 25, 2024
- Check whole process graph for pre-existing `save_result` nodes, not just final node
- Disallow ambiguity of combining explicit `save_result` and download/create_job with format
soxofaan added a commit that referenced this issue Sep 25, 2024
…583, #391)

Make it more consistent with `download`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants