Ensemble reduction and changes to Ensembles

[RondeauG]

Pull Request Checklist:

• pre-commit hooks are installed/active in my local clone ($ pre-commit install)
• This PR addresses an already opened issue (for bug fixes / features)
  • This PR fixes Implement ensemble selection #4
• (If applicable) Documentation has been added / updated (for bug fixes / features)
• If a merge request has been made in parallel to this PR in xscen-notebooks, it is merged and the submodules have been updated.
• HISTORY.rst has been updated (with summary of main changes)
  • Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

• New file reduction.py that contains a function to help build the criteria data and an xclim.ensembles._reduce wrapper.

• Changes to ensemble_stats to:

  • Support xclim.ensembles._robustness functions.
  • Support looping through multiple functions without having to call ensemble_stats every time.
  • Support weights.

• The code that performs common_attrs_only has been moved to clean_up.

• Removed the default to_level in clean_up.

• New function generate_weights that uses metadata in Datasets to guess GCMs and RCMs and create weights accordingly.

• New function unstack_id to reverse-engineer IDs.

• generate_id now also accepts Datasets

Does this PR introduce a breaking change?

• statistics / stats_kwargs have been changed/eliminated in ensemble_stats, respectively.

Other information:

[RondeauG]

Question 1:
I feel that some of the info / weights stuff I implemented in ensemble_stats could be moved in its own function, to make the code cleaner. Thoughts?

Question 2:
kmeans_reduce_ensemble has 3 outputs (selected, clusters, fig_data), while kkz_reduce_ensemble only has 1. Is there a clean way to address that?

[RondeauG]

I added a couple checks before change_significance to make sure that the right data is sent to that function, since it assumes absolute deltas if ref==None. @aulemahal , this is probably something that could improved in xclim, as well as supporting Datasets in a similar way to what is done in ensembles._base.py

[aulemahal]

Beaucoup de commentaires qui ne sont au final que des affaires de styles, donc j'approuve et tu en fais ce que tu veux.

Je ne suis pas certain de voir ce qu'on pourrait porter dans xclim? Le check sur delta_kind ?

[juliettelavoie]

J'ai quelques petits commentaires sur les notebooks, mais sinon c'est beau!

[RondeauG]

Question 2:
kmeans_reduce_ensemble has 3 outputs (selected, clusters, fig_data), while kkz_reduce_ensemble only has 1. Is there a clean way to address that?

Je me rend compte qu'on n'a pas réglé cette question. Des idées ?

[juliettelavoie]

Je pense que toujours retourner 3 trucs c'est ok.
Pour properties_and_measures, on retourne toujours 2 choses, mais si ref_for_measures n'est pas donné, le 2e output est un dataset vide. Peut-être que les 2 outputs optionnels peuvent être dans le bon type mais vide (eg. clusters= {})