Better global tas database and warming level methods #270

aulemahal · 2023-10-06T18:52:26Z

Pull Request Checklist:

This PR addresses an already opened issue (for bug fixes / features)
- This PR fixes Missing simulations in IPCC_annual_global_tas.csv #268
(If applicable) Documentation has been added / updated (for bug fixes / features).
(If applicable) Tests have been added.
This PR does not seem to break the templates.
HISTORY.rst has been updated (with summary of main changes).
- Link to issue (:issue:number) and pull request (:pull:number) has been added.

What kind of change does this PR introduce?

Addition of:

CMIP6_CMCC-ESM2_ssp245_r1i1p1f1
CMIP6_CMCC-ESM2_ssp370_r1i1p1f1
CMIP6_CMCC-ESM2_ssp585_r1i1p1f1
CMIP6_EC-Earth3-CC_ssp245_r1i1p1f1
CMIP6_EC-Earth3-CC_ssp585_r1i1p1f1
CMIP6_KACE-1-0-G_ssp245_r1i1p1f1
CMIP6_KACE-1-0-G_ssp370_r1i1p1f1
CMIP6_KACE-1-0-G_ssp585_r1i1p1f1
CMIP6_TaiESM1_ssp245_r1i1p1f1
CMIP6_TaiESM1_ssp370_r1i1p1f1

to xscen/data/IPCC_annual_global_tas.csv.

Change from CSV to netCDF
Accept DataArrays, DataFrames in get_warming_level, output as those or list.
Multiple models and multiple warming levels in subset_warming_level.

Does this PR introduce a breaking change?

Yes, if get_warming_level is called with a sequence of models, the output is now a list of the same length. The dictionary can be retrieved by passing output='selected'. I found it way more useful this way!

Other information:

The new global tas were computed with the following code:

def get_global_tas(source, exp, member):
    scats = xs.search_data_catalogs(
        'catalogues/simulation.json',
        {'tas': 'MS'},
        other_search_criteria={'source': source, 'experiment': exp, 'member': member, 'processing_level': 'raw'},
        match_hist_and_fut=True
    )
    ds = xs.extract_dataset(scats.popitem()[1])['MS']
    day = ds.tas.resample(time='YS').mean()
    dam = xs.spatial_mean(
        day,
        region='global',
        method='cos-lat',
        spatial_subset=False
    )
    tas = xc.units.convert_units_to(dam, '°C')
    return tas.rename(time='year').assign_coords(year=tas.time.dt.year.values)

See issue #268 for discussion of small differences between this and the official IPCC-provided data.

juliettelavoie · 2023-10-06T19:12:16Z

do we want to add the source to the original data in the csv ?
Technically, it is here: https://github.com/Ouranosinc/xscen/blob/b3c79cd02f01c9f791d082459464b0842b5f4f6c/xscen/extract.py#L861C2-L861C2
(not exactly the github page with the data, but the paper links to the repo)

but maybe it needs to be in the csv also? (while we are here anyway)

aulemahal · 2023-10-06T19:37:07Z

Good idea!

for more information, see https://pre-commit.ci

review-notebook-app · 2023-10-09T20:51:34Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

aulemahal · 2023-10-09T21:52:25Z

Last commits are a rewrite of get_warming_level and subset_warming_level so they're easier to use with arrays.

See changes of the warming level notebook for most explanations. And edit on the root comment.

aulemahal · 2023-10-16T19:23:04Z

@sarahclaude Should I use this PR to add the global tas average for non-r1i1p1f1 models ? Will you need that ?

sarahclaude · 2023-10-16T19:53:23Z

I might need it in the future but we are not dealing with warming levels yets, so not on short-term.

aulemahal · 2023-11-28T19:35:19Z

PR prête pour la revue.

J'ai laissé faire les membres non-r1i1p1f1 pour le moment. Les modèles ajoutés sont essentiellement ceux qui manquaient de ESPO-G6.

RondeauG

Looks pretty good!

xscen/catalog.py

HISTORY.rst

xscen/extract.py

tests/test_extract.py

Co-authored-by: RondeauG <38501935+RondeauG@users.noreply.github.com>

for more information, see https://pre-commit.ci

…globaltas

RondeauG · 2023-11-30T14:48:22Z

There's output to potentially remove, but other than that, this looks good to go!

juliettelavoie

great!

juliettelavoie · 2023-12-05T18:40:14Z

xscen/extract.py

+    output: {'selected', 'requested'}
+        When `realization` is a sequence.
+        If "requested", the function returns a list of the same length as `realization`.
+        If "selected", the function returns a dict, keys are the models selected in the database.


I am fine with a breaking change, returning only a list and removing the output arg.

juliettelavoie · 2023-12-05T19:07:24Z

xscen/extract.py

-        Warming level dataset.
+    xr.Dataset or None
+        Warming level dataset, or None if `ds` can't be subsetted for the requested warming level.
+        The dataset will have a new dimension `warminglevel` with `wl_dim` as coordinates.


what's the point of keeping the fake time axis ?

Maybe I don't understand the question, but as get_warming_level does not perform any statistical computation, there's a N year window for each warming level. The only way to have a multidimensional dataset with time and wl_dim, is to have some sort of time axis that is not the actual horizon (which is different for each warming level and simulation).

my mistake. I forgot that the mean wasn't calculated on the window in that function.

Zeitsperre · 2023-12-07T20:37:46Z

Docs are failing because a file only in this branch isn't present in main. Let me know if you want me to bypass protections to push to main!

Add missing models to csv

03017b9

WIP transitionning to a netcdf

f747a5f

github-actions bot added the CI Modifications to autiomation utilities label Oct 6, 2023

pre-commit-ci bot and others added 4 commits October 6, 2023 22:19

[pre-commit.ci] auto fixes from pre-commit.com hooks

5fda0d5

for more information, see https://pre-commit.ci

Merge and upd restrict_by_wl

3f42c95

Rewrite warming_level stuff for vectorization - upd doc - upd hist

125df00

Rewrite warming_level stuff for vectorization - upd doc - upd hist

2fbfe7d

github-actions bot added the docs label Oct 9, 2023

aulemahal changed the title ~~Better global tas database~~ Better global tas database and warming level methods Oct 9, 2023

aulemahal added 2 commits October 9, 2023 17:55

Fix for tests - allow incomplete subsets

4c8a79f

Merge branch 'main' into upd-globaltas

03d4e46

aulemahal added 3 commits October 25, 2023 14:39

merge

8177f25

Remove csv file

01a1b2e

merge main

187d374

aulemahal requested a review from RondeauG November 28, 2023 19:33

aulemahal added 3 commits November 28, 2023 17:24

Merge branch 'main' into upd-globaltas

5c46aa2

Fix get_warming_level for RCMs and catalogs - include tas nc in package

f52baea

fix nb

cfc773e

RondeauG reviewed Nov 29, 2023

View reviewed changes

aulemahal and others added 4 commits November 29, 2023 18:15

Apply suggestions from code review

2747602

Co-authored-by: RondeauG <38501935+RondeauG@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

958b0e0

for more information, see https://pre-commit.ci

Changes after review - fix no reach in vectorized

66a5844

Merge branch 'upd-globaltas' of github.com:Ouranosinc/xscen into upd-…

bf0bede

…globaltas

RondeauG approved these changes Nov 30, 2023

View reviewed changes

juliettelavoie approved these changes Dec 5, 2023

View reviewed changes

aulemahal and others added 3 commits December 6, 2023 17:14

Remove output arg

1d3c7fd

Add doc on global tas - add observations ds

2594e18

merge main to upd-globaltas

16ed4e7

aulemahal merged commit 1a5fed7 into main Dec 11, 2023
17 of 18 checks passed

aulemahal deleted the upd-globaltas branch December 11, 2023 23:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better global tas database and warming level methods #270

Better global tas database and warming level methods #270

aulemahal commented Oct 6, 2023 •

edited

Loading

juliettelavoie commented Oct 6, 2023

aulemahal commented Oct 6, 2023

review-notebook-app bot commented Oct 9, 2023

aulemahal commented Oct 9, 2023

aulemahal commented Oct 16, 2023

sarahclaude commented Oct 16, 2023

aulemahal commented Nov 28, 2023

RondeauG left a comment

RondeauG commented Nov 30, 2023

juliettelavoie left a comment

juliettelavoie Dec 5, 2023

juliettelavoie Dec 5, 2023

aulemahal Dec 5, 2023

juliettelavoie Dec 5, 2023

Zeitsperre commented Dec 7, 2023

Better global tas database and warming level methods #270

Better global tas database and warming level methods #270

Conversation

aulemahal commented Oct 6, 2023 • edited Loading

Pull Request Checklist:

What kind of change does this PR introduce?

Does this PR introduce a breaking change?

Other information:

juliettelavoie commented Oct 6, 2023

aulemahal commented Oct 6, 2023

review-notebook-app bot commented Oct 9, 2023

aulemahal commented Oct 9, 2023

aulemahal commented Oct 16, 2023

sarahclaude commented Oct 16, 2023

aulemahal commented Nov 28, 2023

RondeauG left a comment

Choose a reason for hiding this comment

RondeauG commented Nov 30, 2023

juliettelavoie left a comment

Choose a reason for hiding this comment

juliettelavoie Dec 5, 2023

Choose a reason for hiding this comment

juliettelavoie Dec 5, 2023

Choose a reason for hiding this comment

aulemahal Dec 5, 2023

Choose a reason for hiding this comment

juliettelavoie Dec 5, 2023

Choose a reason for hiding this comment

Zeitsperre commented Dec 7, 2023

aulemahal commented Oct 6, 2023 •

edited

Loading