jupyter_sample.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1ca2a6f5-bd5a-4520-8380-baf5cb536d41",
   "metadata": {},
   "source": [
    "## ADF Diagnostics In Jupyter\n",
    "This notebook will run the Atmospheric Diagnostic Framework using the settings in a config.yaml file in your ADF directory. \n",
    "\n",
    "Note that it was developed to run on Cheyenne/Caspar JupyterHub *with the NPL (conda) kernel*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7eea8d1a-d6a7-4464-bd6d-8194aa2368d0",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Setup\n",
    "#### Required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cc09d60-42d0-43ac-ab75-92b2f9a3df9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os.path\n",
    "import sys"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6566766f-9d8f-4428-bf01-3c36a1ffeb63",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Paths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1fb67e7-2b56-4136-a703-a18378b66c9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ADF Code path\n",
    "# If it is in your cwd, set adf_code = local_path (initialized below)\n",
    "# otherwise set adf_code appropriately\n",
    "\n",
    "local_path = os.path.abspath('')\n",
    "adf_code = local_path\n",
    "#adf_code = \"/glade/u/home/bundy/diag/ADF_top/ADF\"\n",
    "\n",
    "#set path to ADF lib (main code)\n",
    "lib_path = os.path.join(adf_code,\"lib\")\n",
    "\n",
    "#set path to ADF plotting scripts \n",
    "plotting_scripts_path = os.path.join(adf_code,\"scripts\",\"plotting\")\n",
    "\n",
    "#Add paths to python path:\n",
    "sys.path.append(lib_path)\n",
    "sys.path.append(plotting_scripts_path)\n",
    "\n",
    "print(f\"current working directory = {local_path}\")\n",
    "print(f\"ADF path                  = {adf_code}\")\n",
    "print(f\"ADF source code           = {lib_path}\")\n",
    "print(f\"ADF plotting scripts      = {plotting_scripts_path}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "159be7f2-8593-483a-b200-daf8c3c992c8",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Paths to data that is input or produced by the ADF are set in the config.yaml file. \n",
    "To modify from the defaults, _edit that file_.\n",
    "\n",
    "If there are errors here, it is likely due to path errors above\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd4918d2-dcc0-417b-8867-d368c4cb1d78",
   "metadata": {},
   "outputs": [],
   "source": [
    "#set path to config YAML file:\n",
    "config_file=os.path.join(adf_code,\"config_cam_baseline_example.yaml\")\n",
    "\n",
    "print(f\"Will read settings from \",config_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f553607f-cd31-4aa3-8142-4959b3569a1a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#import ADF diagnostics object\n",
    "from adf_diag import AdfDiag\n",
    "\n",
    "# If this fails, check your paths output in the cells above,\n",
    "# and that you are running the NPL (conda) Kernel\n",
    "# You can see all the paths being examined by un-commenting the following:\n",
    "#sys.path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc194087-e523-4161-9af4-0c45983d268e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Initialize ADF object from the user-set config file\n",
    "#\n",
    "# Note that you will need to set 'user' in your config file. \n",
    "# The file makes some assumptions about output/working data directories \n",
    "# that can be checked by searching for where the variable 'user' is used.\n",
    "#\n",
    "# Any changes that are made to the config file will require re-running this cell\n",
    "# \n",
    "adf = AdfDiag(config_file)\n",
    "print(f\"Reading settings from \",config_file)\n",
    "\n",
    "check_user = adf.read_config_var(\"user\")\n",
    "err_msg = 'ERROR: You need to set user in the config.yaml to your user name'\n",
    "assert check_user != 'USER-NAME-NOT-SET', f'{err_msg}'\n",
    "print(f'user name set to: {check_user}')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1a2939c-dbf8-4cef-8bc9-0f3e756b8075",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pull (and print) some info from the ADF object:\n",
    "baseline_dict = adf.read_config_var(\"diag_cam_climo\")\n",
    "baseline_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3bb53c89-82dc-407b-97a1-6162e8defcc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# More useful to get information with the variable settings (eg cam_case_name) replaced\n",
    "# with their actual values:\n",
    "\n",
    "#Grab a processed variable under \"diag_basic_info\":\n",
    "print(\"ADF plots will be written to\",adf.get_basic_info('cam_diag_plot_loc', required=True))\n",
    "\n",
    "#Grab a processed variable under \"diag_cam_climo\":\n",
    "print(\"case hist files will be read from\",adf.get_cam_info('cam_hist_loc', required=True))\n",
    "\n",
    "#Grab a processed variable under \"diag_cam_baseline_climo\":\n",
    "print(\"baseline time series files will be read from\",adf.get_baseline_info('cam_ts_loc', required=True))\n",
    "\n",
    "#Finally, if you request a non-existent variable, and don't have \"required=True\", it will return None:\n",
    "print(\"The number of pineapples in the config file is\",adf.get_basic_info('pineapples'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7d266a8-f006-468d-96d5-51d9a93dd6dd",
   "metadata": {
    "tags": []
   },
   "source": [
    "### ADF Standard Work Flow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25165dcb-5784-4487-97ab-896b6cf395e2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Create model time series.\n",
    "print(f\"case ts files will be created in\",adf.get_cam_info('cam_ts_loc', required=True))\n",
    "adf.create_time_series()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "924604b7-0da8-4867-9870-b8bb32131483",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Create model baseline time series (if needed):\n",
    "if not adf.compare_obs:\n",
    "    print(f\"baseline ts files will be created in\",adf.get_baseline_info('cam_ts_loc', required=True))\n",
    "    adf.create_time_series(baseline=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85a611c7-a90b-4d24-a518-046407564e35",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Create model climatology (climo) files.\n",
    "print(f\"case climo files will be created in\",adf.get_cam_info('cam_climo_loc', required=True))\n",
    "adf.create_climo()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "012e5b82-f7dd-4fda-84c5-ad1eaeab6406",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#If a user is doing a model vs obs comparison, but\n",
    "#no observations were found, then stop here:\n",
    "if adf.compare_obs and not adf.var_obs_dict:\n",
    "        print('ADF diagnostics has completed successfully.')\n",
    "        sys.exit(0)\n",
    "else:\n",
    "    print('config file did not ask ADF to compare obs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9673eada-6eeb-4c68-b59a-bd7baf62aca1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Regrid model climatology files to match either\n",
    "#observations or CAM baseline climatologies.\n",
    "#This call uses the \"regridding_scripts\" specified\n",
    "#in the config file:\n",
    "adf.regrid_climo()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3d9ff4a-b720-47f1-89e9-10e06dab1d41",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Perform analyses on the simulation(s).\n",
    "#This call uses the \"analysis_scripts\" specified in the\n",
    "#config file:\n",
    "adf.perform_analyses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63f97af2-1c72-467a-9d87-27101506805c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Create the plot requested in the plotting_scripts list in the config file:\n",
    "plot_path = adf.get_basic_info('cam_diag_plot_loc')\n",
    "print(f'plots written to {plot_path}')\n",
    "adf.create_plots()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95161341-a11c-408c-baf6-44191023f55a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Create website.\n",
    "if adf.create_html:\n",
    "    adf.create_website()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8de79bdf-bdf8-466c-9bbe-4dbd9ea98bbd",
   "metadata": {
    "tags": []
   },
   "source": [
    "### ADF Helpful Methods and Structures "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "106a7c03-d3c8-4229-a0f2-9ebd7b3b57fd",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Demonstration of a few methods to get information from the ADF object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19f366ae-6aa1-4e7c-8c09-f4907c1785ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "basic_info_dict = adf.read_config_var(\"diag_basic_info\")\n",
    "print(basic_info_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "367b1c5f-1455-4584-b926-b272605df614",
   "metadata": {},
   "outputs": [],
   "source": [
    "baseline_dict = adf.read_config_var(\"diag_cam_baseline_climo\")\n",
    "print(baseline_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f4d34570-b650-4da6-88a5-2f9942092a2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "case_names = adf.get_cam_info(\"cam_case_name\",required=True)\n",
    "print(case_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66b7555f-d694-4c23-9046-75990c79153f",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_type = basic_info_dict.get('plot_type', 'png')\n",
    "plot_type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb3efbd0-b7c7-442f-b978-644a67db7ab0",
   "metadata": {},
   "outputs": [],
   "source": [
    "case_climo_loc = adf.get_cam_info('cam_climo_loc', required=True)\n",
    "\n",
    "# Variables used in a CAM v CAM (baseline comparison), not required because not used if compare_obs = True\n",
    "data_name = adf.get_baseline_info('cam_case_name', required=False)\n",
    "data_loc = adf.get_baseline_info(\"cam_climo_loc\", required=False)\n",
    "\n",
    "print(case_climo_loc)\n",
    "print(data_name)\n",
    "print(data_loc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67ea0b70-b4b9-4c26-8222-cbc35e7fff78",
   "metadata": {},
   "outputs": [],
   "source": [
    "var_list = adf.diag_var_list\n",
    "print(var_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61d1372c-aade-4c1a-a769-e4b5f4bd2a05",
   "metadata": {},
   "source": [
    " #### Demonstrate how to check for a variable in the list "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65dcd8d2-a8e6-469e-b30c-55cfe5c6f27e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Print all the variables\n",
    "var_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe9ddbe7-4882-4ad2-93aa-f4aafc8dab71",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Check for a variable\n",
    "var_name = 'PRECC'\n",
    "assert var_name in var_list, f'Sorry, you need to include {var_name} to make this plot'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c209c333-99d6-4a46-a46e-5231308dc61f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Check for a variable that isn't in the list\n",
    "var_name = 'PRECT'\n",
    "assert var_name in var_list, f'Sorry, you need to include {var_name} to make this plot'\n",
    "\n",
    "# EXPECT AN ERROR for demonstration purposes! \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fde8debc-d6db-46c3-a8da-293dd88a11e7",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ced2249-5468-4169-8e34-4875cbd99b12",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "NPL (conda)",
   "language": "python",
   "name": "npl-conda"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}