bigquery/tutorial/02-primer-to-ponder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e3a4634b-df94-4e97-96cf-9a42e5e22d8b",
   "metadata": {},
   "source": [
    "# Tutorial 2: Primer to Ponder"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "320599c8-9d85-4ede-b60e-c8f2db4154eb",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\"> <b>Before we get started: </b> \n",
    "    <ul style=\"list-style-type: none;margin: 0;padding: 0;\">\n",
    "        <li>✍️ To run this notebook, you need to have Ponder installed and set up on your machine. If you have not done so already, please refer to our <a href=\"https://docs.ponder.io/getting_started/quickstart.html\">Quickstart guide</a> to get started.</li>\n",
    "        <li>📖 Otherwise, if you're just interested in browsing through the tutorial, keep reading below!</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f2e16db8-8366-4e05-bc86-db5cbe8061c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-05-11 18:59:05 - Creating session yETePRbnYQDX4ScLqp2C6WmwnxqbcfvpZ81o8uZURt\n"
     ]
    }
   ],
   "source": [
    "import ponder; ponder.init()\n",
    "import modin.pandas as pd\n",
    "from google.cloud import bigquery\n",
    "from google.cloud.bigquery import dbapi\n",
    "from google.oauth2 import service_account\n",
    "import json\n",
    "bigquery_con = dbapi.Connection(bigquery.Client(credentials=service_account.Credentials.from_service_account_info(json.loads(open(\"../credential.json\").read()),scopes=[\"https://www.googleapis.com/auth/bigquery\"])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ad0bfb9-6398-4107-9b11-028e240c722b",
   "metadata": {},
   "source": [
    "## What is Ponder?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2a005e4-0e8e-4189-a525-8cf5f2d49cbe",
   "metadata": {},
   "source": [
    "Ponder lets you run your pandas code directly in your data warehouse. This means that you can continue to write pandas, but with the scalability and security benefits of a modern data warehouse. \n",
    "\n",
    "### Key Features\n",
    "\n",
    "- **Data science at all scales**: With Ponder's technology, the same pandas workflows can be run at all scales, from megabytes to terabytes, without changing a single line of code. \n",
    "\n",
    "- **No change to user workflow:** Data scientists can continue running their existing pandas workflows and writing pandas code in their favorite IDE of choice, and benefit from seamless scalability improvements.\n",
    "\n",
    "- **Simplify your data infrastructure:** No need to set up and maintain compute infrastructure required for other parallel processing frameworks (e.g., Spark, Ray, Dask, etc.) to perform large scale data analysis with pandas.\n",
    "\n",
    "- **Guaranteed security:** All your pandas workflows will be executed in BigQuery, thus benefiting from the rigorous security guarantees offered by BigQuery."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "982e72d6-5524-4896-8ee0-6909a2b36ae6",
   "metadata": {},
   "source": [
    "In the following sections, we will showcase some examples of how Ponder works and how it can be used in your work."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ebfe52d-2e42-476c-80ea-e51c3ccddd2e",
   "metadata": {},
   "source": [
    "### Demo: Write SQL no more, Ponder in action!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d0ac7cf-e337-43d3-95d1-ccb9614aee5a",
   "metadata": {},
   "source": [
    "Under the hood, pandas operations are automatically compiled down to SQL queries that get pushed to BigQuery. Queries are executed directly on BigQuery, with users benefiting from the performance, scalability, and security benefits provided by BigQuery as the computation engine.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8c5ee5e-20cc-4f90-8cbe-30053785ddc1",
   "metadata": {},
   "source": [
    "Here is an architecture of how Ponder works: \n",
    "\n",
    "<img src=\"https://ponder.io/wp-content/uploads/2023/04/ponder_architecture.png\" width=\"75%\"></img>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a557836c-9147-4372-a6e4-8ebe1a1d3e4e",
   "metadata": {},
   "source": [
    "To show you that this is actually running in the data warehouse, you can log onto your [BigQuery web interface](https://console.cloud.google.com/bigquery). The pandas operations you execute on Ponder correspond to the SQL queries shown on the `Project History` page in BigQuery web interface."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7ab15648-b41e-4d03-a281-628066ec4899",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_sql(\"PONDER.PONDER_BOOKS\", bigquery_con)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a25e4dd-5d42-4fe4-a010-c2c50d851b87",
   "metadata": {},
   "source": [
    "You can look at the corresponding SQL queries for the pandas operations ran in Ponder by going to `Project History` in your BigQuery web interface. The history page lets you view and drill into the details of queries executed in your BigQuery account in the last 14 days."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13ff8707-2ce5-425e-bf38-64dcd7f885ef",
   "metadata": {},
   "source": [
    "<img src=\"img/bigquery_history.png\" width=\"150%\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41204a3e-48d4-466d-a39a-79dcc8d7fe9b",
   "metadata": {},
   "source": [
    "In this case, you can see that as we connected to the table via `pd.read_sql`, this corresponding SQL query was generated: "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6fa82cc-5a16-477a-8200-34de1c2b8b06",
   "metadata": {},
   "source": [
    "```sql\n",
    "CREATE TEMP TABLE `Ponder_rnzdmeknkw` AS SELECT *, ROW_NUMBER() OVER (ORDER BY 1) -1 AS _PONDER_ROW_NUMBER_ FROM `PONDER.PONDER_BOOKS` \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c415b038-4d16-4db7-97a0-bd8c79c7a82b",
   "metadata": {},
   "source": [
    "You might recall that in the last tutorial, we performed z-score normalization on all the numerical columns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fd3f2587-1b63-4d5c-a997-eca14f47f41d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bookID</th>\n",
       "      <th>average_rating</th>\n",
       "      <th>isbn13</th>\n",
       "      <th>num_pages</th>\n",
       "      <th>ratings_count</th>\n",
       "      <th>text_reviews_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.228041</td>\n",
       "      <td>0.701669</td>\n",
       "      <td>0.065788</td>\n",
       "      <td>1.690193</td>\n",
       "      <td>-0.158898</td>\n",
       "      <td>-0.206879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.733609</td>\n",
       "      <td>1.215243</td>\n",
       "      <td>0.046199</td>\n",
       "      <td>0.728147</td>\n",
       "      <td>-0.158587</td>\n",
       "      <td>-0.204162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.297916</td>\n",
       "      <td>-1.095839</td>\n",
       "      <td>0.067484</td>\n",
       "      <td>0.052226</td>\n",
       "      <td>-0.158409</td>\n",
       "      <td>-0.205715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.528009</td>\n",
       "      <td>-11.224651</td>\n",
       "      <td>0.047099</td>\n",
       "      <td>-0.930554</td>\n",
       "      <td>-0.159493</td>\n",
       "      <td>-0.210372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.345952</td>\n",
       "      <td>-11.224651</td>\n",
       "      <td>0.046189</td>\n",
       "      <td>-0.864206</td>\n",
       "      <td>-0.159493</td>\n",
       "      <td>-0.210372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11118</th>\n",
       "      <td>1.373999</td>\n",
       "      <td>0.872860</td>\n",
       "      <td>0.046125</td>\n",
       "      <td>0.408847</td>\n",
       "      <td>-0.137742</td>\n",
       "      <td>-0.185921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11119</th>\n",
       "      <td>0.186804</td>\n",
       "      <td>0.872860</td>\n",
       "      <td>0.048999</td>\n",
       "      <td>0.587157</td>\n",
       "      <td>-0.158115</td>\n",
       "      <td>-0.208431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11120</th>\n",
       "      <td>-0.193884</td>\n",
       "      <td>0.872860</td>\n",
       "      <td>0.047306</td>\n",
       "      <td>-0.171699</td>\n",
       "      <td>-0.158880</td>\n",
       "      <td>-0.206879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11121</th>\n",
       "      <td>-0.884009</td>\n",
       "      <td>1.586157</td>\n",
       "      <td>0.047147</td>\n",
       "      <td>3.979200</td>\n",
       "      <td>-0.143538</td>\n",
       "      <td>-0.187085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11122</th>\n",
       "      <td>0.476462</td>\n",
       "      <td>1.586157</td>\n",
       "      <td>0.045772</td>\n",
       "      <td>0.661798</td>\n",
       "      <td>-0.141253</td>\n",
       "      <td>-0.112181</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11123 rows x 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         bookID  average_rating    isbn13  num_pages  ratings_count  \\\n",
       "0     -1.228041        0.701669  0.065788   1.690193      -0.158898   \n",
       "1      1.733609        1.215243  0.046199   0.728147      -0.158587   \n",
       "2     -1.297916       -1.095839  0.067484   0.052226      -0.158409   \n",
       "3     -1.528009      -11.224651  0.047099  -0.930554      -0.159493   \n",
       "4      0.345952      -11.224651  0.046189  -0.864206      -0.159493   \n",
       "...         ...             ...       ...        ...            ...   \n",
       "11118  1.373999        0.872860  0.046125   0.408847      -0.137742   \n",
       "11119  0.186804        0.872860  0.048999   0.587157      -0.158115   \n",
       "11120 -0.193884        0.872860  0.047306  -0.171699      -0.158880   \n",
       "11121 -0.884009        1.586157  0.047147   3.979200      -0.143538   \n",
       "11122  0.476462        1.586157  0.045772   0.661798      -0.141253   \n",
       "\n",
       "       text_reviews_count  \n",
       "0               -0.206879  \n",
       "1               -0.204162  \n",
       "2               -0.205715  \n",
       "3               -0.210372  \n",
       "4               -0.210372  \n",
       "...                   ...  \n",
       "11118           -0.185921  \n",
       "11119           -0.208431  \n",
       "11120           -0.206879  \n",
       "11121           -0.187085  \n",
       "11122           -0.112181  \n",
       "\n",
       "[11123 rows x 6 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = df.select_dtypes(include='number').columns\n",
    "(df[x] - df[x].mean())/df[x].std()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee43e51-59ee-463f-8fd1-28aa787b2f05",
   "metadata": {},
   "source": [
    "Now take a look at BigQuery's `Project History`, the corresponding SQL query is 200+ lines long!!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf7f32f7-28b2-4769-9195-7220d30acbc0",
   "metadata": {},
   "source": [
    "```sql\n",
    "SELECT \n",
    "  _PONDER_ROW_LABELS_, \n",
    "  bookID, \n",
    "  average_rating, \n",
    "  isbn13, \n",
    "  num_pages, \n",
    "  ratings_count, \n",
    "  text_reviews_count \n",
    "FROM \n",
    "  (\n",
    "    SELECT \n",
    "      * \n",
    "    FROM \n",
    "      (\n",
    "        SELECT \n",
    "          _PONDER_ROW_NUMBER_, \n",
    "          _PONDER_ROW_LABELS_, \n",
    "          bookID / bookID_ponder_right AS bookID, \n",
    "          average_rating / average_rating_ponder_right AS average_rating, \n",
    "          isbn13 / isbn13_ponder_right AS isbn13, \n",
    "          num_pages / num_pages_ponder_right AS num_pages, \n",
    "          ratings_count / ratings_count_ponder_right AS ratings_count, \n",
    "          text_reviews_count / text_reviews_count_ponder_right AS text_reviews_count \n",
    "        FROM \n",
    "          (\n",
    "            SELECT \n",
    "              _PONDER_ROW_NUMBER_, \n",
    "              _PONDER_ROW_LABELS_, \n",
    "              bookID - bookID_ponder_right AS bookID, \n",
    "              average_rating - average_rating_ponder_right AS average_rating, \n",
    "              isbn13 - isbn13_ponder_right AS isbn13, \n",
    "              num_pages - num_pages_ponder_right AS num_pages, \n",
    "              ratings_count - ratings_count_ponder_right AS ratings_count, \n",
    "              text_reviews_count - text_reviews_count_ponder_right AS text_reviews_count \n",
    "            FROM \n",
    "              (\n",
    "                SELECT \n",
    "                  _PONDER_ROW_NUMBER_, \n",
    "                  _PONDER_ROW_LABELS_, \n",
    "                  bookID, \n",
    "                  average_rating, \n",
    "                  isbn13, \n",
    "                  num_pages, \n",
    "                  ratings_count, \n",
    "                  text_reviews_count \n",
    "                FROM \n",
    "                  (\n",
    "                    SELECT \n",
    "                      bookID, \n",
    "                      title, \n",
    "                      authors, \n",
    "                      average_rating, \n",
    "                      isbn, \n",
    "                      isbn13, \n",
    "                      language_code, \n",
    "                      num_pages, \n",
    "                      ratings_count, \n",
    "                      text_reviews_count, \n",
    "                      publication_date, \n",
    "                      publisher, \n",
    "                      _PONDER_ROW_NUMBER_, \n",
    "                      _PONDER_ROW_NUMBER_ AS _PONDER_ROW_LABELS_ \n",
    "                    FROM \n",
    "                      Ponder_rnzdmeknkw\n",
    "                  )\n",
    "              ) AS _PONDER_LEFT_ CROSS \n",
    "              JOIN (\n",
    "                SELECT \n",
    "                  bookID AS bookID_ponder_right, \n",
    "                  average_rating AS average_rating_ponder_right, \n",
    "                  isbn13 AS isbn13_ponder_right, \n",
    "                  num_pages AS num_pages_ponder_right, \n",
    "                  ratings_count AS ratings_count_ponder_right, \n",
    "                  text_reviews_count AS text_reviews_count_ponder_right \n",
    "                FROM \n",
    "                  (\n",
    "                    SELECT \n",
    "                      0 AS _PONDER_ROW_NUMBER_, \n",
    "                      0 AS _PONDER_ROW_LABELS_, \n",
    "                      AVG(bookID) AS bookID, \n",
    "                      AVG(average_rating) AS average_rating, \n",
    "                      AVG(isbn13) AS isbn13, \n",
    "                      AVG(num_pages) AS num_pages, \n",
    "                      AVG(ratings_count) AS ratings_count, \n",
    "                      AVG(text_reviews_count) AS text_reviews_count \n",
    "                    FROM \n",
    "                      (\n",
    "                        SELECT \n",
    "                          CAST(bookID AS FLOAT64) AS bookID, \n",
    "                          CAST(average_rating AS FLOAT64) AS average_rating, \n",
    "                          CAST(isbn13 AS FLOAT64) AS isbn13, \n",
    "                          CAST(num_pages AS FLOAT64) AS num_pages, \n",
    "                          CAST(ratings_count AS FLOAT64) AS ratings_count, \n",
    "                          CAST(text_reviews_count AS FLOAT64) AS text_reviews_count, \n",
    "                          _PONDER_ROW_LABELS_, \n",
    "                          _PONDER_ROW_NUMBER_ \n",
    "                        FROM \n",
    "                          (\n",
    "                            SELECT \n",
    "                              _PONDER_ROW_NUMBER_, \n",
    "                              _PONDER_ROW_LABELS_, \n",
    "                              bookID, \n",
    "                              average_rating, \n",
    "                              isbn13, \n",
    "                              num_pages, \n",
    "                              ratings_count, \n",
    "                              text_reviews_count \n",
    "                            FROM \n",
    "                              (\n",
    "                                SELECT \n",
    "                                  bookID, \n",
    "                                  title, \n",
    "                                  authors, \n",
    "                                  average_rating, \n",
    "                                  isbn, \n",
    "                                  isbn13, \n",
    "                                  language_code, \n",
    "                                  num_pages, \n",
    "                                  ratings_count, \n",
    "                                  text_reviews_count, \n",
    "                                  publication_date, \n",
    "                                  publisher, \n",
    "                                  _PONDER_ROW_NUMBER_, \n",
    "                                  _PONDER_ROW_NUMBER_ AS _PONDER_ROW_LABELS_ \n",
    "                                FROM \n",
    "                                  Ponder_rnzdmeknkw\n",
    "                              )\n",
    "                          )\n",
    "                      ) \n",
    "                    LIMIT \n",
    "                      1\n",
    "                  )\n",
    "              ) AS _PONDER_RIGHT_\n",
    "          ) AS _PONDER_LEFT_ CROSS \n",
    "          JOIN (\n",
    "            SELECT \n",
    "              bookID AS bookID_ponder_right, \n",
    "              average_rating AS average_rating_ponder_right, \n",
    "              isbn13 AS isbn13_ponder_right, \n",
    "              num_pages AS num_pages_ponder_right, \n",
    "              ratings_count AS ratings_count_ponder_right, \n",
    "              text_reviews_count AS text_reviews_count_ponder_right \n",
    "            FROM \n",
    "              (\n",
    "                SELECT \n",
    "                  0 AS _PONDER_ROW_NUMBER_, \n",
    "                  0 AS _PONDER_ROW_LABELS_, \n",
    "                  STDDEV(bookID) AS bookID, \n",
    "                  STDDEV(average_rating) AS average_rating, \n",
    "                  STDDEV(isbn13) AS isbn13, \n",
    "                  STDDEV(num_pages) AS num_pages, \n",
    "                  STDDEV(ratings_count) AS ratings_count, \n",
    "                  STDDEV(text_reviews_count) AS text_reviews_count \n",
    "                FROM \n",
    "                  (\n",
    "                    SELECT \n",
    "                      _PONDER_ROW_NUMBER_, \n",
    "                      _PONDER_ROW_LABELS_, \n",
    "                      bookID, \n",
    "                      average_rating, \n",
    "                      isbn13, \n",
    "                      num_pages, \n",
    "                      ratings_count, \n",
    "                      text_reviews_count \n",
    "                    FROM \n",
    "                      (\n",
    "                        SELECT \n",
    "                          bookID, \n",
    "                          title, \n",
    "                          authors, \n",
    "                          average_rating, \n",
    "                          isbn, \n",
    "                          isbn13, \n",
    "                          language_code, \n",
    "                          num_pages, \n",
    "                          ratings_count, \n",
    "                          text_reviews_count, \n",
    "                          publication_date, \n",
    "                          publisher, \n",
    "                          _PONDER_ROW_NUMBER_, \n",
    "                          _PONDER_ROW_NUMBER_ AS _PONDER_ROW_LABELS_ \n",
    "                        FROM \n",
    "                          Ponder_rnzdmeknkw\n",
    "                      )\n",
    "                  ) \n",
    "                LIMIT \n",
    "                  1\n",
    "              )\n",
    "          ) AS _PONDER_RIGHT_\n",
    "      ) \n",
    "    WHERE \n",
    "      _PONDER_ROW_NUMBER_ IN (\n",
    "        0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, \n",
    "        14, 15, 16, 17, 18, 19, 20, 21, 22, 23, \n",
    "        24, 25, 26, 27, 28, 29, 30, 11092, 11093, \n",
    "        11094, 11095, 11096, 11097, 11098, \n",
    "        11099, 11100, 11101, 11102, 11103, \n",
    "        11104, 11105, 11106, 11107, 11108, \n",
    "        11109, 11110, 11111, 11112, 11113, \n",
    "        11114, 11115, 11116, 11117, 11118, \n",
    "        11119, 11120, 11121, 11122\n",
    "      )\n",
    "  ) \n",
    "ORDER BY \n",
    "  _PONDER_ROW_NUMBER_ \n",
    "LIMIT \n",
    "  10001\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c101d4a3-e54b-4fc0-8994-20a1c1b75821",
   "metadata": {},
   "source": [
    "In this example, we saw how something as easy to express in pandas in a single line can in fact take *many* lines of SQL to write. \n",
    "\n",
    "Using Ponder leads to huge time-savings since you can think and work natively in pandas when interacting with your data warehouse."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5a06841-eb62-4b01-9465-e84624df20f8",
   "metadata": {},
   "source": [
    "### Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "912d10c4-c4c2-4631-8c51-c908a97ca21e",
   "metadata": {},
   "source": [
    "In this tutorial, we saw how Ponder lets you run pandas on BigQuery. Ponder simplifies your experience in working with data. It does this by translating your pandas queries to corresponding SQL queries to run on your data warehouse. Ponder gives you the flexibility of working in pandas directly and often there are queries that are easier to write in pandas than having to craft hundreds of lines of SQL!\n",
    "\n",
    "As we can see, there are many benefits from being able to leverage the pandas API (over writing SQL directly) on your data warehouse, as summarized in this table. \n",
    "\n",
    "|               | pandas | SQL | Ponder |\n",
    "|---------------|--------|-----|--------|\n",
    "| Easy to use   | ✅      | ❌   | ✅      |\n",
    "| Flexible      | ✅      | ❌   | ✅      |\n",
    "| Scalable      | ❌      | ✅   | ✅      |\n",
    "| Secure access | ❌      | ✅   | ✅      |\n",
    "\n",
    "\n",
    "To learn more about Ponder, check out our product blogpost [here](https://ponder.io/run-pandas-on-1tb-directly-in-your-data-warehouse/). \n",
    "\n",
    "In our [next tutorial](https://github.com/ponder-org/ponder-notebooks/blob/main/bigquery/tutorial/03-reading-data.ipynb), you will learn about how you can use the pandas's I/O methods to read from your database, CSV file, or a Parquet file."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}