foodnames_gan.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recipe Generation with Seq2Seq GAN\n",
    "\n",
    "In this notebook, I train a Seq2Seq autoencoder to encode and decode recipe names. I also train a Generative Adversarial Network (GAN) alongside this to create a generator of fake recipe names and a discriminator of real versus fake recipes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0EsGxJqwgmcg"
   },
   "source": [
    "## Load and preprocess data\n",
    "\n",
    "First, we must acquire the data. For my experiment, I used data from [Eight Portions](https://eightportions.com/datasets/Recipes/), who provide a very useful dataset of recipes including names, ingredients, and directions. I only plan to use the names of the recipes, so I will trim the data for that information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 272
    },
    "colab_type": "code",
    "id": "hbxsRoblagyD",
    "outputId": "dc1c7b3e-41b7-4a60-f203-764217806105"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 50.8M  100 50.8M    0     0  9773k      0  0:00:05  0:00:05 --:--:--  9.7M\n",
      "Archive:  recipes.zip\n",
      "  inflating: recipes_raw_nosource_ar.json  \n",
      "  inflating: recipes_raw_nosource_epi.json  \n",
      "  inflating: recipes_raw_nosource_fn.json  \n",
      "  inflating: LICENSE                 \n"
     ]
    }
   ],
   "source": [
    "# Download the recipes\n",
    "# Source: https://eightportions.com/datasets/Recipes/\n",
    "!curl -o recipes.zip 'https://storage.googleapis.com/recipe-box/recipes_raw.zip'\n",
    "\n",
    "# Unzip without remorse\n",
    "!unzip -o recipes.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "id": "ZETwTGLIasFj",
    "outputId": "0fe6409a-a43c-4df0-efb9-21654ac3bea5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "  \"rmK12Uau.ntP510KeImX506H6Mr6jTu\": {\r\n",
      "    \"title\": \"Slow Cooker Chicken and Dumplings\",\r\n",
      "    \"ingredients\": [\r\n",
      "      \"4 skinless, boneless chicken breast halves ADVERTISEMENT\",\r\n",
      "      \"2 tablespoons butter ADVERTISEMENT\",\r\n",
      "      \"2 (10.75 ounce) cans condensed cream of chicken soup ADVERTISEMENT\",\r\n",
      "      \"1 onion, finely diced ADVERTISEMENT\",\r\n",
      "      \"2 (10 ounce) packages refrigerated biscuit dough, torn into pieces ADVERTISEMENT\",\r\n",
      "      \"ADVERTISEMENT\"\r\n"
     ]
    }
   ],
   "source": [
    "!head recipes_raw_nosource_ar.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When extracting the names, I found that some recipes did not have names. So I had to filter those out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8jTTzDG9bOya"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "# Load the JSON data\n",
    "with open('recipes_raw_nosource_ar.json', 'r') as f:\n",
    "    data = json.load(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "5CV-uc4dbinQ",
    "outputId": "908287f4-7b55-44ab-9a35-f392010b3062"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Slow Cooker Chicken and Dumplings\n",
      "Awesome Slow Cooker Pot Roast\n",
      "Brown Sugar Meatloaf\n",
      "Best Chocolate Chip Cookies\n",
      "Homemade Mac and Cheese Casserole\n",
      "Banana Banana Bread\n",
      "Chef John's Fisherman's Pie\n",
      "Mom's Zucchini Bread\n",
      "The Best Rolled Sugar Cookies\n",
      "Singapore Chili Crabs\n",
      "39522 of 39802\n"
     ]
    }
   ],
   "source": [
    "# Pull out the names\n",
    "names = [data[k]['title'] for k in data if 'title' in data[k]]\n",
    "\n",
    "for r in names[:10]:\n",
    "    print(r)\n",
    "\n",
    "print(len(names), 'of', len(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocess_string(txt):\n",
    "    # Trim non-unicode\n",
    "    for i in range(len(txt)-1, -1, -1):\n",
    "        if ord(txt[i]) > 127:\n",
    "            txt = txt[:i] + txt[i+1:]\n",
    "            \n",
    "    return (txt\n",
    "            .replace('(', ' ( ') # Left parentheses\n",
    "            .replace(')', ' ) ') # Right parentheses\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "names = list(map(preprocess_string, names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1ve_KX6UdPSK"
   },
   "source": [
    "## Tokenize data\n",
    "\n",
    "In order to pass the strings in, we need to create a numerical representation that can be used by the network. I define methods for encoding  a given string of text or decoding label predictions into a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8GG35yAZevnM"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import math, random\n",
    "\n",
    "import tensorflow_datasets as tfds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8qajOFm0b18d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['k', 'b', 'J', '+', 'd', 'f', 'r', 'Y', 'l', 'z', '(', 'V', 'X', '!', '7', 'n', 'e', '$', 'v', 'w', '.', '@', ':', '?', '0', 'C', 'h', 'R', 'H', 'I', '5', 'q', 'o', 'A', '%', 'K', ')', 'p', \"'\", '8', ';', '&', 'W', 'u', 't', 'D', 'M', 'T', 'E', 'N', 'B', 'Q', 'U', '#', 'G', 'S', 'P', 'c', 'g', '4', '3', 'L', 'x', '-', ',', '6', '\\n', '\\t', 'j', '*', 'a', '9', 'O', '2', '/', '\"', 'F', 'Z', 'y', 'i', '1', ' ', 'm', '=', 's']\n"
     ]
    }
   ],
   "source": [
    "# Create a tokenizer\n",
    "chars = set(c for n in names for c in n)\n",
    "words = set(w for n in names for w in n.split())\n",
    "\n",
    "chars.add('\\t')\n",
    "words.add('<start>')\n",
    "\n",
    "chars.add('\\n')\n",
    "words.add('<end>')\n",
    "\n",
    "chars = list(chars)\n",
    "words = list(words)\n",
    "\n",
    "print(chars)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "csIPpyJpdOT1"
   },
   "outputs": [],
   "source": [
    "word_inv_idx = {i+1 : w for i, w in enumerate(words)}\n",
    "word_idx = {w : i+1 for i, w in enumerate(words)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Eyo63GZHMF93",
    "outputId": "c35842c6-87d1-4cc2-a102-0848fd4c4e51"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocab size: 11922\n"
     ]
    }
   ],
   "source": [
    "vocab_size = 1 + len(words)\n",
    "\n",
    "print('Vocab size:', vocab_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can define the tokenization and detokenization behavior. We define two methods:\n",
    "\n",
    "- `str2tok()`: Converts text to tokens\n",
    "- `tok2str()`: Converts tokens to text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QJd2ENtIceNt"
   },
   "outputs": [],
   "source": [
    "def str2tok(txt, length=None):\n",
    "    # Split the string and break into tokens\n",
    "    enc = [word_idx[w] for w in txt.split()]\n",
    "    \n",
    "    if length:\n",
    "        enc += (length - len(enc)) * [0]\n",
    "        \n",
    "    return enc\n",
    "\n",
    "def tok2str(idxs):\n",
    "    # Rejoin\n",
    "    return ' '.join([word_inv_idx[i] for i in idxs if i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using this, we can tokenize the data into the format usable by the neural networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 374
    },
    "colab_type": "code",
    "id": "wRtPoazLeeEg",
    "outputId": "b7cf3446-98c5-4dd9-8352-9a98b41f7204"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Max length: 20\n",
      "[ 6096 11194  6637  7454  7740     0     0     0     0     0     0     0\n",
      "     0     0     0     0     0     0     0     0]\n",
      "[ 1776  6096 11194  6637  7454  7740     0     0     0     0     0     0\n",
      "     0     0     0     0     0     0     0     0     0     0]\n",
      "[ 6096 11194  6637  7454  7740  1437     0     0     0     0     0     0\n",
      "     0     0     0     0     0     0     0     0     0     0]\n",
      "(39522, 20)\n",
      "(39522, 22)\n",
      "(39522, 22)\n"
     ]
    }
   ],
   "source": [
    "# Encode the dataset\n",
    "max_len = max(len(str2tok(n)) for n in names)\n",
    "\n",
    "enc_input = np.array([str2tok(n, max_len) for n in names])\n",
    "dec_input = np.array([str2tok(f\"<start> {n}\", 2+max_len) for n in names])\n",
    "dec_output = np.array([str2tok(f\"{n} <end>\", 2+max_len) for n in names])\n",
    "\n",
    "print('Max length:', max_len)\n",
    "print(enc_input[0])\n",
    "print(dec_input[0])\n",
    "print(dec_output[0])\n",
    "\n",
    "print(enc_input.shape)\n",
    "print(dec_input.shape)\n",
    "print(dec_output.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to define a training/test split. This allows us to check for overfitting of the data during hyperparameter tuning. I do this by generating a list of indices and splitting them into two buckets: one that's used to train and one used solely for evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MTY81LE9qurx"
   },
   "outputs": [],
   "source": [
    "# Train-test-valid split\n",
    "idxs = list(range(len(enc_input)))\n",
    "random.shuffle(idxs)\n",
    "\n",
    "a = int(0.9 * len(idxs))\n",
    "train_idxs = idxs[:a]\n",
    "test_idxs = idxs[a:]\n",
    "idxs = train_idxs\n",
    "\n",
    "a = int(0.9 * len(idxs))\n",
    "train_idxs = idxs[:a]\n",
    "valid_idxs = idxs[a:]\n",
    "del idxs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8FNuEDtLhBaT"
   },
   "source": [
    "## Model\n",
    "\n",
    "I used a Seq2Seq model to convert to and from text. This constitutes an encoder to convert text to an encoding, and a decoder to convert it back to text. I also construct a discriminator that trains to distinguish between real and fake encodings, as well as a generator to create encodings that the discriminator thinks are real."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "gt76wwAygcEF",
    "outputId": "2417573e-c99e-46df-891b-97b87d374e0c"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "from tensorflow.keras.layers import *\n",
    "from tensorflow.keras.models import *\n",
    "from tensorflow.keras.optimizers import *\n",
    "\n",
    "# Silence warnings\n",
    "import logging\n",
    "tf.get_logger().setLevel(logging.ERROR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vvpl_vPehoDP"
   },
   "outputs": [],
   "source": [
    "latent_dim = 64\n",
    "dropout = 0.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Y8x987EykQiR"
   },
   "source": [
    "### Encoder\n",
    "\n",
    "The Seq2Seq encoder includes an embedding layer that converts a token into a representative vector and a recurrent neural network (RNN) component. The RNN takes in a sequence/list of these vectors and processes them one by one to compute a representative encoding of the input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 513
    },
    "colab_type": "code",
    "id": "GQVepj1FhZki",
    "outputId": "943751b5-78f0-4eb4-dfd9-d225c9206e70"
   },
   "outputs": [],
   "source": [
    "enc_in = Input(shape=(None,), name='enc_in')\n",
    "\n",
    "# Apply an embedding to the input\n",
    "emb = Embedding(vocab_size, latent_dim)\n",
    "y = emb(enc_in)\n",
    "\n",
    "# Pass through an RNN\n",
    "rnn = Bidirectional(GRU(latent_dim // 2, return_state=True, dropout=dropout))\n",
    "_, h1, h2 = rnn(y)\n",
    "\n",
    "# Concatenate the output states\n",
    "h = Concatenate()([h1, h2])\n",
    "\n",
    "encoder = Model(enc_in, h, name='encoder')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x8NvFwg_kSE5"
   },
   "source": [
    "### Decoder\n",
    "\n",
    "The decoder goes in the other way; taking the encoding and converting it into a list of characters. However, the network generates probabilities. This allows us to randomize our results or deterministically select the most likely character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 343
    },
    "colab_type": "code",
    "id": "YCkxZFUgieS1",
    "outputId": "a6614fd5-c249-4eb0-bf2b-32df67d8237f"
   },
   "outputs": [],
   "source": [
    "dec_in = Input(shape=(None,), name='dec_in')\n",
    "h = Input(shape=(latent_dim,), name='state_in')\n",
    "\n",
    "# Embed the decoder input\n",
    "emb = Embedding(vocab_size, latent_dim)\n",
    "y = emb(dec_in)\n",
    "\n",
    "# Pass through a generator LSTM\n",
    "rnn = GRU(latent_dim, return_state=True, return_sequences=True, dropout=dropout)\n",
    "y, c = rnn(y, initial_state = h)\n",
    "\n",
    "# Choose the the character by computing a probability distribution\n",
    "dense = Dense(vocab_size, activation='softmax')\n",
    "y = dense(y)\n",
    "\n",
    "decoder = Model([dec_in, h], [y, c], name='decoder')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GAN\n",
    "\n",
    "A Generative Adversarial Network (GAN) consists of a generator and a discriminator. The goal is to train two networks:\n",
    "\n",
    "- A *discriminator* that can classify encodings as real (made from real recipe names) or fake (made by some other process)\n",
    "- A *generator* that can create encodings that the discriminator believes are real."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def wrap(mdl):\n",
    "    \"\"\" Given a Model, create a new Model that computes the same values as\n",
    "    the input model, but cannot be trained. Used for GAN procedures.\n",
    "    \"\"\"\n",
    "    # Build the input(s)\n",
    "    if isinstance(mdl.input_shape, list):\n",
    "        x = [Input(v[1:]) for v in mdl.input_shape]\n",
    "    else:\n",
    "        x = Input(mdl.input_shape[1:])\n",
    "    \n",
    "    # Build the new model\n",
    "    fn = Model(x, mdl(x))\n",
    "    \n",
    "    # Save the trainability of the model\n",
    "    trainable = mdl.trainable\n",
    "    # The new model is untrainable\n",
    "    fn.trainable = False\n",
    "    # The old model stays the way it was\n",
    "    mdl.trainable = trainable\n",
    "    \n",
    "    return fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Discriminator\n",
    "h = Input(shape=(latent_dim,), name='encoding')\n",
    "y = Dense(64, activation='relu')(h)\n",
    "y = Dense(1, activation='sigmoid')(y)\n",
    "discriminator = Model(h, y, name='discriminator')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generator\n",
    "x = Input(shape=(latent_dim,), name='noise')\n",
    "y = Dense(latent_dim, activation='relu')(x)\n",
    "y = Dense(latent_dim, activation='tanh')(y)\n",
    "generator = Model(x, y, name='generator')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IM1b0R73kT3A"
   },
   "source": [
    "### Trainers\n",
    "\n",
    "Now, we can define training procedures. To train, I built four training models:\n",
    "\n",
    "- `ae_train`: Trains the autoencoder component (encoder and decoder)\n",
    "- `gen_train`: Trains the generator to maximize the discriminator's score\n",
    "- `dsc_real_train`: Trains the discriminator to recognize real inputs from the encoder\n",
    "- `dsc_fake_train`: Trains the discriminator to recognize fake inputs from the generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wrappers for all of the previously defined layers are used\n",
    "# to build the trainers. This ensures weights are trained at\n",
    "# the right time.\n",
    "enc_wrap = wrap(encoder)\n",
    "dec_wrap = wrap(decoder)\n",
    "dsc_wrap = wrap(discriminator)\n",
    "gen_wrap = wrap(generator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 309
    },
    "colab_type": "code",
    "id": "mJHF1mBQj9Qn",
    "outputId": "ef4b4d20-3f11-4887-86d6-bf9f04a516a1"
   },
   "outputs": [],
   "source": [
    "# Model to train an autoencoder\n",
    "enc_in = Input(shape=(None,), name='enc_in')\n",
    "dec_in = Input(shape=(None,), name='dec_in')\n",
    "\n",
    "z = encoder(enc_in)\n",
    "y, _ = decoder([dec_in, z])\n",
    "\n",
    "ae_train = Model([enc_in, dec_in], y, name='autoencoder_trainer')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generator trainer\n",
    "noise = Input(shape=(latent_dim,), name='noise')\n",
    "h = generator(noise)\n",
    "y = dsc_wrap(h)\n",
    "gen_train = Model(noise, y, name='gen_trainer')\n",
    "\n",
    "# Discriminator fake trainer\n",
    "noise = Input(shape=(latent_dim,), name='noise')\n",
    "h = gen_wrap(noise)\n",
    "y = discriminator(h)\n",
    "dsc_fake_train = Model(noise, y, name='dsc_fake_trainer')\n",
    "\n",
    "# Discriminator real trainer\n",
    "enc_in = Input(shape=(None,))\n",
    "h = enc_wrap(enc_in)\n",
    "y = discriminator(h)\n",
    "dsc_real_train = Model(enc_in, y, name='dsc_real_trainer')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bARxHfVakwd9"
   },
   "source": [
    "## Model training\n",
    "\n",
    "Now, we can train our model. I define a data generator that provides data one batch at a time. This is done because the actual values used by the network would require a massive amount of memory to store."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XQqaawWOluhG"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.utils import to_categorical\n",
    "\n",
    "def data_gen(idxs, batch_size=64, repeat=True):\n",
    "    not_done = True\n",
    "    \n",
    "    while not_done:\n",
    "        random.shuffle(idxs)\n",
    "\n",
    "        for i in range(batch_size, len(idxs), batch_size):\n",
    "            # Chosen items\n",
    "            i = idxs[i-batch_size:i]\n",
    "\n",
    "            # Input\n",
    "            xe = enc_input[i]\n",
    "            xd = dec_input[i]\n",
    "            \n",
    "            j = 0\n",
    "            while j < max_len:\n",
    "                if all(xe[:,j] == 0):\n",
    "                    break\n",
    "                j += 1\n",
    "\n",
    "            # Output\n",
    "            y = dec_output[i]\n",
    "            data = np.zeros((batch_size, y.shape[1], vocab_size))\n",
    "            for i in range(len(data)):\n",
    "                data[i] = to_categorical(y[i], num_classes=vocab_size)\n",
    "              \n",
    "            xe = xe[:,:j]\n",
    "            xd = xd[:,:j+1]\n",
    "            data = data[:,:j+1]\n",
    "            \n",
    "            yield [xe, xd], data\n",
    "        \n",
    "        not_done = repeat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"encoder\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "enc_in (InputLayer)             [(None, None)]       0                                            \n",
      "__________________________________________________________________________________________________\n",
      "embedding (Embedding)           (None, None, 64)     763008      enc_in[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bidirectional (Bidirectional)   [(None, 64), (None,  18816       embedding[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "concatenate (Concatenate)       (None, 64)           0           bidirectional[0][1]              \n",
      "                                                                 bidirectional[0][2]              \n",
      "==================================================================================================\n",
      "Total params: 781,824\n",
      "Trainable params: 781,824\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "Model: \"decoder\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "dec_in (InputLayer)             [(None, None)]       0                                            \n",
      "__________________________________________________________________________________________________\n",
      "embedding_1 (Embedding)         (None, None, 64)     763008      dec_in[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "state_in (InputLayer)           [(None, 64)]         0                                            \n",
      "__________________________________________________________________________________________________\n",
      "gru_1 (GRU)                     [(None, None, 64), ( 24960       embedding_1[0][0]                \n",
      "                                                                 state_in[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dense (Dense)                   (None, None, 11922)  774930      gru_1[0][0]                      \n",
      "==================================================================================================\n",
      "Total params: 1,562,898\n",
      "Trainable params: 1,562,898\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "Model: \"discriminator\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "encoding (InputLayer)        [(None, 64)]              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 4,225\n",
      "Trainable params: 4,225\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Model: \"generator\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "noise (InputLayer)           [(None, 64)]              0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 64)                4160      \n",
      "=================================================================\n",
      "Total params: 8,320\n",
      "Trainable params: 8,320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "encoder.summary()\n",
    "decoder.summary()\n",
    "discriminator.summary()\n",
    "generator.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"autoencoder_trainer\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "enc_in (InputLayer)             [(None, None)]       0                                            \n",
      "__________________________________________________________________________________________________\n",
      "dec_in (InputLayer)             [(None, None)]       0                                            \n",
      "__________________________________________________________________________________________________\n",
      "encoder (Model)                 (None, 64)           781824      enc_in[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "decoder (Model)                 [(None, None, 11922) 1562898     dec_in[0][0]                     \n",
      "                                                                 encoder[2][0]                    \n",
      "==================================================================================================\n",
      "Total params: 2,344,722\n",
      "Trainable params: 2,344,722\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "Model: \"gen_trainer\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "noise (InputLayer)           [(None, 64)]              0         \n",
      "_________________________________________________________________\n",
      "generator (Model)            (None, 64)                8320      \n",
      "_________________________________________________________________\n",
      "model_2 (Model)              (None, 1)                 4225      \n",
      "=================================================================\n",
      "Total params: 12,545\n",
      "Trainable params: 8,320\n",
      "Non-trainable params: 4,225\n",
      "_________________________________________________________________\n",
      "Model: \"dsc_fake_trainer\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "noise (InputLayer)           [(None, 64)]              0         \n",
      "_________________________________________________________________\n",
      "model_3 (Model)              (None, 64)                8320      \n",
      "_________________________________________________________________\n",
      "discriminator (Model)        (None, 1)                 4225      \n",
      "=================================================================\n",
      "Total params: 12,545\n",
      "Trainable params: 4,225\n",
      "Non-trainable params: 8,320\n",
      "_________________________________________________________________\n",
      "Model: \"dsc_real_trainer\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_6 (InputLayer)         [(None, None)]            0         \n",
      "_________________________________________________________________\n",
      "model (Model)                (None, 64)                781824    \n",
      "_________________________________________________________________\n",
      "discriminator (Model)        (None, 1)                 4225      \n",
      "=================================================================\n",
      "Total params: 786,049\n",
      "Trainable params: 4,225\n",
      "Non-trainable params: 781,824\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "ae_train.summary()\n",
    "gen_train.summary()\n",
    "dsc_fake_train.summary()\n",
    "dsc_real_train.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 105
    },
    "colab_type": "code",
    "id": "CklGcr74kXeN",
    "outputId": "0110908d-41df-4051-e79b-f592d365c4a8"
   },
   "outputs": [],
   "source": [
    "ae_train.compile(optimizer=Adam(lr=1e-3), loss='categorical_crossentropy')\n",
    "gen_train.compile(optimizer=Adam(lr=1e-3), loss='binary_crossentropy')\n",
    "dsc_fake_train.compile(optimizer=Adam(lr=2e-3), loss='binary_crossentropy')\n",
    "dsc_real_train.compile(optimizer=Adam(lr=2e-3), loss='binary_crossentropy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 32\n",
    "batch_size = 64\n",
    "steps_per_epoch = len(train_idxs) // batch_size\n",
    "validation_steps = len(valid_idxs) // batch_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train the GAN architecture, we will need to train all three of the GAN models at the same time. This creates the process of competition between the discriminator and the generator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_iter():\n",
    "    # Training round\n",
    "    loss = [0 for _ in range(3)]\n",
    "    for step, (x, y) in enumerate(data_gen(train_idxs, batch_size=batch_size, repeat=False)):\n",
    "        # Train on real data\n",
    "        loss[0] += dsc_real_train.train_on_batch(x, np.ones((batch_size,)))\n",
    "        \n",
    "        # Train on fake data\n",
    "        noise = np.random.normal(0, 1, size=(batch_size, latent_dim))\n",
    "        loss[1] += gen_train.train_on_batch(noise, np.ones((batch_size,)))\n",
    "        loss[2] += dsc_fake_train.train_on_batch(noise, np.zeros((batch_size,)))\n",
    "        \n",
    "        # Display the loss\n",
    "        print(f'\\r{step+1}/{steps_per_epoch} loss:', end='')\n",
    "        for x in loss:\n",
    "            print(f' {x / (1+step):.4f}', end='')\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 513
    },
    "colab_type": "code",
    "id": "uFQQ_fgBkqvD",
    "outputId": "77f31242-90fa-4ae3-86b0-af515bdda8b0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training autoencoder\n",
      "Epoch 1/32\n",
      "500/500 [==============================] - 186s 372ms/step - loss: 3.5973 - val_loss: 2.6832\n",
      "Epoch 2/32\n",
      "500/500 [==============================] - 167s 333ms/step - loss: 2.5580 - val_loss: 2.4582\n",
      "Epoch 3/32\n",
      "500/500 [==============================] - 184s 367ms/step - loss: 2.3299 - val_loss: 2.2415\n",
      "Epoch 4/32\n",
      "500/500 [==============================] - 175s 350ms/step - loss: 2.1309 - val_loss: 2.0598\n",
      "Epoch 5/32\n",
      "500/500 [==============================] - 160s 321ms/step - loss: 1.9560 - val_loss: 1.9949\n",
      "Epoch 6/32\n",
      "500/500 [==============================] - 154s 309ms/step - loss: 1.8319 - val_loss: 1.8489\n",
      "Epoch 7/32\n",
      "500/500 [==============================] - 163s 326ms/step - loss: 1.7090 - val_loss: 1.7748\n",
      "Epoch 8/32\n",
      "500/500 [==============================] - 172s 343ms/step - loss: 1.6051 - val_loss: 1.6931\n",
      "Epoch 9/32\n",
      "500/500 [==============================] - 161s 321ms/step - loss: 1.5024 - val_loss: 1.6162\n",
      "Epoch 10/32\n",
      "500/500 [==============================] - 159s 317ms/step - loss: 1.4161 - val_loss: 1.6041\n",
      "Epoch 11/32\n",
      "500/500 [==============================] - 168s 336ms/step - loss: 1.3458 - val_loss: 1.5050\n",
      "Epoch 12/32\n",
      "500/500 [==============================] - 170s 341ms/step - loss: 1.2753 - val_loss: 1.4598\n",
      "Epoch 13/32\n",
      "500/500 [==============================] - 171s 341ms/step - loss: 1.2106 - val_loss: 1.4565\n",
      "Epoch 14/32\n",
      "500/500 [==============================] - 169s 337ms/step - loss: 1.1479 - val_loss: 1.3672\n",
      "Epoch 15/32\n",
      "500/500 [==============================] - 176s 351ms/step - loss: 1.0881 - val_loss: 1.3344\n",
      "Epoch 16/32\n",
      "500/500 [==============================] - 171s 343ms/step - loss: 1.0230 - val_loss: 1.2879\n",
      "Epoch 17/32\n",
      "500/500 [==============================] - 172s 343ms/step - loss: 0.9783 - val_loss: 1.2503\n",
      "Epoch 18/32\n",
      "500/500 [==============================] - 169s 337ms/step - loss: 0.9255 - val_loss: 1.2681\n",
      "Epoch 19/32\n",
      "500/500 [==============================] - 173s 345ms/step - loss: 0.8835 - val_loss: 1.1786\n",
      "Epoch 20/32\n",
      "500/500 [==============================] - 170s 341ms/step - loss: 0.8365 - val_loss: 1.1474\n",
      "Epoch 21/32\n",
      "500/500 [==============================] - 172s 345ms/step - loss: 0.7945 - val_loss: 1.1521\n",
      "Epoch 22/32\n",
      "500/500 [==============================] - 175s 351ms/step - loss: 0.7570 - val_loss: 1.1163\n",
      "Epoch 23/32\n",
      "500/500 [==============================] - 173s 346ms/step - loss: 0.7226 - val_loss: 1.1059\n",
      "Epoch 24/32\n",
      "500/500 [==============================] - 176s 352ms/step - loss: 0.6905 - val_loss: 1.0449\n",
      "Epoch 25/32\n",
      "500/500 [==============================] - 169s 339ms/step - loss: 0.6533 - val_loss: 1.0493\n",
      "Epoch 26/32\n",
      "500/500 [==============================] - 165s 329ms/step - loss: 0.6275 - val_loss: 1.0342\n",
      "Epoch 27/32\n",
      "500/500 [==============================] - 181s 362ms/step - loss: 0.5981 - val_loss: 1.0040\n",
      "Epoch 28/32\n",
      "500/500 [==============================] - 178s 356ms/step - loss: 0.5711 - val_loss: 0.9874\n",
      "Epoch 29/32\n",
      "500/500 [==============================] - 190s 380ms/step - loss: 0.5465 - val_loss: 0.9423\n",
      "Epoch 30/32\n",
      "500/500 [==============================] - 186s 372ms/step - loss: 0.5144 - val_loss: 0.9979\n",
      "Epoch 31/32\n",
      "500/500 [==============================] - 188s 377ms/step - loss: 0.4967 - val_loss: 0.9686\n",
      "Epoch 32/32\n",
      "500/500 [==============================] - 198s 396ms/step - loss: 0.4753 - val_loss: 0.9652\n",
      "Training GAN\n",
      "Epoch 1/32\n",
      "500/500 loss: 0.1939 0.7032 0.8967 0.1434 0.8054 0.7996\n",
      "Epoch 2/32\n",
      "500/500 loss: 0.2016 0.5131 1.2110\n",
      "Epoch 3/32\n",
      "500/500 loss: 0.2346 0.3468 1.6379\n",
      "Epoch 4/32\n",
      "500/500 loss: 0.1933 0.6256 1.3615\n",
      "Epoch 5/32\n",
      "500/500 loss: 0.2232 0.4255 1.5950\n",
      "Epoch 6/32\n",
      "500/500 loss: 0.2533 0.2574 1.9724\n",
      "Epoch 7/32\n",
      "500/500 loss: 0.2620 0.3256 1.8985\n",
      "Epoch 8/32\n",
      "500/500 loss: 0.2451 0.5850 1.6240\n",
      "Epoch 9/32\n",
      "500/500 loss: 0.2274 0.4738 1.5607\n",
      "Epoch 10/32\n",
      "500/500 loss: 0.2527 0.3467 1.8419\n",
      "Epoch 11/32\n",
      "500/500 loss: 0.2626 0.3157 1.9967\n",
      "Epoch 12/32\n",
      "500/500 loss: 0.2509 0.3994 1.8541\n",
      "Epoch 13/32\n",
      "500/500 loss: 0.2600 0.4931 1.8350\n",
      "Epoch 14/32\n",
      "500/500 loss: 0.2783 0.2708 2.1145\n",
      "Epoch 15/32\n",
      "500/500 loss: 0.2732 0.2050 2.2425\n",
      "Epoch 16/32\n",
      "500/500 loss: 0.2744 0.1811 2.2848\n",
      "Epoch 17/32\n",
      "500/500 loss: 0.2852 0.3056 2.2130\n",
      "Epoch 18/32\n",
      "500/500 loss: 0.2720 0.6155 1.8588\n",
      "Epoch 19/32\n",
      "500/500 loss: 0.2582 0.2920 1.9363\n",
      "Epoch 20/32\n",
      "500/500 loss: 0.2731 0.2220 2.1656\n",
      "Epoch 21/32\n",
      "500/500 loss: 0.2779 0.1852 2.2943\n",
      "Epoch 22/32\n",
      "500/500 loss: 0.2917 0.3605 2.2603\n",
      "Epoch 23/32\n",
      "500/500 loss: 0.2562 1.0274 1.6834\n",
      "Epoch 24/32\n",
      "500/500 loss: 0.2575 0.3674 1.8736\n",
      "Epoch 25/32\n",
      "500/500 loss: 0.2759 0.2458 2.0341\n",
      "Epoch 26/32\n",
      "500/500 loss: 0.2696 0.2732 1.9613\n",
      "Epoch 27/32\n",
      "500/500 loss: 0.2846 0.2921 2.0834\n",
      "Epoch 28/32\n",
      "500/500 loss: 0.2779 0.3380 2.0908 2.1577\n",
      "Epoch 29/32\n",
      "500/500 loss: 0.2809 0.2990 2.1240\n",
      "Epoch 30/32\n",
      "500/500 loss: 0.2881 0.1884 2.2666 2.2940\n",
      "Epoch 31/32\n",
      "500/500 loss: 0.2863 0.1745 2.3033 2.3030\n",
      "Epoch 32/32\n",
      "500/500 loss: 0.2946 0.3531 2.2219\n"
     ]
    }
   ],
   "source": [
    "# Train the autoencoder\n",
    "print('Training autoencoder')\n",
    "ae_train.fit_generator(data_gen(train_idxs, batch_size=batch_size),\n",
    "                   steps_per_epoch=steps_per_epoch,\n",
    "                   validation_data=data_gen(valid_idxs, batch_size=batch_size),\n",
    "                   validation_steps=validation_steps,\n",
    "                   epochs=epochs)\n",
    "\n",
    "# Train the GAN\n",
    "print('Training GAN')\n",
    "for ep in range(epochs):\n",
    "    print(f'Epoch {ep+1}/{epochs}')\n",
    "    train_iter()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate\n",
    "\n",
    "Now that we have a model, let's make some food! First, I define some utility functions to do the generation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8LGUppSok_WV"
   },
   "outputs": [],
   "source": [
    "def choose_char(p, temperature=0.2):\n",
    "    # Apply temperature\n",
    "    p = np.log(p)\n",
    "    p /= temperature\n",
    "\n",
    "    # Rescale\n",
    "    p = np.exp(p.astype('float64'))\n",
    "    p = p / np.sum(p)\n",
    "\n",
    "    # Randomly choose one from the distribution\n",
    "    p = np.random.multinomial(1, p, 1)\n",
    "\n",
    "    # Choose the most likely character\n",
    "    sampled_token_index = np.argmax(p)\n",
    "    \n",
    "    if sampled_token_index:\n",
    "        token = word_inv_idx[sampled_token_index]\n",
    "    else:\n",
    "        token = False\n",
    "    \n",
    "    return sampled_token_index, token\n",
    "\n",
    "def decode_state(h, temperature=0.2):\n",
    "    x = np.array([[str2tok('<start>')[0]]])\n",
    "    h = np.array([h])\n",
    "\n",
    "    res = []\n",
    "    for _ in range(max_len):\n",
    "        p, h = decoder.predict([x, h])\n",
    "        \n",
    "        i, c = choose_char(p[0][0], temperature=temperature)\n",
    "\n",
    "        if not i or c is False or c == '<end>' or c == '\\n':\n",
    "            # We reached the end of the text\n",
    "            break\n",
    "        else:\n",
    "            # Extend the result with the new token\n",
    "            res.append(i)\n",
    "            # The token to feed in is the one last generated\n",
    "            x[0,0] = i\n",
    "    \n",
    "    # Attach all of the words and return\n",
    "    return tok2str(res)\n",
    "\n",
    "def regenerate(txt, temperature=0.2):\n",
    "    \"\"\" Given text, pass it through the encoder and then the decoder.\n",
    "    \"\"\"\n",
    "    # Tokenize the text\n",
    "    x = np.array([str2tok(txt)])\n",
    "    \n",
    "    # Encode the text\n",
    "    h = encoder.predict(x)[0]\n",
    "    \n",
    "    # Decode and return\n",
    "    return decode_state(h)\n",
    "\n",
    "def gan_generate(temperature=0.2):\n",
    "    \"\"\" Uses the GAN to generate a recipe name.\n",
    "    \"\"\"\n",
    "    x = np.random.normal(0, 1, size=(1, latent_dim))\n",
    "    h = generator.predict(x)[0]\n",
    "    return decode_state(h)\n",
    "\n",
    "def generate(temperature=0.2):\n",
    "    \"\"\" Generate a random recipe from the space of possible encodings.\n",
    "    \"\"\"\n",
    "    res = None\n",
    "    while not res:\n",
    "        # Choose a completely random state\n",
    "        h = np.random.uniform(-1, 1, size=(latent_dim,))\n",
    "        # Use the state to generate some text\n",
    "        res = decode_state(h)\n",
    "    \n",
    "    return res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def classify_food(txt):\n",
    "    \"\"\" Given a recipe name, determine whether or not the name is 'real'\n",
    "    \"\"\"\n",
    "    x = np.array([str2tok(txt)])\n",
    "    h = encoder.predict(x)\n",
    "    y = discriminator.predict(h)[0][0]\n",
    "    return [False, True][int(round(y))]    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we use the network to generate new food, let's see how it works on existing food. Remember that it should produce the same thing we put in. Of course, it won't be perfect, but that's not really a problem here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source: Slow Cooker Chicken and Dumplings\n",
      "Target: Slow Cooker Chicken and Dumplings\n",
      "\n",
      "Source: Awesome Slow Cooker Pot Roast\n",
      "Target: Awesome Slow Cooker Pot Roast\n",
      "\n",
      "Source: Brown Sugar Meatloaf\n",
      "Target: Brown Sugar Meatloaf\n",
      "\n",
      "Source: Best Chocolate Chip Cookies\n",
      "Target: Best Chocolate Chip Cookies\n",
      "\n",
      "Source: Homemade Mac and Cheese Casserole\n",
      "Target: Homemade Macaroni and Cheese Casserole\n",
      "\n",
      "Source: Banana Banana Bread\n",
      "Target: Banana Banana Bread\n",
      "\n",
      "Source: Chef John's Fisherman's Pie\n",
      "Target: Chef John's Margarita Shake\n",
      "\n",
      "Source: Mom's Zucchini Bread\n",
      "Target: Mom's Zucchini Bread\n",
      "\n",
      "Source: The Best Rolled Sugar Cookies\n",
      "Target: The Best Lemon Tea Cookies\n",
      "\n",
      "Source: Singapore Chili Crabs\n",
      "Target: Noodle Pot Stickers\n",
      "\n",
      "Source: Downeast Maine Pumpkin Bread\n",
      "Target: Aunt Wheat Chocolate Bread\n",
      "\n",
      "Source: Best Big, Fat, Chewy Chocolate Chip Cookie\n",
      "Target: Best Big, Fat, Chewy Chocolate Chip Cookie Bars\n",
      "\n",
      "Source: Aimee's Mashed Cauliflower 'Potatoes'\n",
      "Target: Aimee's Mashed Cauliflower 'Potatoes'\n",
      "\n",
      "Source: Irish Lamb Stew\n",
      "Target: Irish Fish Stew\n",
      "\n",
      "Source: To Die For Blueberry Muffins\n",
      "Target: To Die For Blueberry Muffins\n",
      "\n",
      "Source: Broiled Tilapia Parmesan\n",
      "Target: Broiled Tilapia Parmesan\n",
      "\n",
      "Source: Award Winning Soft Chocolate Chip Cookies\n",
      "Target: Award Winning Hot Chocolate Chip Cookies\n",
      "\n",
      "Source: World's Best Lasagna\n",
      "Target: Best Ever Spinach\n",
      "\n",
      "Source: Best Brownies\n",
      "Target: Best Brownies\n",
      "\n",
      "Source: Irish Soda Bread\n",
      "Target: Irish Soda Bread\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Show usage on existing samples\n",
    "for n in names[:20]:\n",
    "    print('Source:', n)\n",
    "    print('Target:', regenerate(n, temperature=0.2))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assuming the previous results look good, we can now try generating new values. One method we can use to generate random recipe names is to randomly select encoding values from the set of possible encodings. In a GRU, the values are bounded between -1 and 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "id": "3Qdt5rAVt6iP",
    "outputId": "9e01417a-4608-48f1-d137-0941eb553256"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*  Creme Fraiche\n",
      "*  Meat Nachos\n",
      "   Pumpkin Pie\n",
      "*+ Wild Kale\n",
      "*  Buckwheat Brownies\n",
      "*+ Cobbler For Hot Dish\n",
      "*  Sriracha\n",
      "*  Fudge I\n",
      "*+ Onions\n",
      "*+ Greens Pasta\n",
      "   Zucchini Pie\n",
      "*+ Authentic la Neige )\n",
      "*  Shepherds Pie\n",
      "*  Shepherd's Sticker Rice\n",
      "*+ Casserole I\n",
      "*  Old Fashioned Sauce\n",
      "*  Crinkle Cookies Brownies\n",
      "*+ Fried Shallots\n",
      "*  Cupcakes\n",
      " + Cocktail Meatballs III\n"
     ]
    }
   ],
   "source": [
    "# Generate a text sample\n",
    "for _ in range(20):\n",
    "    n = generate()\n",
    "    \n",
    "    lbl = '*' if n not in names else ' '\n",
    "    lbl += '+' if classify_food(n) else ' '\n",
    "    \n",
    "    print(lbl, n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's try the GAN model, which was previously trained to be labeled as either real or fake."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*+ The Best Classic Tiramisu\n",
      "*+ Potato and Brie in a Stick\n",
      "*+ Buttery Sopapillas\n",
      "*+ Apple Cookies\n",
      "*  Scallops with Okra ( Kholdnyk )\n",
      "*+ Snickerdoodle I\n",
      "*+ Ultra Easy Frosting\n",
      "*  Barb's Ceviche\n",
      "*  Dragon Tomato Wraps\n",
      "*+ Southwestern Burger with Garlicky\n",
      "*+ Very Best Cinnamon Vanilla Brownies Ever\n",
      "*+ Vegan Pecans III\n",
      "*+ Rocky Cake Cake Brownies\n",
      "*+ Bing Cherry French Toast\n",
      "*  French Onion Triangles\n",
      "*  New Year's Lemonade Cake\n",
      "*+ Pasta and Tomato with Couscous\n",
      " + Sweet Potato Rolls\n",
      "*+ Oven Sweet Turkey Burgers\n",
      "*+ Lime Mango Margarita\n"
     ]
    }
   ],
   "source": [
    "# Generate a text sample\n",
    "for _ in range(20):\n",
    "    n = gan_generate()\n",
    "    \n",
    "    lbl = '*' if n not in names else ' '\n",
    "    lbl += '+' if classify_food(n) else ' '\n",
    "    \n",
    "    print(lbl, n)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "foodnames.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}