Super_Piano_3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Super_Piano_3.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/asigalov61/SuperPiano/blob/master/Super_Piano_3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9opKSK2RSDRg",
        "colab_type": "text"
      },
      "source": [
        "# Super Piano 3: Google Music Transformer\n",
        "## Generating Music with Long-Term structure\n",
        "### Based on 2019 ICLR paper by Cheng-Zhi Anna Huang, Google Brain and Damon Gwinn's code/repo https://github.com/gwinndr/MusicTransformer-Pytorch\n",
        "\n",
        "Huge thanks go out to the following people who contributed the code/repos used in this colab. Additional contributors are listed in the code as well.\n",
        "\n",
        "1) Kevin-Yang https://github.com/jason9693/midi-neural-processor\n",
        "\n",
        "2) gudgud96 for fixing Kevin's MIDI Encoder properly https://github.com/gudgud96\n",
        "\n",
        "2) jinyi12, Zac Koh, Akamight, Zhang https://github.com/COMP6248-Reproducability-Challenge/music-transformer-comp6248\n",
        "\n",
        "Thank you so much for your hard work and for sharing it with the world :)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "05hD19W0hSCP",
        "colab_type": "text"
      },
      "source": [
        "###Setup Environment and Dependencies. Check GPU."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ror_UJUp7wlO",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Check if GPU (driver) is avaiiable (you do not want to run this on CPU, trust me)\n",
        "!nvcc --version\n",
        "!nvidia-smi"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "paYvoZHihtux",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Clone/Install all dependencies\n",
        "!git clone https://github.com/asigalov61/midi-neural-processor\n",
        "!git clone https://github.com/asigalov61/MusicTransformer-Pytorch\n",
        "!pip install tqdm\n",
        "!pip install progress\n",
        "!pip install pretty-midi\n",
        "!pip install pypianoroll\n",
        "!pip install matplotlib\n",
        "!pip install librosa\n",
        "!pip install scipy\n",
        "!pip install pillow\n",
        "!apt install fluidsynth #Pip does not work for some reason. Only apt works\n",
        "!pip install midi2audio\n",
        "!pip install mir_eval\n",
        "!cp /usr/share/sounds/sf2/FluidR3_GM.sf2 /content/font.sf2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VM71tUPVfffi",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Import all needed modules\n",
        "import numpy as np\n",
        "import pickle\n",
        "import os\n",
        "import sys\n",
        "import math\n",
        "import random\n",
        "# For plotting\n",
        "import pypianoroll\n",
        "from pypianoroll import Multitrack, Track\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "#matplotlib.use('SVG')\n",
        "#%matplotlib inline\n",
        "#matplotlib.get_backend()\n",
        "import mir_eval.display\n",
        "import librosa\n",
        "import librosa.display\n",
        "# For rendering output audio\n",
        "import pretty_midi\n",
        "from midi2audio import FluidSynth\n",
        "from google.colab import output\n",
        "from IPython.display import display, Javascript, HTML, Audio"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "34spqHYPJtTJ",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title (Optional) Pre-trained models download (2 models trained for 100 epochs to 1.968 FLoss and 0.420 acc)\n",
        "!mkdir /content/MusicTransformer-Pytorch/rpr\n",
        "!mkdir /content/MusicTransformer-Pytorch/rpr/results\n",
        "%cd /content/MusicTransformer-Pytorch/rpr/results\n",
        "!wget 'https://superpiano.s3-us-west-1.amazonaws.com/SuperPiano3models.zip'\n",
        "!unzip SuperPiano3models.zip\n",
        "%cd /content/MusicTransformer-Pytorch/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4kE-VhygOPuG",
        "colab_type": "text"
      },
      "source": [
        "#Please note that you MUST DOWNLOAD AND PROCESS ONE OF THE DATASETS TO TRAIN OR TO USE PRE-TRAINED MODEL as it primes the model from DATASET files."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9gd-O5LZyJGD",
        "colab_type": "text"
      },
      "source": [
        "#Option 1: MAESTRO DataSet"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bGqw8o6oxUY",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Download Google Magenta MAESTRO v.2.0.0 Piano MIDI Dataset (~1300 MIDIs)\n",
        "%cd /content/MusicTransformer-Pytorch/dataset/\n",
        "!wget 'https://storage.googleapis.com/magentadata/datasets/maestro/v2.0.0/maestro-v2.0.0-midi.zip'\n",
        "!unzip maestro-v2.0.0-midi.zip\n",
        "%cd /content/MusicTransformer-Pytorch/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yXiyUuuonMqM",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Prepare directory sctructure and MIDI processor\n",
        "%cd /content/\n",
        "!mv midi-neural-processor midi_processor\n",
        "%cd /content/MusicTransformer-Pytorch/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vN-bpkEGxSMY",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Process MAESTRO MIDI DataSet\n",
        "!python3 preprocess_midi.py '/content/MusicTransformer-Pytorch/dataset/maestro-v2.0.0'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XKz4SKoeXYWc",
        "colab_type": "text"
      },
      "source": [
        "#Option 2: Your own Custom MIDI DataSet"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "27zmkAM9vEGX",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Create directory structure for the DataSet and prep MIDI processor\n",
        "\n",
        "!mkdir '/content/MusicTransformer-Pytorch/dataset/e_piano/'\n",
        "!mkdir '/content/MusicTransformer-Pytorch/dataset/e_piano/train'\n",
        "!mkdir '/content/MusicTransformer-Pytorch/dataset/e_piano/test'\n",
        "!mkdir '/content/MusicTransformer-Pytorch/dataset/e_piano/val'\n",
        "!mkdir '/content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis'\n",
        "\n",
        "%cd /content/\n",
        "!mv midi-neural-processor midi_processor\n",
        "%cd /content/MusicTransformer-Pytorch/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KUXaozztvGU2",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Upload your custom MIDI DataSet to created \"dataset/e_piano/custom_midis\" folder through this cell or manually through any other means. You can also use ready-to-use DataSets below\n",
        "from google.colab import files\n",
        "%cd '/content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis'\n",
        "uploaded = files.upload()\n",
        "\n",
        "for fn in uploaded.keys():\n",
        "  print('User uploaded file \"{name}\" with length {length} bytes'.format(\n",
        "      name=fn, length=len(uploaded[fn])))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uDsmK-vkwOgl",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title (The Best Choice/Works best stand-alone) Super Piano 2 Original 2500 MIDIs of Piano Music\n",
        "%cd /content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis\n",
        "!wget 'https://github.com/asigalov61/SuperPiano/raw/master/Super_Piano_2_MIDI_DataSet_CC_BY_NC_SA.zip'\n",
        "!unzip -j 'Super_Piano_2_MIDI_DataSet_CC_BY_NC_SA.zip'\n",
        "!rm Super_Piano_2_MIDI_DataSet_CC_BY_NC_SA.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "feA03YKvwgaV",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title (Second Best Choice/Works best stand-alone) Alex Piano Only Original 450 MIDIs \n",
        "%cd /content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis\n",
        "!wget 'https://github.com/asigalov61/AlexMIDIDataSet/raw/master/AlexMIDIDataSet-CC-BY-NC-SA-Piano-Only.zip'\n",
        "!unzip -j 'AlexMIDIDataSet-CC-BY-NC-SA-Piano-Only.zip'\n",
        "!rm AlexMIDIDataSet-CC-BY-NC-SA-All-Drafts-Piano-Only.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e-jAV_Qv5Fn_",
        "colab_type": "text"
      },
      "source": [
        "For now, we are going to split the dataset by random into \"test\"/\"val\" dirs which is not ideal. So feel free to modify the code to your liking to achieve better training results with this implementation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IZ5c6d5lXemo",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Process your custom MIDI DataSet :)\n",
        "%cd /content/MusicTransformer-Pytorch\n",
        "from processor import encode_midi\n",
        "\n",
        "import os\n",
        "import random\n",
        "\n",
        "\n",
        "\n",
        "%cd '/content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis'\n",
        "\n",
        "custom_MIDI_DataSet_dir = '/content/MusicTransformer-Pytorch/dataset/e_piano/custom_midis'\n",
        "\n",
        "train_dir = '/content/MusicTransformer-Pytorch/dataset/e_piano/train' # split_type = 0\n",
        "test_dir = '/content/MusicTransformer-Pytorch/dataset/e_piano/test' # split_type = 1  \n",
        "val_dir = '/content/MusicTransformer-Pytorch/dataset/e_piano/val' # split_type = 2\n",
        "\n",
        "total_count = 0\n",
        "train_count = 0\n",
        "val_count   = 0\n",
        "test_count  = 0\n",
        "\n",
        "f_ext = '.pickle'\n",
        "fileList = os.listdir(custom_MIDI_DataSet_dir)\n",
        "for file in fileList:\n",
        "     # we gonna split by a random selection for now\n",
        "    \n",
        "    split = random.randint(1, 2)\n",
        "    if (split == 0):\n",
        "         o_file = os.path.join(train_dir, file+f_ext)\n",
        "         train_count += 1\n",
        "\n",
        "    elif (split == 2):\n",
        "         o_file0 = os.path.join(train_dir, file+f_ext)\n",
        "         train_count += 1\n",
        "         o_file = os.path.join(val_dir, file+f_ext)\n",
        "         val_count += 1\n",
        "\n",
        "    elif (split == 1):\n",
        "         o_file0 = os.path.join(train_dir, file+f_ext)\n",
        "         train_count += 1\n",
        "         o_file = os.path.join(test_dir, file+f_ext)\n",
        "         test_count += 1\n",
        "    try:\n",
        "      prepped = encode_midi(file)\n",
        "      o_stream = open(o_file0, \"wb\")\n",
        "      pickle.dump(prepped, o_stream)\n",
        "      o_stream.close()\n",
        "\n",
        "      prepped = encode_midi(file)\n",
        "      o_stream = open(o_file, \"wb\")\n",
        "      pickle.dump(prepped, o_stream)\n",
        "      o_stream.close()\n",
        "   \n",
        "      print(file)\n",
        "      print(o_file)\n",
        "      print('Coverted!')  \n",
        "    except KeyboardInterrupt: \n",
        "      raise   \n",
        "    except:\n",
        "      print('Bad file. Skipping...')\n",
        "\n",
        "print('Done')\n",
        "print(\"Num Train:\", train_count)\n",
        "print(\"Num Val:\", val_count)\n",
        "print(\"Num Test:\", test_count)\n",
        "print(\"Total Count:\", train_count)\n",
        "\n",
        "%cd /content/MusicTransformer-Pytorch"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JwCQIziNwHxe",
        "colab_type": "text"
      },
      "source": [
        "#Train the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hwisXl2Iy_Xf",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Activate Tensorboard Graphs/Stats to monitor/evaluate model perfomance during and after training runs\n",
        "# Load the TensorBoard notebook extension\n",
        "%reload_ext tensorboard\n",
        "import tensorflow as tf\n",
        "import datetime, os\n",
        "%tensorboard --logdir /content/MusicTransformer-Pytorch/rpr"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sbv_sJyLq5om",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Start to Train the Model\n",
        "batch_size = 4 #@param {type:\"slider\", min:0, max:8, step:1}\n",
        "number_of_training_epochs = 150 #@param {type:\"slider\", min:0, max:200, step:1}\n",
        "maximum_output_MIDI_sequence = 2048 #@param {type:\"slider\", min:0, max:8192, step:128}\n",
        "!python3 train.py -output_dir rpr --rpr -batch_size=$batch_size -epochs=$number_of_training_epochs -max_sequence=$maximum_output_MIDI_sequence #-n_layers -num_heads -d_model -dim_feedforward"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9VLdhokSGUAu",
        "colab_type": "code",
        "colab": {},
        "cellView": "form"
      },
      "source": [
        "#@title Re-Start Training from a certain checkpoint and epoch\n",
        "batch_size = 4 #@param {type:\"slider\", min:0, max:8, step:1}\n",
        "number_of_training_epochs = 150 #@param {type:\"slider\", min:0, max:200, step:1}\n",
        "maximum_output_MIDI_sequence = 2048 #@param {type:\"slider\", min:0, max:8192, step:128}\n",
        "saved_checkpoint_full_path = \"/content/MusicTransformer-Pytorch/rpr/weights/epoch_0033.pickle\" #@param {type:\"string\"}\n",
        "continue_epoch_number = 33 #@param {type:\"integer\"}\n",
        "\n",
        "!python3 train.py -output_dir rpr --rpr -batch_size=$batch_size -epochs=$number_of_training_epochs -max_sequence=$maximum_output_MIDI_sequence -continue_weights $saved_checkpoint_full_path -continue_epoch $continue_epoch_number #-n_layers -num_heads -d_model -dim_feedforward"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k1D-o-E-TnI8",
        "colab_type": "text"
      },
      "source": [
        "###Evaluate the resulted models"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qQLOmv7wrOos",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Evaluate Best Resulting Accuracy Model (best_acc_weights.pickle)\n",
        "!python3 evaluate.py -model_weights rpr/results/best_acc_weights.pickle --rpr"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c7QftGOfTyx2",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Evaluate Best Resulting Loss Model (best_loss_weights.pickle)\n",
        "!python3 evaluate.py -model_weights rpr/results/best_loss_weights.pickle --rpr"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MusrrrOxt1uy",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Graph the results\n",
        "import argparse\n",
        "import os\n",
        "import csv\n",
        "import math\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "RESULTS_FILE = \"results.csv\"\n",
        "EPOCH_IDX = 0\n",
        "LR_IDX = 1\n",
        "EVAL_LOSS_IDX = 4\n",
        "EVAL_ACC_IDX = 5\n",
        "\n",
        "SPLITTER = '?'\n",
        "\n",
        "\n",
        "def graph_results(input_dirs=\"/content/MusicTransformer-Pytorch/rpr/results\", output_dir=None, model_names=None, epoch_start=0, epoch_end=None):\n",
        "    \"\"\"\n",
        "    ----------\n",
        "    Author: Damon Gwinn\n",
        "    ----------\n",
        "    Graphs model training and evaluation data\n",
        "    ----------\n",
        "    \"\"\"\n",
        "\n",
        "    input_dirs = input_dirs.split(SPLITTER)\n",
        "\n",
        "    if(model_names is not None):\n",
        "        model_names = model_names.split(SPLITTER)\n",
        "        if(len(model_names) != len(input_dirs)):\n",
        "            print(\"Error: len(model_names) != len(input_dirs)\")\n",
        "            return\n",
        "\n",
        "    #Initialize Loss and Accuracy arrays\n",
        "    loss_arrs = []\n",
        "    accuracy_arrs = []\n",
        "    epoch_counts = []\n",
        "    lrs = []\n",
        "\n",
        "    for input_dir in input_dirs:\n",
        "        loss_arr = []\n",
        "        accuracy_arr = []\n",
        "        epoch_count = []\n",
        "        lr_arr = []\n",
        "\n",
        "        f = os.path.join(input_dir, RESULTS_FILE)\n",
        "        with open(f, \"r\") as i_stream:\n",
        "            reader = csv.reader(i_stream)\n",
        "            next(reader)\n",
        "\n",
        "            lines = [line for line in reader]\n",
        "\n",
        "        if(epoch_end is None):\n",
        "            epoch_end = math.inf\n",
        "\n",
        "        epoch_start = max(epoch_start, 0)\n",
        "        epoch_start = min(epoch_start, epoch_end)\n",
        "\n",
        "        for line in lines:\n",
        "            epoch = line[EPOCH_IDX]\n",
        "            lr = line[LR_IDX]\n",
        "            accuracy = line[EVAL_ACC_IDX]\n",
        "            loss = line[EVAL_LOSS_IDX]\n",
        "\n",
        "            if(int(epoch) >= epoch_start and int(epoch) < epoch_end):\n",
        "                accuracy_arr.append(float(accuracy))\n",
        "                loss_arr.append(float(loss))\n",
        "                epoch_count.append(int(epoch))\n",
        "                lr_arr.append(float(lr))\n",
        "\n",
        "        loss_arrs.append(loss_arr)\n",
        "        accuracy_arrs.append(accuracy_arr)\n",
        "        epoch_counts.append(epoch_count)\n",
        "        lrs.append(lr_arr)\n",
        "\n",
        "    if(output_dir is not None):\n",
        "        try:\n",
        "            os.mkdir(output_dir)\n",
        "        except OSError:\n",
        "            print (\"Creation of the directory %s failed\" % output_dir)\n",
        "        else:\n",
        "            print (\"Successfully created the directory %s\" % output_dir)\n",
        "\n",
        "    ##### LOSS #####\n",
        "    for i in range(len(loss_arrs)):\n",
        "        if(model_names is None):\n",
        "            name = None\n",
        "        else:\n",
        "            name = model_names[i]\n",
        "\n",
        "        #Create and save plots to output folder\n",
        "        plt.plot(epoch_counts[i], loss_arrs[i], label=name)\n",
        "        plt.title(\"Loss Results\")\n",
        "        plt.ylabel('Loss (Cross Entropy)')\n",
        "        plt.xlabel('Epochs')\n",
        "        fig1 = plt.gcf()\n",
        "\n",
        "    plt.legend(loc=\"upper left\")\n",
        "\n",
        "    if(output_dir is not None):\n",
        "        fig1.savefig(os.path.join(output_dir, 'loss_graph.png'))\n",
        "\n",
        "    plt.show()\n",
        "\n",
        "    ##### ACCURACY #####\n",
        "    for i in range(len(accuracy_arrs)):\n",
        "        if(model_names is None):\n",
        "            name = None\n",
        "        else:\n",
        "            name = model_names[i]\n",
        "\n",
        "        #Create and save plots to output folder\n",
        "        plt.plot(epoch_counts[i], accuracy_arrs[i], label=name)\n",
        "        plt.title(\"Accuracy Results\")\n",
        "        plt.ylabel('Accuracy')\n",
        "        plt.xlabel('Epochs')\n",
        "        fig2 = plt.gcf()\n",
        "\n",
        "    plt.legend(loc=\"upper left\")\n",
        "\n",
        "    if(output_dir is not None):\n",
        "        fig2.savefig(os.path.join(output_dir, 'accuracy_graph.png'))\n",
        "\n",
        "    plt.show()\n",
        "\n",
        "    ##### LR #####\n",
        "    for i in range(len(lrs)):\n",
        "        if(model_names is None):\n",
        "            name = None\n",
        "        else:\n",
        "            name = model_names[i]\n",
        "\n",
        "        #Create and save plots to output folder\n",
        "        plt.plot(epoch_counts[i], lrs[i], label=name)\n",
        "        plt.title(\"Learn Rate Results\")\n",
        "        plt.ylabel('Learn Rate')\n",
        "        plt.xlabel('Epochs')\n",
        "        fig2 = plt.gcf()\n",
        "\n",
        "    plt.legend(loc=\"upper left\")\n",
        "\n",
        "    if(output_dir is not None):\n",
        "        fig2.savefig(os.path.join(output_dir, 'lr_graph.png'))\n",
        "\n",
        "    plt.show()\n",
        "graph_results('/content/MusicTransformer-Pytorch/rpr/results', model_names='rpr')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fxHrTsFUdn-r",
        "colab_type": "text"
      },
      "source": [
        "To have the model continue your custom MIDI enter the following into the custom_MIDI field below:\n",
        "\n",
        "-primer_file '/content/some_dir/some_seed_midi.mid'\n",
        "\n",
        "For example: -primer_file '/content/MusicTransformer-Pytorch/seed.mid'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EJXWoBMWL3ph",
        "colab_type": "text"
      },
      "source": [
        "# Generate and Explore the output :)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "czNulONr4tB6",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Generate, Plot, Graph, Save, Download, and Render the resulting output\n",
        "number_of_tokens_to_generate = 1023 #@param {type:\"slider\", min:1, max:2048, step:1}\n",
        "priming_sequence_length = 65 #@param {type:\"slider\", min:1, max:2048, step:8}\n",
        "maximum_possible_output_sequence = 2048 #@param {type:\"slider\", min:0, max:2048, step:8}\n",
        "select_model = \"/content/MusicTransformer-Pytorch/rpr/results/best_loss_weights.pickle\" #@param [\"/content/MusicTransformer-Pytorch/rpr/results/best_acc_weights.pickle\", \"/content/MusicTransformer-Pytorch/rpr/results/best_loss_weights.pickle\"]\n",
        "custom_MIDI = \"\" #@param {type:\"string\"}\n",
        "\n",
        "import processor\n",
        "from processor import encode_midi, decode_midi\n",
        "\n",
        "!python generate.py -output_dir output -model_weights=$select_model --rpr -target_seq_length=$number_of_tokens_to_generate -num_prime=$priming_sequence_length -max_sequence=$maximum_possible_output_sequence $custom_MIDI #\n",
        "\n",
        "print('Successfully exported the output to output folder. To primer.mid and rand.mid')\n",
        "\n",
        "# set the src and play\n",
        "FluidSynth(\"/content/font.sf2\").midi_to_audio('/content/MusicTransformer-Pytorch/output/rand.mid', '/content/MusicTransformer-Pytorch/output/output.wav')\n",
        "\n",
        "from google.colab import files\n",
        "files.download('/content/MusicTransformer-Pytorch/output/rand.mid')\n",
        "files.download('/content/MusicTransformer-Pytorch/output/primer.mid')\n",
        "\n",
        "Audio('/content/MusicTransformer-Pytorch/output/output.wav')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IG48uyKGzcTI",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Plot and Graph the Output :)\n",
        "graphs_length_inches = 18 #@param {type:\"slider\", min:0, max:20, step:1}\n",
        "notes_graph_height = 6 #@param {type:\"slider\", min:0, max:20, step:1}\n",
        "highest_displayed_pitch = 92 #@param {type:\"slider\", min:1, max:128, step:1}\n",
        "lowest_displayed_pitch = 24 #@param {type:\"slider\", min:1, max:128, step:1}\n",
        "piano_roll_color_map = \"Blues\"\n",
        "\n",
        "import librosa\n",
        "import numpy as np\n",
        "import pretty_midi\n",
        "import pypianoroll\n",
        "from pypianoroll import Multitrack, Track\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "#matplotlib.use('SVG')\n",
        "# For plotting\n",
        "import mir_eval.display\n",
        "import librosa.display\n",
        "%matplotlib inline\n",
        "\n",
        "\n",
        "midi_data = pretty_midi.PrettyMIDI('/content/MusicTransformer-Pytorch/output/rand.mid')\n",
        "\n",
        "def plot_piano_roll(pm, start_pitch, end_pitch, fs=100):\n",
        "    # Use librosa's specshow function for displaying the piano roll\n",
        "    librosa.display.specshow(pm.get_piano_roll(fs)[start_pitch:end_pitch],\n",
        "                             hop_length=1, sr=fs, x_axis='time', y_axis='cqt_note',\n",
        "                             fmin=pretty_midi.note_number_to_hz(start_pitch))\n",
        "\n",
        "\n",
        "\n",
        "roll = np.zeros([int(graphs_length_inches), 128])\n",
        "# Plot the output\n",
        "\n",
        "track = Multitrack('/content/MusicTransformer-Pytorch/output/rand.mid', name='track')\n",
        "plt.figure(figsize=[graphs_length_inches, notes_graph_height])\n",
        "fig, ax = track.plot()\n",
        "fig.set_size_inches(graphs_length_inches, notes_graph_height)\n",
        "plt.figure(figsize=[graphs_length_inches, notes_graph_height])\n",
        "ax2 = plot_piano_roll(midi_data, int(lowest_displayed_pitch), int(highest_displayed_pitch))\n",
        "plt.show(block=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VFMWSDq7ZKM2",
        "colab_type": "text"
      },
      "source": [
        "### Save to Google Drive (Standard GD connect code)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n-A3ju-Fz0Eh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}