{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training MNIST data classifier in PyGlow\n",
    "This notebook is to make you realize how much PyGlow shares with Keras API structure. The purpose of this type of API is to make anyone just entering this field to quickly realize the power of these tools rather than get overwhelmed by the complexity of the theory itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing glow modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from glow.layers import Dense, Dropout, Conv2d, Flatten\n",
    "from glow.models import Sequential\n",
    "from glow.datasets import mnist\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load dataset\n",
    "Currently PyGlow supports two standard datasets namely mnist, cifar10 so you can use either of them for experimention."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyperparameter\n",
    "batch_size = 64\n",
    "num_workers = 3\n",
    "validation_split = 0.2\n",
    "num_epochs = 2\n",
    "\n",
    "# load the dataset\n",
    "train_loader, val_loader, test_loader = mnist.load_data(\n",
    "    batch_size=batch_size, num_workers=num_workers, validation_split=validation_split\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Keras like Sequential Model\n",
    "We use the basic keras-like Sequential model to build our model in the 'keras' way !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on CUDA enabled device !\n"
     ]
    }
   ],
   "source": [
    "model = Sequential(input_shape=(1, 28, 28), gpu=True)\n",
    "model.add(Conv2d(filters=16, kernel_size=3, stride=1, padding=1, activation='relu'))\n",
    "model.add(Flatten())\n",
    "model.add(Dropout(0.4))\n",
    "model.add(Dense(500, activation='relu'))\n",
    "model.add(Dropout(0.4))\n",
    "model.add(Dense(200, activation='relu'))\n",
    "model.add(Dropout(0.2))\n",
    "model.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compile the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='SGD', loss='cross_entropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|          | 0/750 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Epoch 1/2\n",
      "Training loop: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 750/750 [00:34<00:00, 19.50it/s]\n",
      "  0%|          | 0/188 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "loss: 2.24 - acc: 0.39\n",
      "Validation loop: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 188/188 [00:02<00:00, 93.60it/s]\n",
      "  0%|          | 0/750 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "loss: 1.99 - acc: 0.50\n",
      "\n",
      "\n",
      "Epoch 2/2\n",
      "Training loop: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 750/750 [00:32<00:00, 22.96it/s]\n",
      "  0%|          | 0/188 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "loss: 1.81 - acc: 0.83\n",
      "Validation loop: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 188/188 [00:01<00:00, 105.00it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "loss: 1.66 - acc: 0.84\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# training the model\n",
    "model.fit_generator(train_loader, val_loader, num_epochs, show_plot=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}