Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning Course Notes

Some weeks ago I started the last course by Andrew Ng and Laurende Moroney about Tensorflow.
The course it's called Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning and has a very practical approach. It gave me the opportunity to review the foundamentals of deep learning I learned from Andrew's past courses and I almost forgot.
I wrote these notes to have an easy way to review all the stuff I've got there.

Contents

• Very basic intro : A very basic introduction concepts on Machine Learning and Deep learning Neural Networks.
• Basic Tensorflow functions: A very basic TensorFlow methods to create a basic Neural Network using keras.
• Keras Callbacks: You can define a callback that will be invoked during training your NN when an event occurs.
• Convolutional Neural Network Intro: We add some layers to our NN in order to apply some filters called Convolutions. We can reduce our data applying Pooling

A very basic intro to deep learning

Machine Learning

Thanks to an input data and answers you obtain a rules to apply to new data.

Neural Network

Network of neurons that computes an input to produce an output. There's an input layer, a defined number of hidden layers and output layer. Every layer has a number of units, called neurons.
Every neuron has what's called activation function. An activation function is something like a filter that decides how much every neuron will participate in the whole network based on the values receiving.

Training NN

You need to train a NN in order to obtain the best rules that produce your desired output. To train a NN you need a dataset with training labeled data.
Training a NN means to use your NN on the training data and check the results you obtain thanks to your actual rules. Then you check your differences between the output you obtained and the output you want to achieve. We apply to this difference what's called a loss function. loss function allows you to parametrize your difference in some way in order to be more computable or normalized to be comparable. This loss function will be your rating value to know how good your NN is doing. This is done using a loop. This loop steps are the following

1. Compute your inputs with your NN to obtain an output
2. Get your loss value
3. Modify your parameters slightly to improve your results
4. Back to step 1. until you consider your result is good enough.

The way you retouch your parameters in your NN is not random. You use what's called back propagation. back propagation uses mathematical tool (derivatives mainly) in order to obtain in which direction you hace to retouch your parameters in order to improve your result.

Optimizers

An optimizer is an algorithm that improve the way you retouch your parameters on every iteration of the training loops.

Loss Function

An algorithm that allow you to know how good your rules are.

Basic Tensorflow Functions

The following lines recaps the basic functions to deploy a simple Neural Network using TensorFlow and keras on python. Steps to code a full simple NN.

The Steps

The following are the necessary steps for a basic Neural Network.

• Imports
• Load Datasets
• Normalization
• Model definition
• Learning process definition
• Training model
• Evaluation the model
• Predict

The imports

Define the tools you will use:

• TensorFlow framework - tensorflow
• MatPlot Library - matplotlib
• Numpy Library - numpy

# TensorFlow Framework
import tensorflow as tf
#  Pyplot to plot images
import matplotlib.pyplot as plt
# Numpy Library
import numpy as np

Load Dataset (Using MNIST)

• Datasets - tf.keras.datasets.fashion_mnist
• Load Data - mnist.load_data()

# Mnist
mnist = tf.keras.datasets.fashion_mnist
# Load data
(training_images, training_labels), (test_images, test_labels) = mnist.load_data()

Plot Data

• Show Image - plt.imshow()

plt.imshow(training_images[23])

Normalization

training_images = training_images / 255.0
test_images = test_images / 255.0

Model Definition

Define the Neural Network composition. Layers, units, activation functions.

• Model Definition - models.Sequential()
• Learning Process Definition - model.compile()

# model definition
model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), 
                                    tf.keras.layers.Dense(512, activation=tf.nn.relu), 
                                    tf.keras.layers.Dense(10, activation=tf.nn.softmax)])
# model compilation
model.compile(optimizer = tf.train.AdamOptimizer(),
              loss = 'sparse_categorical_crossentropy',
              metrics=['accuracy'])

Training

• Process the training - model.fit()

# Train the model
# epochs define the number of iterations for the lerning process
model.fit(training_images, training_labels, epochs=40)

Evaluate

• Evaluate test data - model.evaluate()

model.evaluate(test_images, test_labels)

Predict. Use the Neural Network

• Use the Neural Network - model.predict()

model.predict(sample[3]);

NOTES -

• For simple data, increase the number of layers usually has no significant impact
• For more complex data, like color images, adding more layers will be necessary
• For simple data, adding neurons can usually have a good impact.

Keras Callbacks

Keras allows us to define callbacks based on an event. During the training, every time the event occurs our callback is invoked. You can actuate on your train in your callback

The following steps are needed

• Define your class for your callback - tf.keras.callbacks.Callback
• Define the event that will invoke your callback - def on_epoch_end
• Check a condition. logs.get()
• Do something based on the condition

class myCallback(tf.keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs={}):
    if(logs.get('acc') > 0.99):
      print("Reached 99% accuracy so cancelling training")
      self.model.stop_training = True

callbacks = myCallback()

# ...

# Add your callback when you call .fit method
model.fit(x_train, y_train, epochs=5, callbacks = [callbacks])

Convolutional Neural Networks

CNN is a technique we apply to our basic Neural Netowrks in order to improve the analysis of our data. We try to detect features in a better way and we also try to reduce our data to improve the speed of our algorithms.

Concepts

Convolution

A convolution is a filter we apply to our data (images) in order to emphasize some aspects of it. We define a custom size of the filter to apply on every pixel and its neighbours.

Pooling

Pooling allows us to reduce/compress our data but without removing the features on it. We apply some pool of defined sized and apply some algorithm on it. For instance we can get the most significant value (the highest) of the pool.

How to apply on TensorFlow

Convolution

Keras layers gives us a method to apply Convolutional layers to our NNs.
Conv2D() - Accepts the following basic parameters:

1. Number of filters
2. Shape of the filter. (width, height)
3. Activation function - activation=
4. Input shape - input_shape=

Usage:

tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1))

SHAPES: When we apply a Convolution to images, we can't apply to every corner and edges of the images due to fact that edge pixels has no enough pixels in the neighbourhood to apply the entire box. For instance, we can't apply a 3x3 filter on the pixel (0,0). We need to start applying on the (1,1), loosing 1 pixel on the top, bottom and sides of the image. Is for this reason that an Convolution input shape of (28, 28) will output a shape of (26, 26). Exists some tricks to add fake pixels on edges to allow us starting for the pixel (0, 0)

Pooling

Keras layers gives us a method to add some Polling layers to our NN.
MaxPooling2D() - Accepts the following basic parameters:

1. Shape of the pool. (width, height)

Usage:

tf.keras.layers.MaxPooling(2, 2)

SHAPES: The resulting shape after appling some pooling can be calculated based on the shape of the pool and the input shape. output_shape = input_shape / pool_shape

Model Summary

When you have defined your Neural Network you can print all the details with keras using model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param nums   
=================================================================
conv2d_3 (Conv2D)            (None, 26, 26, 32)        320       
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 13, 13, 32)        0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 5408)              0         
_________________________________________________________________
dense_6 (Dense)              (None, 256)               1384704   
_________________________________________________________________
dense_7 (Dense)              (None, 10)                2570      
=================================================================
Total params: 1,387,594
Trainable params: 1,387,594
Non-trainable params: 0
_________________________________________________________________

About me

I'm David Ibañez from Barcelona. Feel free to get in touch, here or:

• You can get in touch with me on Twitter

Sources

• Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning
• Andrew Ng on Twitter
• Laurende Moroney on Twitter