
Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning Course Notes
Some weeks ago I started the last course by Andrew Ng and Laurende Moroney about Tensorflow.
The course it's called Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning and has a very practical approach. It gave me the opportunity to review the foundamentals of deep learning I learned from Andrew's past courses and I almost forgot.
I wrote these notes to have an easy way to review all the stuff I've got there.
Contents
- Very basic intro : A very basic introduction concepts on Machine Learning and Deep learning Neural Networks.
- Basic Tensorflow functions: A very basic
TensorFlow
methods to create a basic Neural Network using keras. - Keras Callbacks: You can define a callback that will be invoked during training your NN when an event occurs.
- Convultational Neural Network Intro: We add some layers to our NN in order to apply some filters called
Convulations
. We can reduce our data applyingPooling
A very basic intro to deep learning
Machine Learning
Thanks to an input data and answers you obtain a rules to apply to new data.
Neural Network
Network of neurons that computes an input to produce an output. There's an input layer, a defined number of hidden layers and output layer. Every layer has a number of units, called neurons.
Every neuron has what's called activation function
. An activation function
is something like a filter that decides how much every neuron will participate in the whole network based on the values receiving.
Training NN
You need to train a NN in order to obtain the best rules that produce your desired output. To train a NN you need a dataset with training labeled data.
Training a NN means to use your NN on the training data and check the results you obtain thanks to your actual rules. Then you check your differences between the output you obtained and the output you want to achieve. We apply to this difference what's called a loss function
. loss function
allows you to parametrize your difference in some way in order to be more computable or normalized to be comparable. This loss function
will be your rating value to know how good your NN is doing. This is done using a loop
. This loop steps are the following
- Compute your inputs with your NN to obtain an output
- Get your
loss
value - Modify your parameters slightly to improve your results
- Back to step 1. until you consider your result is good enough.
The way you retouch your parameters in your NN is not random. You use what's called back propagation
. back propagation
uses mathematical tool (derivatives mainly) in order to obtain in which direction you hace to retouch your parameters in order to improve your result.
Optimizers
An optimizer is an algorithm that improve the way you retouch your parameters on every iteration of the training loops.
Loss Function
An algorithm that allow you to know how good your rules are.
Basic Tensorflow Functions
The following lines recaps the basic functions to deploy a simple Neural Network using TensorFlow
and keras
on python
. Steps to code a full simple NN.
The Steps
The following are the necessary steps for a basic Neural Network.
- Imports
- Load Datasets
- Normalization
- Model definition
- Learning process definition
- Training model
- Evaluation the model
- Predict
The imports
Define the tools you will use:
- TensorFlow framework -
tensorflow
- MatPlot Library -
matplotlib
- Numpy Library -
numpy
# TensorFlow Framework
import tensorflow as tf
# Pyplot to plot images
import matplotlib.pyplot as plt
# Numpy Library
import numpy as np
Load Dataset (Using MNIST)
- Datasets -
tf.keras.datasets.fashion_mnist
- Load Data -
mnist.load_data()
# Mnist
mnist = tf.keras.datasets.fashion_mnist
# Load data
(training_images, training_labels), (test_images, test_labels) = mnist.load_data()
Plot Data
- Show Image -
plt.imshow()
plt.imshow(training_images[23])
Normalization
training_images = training_images / 255.0
test_images = test_images / 255.0
Model Definition
Define the Neural Network composition. Layers, units, activation functions.
- Model Definition -
models.Sequential()
- Learning Process Definition -
model.compile()
# model definition
model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)])
# model compilation
model.compile(optimizer = tf.train.AdamOptimizer(),
loss = 'sparse_categorical_crossentropy',
metrics=['accuracy'])
Training
- Process the training -
model.fit()
# Train the model
# epochs define the number of iterations for the lerning process
model.fit(training_images, training_labels, epochs=40)
Evaluate
- Evaluate test data -
model.evaluate()
model.evaluate(test_images, test_labels)
Predict. Use the Neural Network
- Use the Neural Network -
model.predict()
model.predict(sample[3]);
NOTES -
- For simple data, increase the number of layers usually has no significant impact
- For more complex data, like color images, adding more layers will be necessary
- For simple data, adding neurons can usually have a good impact.
Keras Callbacks
Keras allows us to define callbacks based on an event. During the training, every time the event occurs our callback is invoked. You can actuate on your train in your callback
The following steps are needed
- Define your
class
for yourcallback
-tf.keras.callbacks.Callback
- Define the event that will invoke your callback -
def on_epoch_end
- Check a condition.
logs.get()
- Do something based on the condition
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if(logs.get('acc') > 0.99):
print("Reached 99% accuracy so cancelling training")
self.model.stop_training = True
callbacks = myCallback()
# ...
# Add your callback when you call .fit method
model.fit(x_train, y_train, epochs=5, callbacks = [callbacks])
Convulational Neural Networks
CNN is a technique we apply to our basic Neural Netowrks in order to improve the analysis of our data. We try to detect features in a better way and we also try to reduce our data to improve the speed of our algorithms.
Concepts
Convulation
A convulation is a filter we apply to our data (images) in order to emphasize some aspects of it. We define a custom size of the filter to apply on every pixel and its neighbours.
Pooling
Pooling allows us to reduce/compress our data but without removing the features on it. We apply some pool of defined sized and apply some algorithm on it. For instance we can get the most significant value (the highest) of the pool.
How to apply on TensorFlow
Convulation
Keras layers gives us a method to apply Convulational layers to our NNs.
Conv2D() - Accepts the following basic parameters:
- Number of filters
- Shape of the filter. (width, height)
- Activation function -
activation=
- Input shape -
input_shape=
Usage:
tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1))
SHAPES: When we apply a Convulation to images, we can't apply to every corner and edges of the images due to fact that edge pixels has no enough pixels in the neighbourhood to apply the entire box. For instance, we can't apply a 3x3 filter on the pixel (0,0). We need to start applying on the (1,1), loosing 1 pixel on the top, bottom and sides of the image. Is for this reason that an Convulation input shape of (28, 28) will output a shape of (26, 26). Exists some tricks to add fake pixels on edges to allow us starting for the pixel (0, 0)
Pooling
Keras layers gives us a method to add some Polling layers to our NN.
MaxPooling2D() - Accepts the following basic parameters:
- Shape of the pool. (width, height)
Usage:
tf.keras.layers.MaxPooling(2, 2)
SHAPES: The resulting shape after appling some
pooling
can be calculated based on the shape of the pool and the input shape.output_shape = input_shape / pool_shape
Model Summary
When you have defined your Neural Network you can print all the details with keras
using model.summary()
_________________________________________________________________
Layer (type) Output Shape Param nums
=================================================================
conv2d_3 (Conv2D) (None, 26, 26, 32) 320
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 13, 13, 32) 0
_________________________________________________________________
flatten_3 (Flatten) (None, 5408) 0
_________________________________________________________________
dense_6 (Dense) (None, 256) 1384704
_________________________________________________________________
dense_7 (Dense) (None, 10) 2570
=================================================================
Total params: 1,387,594
Trainable params: 1,387,594
Non-trainable params: 0
_________________________________________________________________
About me
I'm David Ibañez from Barcelona. Feel free to get in touch, here or:
- You can get in touch with me on Twitter