Image Classification¶
In this project, we'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. we'll preprocess the images, then train a convolutional neural network on all the samples. The images need to be normalized and the labels need to be one-hot encoded. we'll build a convolutional, max pooling, dropout, and fully connected layers. At the end, we'll get to see your neural network's predictions on the sample images.Get the Data¶
Run the following cell to download the CIFAR-10 dataset for python.
In [2]:
from urllib.request import urlretrieve from os.path import isfile, isdir from tqdm import tqdm import problem_unittests as tests import tarfile cifar10_dataset_folder_path = 'cifar-10-batches-py' # Use Floyd's cifar-10 dataset if present floyd_cifar10_location = '/input/cifar-10/python.tar.gz' if isfile(floyd_cifar10_location): tar_gz_path = floyd_cifar10_location else: tar_gz_path = 'cifar-10-python.tar.gz' class DLProgress(tqdm): last_block = 0 def hook(self, block_num=1, block_size=1, total_size=None): self.total = total_size self.update((block_num - self.last_block) * block_size) self.last_block = block_num if not isfile(tar_gz_path): with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar: urlretrieve( 'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz', tar_gz_path, pbar.hook) if not isdir(cifar10_dataset_folder_path): with tarfile.open(tar_gz_path) as tar: tar.extractall() tar.close() tests.test_folder_path(cifar10_dataset_folder_path)
Explore the Data¶
The dataset is broken into batches to prevent your machine from running out of memory. The CIFAR-10 dataset consists of 5 batches, nameddata_batch_1, data_batch_2, etc.. Each batch contains the labels and images that are one of the following:- airplane
- automobile
- bird
- cat
- deer
- dog
- frog
- horse
- ship
- truck
batch_id and sample_id. The batch_id is the id for a batch (1-5). The sample_id is the id for a image and label pair in the batch.Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?". Answers to questions like these will help you preprocess the data and end up with better predictions.
In [3]:
%matplotlib inline %config InlineBackend.figure_format = 'retina' import helper import numpy as np # Explore the dataset batch_id = 1 sample_id = 5 helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)
In [4]:
def normalize(x): """ Normalize a list of sample image data in the range of 0 to 1 : x: List of image data. The image shape is (32, 32, 3) : return: Numpy array of normalize data """ # TODO: Implement Function #print(x.shape) normalized_images=np.zeros((x.shape)) nbr_images=x.shape[0] min_image,max_image=x.min(),x.max() #print(max_image) for image in range(nbr_images): normalized_images[image,...]=((x[image,...]-min_image)-float(min_image))/(float(max_image-min_image)) return normalized_images tests.test_normalize(normalize)
One-hot encode¶
Just like the previous code cell, we'll be implementing a function for preprocessing. This time, we'll implement theone_hot_encode function. The input, x, are a list of labels. Function returns the list of labels as One-Hot encoded Numpy array. The possible values for labels are 0 to 9. The one-hot encoding function return the same encoding for each value between each call to one_hot_encode.
In [5]:
from sklearn.preprocessing import OneHotEncoder from sklearn import preprocessing lb=None def one_hot_encode(x): """ One hot encode a list of sample labels. Return a one-hot encoded vector for each label. : x: List of sample Labels : return: Numpy array of one-hot encoded labels """ # TODO: Implement Function global lb if lb is None: lb = preprocessing.LabelBinarizer() lb.fit(x) encodings = lb.transform(x) return encodings tests.test_one_hot_encode(one_hot_encode)
Preprocess all the data and save it¶
Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.
In [6]:
# Preprocess Training, Validation, and Testing Data helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)
Check Point¶
This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.
In [7]:
import pickle import problem_unittests as tests import helper # Load the Preprocessed Validation data valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))
Build the network¶
For the neural network, we'll build each layer into a function.Let's begin!
Input¶
The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. we'll Implement the following functionsneural_net_image_input- Return a TF Placeholder
- Set the shape using
image_shapewith batch size set toNone. - Name the TensorFlow placeholder "x" using the TensorFlow
nameparameter in the TF Placeholder.
neural_net_label_input- Return a TF Placeholder
- Set the shape using
n_classeswith batch size set toNone. - Name the TensorFlow placeholder "y" using the TensorFlow
nameparameter in the TF Placeholder.
neural_net_keep_prob_input- Return a TF Placeholder for dropout keep probability.
- Name the TensorFlow placeholder "keep_prob" using the TensorFlow
nameparameter in the TF Placeholder.
Note:
None for shapes in TensorFlow allow for a dynamic size.
In [8]:
import tensorflow as tf def neural_net_image_input(image_shape): """ Return a Tensor for a batch of image input : image_shape: Shape of the images : return: Tensor for image input. """ # TODO: Implement Function return tf.placeholder(tf.float32, shape=[None]+list(image_shape),name="x") def neural_net_label_input(n_classes): """ Return a Tensor for a batch of label input : n_classes: Number of classes : return: Tensor for label input. """ # TODO: Implement Function #print(n_classes) return tf.placeholder(tf.float32, shape=[None,n_classes],name="y") def neural_net_keep_prob_input(): """ Return a Tensor for keep probability : return: Tensor for keep probability. """ # TODO: Implement Function return tf.placeholder(tf.float32, shape=(None),name="keep_prob") tf.reset_default_graph() tests.test_nn_image_inputs(neural_net_image_input) tests.test_nn_label_inputs(neural_net_label_input) tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)
Convolution and Max Pooling Layer¶
Convolution layers have a lot of success with images. For this code cell, we'll implement the functionconv2d_maxpool to apply convolution then max pooling:- Create the weight and bias using
conv_ksize,conv_num_outputsand the shape ofx_tensor. - Apply a convolution to
x_tensorusing weight andconv_strides. - Add bias
- Add a nonlinear activation to the convolution.
- Apply Max Pooling using
pool_ksizeandpool_strides.
In [9]:
def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides): """ Apply convolution then max pooling to x_tensor :param x_tensor: TensorFlow Tensor :param conv_num_outputs: Number of outputs for the convolutional layer :param conv_ksize: kernal size 2-D Tuple for the convolutional layer :param conv_strides: Stride 2-D Tuple for convolution :param pool_ksize: kernal size 2-D Tuple for pool :param pool_strides: Stride 2-D Tuple for pool : return: A tensor that represents convolution and max pooling of x_tensor """ # TODO: Implement Function shape=[conv_ksize[0], conv_ksize[1], ((x_tensor.get_shape())[3]), conv_num_outputs] print((list(x_tensor.get_shape())[3])) weights =tf.Variable(tf.truncated_normal([conv_ksize[0], conv_ksize[1], x_tensor.get_shape().as_list()[3], conv_num_outputs], mean=0.0,stddev = .1)) bias=tf.Variable(tf.zeros(conv_num_outputs)) x = tf.nn.conv2d(x_tensor, weights, strides=[1, conv_strides[0], conv_strides[1], 1], padding='SAME') x = tf.nn.bias_add(x, bias) x= tf.nn.relu(x) output=tf.nn.max_pool( x, ksize=[1, pool_ksize[0], pool_ksize[1], 1], strides=[1, pool_strides[0], pool_strides[1], 1], padding='SAME') return output tests.test_con_pool(conv2d_maxpool)
Flatten Layer¶
Here we'll Implement theflatten function to change the dimension of x_tensor from a 4-D tensor to a 2-D tensor. The output should be the shape (Batch Size, Flattened Image Size). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer.
In [10]:
def flatten(x_tensor): """ Flatten x_tensor to (Batch Size, Flattened Image Size) : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions. : return: A tensor of size (Batch Size, Flattened Image Size). """ # TODO: Implement Function x_tensor_shape=( x_tensor.get_shape().as_list()) x_tensor_ht=x_tensor_shape[1] x_tensor_wt=x_tensor_shape[2] x_tensor_channels=x_tensor_shape[3] pool2_flat = tf.reshape(x_tensor, [-1, x_tensor_ht *x_tensor_wt*x_tensor_channels]) return pool2_flat tests.test_flatten(flatten)
Fully-Connected Layer¶
IHere we'll mplement thefully_conn function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer.
In [11]:
def fully_conn(x_tensor, num_outputs): """ Apply a fully connected layer to x_tensor using weight and bias : x_tensor: A 2-D tensor where the first dimension is batch size. : num_outputs: The number of output that the new tensor should be. : return: A 2-D tensor where the second dimension is num_outputs. """ # TODO: Implement Function x_tensor_shape=np.array(x_tensor.get_shape().as_list()[1:]).prod() weights =tf.Variable(tf.truncated_normal([x_tensor_shape, num_outputs],mean=0.0,stddev = .1)) biases=tf.Variable(tf.zeros(num_outputs)) c1 = tf.add(tf.matmul(x_tensor, weights), biases) c1 = tf.nn.relu(c1) return c1 tests.test_fully_conn(fully_conn)
Output Layer¶
we'll Implement theoutput function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.
In [12]:
def output(x_tensor, num_outputs): """ Apply a output layer to x_tensor using weight and bias : x_tensor: A 2-D tensor where the first dimension is batch size. : num_outputs: The number of output that the new tensor should be. : return: A 2-D tensor where the second dimension is num_outputs. """ # TODO: Implement Function x_tensor_shape=np.array(x_tensor.get_shape().as_list()[1:]).prod() print(x_tensor_shape) weights =tf.Variable(tf.truncated_normal([x_tensor_shape, num_outputs],mean=0.0,stddev = .1)) biases=tf.Variable(tf.zeros(num_outputs)) c1 = tf.add(tf.matmul(x_tensor, weights), biases) return c1 tests.test_output(output)
Create Convolutional Model¶
we'll Implement the functionconv_net to create a convolutional neural network model. The function takes in a batch of images, x, and outputs logits. Use the layers you created above to create this model:- Apply 1, 2, or 3 Convolution and Max Pool layers
- Apply a Flatten Layer
- Apply 1, 2, or 3 Fully Connected Layers
- Apply an Output Layer
- Return the output
- Apply TensorFlow's Dropout to one or more layers in the model using
keep_prob.
In [13]:
def conv_net(x, keep_prob): """ Create a convolutional neural network model : x: Placeholder tensor that holds image data. : keep_prob: Placeholder tensor that hold dropout keep probability. : return: Tensor that represents logits """ # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers # Play around with different number of outputs, kernel size and stride # Function Definition from Above: # conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides) conv = conv2d_maxpool(x, conv_num_outputs=16, conv_ksize=[5,5], conv_strides=[1,1], pool_ksize=[3,3], pool_strides=[2,2]) conv = conv2d_maxpool(conv, conv_num_outputs=32, conv_ksize=[5,5], conv_strides=[1,1], pool_ksize=[3,3], pool_strides=[2,2]) # conv = conv2d_maxpool(conv, # conv_num_outputs=64, # conv_ksize=[5,5], # conv_strides=[1,1], # pool_ksize=[3,3], # pool_strides=[2,2]) # TODO: Apply a Flatten Layer # Function Definition from Above: # flatten(x_tensor) flattened=flatten(conv) # TODO: Apply 1, 2, or 3 Fully Connected Layers # Play around with different number of outputs # Function Definition from Above: # fully_conn(x_tensor, num_outputs) fully_c= fully_conn(flattened,512) fully_c= fully_conn(fully_c,250) #fully_c= fully_conn(fully_c,50) # TODO: Apply an Output Layer # Set this to the number of classes # Function Definition from Above: # output(x_tensor, num_outputs) fully_c=tf.nn.dropout(fully_c,keep_prob) output_cn=output(fully_c, 10) # TODO: return output return output_cn ############################## ## Build the Neural Network ## ############################## # Remove previous weights, bias, inputs, etc.. tf.reset_default_graph() # Inputs x = neural_net_image_input((32, 32, 3)) y = neural_net_label_input(10) keep_prob = neural_net_keep_prob_input() # Model logits = conv_net(x, keep_prob) # Name logits Tensor, so that is can be loaded from disk after training logits = tf.identity(logits, name='logits') # Loss and Optimizer cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y)) optimizer = tf.train.AdamOptimizer().minimize(cost) # Accuracy correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1)) accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy') tests.test_conv_net(conv_net)
Train the Neural Network¶
Single Optimization¶
We'll Implement the functiontrain_neural_network to do a single optimization. The optimization should use optimizer to optimize in session with a feed_dict of the following:xfor image inputyfor labelskeep_probfor keep probability for dropout
tf.global_variables_initializer() has already been called.Note: Nothing needs to be returned. This function is only optimizing the neural network.
In [14]:
def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch): """ Optimize the session on a batch of images and labels : session: Current TensorFlow session : optimizer: TensorFlow optimizer function : keep_probability: keep probability : feature_batch: Batch of Numpy image data : label_batch: Batch of Numpy label data """ # TODO: Implement Function session.run(optimizer, feed_dict={ x: feature_batch, y: label_batch, keep_prob: keep_probability}) tests.test_train_nn(train_neural_network)
Show Stats¶
We'll Implement the functionprint_stats to print loss and validation accuracy. We'll Use the global variables valid_features and valid_labels to calculate validation accuracy. Use a keep probability of 1.0 to calculate the loss and validation accuracy.
In [15]:
def print_stats(session, feature_batch, label_batch, cost, accuracy): """ Print information about loss and validation accuracy : session: Current TensorFlow session : feature_batch: Batch of Numpy image data : label_batch: Batch of Numpy label data : cost: TensorFlow cost function : accuracy: TensorFlow accuracy function """ # TODO: Implement Function loss=sess.run(cost,feed_dict={x:valid_features, y:valid_labels, keep_prob:1}) valid_acc=sess.run(accuracy,feed_dict={x:valid_features, y:valid_labels, keep_prob:1}) print('Loss: {:>10.4f} Validation Accuracy: {:.6f}'.format(loss, valid_acc))
Hyperparameters¶
We'll Tune the following parameters:- Set
epochsto the number of iterations until the network stops learning or start overfitting - Set
batch_sizeto the highest number that your machine has memory for. Most people set them to common sizes of memory:- 64
- 128
- 256
- ...
- Set
keep_probabilityto the probability of keeping a node using dropout
In [18]:
# TODO: Tune Parameters epochs = 75 batch_size = 256 keep_probability = .75
Train on a Single CIFAR-10 Batch¶
Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.
In [20]:
print('Checking the Training on a Single Batch...') with tf.Session() as sess: # Initializing the variables sess.run(tf.global_variables_initializer()) # Training cycle for epoch in range(epochs): batch_i = 1 for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size): train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels) print('Epoch {:>2}, CIFAR-10 Batch {}: '.format(epoch + 1, batch_i), end='') print_stats(sess, batch_features, batch_labels, cost, accuracy)
Fully Train the Model¶
Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.
In [21]:
save_model_path = './image_classification' print('Training...') with tf.Session() as sess: # Initializing the variables sess.run(tf.global_variables_initializer()) # Training cycle for epoch in range(epochs): # Loop over all batches n_batches = 5 for batch_i in range(1, n_batches + 1): for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size): train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels) print('Epoch {:>2}, CIFAR-10 Batch {}: '.format(epoch + 1, batch_i), end='') print_stats(sess, batch_features, batch_labels, cost, accuracy) # Save Model saver = tf.train.Saver() save_path = saver.save(sess, save_model_path)
In [22]:
%matplotlib inline %config InlineBackend.figure_format = 'retina' import tensorflow as tf import pickle import helper import random # Set batch size if not already set try: if batch_size: pass except NameError: batch_size = 64 save_model_path = './image_classification' n_samples = 4 top_n_predictions = 3 def test_model(): """ Test the saved model against the test dataset """ test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb')) loaded_graph = tf.Graph() with tf.Session(graph=loaded_graph) as sess: # Load model loader = tf.train.import_meta_graph(save_model_path + '.meta') loader.restore(sess, save_model_path) # Get Tensors from loaded model loaded_x = loaded_graph.get_tensor_by_name('x:0') loaded_y = loaded_graph.get_tensor_by_name('y:0') loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0') loaded_logits = loaded_graph.get_tensor_by_name('logits:0') loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0') # Get accuracy in batches for memory limitations test_batch_acc_total = 0 test_batch_count = 0 for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size): test_batch_acc_total += sess.run( loaded_acc, feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0}) test_batch_count += 1 print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count)) # Print Random Samples random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples))) random_test_predictions = sess.run( tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions), feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0}) helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions) test_model()
Why 67% Accuracy?¶
You might be wondering why we can't get an accuracy any higher. First things first, 67% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. However, you might notice people are getting scores well above 80%. That's because we haven't learned all there is to know about neural networks. We still need to cover a few more techniques.
In [ ]:
No comments:
Post a Comment