Background

---

To find out what Deep Learning is and how it is different from most of Artificial Intelligence please go and check my last post.

Now.. What the heck is a Deep Convolutional Neural Network?

---

To answer this, we must first look at the operation of convolution. This is mainly used in mathematics, and it's an integral that expresses the amount of overlap of one function g as it is being shifted over another function f. We could say that the convolution "blends" one function with another.

Convolution sample 1	Convolution sample 2

We're not going to get too much into the mathematics behind this, but let's go a bit into the history of DCNNs:

Deep Convolutional Neural Networks have been the go-to solution for machine vision projects in the last few years. Early in 2015 after a series of breakthroughs by Microsoft, Google and Baidu, a machine was able to beat the human at an object recognition challenge for the first time in the history of AI.

It's hard to mention the CNN without touching on the ImageNet challenge. This is a project that was inspired by the need of high quality data in the image processing space. Every year the top teams in the world compete against each other to create the best possible object recognition software. Going back to 2012 when AlexNet took first place in the challenge, every single winner since then has used a Convolutional Network as their model - which isn't surprising since the error rate of image detection tasks has dropped significantly with CNNs.

Deep Convolutional Neural Networks, as any Deep Neural Network, contain many hidden layers. When looking at a DCNN we can see there is at least one convolutional layer, one activation ReLU layer, and one pooling layer - in this order; and they can repeat again - always ending with one fully-connected layer.

Example

---

Say we have an image of resolution 32x32 and we wanted to classify the image using CIFAR-10 (categorising any input image into one of 10 different categories).

Initially we have an input layer of a given volume - holding raw pixel values - in this case, say an image of width 32, height 32 and three color channels red, green and blue - which would be a volume of [32x32x3]

The convolutional layer applies a convolution operation to the input, computing the output of neurons that are connected to local regions in the input, each having to compute a dot product between their weights and a small region to which they are connected in the input volume. This operation would result in a volume of [32x32x12] if we wanted to use 12 filters.

The ReLU layer applies a rather ordinary elementwise activation function, max(0, x) thresholding at zero. This will leave the size of the volume unchanged.

The pool layer simply performs a downsampling operation on the spatial dimensions (w, h) resulting in a halved volume on the specified dimensions [16x16x12].

The fully-connected layer computes the class scores, then resulting in a volume size [1x1x10], where each of the 10 output numbers are linked to a class score - such as among the CIFAR-10 categories. The ten values would add up to 100% and the highest value would represent the prediction of the algorithm as to which category the image belongs to.

Of course, you can stack multiple Conv/ReLU/Pool layers, and even change the architecture of the Deep Convolutional Neural Network to better suit certain cases, as we can see in the image below - but this is out of the scope of this post:

So, what are the applications of Deep Convolutional Neural Networks you ask?

---

Deep Convolutional Neural Networks are currently being used in producing autonomous cars with the help of NVidia's Drive PX, diagnosing cancer in extremely early stages with a very high success rate, detecting certain facial expressions in various applications, etc. Therefore, DCNNs can be used on anything that provides visual input and is expected to be classified - your imagination is the limit.

Credits

---

The images used in this post are taken from Google search and the credits for them go to their original creators.

Thank you for reading.