**What is Deep Learning?**

Traditional Machine Learning had used handwritten features and modality-specific machine learning to classify images, text or recognize voices. Deep learning / Neural network identifies features and finds different patterns automatically. Time to build these complex tasks has been drastically reduced and accuracy has exponentially increased because of advancements in Deep learning. Neural networks have been partly inspired from how 86 billion neurons work in a human and become more of a mathematical and a computer problem. We will see by the end of the blog how neural networks can be intuitively understood and implemented as a set of matrix multiplications, cost function, and optimization algorithms.

**Biological analogy of Neural Network**

**Mathematical/Functional analogy of a Neural Network**

**Note: A neuron is also called as a Unit or a function since a human neuron in the real sense is much more complex in nature than the one we use in our Neural Networks.**

**History of Deep** **Learning**

Though DL is being adopted across various enterprises in the last few years, the theory and techniques back to 1940.DL was called with various names.

Cybernetics 1940–1960

Connectionism 1980–1960

Deep learning 2006-*

**Why now?**

GPU from the gaming industry, Computation power on demand, and advancements in optimization algorithms have made it the right time for building applications using Deep learning.

- DL at a high level can be seen as a mathematical function and the nature of DL is that there exists a neural network for every problem. This is often called universality.Though there is no guarantee that we can find one. (More on: An intuitive explanation from Michael Nielsen)
- An algorithm which fine tunes the parameters of the function. Thanks to Back Propagation and Gradient descent and various versions of it which precisely does that.
- The above 2 steps which turns out to be a set of matrix multiplications and derivative calculations, which can be executed much faster on a GPU. Thanks to the Gaming industry for popularizing and a giving us faster GPU’s which was previously used for performing matrix operations to manipulate the pixels on the game screens.
- Thanks to cloud providers like Amazon, Microsoft for allowing customers to use GPU-based instances to build deep learning models on demand.

**The growth of Deep Learning**

A recent talk by Jeff Dean has highlighted the massive increase in Deep learning at google. At google, DL is being used in various departments and usage has increased exponentially in the last few years.

DL is also being applied across multiple impactful applications by different teams across the globe implying its ease of use. Few of them are

**Classification of cucumber:**It’s astonishing to know how a farmer from Japan used deep learning and TensorFlow to build a machine that uses image recognition to segregate cucumbers. Which previously used to take his mother 8 hours of work for few days.**Classification of skin Lesions:**Using a pre-trained Convolutional Neural Network, the group created a model that could accurately classify skin lesions by 60%, a four-fold increase over the previous benchmark of 15.6%. That’s a massive improvement!**Heart condition diagnosis:**Two hedge fund analysts built this model to diagnose heart diseases with accuracy matching that of doctors.**Clothes Classification:**This group successfully built a model that could recognize articles of clothing, as well as their style.

The applications that are being built has the ability to transform health sector, fashion industry and much more.

From Andrew Ng’s point of view. “AI with deep learning is the new electricity, Just as 100 years ago electricity transformed industry after industry, AI will now do the same.”

**Popular tools that ease the use of Deep Learning**

There are many tools that are available today that are helping companies adopt Deep learning faster. Few of them are

- Keras: We love keras, it allows us in building and executing our DL models very quickly. It provides a clean API in Python and runs either on TensorFlow and Theano. It was built with an aim of building DL models easier and faster. It comes with a support for algorithms like Convolutional Networks, Recurrent Networks and both.
- TensorFlow: TensorFlow is an open source software from google for numerical computing using data flow graphs. It helps researchers in experimenting and building new machine learning algorithms and new specialized neural network models. In addition to that, it provides a higher level tool called TF-learn which used for building deep neural networks quicker. Google in 2016 has open sourced the distributed version of tensor flow making it scalable. TensorFlow provides you for shipping models to android/ios devices thus reducing the time to deploy. It also comes with a serving layer which exposes the DL models to other systems.
- Theano: Theano has similar capabilities like Tensor Flow. We have observed a close tie between TensorFlow and Theano capabilities. Time has to say which is going to be better.

## Types of Neural Networks

Let us start with understanding how a single neural network works.

In the image above we have a neuron which takes an inputs X along with weights W and passes it to function f which acts as a classifier. It is very similar to logistic regression or SVM. We have the three important parts of building a neural network.

- Score function (f): The score function applies the activation function on the inputs along with weights. In this example, the activation function is a sigmoid which outputs values between 0 to 1, which acts as probabilities for our binary classifier network.
- Cost function: To evaluate how the neural network has performed against the ground truth we need to use an evaluation metric. We can use binary cross entropy for the same which tells us how good our network has performed.
- Optimization algorithm: Our function has input variables which are going to remain constant and the goal is to learn the weights to reduce the cost thus improving accuracy. There are different ways like searching the weights randomly or using algorithms like SGD (stochastic gradient descent) which helps in finding the optimal weights for the particular problem.

All the above steps are common to most of the machine learning algorithms. Let’s discuss few more jargons which are typically used in the deep learning domain before looking at different types of network.

- Hidden Layer: The name sounds too crazy when I heard it first but it is quite simple. Any layer that is between input and output is called hidden layers.
- Deep Networks: Any network that has more than 1 layer is called deep networks and as the depth increases the computational complexity also increases.
- Activation function: We have different activation functions available like sigmoid, tanh, Rectified Linear unit(relu) — applied max(0,-), leaky relu to fix some drawbacks that come with relu, max out. The most predominantly used ones are relu and max out usage has also picked up in the recent months.

### Feed Forward Neural network

Let us take an example of predicting whether a customer is planning a trip. We build multiple ML models which capture different pattern in the data. Then we use the output from these models to run another ML model to discover patterns which were not captured by the previous models. It becomes more complex when we keep on adding more models to the prediction system. Let us look how the stacking of multiple models could look like.

Feed forward neural network does this exactly.It helps in stacking up multiple models together and discovers hidden patterns. When observed closely the above network can be represented as nested matrix multiplication and can be represented as below

- Input Layer — X, Weights to hidden layer 1 — W where W,X are represented in the form of a matrix.
- Output to hidden layer 2 — Activation function f1(WX)
- 2nd Hidden layer can be represented as f2(f1(wx))

When the network becomes deep that is more than one layer, a new challenge arises. How do I associate the error with the cost function to multiple layers? Back propagation algorithm helps in propagating the error to all the weights in the previous layers. The best part is most of the today’s deep learning tool take care of the back propagation automatically. All we need to specify is build the topology of the network that is the number of layers, a number of nodes, what activation function to be used, how many epochs/iterations to be executed.

## Convolutional Neural Network

Feed forward neural network comes with its limitations when applied to problems like image recognition.Computers understand an image as a matrix with dimensions as height, width, channels (RGB). To use images in FFN we need to flatten the matrix into a vector of pixels. On simple image recognition like MNIST data set this gives accuracy around 95% but for complex image recognition the accuracy drops drastically. When images are flattened as vectors the spatial information about the data is lost. Convolutional networks help in capturing the spatial information.

## 1,921 Comments “Deep Learning Demystified”

I went over this website and I think you have plenty of excellent

information, saved to faves.

VK

I’m glad I found your article. I would never have

made sense of this topic on my own. I’ve read a number of articles on this topic, yet I

was baffled right up until I read yours.

I have been exploring for a little for any high quality

articles or blog posts in this kind of area .

Exploring the web I eventually came across this site. Reading this

information I’m happy to show that I have an incredibly

good weird feeling that I came upon exactly what I wanted.

Thanks for all your efforts that you have placed in this very interesting information.

It’s an amazing piece of writing in favor of all the internet viewers; they will obtain advantage

from it I am sure.

Greetings! Very helpful advice in this particular article!

It’s the little changes which will make the most important changes.

Thanks a lot for sharing!

Super-Duper website! I’m loving it!! Will be back later to

read some more. I am bookmarking your feeds as well.

I am definitely bookmarking this website and sharing it with my friends.

You’ll be getting a lot of visitors to your site from me!

I have wanted to write about something like this on my

webpage and you gave me an idea. Regards.

Wow! This could be one of the most useful blogs we’ve ever arrive across on this subject.Excellent.

I’m also an expert in this topic hence I can understand your effort.

Wow! This could be one of the most helpful blogs we’ve ever arrive across about this subject.Excellent.

I’m also an expert in this topic thus I can understand your effort.

Fantastic article. I appreciate you for sharing it to us.

I have learned a lot.

Does your website have a contact page? I’m having a tough time locating it but, I’d

like to send you an email. I’ve got some ideas for your blog you

might be interested in hearing. Either way, great site and I look forward to seeing it grow over

time.

You made some good points there. I looked on the internet for the issue and found most people will

go along with your website.

Spot on with this write-up, I truly believe that this

amazing site needs a great deal more attention. I’ll probably be returning

to see more, thanks for the information!

Amazing! This could be among the most useful blogs we’ve came across

on this subject. Wonderful. I am also a specialist in this topic so I could understand your effort.

I enjoy your article. It’s evident that you have plenty of knowledge about

this topic. Your points are well made and relatable.

Thanks for writing engaging and useful material.

Thanks for sharing exceptional informations. Your site is extremely cool.

I’m stunned at the details which you’ve put on this site.

It unveils how perfectly you recognize this subject.

Bookmarked this web site, will keep coming back for more articles.

You, amigo, ROCK! I found simply the information I already searched in all places

and simply couldn’t come across. What an ideal

website.

Yours is a prime example of informative writing.

I think my students could learn a whole lot from your writing style and your content.

I may share this post with them.

Sex during periods is also healthy for a woman; becauuse the orgasm releases some of the muscular tension in thhe vagina, and will also “clean out” all excess fluids that might build upp during the use oof a tampon or pad.

Being a virgin at age 70 is tough (just remember how tough

iit was for The 40-year-old virgin) but Shaw

is hopeful that her Prince Charming is still out there somewhere.

“She is not Victoria’s ‘muse’ and there is no ‘face of’ the advertising campaign.

Hello there, Neat post. I’ll be your frequent visitor.

Thanks for this post, I am a big fan of this web site would like to

be updated.

I just like this site so much, saved to bookmarks.

It’s going to be ending of mine day, but before ending I

am reading this enormous article to increase my knowledge.

Many thanks for sharing. I am fascinated with your website.

I’ll publish this to my facebook wall.

Hello Web Admin, I noticed that your On-Page SEO is is missing a few factors, for one you do not use all three H tags in your post, also I notice that you are not using bold or italics properly in your SEO optimization. On-Page SEO means more now than ever since the new Google update: Panda. No longer are backlinks and simply pinging or sending out a RSS feed the key to getting Google PageRank or Alexa Rankings, You now NEED On-Page SEO. So what is good On-Page SEO?First your keyword must appear in the title.Then it must appear in the URL.You have to optimize your keyword and make sure that it has a nice keyword density of 3-5% in your article with relevant LSI (Latent Semantic Indexing). Then you should spread all H1,H2,H3 tags in your article.Your Keyword should appear in your first paragraph and in the last sentence of the page. You should have relevant usage of Bold and italics of your keyword.There should be one internal link to a page on your blog and you should have one image with an alt tag that has your keyword….wait there’s even more Now what if i told you there was a simple WordPress plugin that does all the On-Page SEO, and automatically for you? That’s right AUTOMATICALLY, just watch this 4minute video for more information at. Seo Plugin

I’m truly enjoying the design and layout of your blog.

It’s a very easy on the eyes which makes it much more

pleasant for me to come here and visit more often. Did you hire out a developer to create your theme?

Excellent work!cheap nhl jerseys

I like this web site very much, Its a very nice situation to read and

find info.

Hello! Great stuff here, please keep us posted.