While studying backpropagation, I came across this great mathematical explanation or breakdown of how it goes. Matt Mazur, a data scientist working for Help Scout, wrote about it on his personal blog.
I am so excited about the mathematical details that power neural networks that I enjoy writing down on a piece of paper how the actual computations work, both for forward propagation (the logits and the activations) and for backpropagation with gradient descent (the partial derivatives).
This passion and excitement rose inside of me as doing the courses in the Deep Learning specializations of Professor Andrew Ng at Coursera. This part of neural networks is greatly explain in the first course.
Anyway, backpropagation simply means back propagating the error you get as you reach the output of a network, with the purpose of updating the weights to minimize this error. Matt details both the forward pass and the backward pass through a neural network, with the equations and derivatives that they imply.
So, if you're a geek that's excited about this stuff, I'd recommend taking a piece of paper and replicating the work done by Matt on his blog. Also, do check the visualizations he provides as they enhance the understanding of backprop.
To stay in touch with me, follow @cristi
Cristi Vlad Self-Experimenter and Author
I always use chain rule of differentiation to understand backpropagation. Since neural network can be represented as f(g(h(...(input)...))) where each function is a layer of the neural network, while trying to minimize the error function which is another function of (predicted-actual), we differentiate this function of function representation, resulting in usage of chain rule.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
@cristi can you help me out in derivatives and integration i will be happy ?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Sure, can you be more specific?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Very interesting @cristi, as a mathematician Neural Networks and Genetical Algorithms are one my top list of Topics.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
You got yourself a new follower :) Saludos!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I new follower. Thanks
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
You got yourself one too!!!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Super. Very nice
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks for the link, looks very interesting.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
great information the one of your publication, generally I practice at every moment the mathematics since my university career is based on pure calculations. Greetings cristi
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I always take very good with math since I was in the school of Christ. regards
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit