DNN Wiki

Deep Neural Network Personal Wiki
Learning
Published

July 22, 2020

What to learn?

I wanted to develop glossary/Wiki of DNN related topics and my explanation of them so that I can be sure that I know the topics. But for that I needed a list of relevant topics. DNN is an exponentially exploding field and with low signa to noise ratio. So it becomes really difficult in fooling up with the new work without having a firm understanding of what is firm knowledge. This selection of topics should help any new commer to be sure that if I know what these topics are then you can claim that you know a little about deep learning.

  • Deep Learning vs Other Machine Learning Approaches
  • The Essential Math of Artificial Neurons
  • The Essential Math of Neural Networks
  • Activation Functions
  • Cost/Loss Functions
  • Stochastic Gradient Descent
  • Backpropagation
  • Mini-Batches
  • Learning Rate
  • Optimizers (e.g., Adam, Nadam)
  • Glorot/He Weight Initialization
  • Dense Layers
  • Softmax Layers
  • Dropout
  • Data Augmentation

References

  1. https://aiplus.odsc.com/courses/deep-learning-with-tensorflow-2-and-pytorch-1