Different types of uncertainty in DNN systems?

What are the uncertainties in DNN?

This question is easily answered if you have read a little bit of DNN literature. The most popular answer is - Aleatoric uncertainty - Epistemic uncertainty

An additional statement is always added to these different types of uncertainty which will claim that - Aleatoric uncertatinty is model uncertainty - Epistemic uncertatinty is data uncertainty

But what actually are these values and what do they explain ? Lets do a brief overview of different literature to answer these questions.

Alternative definitions

“Aleatory uncertainty is also referred to in the literature as variability, irreducible uncertainty, inherent uncertainty, and stochastic uncertainty. Epistemic uncertainty is also termed reducible uncertainty, subjective uncertainty, and state-of-knowledge uncertainty.” [2]

“Aleatory uncertainties are described as arising from inherent variabilities or randomness in systems, whereas epistemic uncertainties are due to imperfect knowledge.” [1]

Important points

1. Epistemic sources of uncertatinty is reducible but aleatory uncertainty is not reducible[1]

References

[1] Probability is Perfect, but we Can’t Elicitit Perfectly Anthony O’Hagan & Jeremy E. Oakley http://www.yaroslavvb.com/papers/epistemic.pdf

[2] Challenge problems: uncertainty in system response given uncertain parameters Author links open overlay panelWilliam L.Oberkam https://www.sciencedirect.com/science/article/pii/S0951832004000493