January 2, 2020

# graykode/distribution-is-all-you-need

The basic distribution probability Tutorial for Deep Learning Researchers

repo name graykode/distribution-is-all-you-need
homepage
language Python
size (curr.) 1288 kB
stars (curr.) 753
created 2019-09-06

## distribution-is-all-you-need

distribution-is-all-you-need is the basic distribution probability tutorial for most common distribution focused on Deep learning using python library.

## distribution probabilities and features

1. Uniform distribution(continuous), code
• Uniform distribution has same probaility value on [a, b], easy probability.
1. Bernoulli distribution(discrete), code
• Bernoulli distribution is not considered about prior probability P(X). Therefore, if we optimize to the maximum likelihood, we will be vulnerable to overfitting.
• We use binary cross entropy to classify binary classification. It has same form like taking a negative log of the bernoulli distribution.
1. Binomial distribution(discrete), code
• Binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments.
• Binomial distribution is distribution considered prior probaility by specifying the number to be picked in advance.
1. Multi-Bernoulli distribution, Categorical distribution(discrete), code
• Multi-bernoulli called categorical distribution, is a probability expanded more than 2.
• cross entopy has same form like taking a negative log of the Multi-Bernoulli distribution.
1. Multinomial distribution(discrete), code
• The multinomial distribution has the same relationship with the categorical distribution as the relationship between Bernoull and Binomial.
1. Beta distribution(continuous), code
• Beta distribution is conjugate to the binomial and Bernoulli distributions.
• Using conjucation, we can get the posterior distribution more easily using the prior distribution we know.
• Uniform distiribution is same when beta distribution met special case(alpha=1, beta=1).
1. Dirichlet distribution(continuous), code
• Dirichlet distribution is conjugate to the MultiNomial distributions.
• If k=2, it will be Beta distribution.
1. Gamma distribution(continuous), code
• Gamma distribution will be beta distribution, if `Gamma(a,1) / Gamma(a,1) + Gamma(b,1)` is same with `Beta(a,b)`.
• The exponential distribution and chi-squared distribution are special cases of the gamma distribution.
1. Exponential distribution(continuous), code
• Exponential distribution is special cases of the gamma distribution when alpha is 1.
1. Gaussian distribution(continuous), code
• Gaussian distribution is a very common continuous probability distribution
1. Normal distribution(continuous), code
• Normal distribution is standarzed Gaussian distribution, it has 0 mean and 1 std.
1. Chi-squared distribution(continuous), code
• Chi-square distribution with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.
• Chi-square distribution is special case of Beta distribution
1. Student-t distribution(continuous), code
• The t-distribution is symmetric and bell-shaped, like the normal distribution, but has heavier tails, meaning that it is more prone to producing values that fall far from its mean.

## Author

If you would like to see the details about relationship of distribution probability, please refer to this.

• Tae Hwan Jung @graykode, Kyung Hee Univ CE(Undergraduate).
• Author Email : nlkey2022@gmail.com
• If you leave the source, you can use it freely.