Understanding the Gamma Distribution: Definition, Applications, Intuition, and Mean Derivation

What Is the Gamma Distribution?

• A continuous probability distribution defined for (Y \ge 0).
• Parameters: (\alpha>0) (shape) and (\beta>0) (rate).
• Probability density function (PDF): [ f_{Y}(y\mid\alpha,\beta)=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\,y^{\alpha-1}e^{-\beta y},\qquad y\ge0 ]
• (\Gamma(\alpha)) is the gamma function, the continuous analogue of the factorial.

Why Use the Gamma Distribution in Bayesian Inference?

• Modeling non‑negative quantities: It is ideal for variables that represent rates, waiting times, or any quantity that cannot be negative.
• Prior for Poisson rate ((\lambda)): When the data follow a Poisson distribution, a Gamma prior on (\lambda) is conjugate, leading to analytically tractable posteriors.
• Prior for precision ((\tau = 1/\sigma^2)): In normal‑likelihood models, a Gamma prior on precision simplifies calculations because precision is also non‑negative.

Intuition Behind the Two Parameters

Shape Parameter (\alpha)

• (\alpha = 1): The PDF reduces to an exponential decay (e^{-\beta y}); the distribution peaks at zero.
• Increasing (\alpha) adds a polynomial factor (y^{\alpha-1}) that initially pushes the density away from zero, creating a hump. As (\alpha) grows, the peak moves rightward and the distribution becomes more symmetric.
• Examples:
• (\alpha=2): PDF (y e^{-\beta y}) – starts low, rises to a modest peak, then decays.
• (\alpha=3): PDF (y^{2} e^{-\beta y}) – higher, later peak.

Rate Parameter (\beta)

• Controls scale (inverse of the mean). Larger (\beta) makes the distribution taller and narrower (sharper peak) because:
• The factor (\beta^{\alpha}) raises the overall height.
• The exponential term (e^{-\beta y}) forces a faster decay.
• Visual effect: With (\alpha) fixed, raising (\beta) squeezes the distribution toward zero while increasing its maximum.

Visual Exploration (Conceptual)

• Alpha = 1, Beta = 1 → Simple exponential curve.
• Alpha = 2, Beta = 1 → Hump appears; peak moves right of zero.
• Alpha = 3, Beta = 1 → Higher, more right‑shifted hump.
• Alpha = 3, Beta = 2 → Same shape but taller and sharper; the tail drops off more quickly.
• Computational tools (e.g., MATLAB) can plot these families to see how the PDF morphs with parameter changes.

Deriving the Mean of a Gamma Distribution

1. Start with the expectation definition: [ \mathbb{E}[Y]=\int_{0}^{\infty} y\,f_{Y}(y\mid\alpha,\beta)\,dy ]
2. Insert the PDF and pull out constants: [ \mathbb{E}[Y]=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\int_{0}^{\infty} y^{\alpha}e^{-\beta y}\,dy ]
3. Recognize the integral as a Gamma function with shape (\alpha+1) and rate (\beta): [ \int_{0}^{\infty} y^{\alpha}e^{-\beta y}\,dy = \frac{\Gamma(\alpha+1)}{\beta^{\alpha+1}} ]
4. Combine constants: [ \mathbb{E}[Y]=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\cdot\frac{\Gamma(\alpha+1)}{\beta^{\alpha+1}} = \frac{\Gamma(\alpha+1)}{\Gamma(\alpha)}\cdot\frac{1}{\beta} ]
5. Use the property (\Gamma(\alpha+1)=\alpha\Gamma(\alpha)) to simplify: [ \mathbb{E}[Y]=\frac{\alpha}{\beta} ]
6. The result holds for any positive (\alpha), integer or not.

Key Takeaways

• The Gamma distribution is a flexible tool for modeling positive continuous data and serves as a conjugate prior for Poisson rates and precision parameters.
• Shape (\alpha) determines the location and existence of a hump; rate (\beta) controls scale, making the distribution taller and narrower as it increases.
• The mean of a Gamma((\alpha,\beta)) distribution is simply (\alpha/\beta), derived via a neat trick that leverages the Gamma function’s integral definition.

The Gamma distribution’s simple yet powerful form—characterized by shape and rate—makes it indispensable for Bayesian modeling of non‑negative quantities, and its mean is elegantly given by the ratio of its parameters, (\alpha/\beta).