Dr. Xiaozhou Li

Dr. Xiaozhou Li

Mathematics. Strength. Long-Term Mindset. Think Hard. Train Harder. Live Long.

Lecture Note: §3 Random Variables and Discrete Distributions

Published:

Last Update: 2025-11-24

Random Variable

  • A variable whose value is randomly decided.

  • A random variable is a real-valued function defined on a sample space.

  • Random variables are the main tools used for modeling unknown quantities in statistical analyses.

  • Deterministic variables.

  • 随机事件这个概念实际上是包容在随机变量这个更广的概念之内。随机事件是从静态的观点来研究随机现象，而随机变量则是一种动态的观点，一如数学分析中的常量与变量的区分那样。变量概念是高等数学有别于初等数学的基础概念。同样，概率论能从计算一些孤立事件的概念发展为 一个更高的理论体系，其基础概念是随机变量。

§3.1 Random Variables and Discrete Distributions

  • Random Variable

    • Let \(S\) be the sample space for an experiment. A real-valued function that is defined on \(S\) is called a random variable.
  • Discrete Distribution/Random Variable
    • It takes a finite number \(k\) of different values \(x_1, x_2, \ldots, x_k\), or, at most, an infinite sequence of different values \(x_1, x_2, \ldots\).

  • Probability Function/p.f./Support

    • If a random variable \(X\) has a discrete distribution, the probability function (p.f.) of \(X\) is defined as the function \(f\) such that for every real number \(x\), \(f(x) = \Pr(X =x)\).

    • The closure of the set \({x : f (x) > 0}\) is called the support of (the distribution of) \(X\).

    • Properties:

    - \(f(x)\geq 0\), - \(\sum_{i=1}^{\infty} f(x_i) = 1\), - \(\Pr(X \in C) = \sum_{x_i \in C} f(x_i)\)

  • Typical Discrete Distributions

    • Bernoulli Distribution

    • Uniform Distribution

    • Binomial Distribution

    • Poisson Distribution

    • The Hypergeometric Distributions

    • The Negative Binomial Distributions

§3.2 Continuous Distributions

  • Continuous Distribution/Random Variable
    • The probability density function (p.d.f) \(f(x)\):
    - \(\Pr(a\leq X\leq b) = \int_{a}^{b} f(x)\,dx\).
    • As the size of the interval gets smaller, we can write \(\Pr(a\leq X\leq a + dx) \approx f(a)dx\)
    - Continuous distributions assign probability \(0\) to individual values.
    • Properties:
    - \(f(x)\geq 0\), - \(\int_{-\infty}^\infty f(x)\,dx = 1\)

  • Typical Continuous Distributions

    • Uniform Distribution

    • Normal Distribution

    • Exponential Distributions

§3.3 The Cumulative Distribution Function

Although a discrete distribution is characterized by its p.f. and a continuous distribution is characterized by its p.d.f., every distribution has a common characterization through its (cumulative) distribution function (c.d.f.).

  • (Cumulative) Distribution Function: the distribution function or cumulative distribution function (abbreviated c.d.f.) of a random variable is the function

    • \(F(x) = \Pr(X \leq x) \quad \text{ for } -\infty < x < \infty\)

    • Properties of c.d.f

      • Nondecreasing

      • \(\lim _{x \mapsto -\infty} F(x)=0\) and \(\lim _{x \mapsto \infty} F(x)=1\)

      • Continuity from the right \(F(x)= F(x^{+})\)

      • \(\Pr (X > x) = 1 - F(x)\)

      • \(\Pr(x_1 < X \leq x_2) = F(x_2) - F(x_1)\)

      • \(\Pr(X = x) = F(x) - F(x^{-})\)

  • The c.d.f. of a Discrete Distribution

    • \(F(x)\) will have a jump of magnitude \(f(x_i)\) at each possible value \(x_i\) of \(X\).

  • The c.d.f of a Continuous Distribution

    • \(F(x)\) is continuous at every \(x\), \(F(x)=\int_{-\infty}^{x} f(t)\, dt\)

    • \(\frac{d F(x)}{d x}=f(x)\) at all \(x\) such that \(f\) is continuous.

  • Quantiles/Percentiles

§3.4 Bivariate Distributions

We generalize the concept of distribution of a random variable to the joint distribution of two random variables.

  • Discrete Joint Distribution

    • Joint Probability Function (p.f.): \(f (x, y) = \Pr (X = x \text{ and } Y = y)\)

    • \(0 \leq f(x,y) \leq 1\), \(\sum_{\text{all }(x,y)}f(x,y)=1\)

    • \(\Pr[(X,Y)\in C]= \sum_{(x,y)\in C}f(x,y)\).

  • Continuous Joint Distribution

    • Joint p.d.f: \(\Pr[(X, Y ) \in C] = \int_C \int f (x, y) dx dy\),

    • \(f(x,y) \geq 0\), \(\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f(x,y) dxdy = 1\)

  • Mixed Bivariate Distributions

    • Joint p.f/p.d.f: \(\Pr(X\in A \text{ and } Y\in B) = \int_B \sum_{x\in A} f (x, y) dy\)
  • Joint (Cumulative) Distribution Function/c.d.f
    • \(F(x,y) = \Pr (X\leq x \text{ and } Y\leq y)\)

§3.5 Marginal Distributions

  • Marginal c.d.f

    • \(F_1 (x) = \lim_{y \mapsto \infty} F (x, y)\) and \(F_2 (y) = \lim_{x \mapsto \infty} F (x, y)\)

  • Deriving a Marginal p.f from a Joint p.f

    • \(f_1(x)= \sum_{\text{All }y}f(x,y)\) and \(f_2(y)= \sum_{\text{All }x}f(x,y)\)

  • Deriving a Marginal p.d.f from a Joint p.d.f

    • \(f_1(x) = \int_{-\infty}^{\infty} f(x,y)dy\) and \(f_2(y) = \int_{-\infty}^{\infty} f(x,y)dx\)

  • Deriving a Marginal p.f/p.d.f from a Joint p.f/p.d.f

§3.6 Conditional Distributions

  • Discrete/Continuous Conditional Distribution

    • The conditional p.f./p.d.f. of \(X\) given \(Y\): \(g_1(x \vert y)= \dfrac{f(x,y)}{f_2(y)}\)

  • Multiplication Rule for Distributions

    • \(f (x, y) = g_1(x \vert y)f_2(y)\) and \(f (x, y) = f_1(x)g_2(y \vert x)\)

  • Law of Total Probability for Random Variables

    • \(f_1(x) = \sum_y g_1 (x \vert y) f_2 (y)\)

    • \(f_1(x) = \int_{-\infty}^{\infty} g_1 (x \vert y) f_2 (y)dy\)

  • Bayes’ Theorem for Random Variables

    • \(g_2(y \vert x) = \dfrac{g_1 (x \vert y) f_2 (y)}{f_1(x)}\)

    • \(g_1(x \vert y) = \dfrac{g_2 (y \vert x) f_1 (y)}{f_2(x)}\)

  • Independent Random Variables
    • Definition:
    - \(\Pr(X\in A \text{ and }Y\in B) = \Pr(X \in A)\Pr(Y \in B) \) - \(g_1(x \vert y) = f_1(x)\)
    • Properties:
    - \(F(x,y) = F_1(x)F_2(y)\) - \(f(x,y) = f_1(x)f_2(y)\)

§3.7 Multivariate Distributions

  • Joint Distribution Function/c.d.f
    • \(F(x_1, \ldots, x_n) = \Pr (X_1 \leq x_1, X_2\leq x_2, \ldots, X_n\leq x_n)\)
  • Joint Discrete Distribution
    • \(f(x_1, \ldots, x_n) = \Pr (X_1 = x_1, \ldots, X_n= x_n)\)
  • Joint Continuous Distribution

    • \(\Pr[(X_1 ,\ldots,X_n) \in C]=\int\cdots\int\limits_{C} f(x_1,…,x_n) dx_1\cdots dx_n\)

    • \(f(x_1, \ldots, x_n) = \dfrac{\partial^n F(x_1,\ldots,x_n)}{\partial x_1\cdots \partial x_n}\)

  • Marginal Distributions/c.d.f
    • \(F_1(x_1) = \Pr(X_1\leq x_1) = \Pr(X_1 \leq x_1, X_2\leq \infty, \ldots, X_n\leq \infty) = \lim_{x_2,\ldots, x_n \mapsto \infty}F(x_1,x_2,\ldots,x_n) \)
  • Independent Random Variables

    • \( \Pr(X_1 \in A_1, X_2 \in A_2,…,X_n \in A_n) =\Pr (X_1 \in A_1)\Pr (X_2 \in A_2)\cdots \Pr(X_n \in A_n)\)

    • Independent and Identically Distributed (i.i.d)

  • Conditional Distributions

    • Multivariate Law of Total Probability and Bayes’ Theorem

    • Conditionally Independent Random Variables

§3.8 Functions of a Random Variable

  • Discrete Distribution \(Y = r(X)\):

    • \(g(y)= \Pr(Y=y)=\Pr[r(X)=y]= \sum_f(x)\)

  • Continuous Distribution

    • Linear Function \(Y = aX + b\)

    • \(Y = X^{2}\)

    • General discussion: \(Y = r(X)\)

  • Functions of Two or More Random Variables

    • Discrete Distribution

      • let denote the set of all points \((x_1, . . . , x_n)\) such that \(r_1(x_1,\ldots, x_{n}) = y_1\), \(r_2(x_1,\ldots, x_{n}) = y_2\), \(\ldots, r_m(x_1,\ldots, x_{n}) = y_m\):

        • \(g(y_1, . . . , y_m) = \sum_{(x_1,\ldots,x_n)\in A}f(x_1, . . . , x_n)\)
    - Binomial Distribution - If the random variables \(X_1,\ldots,X_n\) are i.i.d Bernoulli distribution with parameter \(p\), and if \(X = X_1 +\cdots + X_n\), then \(X\) has the binomial distribution with parameters \(n\) and \(p\).

    • Continuous Distribution

      • Linear Function \(Y = a_1 X_1 + a_2 X_2 + b\)

      • If (\(X_1\) and \(X_2)\) are independent, the convolution \(Y = X_1 + X_2\),

        • \(g(y) = \int_{-\infty}^{\infty} f_1(y-z)f_2(z)dz\)

        • or \(g(y) = \int_{-\infty}^{\infty} f_1(z)f_2(y-z)dz\)