Introduction to Continuous Probability Distributions

1 Introduction to Continuous Probability Distributions

Probability theory is broadly divided into two major branches depending on the nature of the random variable being studied: discrete and continuous. While many introductory statistics courses begin with discrete distributions — counting heads in coin flips, tallying defective products on an assembly line, or recording the number of customers arriving at a bank — a vast range of real-world phenomena cannot be adequately described by counting. Measurements of time, weight, temperature, distance, and voltage all share a common feature: they exist on a continuous scale, where values flow seamlessly from one to the next without gaps. Understanding continuous probability distributions requires a fundamental shift in perspective, moving from assigning probabilities to individual outcomes to reasoning about probabilities over intervals. This topic builds that foundation carefully, explaining why the tools developed for discrete variables must be reimagined for the continuous case.

Discrete vs. Continuous Random Variables

A random variable is a function that assigns a numerical value to each outcome of a random experiment. The critical distinction between discrete and continuous random variables lies in the set of values they can take.

A discrete random variable takes on a countable number of distinct values. "Countable" here means you could, in principle, list every possible outcome: 0, 1, 2, 3, and so on. Examples include the number of cars passing through an intersection in an hour, the number of defective items in a batch of 100, or the result of rolling a six-sided die. For discrete variables, it is entirely meaningful — and often straightforward — to assign a specific, nonzero probability to each individual outcome. The probability distribution is described by a probability mass function (PMF), which maps each possible value to its probability, and the probabilities across all outcomes sum to exactly 1.

A continuous random variable, by contrast, can take on any value within a given interval or range. The possible outcomes form an uncountably infinite set. Consider measuring the exact time it takes for a chemical reaction to complete, or the precise weight of a newborn baby, or the temperature of a furnace at a randomly selected moment. Between any two values — say 2.5 seconds and 2.6 seconds — there are infinitely many possible measurements: 2.51, 2.507, 2.5071, and so on without end. No finite list can exhaust them.

This distinction has profound implications. Common continuous measurements include:

• Time: The lifespan of a light bulb, the time between arrivals of buses, the duration of a phone call.
• Physical measurements: The height or weight of individuals in a population, the diameter of a manufactured part.
• Environmental data: Daily rainfall totals, wind speed, air temperature.
• Financial data: Daily percentage changes in stock prices, loan repayment amounts.

For all of these, the strategy used to compute probabilities for discrete variables — summing up probabilities at individual points — completely breaks down, and a new mathematical framework is required.

Why Individual Point Probabilities Are Zero

This is one of the most conceptually important and initially surprising results in continuous probability theory: for any continuous random variable X, the probability of X taking on any single, exact value is zero. Formally, P(X = a) = 0 for every specific value a.

To understand why, consider what it would mean if this were not the case. Suppose the probability of landing exactly on any specific value were some tiny but positive number, say 0.0001. Since there are infinitely many values in any interval — an uncountably infinite number — the sum of all these individual probabilities would be infinite, not 1. The only way probabilities can remain consistent (summing to 1 over the entire range) when spread across infinitely many values is if each individual point carries zero probability.

Think of it this way: if you randomly select a real number from the interval [0, 1], asking "what is the probability that I selected exactly 0.37291847...?" is like asking what fraction of the interval a single geometric point occupies. A point has no width, so it contributes zero area — and zero probability.

This has a critically useful practical consequence: for continuous distributions, strict and non-strict inequalities are interchangeable.

• P(X < a) = P(X ≤ a), because adding the single point a contributes zero additional probability.
• Similarly, P(a < X < b) = P(a ≤ X ≤ b) = P(a ≤ X < b) = P(a < X ≤ b).

This is in sharp contrast to discrete distributions. For a discrete variable, the distinction between P(X < 3) and P(X ≤ 3) is critical — P(X = 3) may be substantial, say 0.25, and omitting or including that point changes the probability significantly. In the continuous world, this concern disappears entirely because boundary points carry no weight.

The Probability Density Function (PDF)

Since we cannot meaningfully talk about the probability of landing at a single point, continuous distributions are instead described by a probability density function, typically denoted f(x). The PDF is the central mathematical object for characterizing a continuous distribution, and understanding it correctly is essential.

The PDF describes the density of probability at each point — that is, how concentrated or spread out the probability is in different regions. A higher value of f(x) at some point x means that values near x are more likely to be observed than values near a point where f(x) is lower. However, f(x) itself is not a probability. This distinction is crucial:

• For a discrete variable, the PMF gives P(X = x), which must lie between 0 and 1.
• For a continuous variable, f(x) gives probability density — the rate at which probability accumulates per unit of x. The value of f(x) can exceed 1, and this is perfectly valid. For instance, the PDF of a uniform distribution on the interval [0, 0.5] equals 2 everywhere on that interval — well above 1 — and this is entirely legitimate.

Every valid PDF must satisfy exactly two conditions:

• Non-negativity: f(x) ≥ 0 for all x. Density cannot be negative.
• Total area equals 1: The integral of f(x) over its entire domain equals 1, reflecting the certainty that the variable will take some value.

Different families of continuous distributions are defined by the specific mathematical form of their PDF. For example:

• The uniform distribution on [a, b] has PDF f(x) = 1/(b−a) for a ≤ x ≤ b, and 0 elsewhere — a flat, rectangular shape.
• The normal distribution with mean μ and standard deviation σ has the familiar bell-shaped PDF: f(x) = (1/(σ√(2π))) × e^(−(x−μ)²/(2σ²)).
• The exponential distribution with rate parameter λ has PDF f(x) = λe^(−λx) for x ≥ 0, a curve that starts high and decays rapidly.

Each of these formulas defines a uniquely shaped curve, but all satisfy the two fundamental requirements: non-negativity and total area of 1.

Probability as Area Under the Curve

The mechanism for computing probabilities from a PDF is geometric: probability equals area under the PDF curve. Specifically, the probability that a continuous random variable X falls between two values a and b is the area under the PDF curve between x = a and x = b.

Mathematically, this is expressed using the definite integral:

P(a ≤ X ≤ b) = ∫[a to b] f(x) dx

This formula is the cornerstone of continuous probability calculations. Let's unpack it with a concrete example. Suppose X follows a uniform distribution on the interval [0, 10], so f(x) = 1/10 = 0.1 for all x in [0, 10]. What is the probability that X falls between 3 and 7?

P(3 ≤ X ≤ 7) = ∫[3 to 7] 0.1 dx = 0.1 × (7 − 3) = 0.1 × 4 = 0.4

Intuitively, the interval [3, 7] covers 40% of the total range [0, 10], so there is a 40% chance of falling there. The area under the flat PDF curve between 3 and 7 is simply a rectangle of width 4 and height 0.1, giving area 0.4.

For more complex PDFs like the normal distribution, the integral does not have a simple closed form, and probabilities are typically obtained from tables or software. But the conceptual principle remains the same: shade the region under the curve between two values, and the area of that shaded region is the probability.

A few key properties follow directly from this area-based interpretation:

• The total area under the entire PDF curve, from the lowest to highest possible value of X, always equals exactly 1. This ensures all probabilities account for every possible outcome.
• The probability of X being less than some value a equals the area under the curve to the left of a.
• The probability of X exceeding some value b equals the area under the curve to the right of b, which equals 1 minus the area to the left of b.
• Adding or removing a single point from an interval changes the area by zero, which is why strict and non-strict inequalities give the same result.

The Cumulative Distribution Function (CDF)

Working directly with integrals each time you need a probability can be cumbersome. The cumulative distribution function (CDF), denoted F(x), is a related function that packages the work of integration into a convenient tool for probability calculations.

The CDF is defined as the probability that the random variable X takes a value less than or equal to x:

F(x) = P(X ≤ x) = ∫[−∞ to x] f(t) dt

In other words, F(x) accumulates probability from the far left of the distribution up to the point x. Because it represents a probability, F(x) always satisfies 0 ≤ F(x) ≤ 1.

The CDF has several essential properties:

• Non-decreasing: As x increases, probability only accumulates — it never decreases. So F is a non-decreasing (monotonically increasing) function.
• Boundary behavior: As x approaches the lower limit of the distribution, F(x) approaches 0. As x approaches the upper limit (or positive infinity), F(x) approaches 1.
• Interval probabilities from the CDF: The probability that X falls between a and b is simply the difference of CDF values: P(a ≤ X ≤ b) = F(b) − F(a). This avoids re-integrating and makes the CDF especially practical.

For example, suppose the CDF for some distribution gives F(7) = 0.70 and F(3) = 0.30. Then:

P(3 ≤ X ≤ 7) = F(7) − F(3) = 0.70 − 0.30 = 0.40

This matches what we computed directly via integration in the previous section. The CDF approach is particularly valuable for the normal distribution, where direct integration is impossible without numerical methods. Standard normal tables are nothing more than tabulations of the normal CDF — they give F(z) for various values of z, and virtually all normal distribution problems reduce to computing differences of these CDF values.

The relationship between the PDF and CDF is the fundamental theorem of calculus: the PDF is the derivative of the CDF, and the CDF is the integral of the PDF. If you know one, you can find the other.

f(x) = d/dx F(x)     (PDF is the derivative of the CDF)
F(x) = ∫[−∞ to x] f(t) dt     (CDF is the integral of the PDF)

Key Parameters of Continuous Distributions

Just as a discrete distribution can be characterized by summary numbers, continuous distributions are described by parameters — numerical constants that appear in the PDF formula and determine the distribution's location, shape, and spread.

The two most fundamental parameters are:

• Mean (μ): Also called the expected value, the mean describes the center or balance point of the distribution. For a continuous variable, it is computed as the integral of x weighted by the PDF: μ = ∫ x · f(x) dx. It indicates where the distribution is located on the number line.
• Variance (σ²): The variance measures the spread or dispersion of the distribution around the mean. It is computed as σ² = ∫ (x − μ)² · f(x) dx. The square root of the variance, σ, is the standard deviation, which is expressed in the same units as X and is more intuitively interpretable.

Different families of distributions use different parameters. The table below summarizes the key parameters for three common continuous distributions:

An important insight about parameters is that changing them transforms the PDF curve in predictable ways without violating the fundamental rules of a valid PDF. For example:

• Changing μ in a normal distribution shifts the bell curve left or right along the x-axis, changing where it is centered but not its shape.
• Increasing σ in a normal distribution flattens and widens the bell curve, while decreasing σ makes it taller and narrower. In both cases, the total area under the curve remains exactly 1.
• For the exponential distribution, a larger rate λ produces a steeper drop-off, meaning shorter waiting times are far more likely.

In applied statistics, the true parameters of a population distribution are almost never known exactly. Instead, they are estimated from sample data — the sample mean x̄ estimates μ, and the sample variance s² estimates σ². These estimates allow practitioners to fit a continuous distribution to real-world data and use it to make predictions, set tolerances, or calculate probabilities relevant to their specific problem. For instance, a manufacturer might measure the diameters of 200 machined parts, estimate μ and σ from this sample, fit a normal distribution, and then compute the probability that a randomly produced part falls outside acceptable tolerances.

Mastering these foundational concepts — the distinction between discrete and continuous variables, the meaning of zero point probability, the role of the PDF and CDF, and the significance of distribution parameters — forms the bedrock upon which all more advanced work with continuous distributions rests.