Bernoulli Distribution

The Bernoulli Distribution is the mathematical framework for the simplest kind of experiment: one where there are only two possible ways things can end.

Think of it as the “Either/Or” model. In statistical language, we typically label these two outcomes as 1 (Success) and 0 (Failure).
 

• Success (1): This doesn't necessarily mean “good”—it just means the specific thing you were looking for happened. We represent the chance of this happening with the letter \(p.\)
   
• Failure (0): This just means the other option happened. The probability of this is whatever is left over, written as \(q = 1 - p.\)
   

This concept, known as a Bernoulli trial, appears frequently in daily life. You are running a Bernoulli trial every time you:
 

• Flip a coin (Heads vs. Tails).
   
• Guess on a True/False quiz question.
   
• Quality-check a product (Working vs. Defective).
   

Why is it important?

The Bernoulli distribution is famous not because it's complex, but because it is the foundation for much larger statistical concepts. It is the “atom” of discrete probability.
 

• If you repeat a Bernoulli trial multiple times (like flipping a coin 10 times) and count the wins, you build a Binomial distribution.
   
• If you keep flipping the coin until you finally get a “Head,” you are building a Geometric distribution.
   

Without Bernoulli, these more complex models wouldn't exist.

Example:

Imagine you are rolling a standard die. Let's say you decide that landing on Six counts as a “success.”
 

• Your random variable (X) has a \(\frac{1}{6}\times100=16.66\%\) chance of being a success.
   
• Mathematically, that is \(P(X=1) = 0.1666.\)
   
• Consequently, getting anything other than Six (failure) is \(1-0.1666=0.8334.\)
   

While the math is simple, this model gives statisticians a standardized way to describe binary outcomes in everything from gambling to manufacturing.

Before diving deeper into Bernoulli Distribution, let us look into the basic terminologies:
 

• Success and Failure: In a Bernoulli trial, the two possible outcomes are labeled as success (1) and Failure (0).
   
• Probability of Success (p): This is the likelihood of a success occurring in a single Bernoulli trial, and it always lies between 0 and 1.
   
• Probability of Failure (q): The probability of Failure is the complement of the success, given by \(q = 1 − p.\)
   
• Random Variable (X): In a Bernoulli Distribution, the random variable \(X\) can take only two values—1 for success and 0 for Failure.
   
• Bernoulli Trial: A single experiment or test that results in exactly two possible outcomes.
   
• Bernoulli Parameter: The parameter p represents the probability of success in the Bernoulli distribution.
   
• Expected Value (Mean): The expected value, written as \(E[X]\), represents the average outcome over many Bernoulli trials.
   
• Variance: This indicates how much the values of the random variable \(X\) deviate from the mean.
   
• Binomial Distribution: A distribution formed by performing multiple independent Bernoulli trials and counting the number of successes.
   
• Geometric Distribution: A distribution derived from Bernoulli trials that measures how many trials are needed to obtain the first success.
   
• Negative Binomial Distribution: Another distribution based on Bernoulli trials, describing the number of trials required to achieve a fixed number of successes.

Learning the properties of Bernoulli distribution helps students understand the situations with only two possible outcomes. It is a simple way to know how probability works in real life and also in subjects, such as economics, mathematics, statistics, and computer science. The properties of Bernoulli distribution are listed below:
 

• Binary Outcomes: There are strictly two possible results for every experiment: Success (labeled 1) or Failure (labeled 0).
   
• Fixed Odds: The probability of getting a “Success” \((p)\) is locked in; it never changes from one trial to the next.
   
• No Memory (Independence): Every attempt is a fresh start. The result of your previous try has absolutely no effect on the current one.
   
• The “Leftover” Rule: The chance of Failure \((q)\) is mathematically just whatever is left over after accounting for success \((1 - p).\)
   
• Discrete Nature: Because you can only land on specific whole numbers (0 or 1) rather than a continuous range, it is a discrete distribution.
   
• The Average (Mean): The expected value is equal to the probability of success: \((p).\)
   
• The Spread (Variance): The variance, which measures how much results differ from the average, is calculated by multiplying the chance of success by the opportunity of Failure: \(p(1 - p).\)

The Bernoulli distribution formula expresses the probability of two possible results. It shows the chances of success and failure.
 

The Bernoulli distribution has two important functions, probability distribution function (PDF) and cumulative distribution function (CDF). The PDF gives the probability of success and failure. The CDF, on the other hand, describes the probability of getting a value less than or equal to 1 or 0.  

The PDF for Bernoulli distribution is:

\(P (X = x) = px (1 − p)1 − x\)

Here, 

\(P (X = x) =\) The chance of getting the possible outcome, like success or failure.

\(X =\) The outcome 

Where \(x = 1\) means success and \(x = 0\) means failure

\(p =\) Probability of success.

\(1 - p =\) Probability of failure.
 

Now, let us take a closer look at this formula with an example:

In a running competition, if the probability of winning is 0.7, then the probability of losing is:
 

\(1 − 0.7 = 0.3\)

By using the Bernoulli distribution formula, we can find the probability.

Probability of success:

\(​ P (X = x) = px (1 − p) − x ​\)

\(​ P (X = 1) = (0.7) (0.3) − 1 ​\)

\(0.7 × 1 = 0.7\)

So, the probability of winning is 0.7

Probability of failure:

\(​ P (X = x) = px (1 − p) − x ​\)

\(​ P (X = 0) = (0.7) (0.3)1 − 0 ​\)

\(1 × 0.3 = 0.3\)

Therefore, the probability of losing is 0.3

The cumulative distribution function for the Bernoulli distribution is:

\(F(x) = 0\), when \(x < 0\).

\(F(x) = 1 − p\), if \(0 ≤ x < 1\), and

\(F(x) = 1\), if \(x ≥ 1\). 

\(F(x) =\) The cumulative distribution function and the random variable \(X\) is less than or equal to \(x\). 

\(P (X ≤ x) =\) Probability of \(X\) is less than or equal to \(x\). 

The mean represents the expected average outcome for a distribution, telling you what value to expect over the long run. Variance measures the distribution of uncertainty around the mean, quantifying how likely the actual results are to deviate from the expected average.
 

1. Mean \((\mu)\):
   
   \(E[X] = p\)
    
   • Since the outcome is either 1 (with probability \(p\)) or 0 (with probability\(1-p\)), the weighted average is simply \(1(p) + 0(1-p) = p\)
      
2. Variance \((\sigma^2)\):
   
   \(​​​​​​​Var(X) = p(1 - p)\)
    
   • Variance measures how much the outcomes “spread” from the mean.
      
   • It is calculated as \(E[X^2] - (E[X])^2\)
      
   • Since \(1^2 = 1\) and\( 0^2 = 0\), \(E[X^2]\) is also just \(p\).
      
   • Therefore, \(Var(X) = p - p^2 = p(1-p)\)
      

The Probability Mass Function (PMF) of a Bernoulli distribution specifies the probability that a random variable X takes on a specific value x. Since the Bernoulli distribution only has two possible outcomes (Success or Failure), the PMF is quite simple. It is typically expressed as:

\(P(X=x) =  \begin{cases}  p & \text{if } x = 1 \text{ (Success)} \\ 1-p & \text{if } x = 0 \text{ (Failure)} \\ 0 & \text{otherwise} \end{cases}\)

• p: The probability of success (e.g., 0.5 for a fair coin).
   
• 1-p: The probability of failure (often denoted as q).

The Cumulative Distribution Function (CDF) calculates the probability that a random variable X will take a value less than or equal to a specific number x. For a Bernoulli distribution (where outcomes are strictly 0 or 1), the CDF is a step function that increases in jumps at \(x=0\) and \(x=1\).

\(F(x) = P(X \le x) =  \begin{cases}  0 & \text{if } x < 0 \\ 1 - p & \text{if } 0 \le x < 1 \\ 1 & \text{if } x \ge 1 \end{cases}\)
 

Where:
 

• If \(x < 0\): Since the lowest possible outcome is 0, it is impossible to get a value lower than that, so the cumulative probability is 0.
   
• If \(0 \le x < 1\): At this stage, you have only captured the “Failure” outcome (0), so the running total equals the probability of failure \((1-p)\).
   
• If \(x \ge 1\): By reaching 1, you have captured all possible outcomes (both Failure and Success), so the total probability reaches 1 (100%).

The Bernoulli distribution graphs depict a single binary experiment, with the PMF showing two distinct bars representing the probabilities of failure and success. This is paired with the CDF, a staircase-like step function that depicts the cumulative probability, rising at each outcome until it reaches 100 percent.
 

Probability Mass Function (PMF): This graph shows the probabilities of the two possible outcomes. You will see two distinct bars.
 

• At x=0 (Failure): The height is 1-p (or q).
   
• At x=1 (Success): The height is p.

Cumulative Distribution Function (CDF): This graph shows the running total of probability. It looks like a staircase.
 

• It stays at 0 for x < 0.
   
• It jumps to 1-p at x=0.
   
• It jumps to 1 at x=1 and stays there.
   

A Bernoulli trial is an experiment with only two possible outcomes: success or failure. It is named after the mathematician James Bernoulli.

Common Examples:
 

• Flipping a coin (Heads or Tails).
   
• Checking for rain (Yes or No).
   
• Newborn gender (Boy or Girl).
   

The 3 Essential Rules:
 

1. Two Options Only: The result must be binary (like 0 or 1); there are no third options.
    
2. Independent: The result of one try does not change the next (e.g., a coin has no memory).
    
3. Constant Odds: The probability of success ($p$) stays the exact same for every single trial.

The Bernoulli distribution is the simplest probability model with only two outcomes, success or failure. Mastering it builds the foundation for understanding more complex probability distributions.

• A Bernoulli distribution has only two outcomes, success (1) and failure (0), so understanding this basic idea is essential.
   
• The key parameter p represents the probability of success and determines how likely each outcome is.
   
• Remember that the mean is p and the variance is p(1 – p), as these formulas are commonly used in problems.
   
• It is a special case of the binomial distribution where n = 1, which helps in connecting related concepts.
   
• Practice with real-life examples like coin tosses or pass/fail situations to build a strong intuition.
   
• To reduce anxiety, call it the “One-Shot Experiment” instead of using the fancy math name. It is just one action with one result.
   
• Explain that a Bernoulli trial is analogous to a single Lego brick. If you stack many bricks together, you get a Binomial distribution, but we're only looking at a single brick right now.
   
• Students frequently assume that the odds are 50/50. Show them “lopsided” examples, such as rolling a six on a die (which is difficult to win but has a Yes/No outcome).
   
• Remind them that the die (or coin) has no memory. Even if you get “Heads” five times in a row, the next flip is a new start.
   
• In math, “success” simply means that something happened; it does not imply “good”. Finding a broken toy in a factory line is a “success” if you're counting.

The concept of Bernoulli distribution can be applied to predict the outcome of events. Probability plays a major role even in our day-to-day lives. Here are some examples where it is used.

• In logistics, the Bernoulli distribution helps predict whether an event occurs or not. For example, it can be used when a delivery company or its clients want to know whether their package can arrive on time or not.

• It is used extensively in the field of medicine, especially during the launch of a new vaccine or drug. For instance, it helps check the probability of success of a newly-launched vaccine. 

• The Bernoulli distribution can be applied to predict the weather. For e.g., to check the probability of rain.

• The concept of Bernoulli distribution also plays a crucial role in fields of machine learning and AI.

• In A/B testing, Bernoulli distribution helps determine whether a new feature leads to user success (conversion) or not.