For understanding Bayes’ theorem better one should always know about conditional probability. Conditional probability is the probability of occurrence of an event with condition that some other event has already occurred.

It is denoted by $P (B|A)$ and is read as probability of B given that A has occurred. It may also be written as $P_B (A)$.

For example consider event X of taking a spade card out of a complete deck of cards. Next consider event Y of drawing a black color card from the deck. Then the probability of P (X|Y) will be the probability that a black color card is drawn and it is a spade.

Now when we talk about $P (Y|X)$, which is the probability that a spade is drawn and it is black, will always be 1 as spade is always black.
Bayes’ theorem is a relation that that is relating the current belief with original or prior belief. It can also be relating evidence just as belief. It is basically making it understand the way the probability of occurrence of an event gets affected by some new evidence discovered.

Conditional Probability

Let us assume that $X$ and $Y_i$ are sets.
Then according to conditional probability we have,

$P (X \cap  Y_i)$ = $P (X) P (Y_i|X)$                    (i)

[‘$\cap$’ implies intersection]

Also we will have,

$P (X \cap  Y_i)$ = $P (Y_i \cap  X)$ = $P (Y_i) P(X|Y_i)$            (ii)

From (i) and (ii) we get,

 $P (X) P (Y_i|X)$ = $P (Y_i) P(X|Y_i)$

$P (Y_i|X)$ = $\frac{[P (Y_i) P (X|Y_i)]}{P (X)}$
In general the Bayes’s theorem formula states that:

$P (X|Y)$ = $\frac{[P (X) P (Y|X)]}{P (Y)}$

Where: $X$ and $Y$ are two sets
$P (X)$ is the prior probability

$P (X|Y)$ is the conditional probability of $X$ given that $Y$ has occurred.

We can write $Y$ = $X \cap Y$ and $\overline{X} \cap Y$, so we get,

$P (Y)$ = $P (X) P (X|Y) + P (\overline{X}) P (\overline{X}|Y)$
Hence another version of Bayes’ formula can be:

$P (X|Y)$ = $\frac{[P (X) P (Y|X)]}{[P (X) P (X|Y) + P (\overline{X}) P (\overline{X}|Y)]}$


Below are some examples on topic Bay's formula:

Example 1:   A bag contains 30% brown, 20% red, 20% yellow, 10% orange, 10% green, 10% tan candies. Another similar kind of new bag including blue candies with 20% green, 24% blue, 14% yellow, 16% orange, 13% brown, 13% red is also available. If we take out two candies one from each bag, one green and one yellow, find the probability that the yellow one came from the old bag.

Solution: The first scenario is which bag is selected first. Let $X$: bag 1 is old and bag 2 is new and $Y$: bag 1 is new and bag 2 is old.

So we get, $P (X) = P (Y)$ = $\frac{1}{2}$.

According to question, the belief is, B: yellow candy from old bag and green candy from new bag.

$P (B|X)$ = (0.2) (0.2) = 0.04

$P (B|Y)$ = (0.1) (0.14) = 0.14

We need to find the probability that the green one came from the new bag that is,

 $P (X|B)$ = $\frac{P (X) P (B|X)}{ P (B)}$ = 0.74 (appox.)

Example 2: A bucket, A, contains 5 red marbles and 4 blue marbles and another bucket B contains 7 red marbles and 5 blue marbles. One bucket is selected at random and a marble is drawn from it. If the marble drawn is found red, find the probability that the bucket chosen was A.

Let probability of selecting bucket A and bucket B is P(C) and P(D)

Then P(C) = P(D) = $\frac{1}{2}$

 Let R be the event that the marble chosen from the selected bucket is red.

Then P(R|C) = $\frac{5}{9}$ and P(R|D) = $\frac{7}{12}$

By Baye's theorem P(C|R) = $\frac{P(C)P(R|C)}{P(C)P(R|C)+P(D)P(R|D)}$

= $\frac{\frac{1}{2} \times \frac{5}{9}}{\frac{1}{2} \times \frac{5}{9}+\frac{1}{2} \times \frac{7}{12}}$

= $\frac{\frac{5}{18}}{ \frac{5}{18} + \frac{7}{24}}$

= $\frac{\frac{5}{18}}{\frac{41}{72}}$

= $\frac{20}{41}$