Conditional dependence

In probability theory, conditional dependence is a relationship between two or more events that are dependent when a third event occurs. It is the opposite of conditional independence. For example, if $A$ and $B$ are two events that individually increase the probability of a third event $C,$ and do not directly affect each other, then initially (when it has not been observed whether or not the event $C$ occurs) $P (A ∣ B) = P (A) and P (B ∣ A) = P (B)$ ( $A and B$ are independent).

But suppose that now $C$ is observed to occur. If event $B$ occurs then the probability of occurrence of the event $A$ will decrease because its positive relation to $C$ is less necessary as an explanation for the occurrence of $C$ (similarly, event $A$ occurring will decrease the probability of occurrence of $B$ ). Hence, now the two events $A$ and $B$ are conditionally negatively dependent on each other because the probability of occurrence of each is negatively dependent on whether the other occurs. We have $P (A ∣ C and B) < P (A ∣ C) .$

Conditional dependence of A and B given C is the logical negation of conditional independence $((A ⊥ ⊥ B) ∣ C)$ . In conditional independence two events (which may be dependent or not) become independent given the occurrence of a third event.

Example

In essence probability is influenced by a person's information about the possible occurrence of an event. For example, let the event $A$ be 'I have a new phone'; event $B$ be 'I have a new watch'; and event $C$ be 'I am happy'; and suppose that having either a new phone or a new watch increases the probability of my being happy. Let us assume that the event $C$ has occurred – meaning 'I am happy'. Now if another person sees my new watch, he/she will reason that my likelihood of being happy was increased by my new watch, so there is less need to attribute my happiness to a new phone.

To make the example more numerically specific, suppose that there are four possible states $Ω = {s_{1}, s_{2}, s_{3}, s_{4}},$ given in the middle four columns of the following table, in which the occurrence of event $A$ is signified by a $1$ in row $A$ and its non-occurrence is signified by a $0,$ and likewise for $B$ and $C .$ That is, $A = {s_{2}, s_{4}}, B = {s_{3}, s_{4}},$ and $C = {s_{2}, s_{3}, s_{4}} .$ The probability of $s_{i}$ is $1 / 4$ for every $i .$

Event	$P (s_{2}) = 1 / 4$	$P (s_{3}) = 1 / 4$	$P (s_{4}) = 1 / 4$	Probability of event
$A$	1	0	1	$\frac{1}{2}$
$B$	0	1	1	$\frac{1}{2}$
$C$	1	1	1	$\frac{3}{4}$

and so

Event	$s_{2}$	$s_{3}$	$s_{4}$	Probability of event
$A \cap B$	0	0	1	$\frac{1}{4}$
$A \cap C$	1	0	1	$\frac{1}{2}$
$B \cap C$	0	1	1	$\frac{1}{2}$
$A \cap B \cap C$	0	0	1	$\frac{1}{4}$

In this example, $C$ occurs if and only if at least one of $A, B$ occurs. Unconditionally (that is, without reference to $C$ ), $A$ and $B$ are independent of each other because $P (A)$ —the sum of the probabilities associated with a $1$ in row $A$ —is $\frac{1}{2},$ while $P (A ∣ B) = P (A and B) / P (B) = \frac{1 / 4}{1 / 2} = \frac{1}{2} = P (A) .$ But conditional on $C$ having occurred (the last three columns in the table), we have $P (A ∣ C) = P (A and C) / P (C) = \frac{1 / 2}{3 / 4} = \frac{2}{3}$ while $P (A ∣ C and B) = P (A and C and B) / P (C and B) = \frac{1 / 4}{1 / 2} = \frac{1}{2} < P (A ∣ C) .$ Since in the presence of $C$ the probability of $A$ is affected by the presence or absence of $B, A$ and $B$ are mutually dependent conditional on $C .$

References

↑ Husmeier, Dirk. "Introduction to Learning Bayesian Networks from Data". In Husmeier, Dirk; Dybowski, Richard; Roberts, Stephen (eds.). Probabilistic Modeling in Bioinformatics and Medical Informatics. Advanced Information and Knowledge Processing. Springer-Verlag. pp. 17–57. doi:10.1007/1-84628-119-9_2. ISBN 1852337788.
↑ Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid" Archived 2013-12-27 at the Wayback Machine
↑ Probabilistic independence on Britannica "Probability->Applications of conditional probability->independence (equation 7) "
↑ Introduction to Artificial Intelligence by Sebastian Thrun and Peter Norvig, 2011 "Unit 3: Explaining Away"^{[permanent dead link]}
↑ Bouckaert, Remco R. (1994). "11. Conditional dependence in probabilistic networks". In Cheeseman, P.; Oldford, R. W. (eds.). Selecting Models from Data, Artificial Intelligence and Statistics IV. Lecture Notes in Statistics. Vol. 89. Springer-Verlag. pp. 101–111, especially 104. ISBN 978-0-387-94281-0.
↑ Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid Archived 2013-12-27 at the Wayback Machine

[1] Husmeier, Dirk. "Introduction to Learning Bayesian Networks from Data". In Husmeier, Dirk; Dybowski, Richard; Roberts, Stephen (eds.). Probabilistic Modeling in Bioinformatics and Medical Informatics. Advanced Information and Knowledge Processing. Springer-Verlag. pp. 17–57. doi:10.1007/1-84628-119-9_2. ISBN 1852337788.

[2] Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid" Archived 2013-12-27 at the Wayback Machine

[3] Probabilistic independence on Britannica "Probability->Applications of conditional probability->independence (equation 7) "

[4] Introduction to Artificial Intelligence by Sebastian Thrun and Peter Norvig, 2011 "Unit 3: Explaining Away"^{[permanent dead link]}

[5] Bouckaert, Remco R. (1994). "11. Conditional dependence in probabilistic networks". In Cheeseman, P.; Oldford, R. W. (eds.). Selecting Models from Data, Artificial Intelligence and Statistics IV. Lecture Notes in Statistics. Vol. 89. Springer-Verlag. pp. 101–111, especially 104. ISBN 978-0-387-94281-0.

[6] Conditional Independence in Statistical theory "Conditional Independence in Statistical Theory", A. P. Dawid Archived 2013-12-27 at the Wayback Machine

Conditional dependence

Example

See also

References