Science > Physics > Statistical Dependence in PI vs Conditional Probability/Bayesian
| Topic: |
Science > Physics |
| User: |
"OsherD" |
| Date: |
16 Jan 2006 09:08:46 PM |
| Object: |
Statistical Dependence in PI vs Conditional Probability/Bayesian |
From Osher Doctorow
COPYRIGHT NOTICE
Statistical Dependence in PI vs Conditional Probability/Bayesian
Copyright By Owner Osher Doctorow Ph.D.
First Published 2006
If continuous random variables X and Y are statistically independent,
then we have:
1) F(x, y) = FX(x)FY(y)
where F(x, y) = P(X < = x, Y < = y), FX(x) = P(X < = x), etc. These
are also called respectively the joint cumulative distribution
functions (cfds) and (marginal) cdfs for X and for Y.
Let's compare PI P(X-->Y)(x, y) = 1 + F(x, y) - FX(x) with conditional
probability/Bayesian F(Y/X=x) = F(x, y)/FX(x). To do this, let
(P(X-->Y)(x,y)_IND be P(X-->Y)(x,y) when X and Y are statistically
independent, so that:
2) P(X-->Y)(x,y)_IND = 1 + FX(x)FY(y) - FX(x)
Let F(Y/X=x)_IND be F(Y/X=x) when X, Y are statistically independent,
so that:
3) F(Y/X=x)_IND = FX(x)FY(y)/FX(x) = FY(y)
Therefore we have:
4) P(X-->Y)(x, y) - P(X-->Y)(x, y)_IND = F(x, y) - FX(x)FY(y)
5) F(Y/X=x) - F(Y/X=x)_IND = (F/FX) - FY = [(F(x, y) -
FX(x)FY(y)]/FX(x)
These results are remarkably close, but probability-statistics people
know that the expression on the right hand side of (4) is key to
positive versus negative statistical quadrant dependence defined by:
6) X and Y are (statistically) positively quadrant dependent iff F(x,
y) - FX(x)FY(y) > 0
7) X and Y are (statistically) negatively quadrant dependent iff F(x,y)
- FX(x)FY(y) < 0
X and Y are statistically independent iff F(x, y) = FX(x)FY(y). See
E. Lehmann (1967) who developed the concepts of (6) and (7).
Notice that (6) and (7) do not actually evaluate the specific
magnitudes of F(x, y) - FX(x)FY(y) but only their signs. However, from
the definitions of positive and negative dependence and independence,
clearly a larger magnitude of the above difference is further away from
0 (independence) and so indicates intuitively a larger positive
dependence.
So equation (4) measures the positive quadrant dependence of X and Y.
Equation (5), the conditional probability or Bayesian expression,
divides the above result by FX(x) and therefore distorts the result by
FX(x). So the PI expression (4) is better than the conditional
probability or Bayesian expression (5). Furthermore, since x is
"fixed", "given", or "constant" in conditional probability, dividing by
FX(x) in (5) further distorts the results.
Osher Doctorow
.
|
|
| User: "OsherD" |
|
| Title: Re: Statistical Dependence in PI vs Conditional Probability/Bayesian |
16 Jan 2006 10:25:35 PM |
|
|
From Osher Doctorow
So we see that if DEP(x, y) is defined as:
1) DEP(x, y) = statistical quadrant dependence as magnitude of F(x, y)
- FX(x)FY(y) (positive, 0, or negative)
then we can write:
2) P(X-->Y)(x, y) = DEP(x, y) + P(X-->Y)(x, y)_IND
with the first term on the right hand side indicating the dependence
and the secod term on the right hand side indicating the
"non-dependence". Note very carefully that "statistical independence"
and "non-dependence" are not the same thing. I'll try to come back to
this later.
An analogous attempt with F(Y|X=x) fails, because:
3) F(Y|X=x) - F(Y|X=x)_IND = DEP/FX(x)
So the decomposition of F(Y|X=x) fails because of the denominator of
the right hand side, FX(x). That is to say:
4) F(Y|X=x) = DEP/FX(x) + F(Y|X=x)_IND
and the first term on the right hand side is distorted away from DEP by
denominator FX(x).
Osher Doctorow
.
|
|
|
| User: "OsherD" |
|
| Title: Re: Statistical Dependence in PI vs Conditional Probability/Bayesian |
16 Jan 2006 10:54:19 PM |
|
|
From Osher Doctorow
The "Non-Dependence" of X and Y in PI is, from the second posting back:
1) P(X-->Y)(x, y)_IND = 1 + FX(x)FY(y) - FX(x)
This is not the "independence" or "statistical independence" of X and Y
in PI or in anywhere else. "Independence" in probability-statistics is
not measured on a continuous scale but on an "all or nothing" scale, as
some 0 level of dependence (DEP) measured in some continuous way or
simply as the applicability of the equation F(x, y) = FX(x)FY(y), which
applicability is likewise not continuous but "all or none". We have
seen that dependence (DEP) is differently measured in PI and in
conditional/Bayesian probability-statistics.
To recapitulate the situation for F(Y|X=x)_IND, it is:
2) F(Y|X=x)_IND = FY(y)
Whatever it is that F(X-->Y)(x,y)_IND measures, it is at least as large
in magnitude as F(Y|X=x)_IND, because:
3) 1 + FX(x)FY(y) - FX(x) > = FY
since (3) is equivalent to:
4) 1 - FX(x) > = FY(y)(1 - FX(x))
Inequality (4) is true because if FX(x) is not 1, then it just says 1 >
= FY(y) = P(Y < = y) which is true since all probabilities are between
0 and 1. If FX(x) = 1, then (4) just says 0 > = 0.
Intuitively speaking, objects or expressions or processes take on an
_IND subscript when they "split" or "separate as variables"
multiplicatively according to F(x, y) = FX(x)FY(y) or an analogous
equation. This is a dangerous warning concerning solving partial
differential equations (PDEs) by separation of variables, but that
isn't the main intent of this thread or this part of the thread anyway.
The most important point is that "independence" as obeying F(x, y) =
FX(x)FY(y) is one type of all-or-none "thing", while the effect of that
type of thing on continuous measures such as P(X-->Y)(x,y)_IND or
F(Y|X=x)_IND is in fact no longer an all-or-none "thing" but the
particular continuous measure itself.
Osher Doctorow
.
|
|
|
| User: "OsherD" |
|
| Title: Re: Statistical Dependence in PI vs Conditional Probability/Bayesian |
16 Jan 2006 11:30:51 PM |
|
|
From Osher Doctorow
"Independence" in relationship to (probable) causation, whether
"unfrozen" or "frozen", does not mean that "there is no (probable)
causation". Let me explain this with a toss of two "independent"
coins. Throw two coins, both not loaded, in an "identical" or "almost
identical" (except possibly for location, for example) manner. Roughly
(very roughly!) speaking, one coin doesn't influence the other, and a
throw of one doesn't influence a throw of the other. But the
probability that the second coin lands "heads (H)" given (freezing,
holding constant) that the first coin lands "tails" isn't 0. That
probability, analogous to the continuous F(Y|X=x)_IND = FY(y), is just
the probability of the second coin landing heads, namely 1/2.
Likewise, we wouldn't expect that the discrete version of
P(X-->Y)(x,y)_IND = 1 + FX(x)FY(y) - FX(x) is 0, and in fact it is 1 +
(1/2)(1/2) - (1/2) = 3/4 = 0.75.
How is this possible? If the first coin and/or its toss doesn't
influence the second coin and/or its toss, how could the probability of
the first coin (or toss) influencing the second coin (or toss) be 0.75
(or 3/4)? And how could the probability of the second coin (or toss)
"given" the first coin (or toss) be 1/2? The simplest answer is that
"doesn't influence" in the above is an incorrect description. The
"real" "doesn't influence" should be calculated by:
1) P(X-->Y)(x, y) = 1 + F(x, y) - FX(x) = 0
or, using conditional/Bayesian probability-statistics, by:
2) F(Y|X = x) = F(x, y)/FX(x) = 0
The only solution to (1) is FX(x) = 1 and F(x, y) = 0 because F(x, y) <
= FX(x). So FX(x) = P(X < = x) = 1. For a continuous random variable
X to have P(X < = x) = 1, the real number x must be above the "support"
of X, that is to say a value of X such that all the non-zero
probabilities or densities or cumulative distribution functions are
below and/or at that value. For F(x, y) = 0, for example for the
"independent" cases we would get FY(y) = 0 so that P(Y < = y) = 0,
which means that y is below the "support of Y.
For conditional/Bayesian probability-statistics, for the "independent"
case we get FY(y) = 0 just as in the last paragraph but with no
condition on X other than FX(x) is not 0.
Osher Doctorow
.
|
|
|
| User: "OsherD" |
|
| Title: Re: Statistical Dependence in PI vs Conditional Probability/Bayesian |
17 Jan 2006 12:03:10 AM |
|
|
From Osher Doctorow
So what does it mean to say that the only "non-influencing" scenarios
for two fair coins tossed similarly is when the second coin has values
"below its range of values"? It is roughly like saying that "only
impossibilities have no influence." This is not always the case for
other variables. For example, the probability of a point in 3
dimensions is 0 if a continuous random variable describes a volume of
space containing the point. The probability of an edge or a planar
object or a two-dimensional surface is likewise 0 in 3 dimensions under
the above conditions. So 0 probability events/processes/things don't
necessarily involve impossibilities.
The best way to look at it, I think, is this. "Statistically
independent" coins, dice, etc., have little influence on each other.
But it's best to avoid "no influence" descriptions. Even the Null Set
in the sense of "set with no elements" has influence. In fact, the
Null Set (N) has probable influence on every event B because:
1) P(N-->B) = P(N' U B) = P(Universe U B) = 1
Let's calculate this for conditional probabiity:
2) P(B|N) = P(BN)/P(N)
Oops! P(N) = 0, so conditional probability isn't defined for the Null
Set. And that's the only place where PI (Probable Influence) is
defined but conditional/Bayesian probability-statistics isn't defined.
And that's why it hasn't been discovered till recently.
Let's look at the general case in PI:
3) P(A-->B) = 1 + P(AB) - P(A) = 0 iff P(A) = 1, P(AB) = 0
Here A and B don't intersect (P(AB) = 0) or else P(B) = 0 (hence P(AB)
= 0). If A is the first coin toss (outcome) and B is the second coin
toss (outcome), then if they both occur they are ordinarily described
as intersecting not in the sense of influencing each other but in the
sense of both occurring "simultaneously", that is to say in the same
experiment or real scenario. So we aren't dealing with P(AB) = 0.
Likewise, with P(B|A), we get in general:
4) P(B|A) = P(AB)/P(A) = 0 iff P(AB) = 0, P(A) not 0
so we're not dealing with the P(AB) = 0 case.
So although we're using continuous random variables mostly, the general
idea for all random variables is correct, namely Nature builds
"influence" into almost everything except (a) alternatives that don't
occur in the same experiment and (b) (probable) causes which are
"certain" to occur provided that the question is asked whether they
influence non-simulaneously occurring events. For everything else,
there's a "minimum non-zero influence," somewhat like a "minimum
non-zero energy", and "independence" is something like that minimum
more or less (and sometimes less).
Osher Doctorow
.
|
|
|
|
|
|
|
|
Related Articles |
|
|
