| Topic: |
Science > Physics |
| User: |
"KY" |
| Date: |
28 Oct 2004 12:14:08 AM |
| Object: |
Q: properties of covariance matrices |
Let x and y be n-tuplets of normally distributed random variables, and let
C = cov(x,y') be the nxn covariance matrix of x and y.
Let z be some fixed n-tuplet whose elements are all strictly positive.
1) Are all the elements of the n-vector, Cz, non-negative?
2) Is z'Cz > 0?
Where can I read more about bounds on the elements of covariance matrices?
.
|
|
| User: "Robert Israel" |
|
| Title: Re: Q: properties of covariance matrices |
28 Oct 2004 01:52:33 AM |
|
|
In article <6b5592.0410272114.5012f641@posting.google.com>,
KY <kky2001@columbia.edu> wrote:
Let x and y be n-tuplets of normally distributed random variables, and let
C = cov(x,y') be the nxn covariance matrix of x and y.
I don't think that's what you mean. The covariance matrix of an
n-tuple of random variables X_1, ..., X_n is the n x n matrix
C with entries C_{ij} = Cov(X_i, X_j). For two different
n-tuples, the matrix with entries Cov(X_i, Y_j) is not called
a covariance matrix AFAIK.
A covariance matrix (as I defined it above) is positive semidefinite, and
every positive semidefinite (real) matrix is the covariance matrix of some
set of random variables. With your definition the covariance matrix
would just be any arbitrary n x n real matrix, with no particular
properties.
Let z be some fixed n-tuplet whose elements are all strictly positive.
1) Are all the elements of the n-vector, Cz, non-negative?
No. For example, [use fixed-width font]
[ 2 -1 ] [ 3 ] [ 5 ]
C = [-1 2 ], z = [ 1 ], C z = [ -1 ]
2) Is z'Cz > 0?
It's >= 0 because C is positive semidefinite. If C is positive definite
it would be > 0, but if not there may be z for which Cz = 0, e.g.
[ 1 -1 ] [ 1 ]
C = [-1 1 ], z = [ 1 ]
Where can I read more about bounds on the elements of covariance matrices?
All inequalities on the elements are consequences of the fact that
every principal minor (i.e. the determinant of the matrix formed
by taking some set of rows and the corresponding set of columns)
is nonnegative. Any real symmetric matrix whose principal minors
are all nonnegative is positive semidefinite.
Robert Israel
Department of Mathematics http://www.math.ubc.ca/~israel
University of British Columbia Vancouver, BC, Canada
.
|
|
|
| User: "Ken S. Tucker" |
|
| Title: Re: Q: properties of covariance matrices |
29 Oct 2004 12:03:47 PM |
|
|
(Robert Israel) wrote in message news:<clq4vh$5f7$1@nntp.itservices.ubc.ca>...
In article <6b5592.0410272114.5012f641@posting.google.com>,
KY <kky2001@columbia.edu> wrote:
Let x and y be n-tuplets of normally distributed random variables, and let
C = cov(x,y') be the nxn covariance matrix of x and y.
I don't think that's what you mean. The covariance matrix of an
n-tuple of random variables X_1, ..., X_n is the n x n matrix
C with entries C_{ij} = Cov(X_i, X_j). For two different
n-tuples, the matrix with entries Cov(X_i, Y_j) is not called
a covariance matrix AFAIK.
A covariance matrix (as I defined it above) is positive semidefinite, and
every positive semidefinite (real) matrix is the covariance matrix of some
set of random variables. With your definition the covariance matrix
would just be any arbitrary n x n real matrix, with no particular
properties.
Let z be some fixed n-tuplet whose elements are all strictly positive.
1) Are all the elements of the n-vector, Cz, non-negative?
No. For example, [use fixed-width font]
[ 2 -1 ] [ 3 ] [ 5 ]
C = [-1 2 ], z = [ 1 ], C z = [ -1 ]
2) Is z'Cz > 0?
It's >= 0 because C is positive semidefinite. If C is positive definite
it would be > 0, but if not there may be z for which Cz = 0, e.g.
[ 1 -1 ] [ 1 ]
C = [-1 1 ], z = [ 1 ]
Where can I read more about bounds on the elements of covariance matrices?
All inequalities on the elements are consequences of the fact that
every principal minor (i.e. the determinant of the matrix formed
by taking some set of rows and the corresponding set of columns)
is nonnegative. Any real symmetric matrix whose principal minors
are all nonnegative is positive semidefinite.
Robert Israel
Department of Mathematics http://www.math.ubc.ca/~israel
University of British Columbia Vancouver, BC, Canada
Hi Robert
I'm studying the distinction of covariant and
contravariant tensors from the PoV of physics.
I think mathematicians like a more basic
approach, so to generalize let me introduce
my ignorance this way,
s ds = X dX (covariant)
S dX = X dS (contravariant)
Consider "X" to be length in an orthogonal
geometry where cartesian CS's are valid.
I'll solve these using a constant "k = K"
with the different names to distinguish
context by,
s = X + k (covariant)
S = K*X (contravariant)
k = K
Now let's try to convert these into vectors.
-> -> ->
s = X + k (covariant)
works ("X>" is lazy ascii notation for vector X) when
X>.k> = 0 , (scalar product)
rendering pythagoras (k> being at right angles to X>),
and retains the covariant,
s ds = X dX
when s^2 = X^2 + k^2 is solved....no sweat.
In the "contravariant projection field", "S=K*X"
(it appears to me) the K is a scale factor, and
that field does not admit S>, K> and X> vectors,
simultaneously as the "covariant projection field"
"s=X+k" does.
I find K is a magnitude, (scalar invariant),
however k> is a vector.
Reverting to tensor analysis, we should find
vector components expressed by the covariant,
"A_u" and magnitudes expressed by "A^u".
I'll stop here pending comments/replies.
Regards
Ken S. Tucker
PS: I'm in the Okanagan Valley, my kids
live in Vancouver. My daughter attended
UBC, she's the skinny blue eyed blonde
that smiles all the time, you the one.
kst
.
|
|
|
|
| User: "KY" |
|
| Title: Re: Q: properties of covariance matrices |
28 Oct 2004 11:25:47 AM |
|
|
Let x and y be n-tuplets of normally distributed random variables, and let
C = cov(x,y') be the nxn covariance matrix of x and y.
I don't think that's what you mean. The covariance matrix of an
n-tuple of random variables X_1, ..., X_n is the n x n matrix
C with entries C_{ij} = Cov(X_i, X_j). For two different
n-tuples, the matrix with entries Cov(X_i, Y_j) is not called
a covariance matrix AFAIK.
With your definition the covariance matrix
would just be any arbitrary n x n real matrix, with no particular
properties.
Unforunately, I really did mean cov(x,y'). So cov(x,y') has no
special properties? Too bad.
How about the matrix sum
D = cov(x,x') + cov(x,y')
or
E = D * Inv{cov(x+y,x'+y')]
Is there a special name for the NxN matrix E?
Let z be some fixed n-tuplet whose elements are all strictly positive.
1) Are all the elements of the n-vector Dz non-negative?
2) Are all elements of the n-vector Ez non-negative?
Thanks in advance.
Ricky
.
|
|
|
| User: "zigoteau" |
|
| Title: Re: Q: properties of covariance matrices |
29 Oct 2004 03:34:16 AM |
|
|
(KY) wrote in message news:<6b5592.0410280825.15e163bf@posting.google.com>...
Hi, KY,
Let x and y be n-tuplets of normally distributed random variables, and let
C = cov(x,y') be the nxn covariance matrix of x and y.
I don't think that's what you mean. The covariance matrix of an
n-tuple of random variables X_1, ..., X_n is the n x n matrix
C with entries C_{ij} = Cov(X_i, X_j). For two different
n-tuples, the matrix with entries Cov(X_i, Y_j) is not called
a covariance matrix AFAIK.
With your definition the covariance matrix
would just be any arbitrary n x n real matrix, with no particular
properties.
Unforunately, I really did mean cov(x,y'). So cov(x,y') has no
special properties? Too bad.
It may not be precisely the covariance matrix, but it is the
correlation matrix, and there is a strong connection to the covariance
= autocorrelation matrix. All you have to do is to consider a set of
random variables, say Z, which is the union of X and Y:
Z_i = X_i if 0<i<N
= Y_{i-N} if N<i<2*N
Then your matrix is just an off-diagonal submatrix of the covariance
matrix of Z
How about the matrix sum
D = cov(x,x') + cov(x,y')
or
E = D * Inv{cov(x+y,x'+y')]
Is there a special name for the NxN matrix E?
Let z be some fixed n-tuplet whose elements are all strictly positive.
1) Are all the elements of the n-vector Dz non-negative?
2) Are all elements of the n-vector Ez non-negative?
This is getting a bit complicated. Is it involved in the solution of a
natural problem?
Cheers,
Zigoteau.
.
|
|
|
| User: "KY" |
|
| Title: Re: Q: properties of covariance matrices |
30 Oct 2004 12:12:08 PM |
|
|
It may not be precisely the covariance matrix, but it is the
correlation matrix, and there is a strong connection to the covariance
= autocorrelation matrix. All you have to do is to consider a set of
random variables, say Z, which is the union of X and Y:
Z_i = X_i if 0<i<N
= Y_{i-N} if N<i<2*N
Then your matrix is just an off-diagonal submatrix of the covariance
matrix of Z
This is getting a bit complicated. Is it involved in the solution of a
natural problem?
Cheers,
Zigoteau,
I'm interested in the following measurement problem.
We want to measure
A = B + C, where B and C are normally distributed n-tuplets.
However, we do not observe A, B, or C directly.
Instead, we observe
D = B + F, where F is another normally distributed n-tuplet.
So, I want to construct a definition of how good a "measure"
that D is of A. Obviously, this is related to the covariance
between D and A. Second, I want to get a sense of whether the various
matrix elements of this covariance matrix must be positive or negative.
-- Ricky
.
|
|
|
| User: "zigoteau" |
|
| Title: Re: Q: properties of covariance matrices |
31 Oct 2004 08:13:26 AM |
|
|
(KY) wrote in message news:<6b5592.0410300912.7e730ab1@posting.google.com>...
Hi, Ricky,
This is getting a bit complicated. Is it involved in the solution of a
natural problem?
I'm interested in the following measurement problem.
We want to measure
A = B + C, where B and C are normally distributed n-tuplets.
However, we do not observe A, B, or C directly.
Instead, we observe
D = B + F, where F is another normally distributed n-tuplet.
Why can't you say
A = D + G
where G=C-F. Is there a need to consider F and C separately?
So, I want to construct a definition of how good a "measure"
that D is of A. Obviously, this is related to the covariance
between D and A.
Yes, I would have thought that the correlation matrix of the
measurement error <G^T*G> was what you needed, with the possible
addition of <D^T*D> and <D^T*G> for comparison. If <D^T*G> is nonzero,
then the measurement is biased and you can get a better answer by
combining measurements. Any attempt to produce a single "goodness of
measurement" parameter would eliminate useful distinctions.
Second, I want to get a sense of whether the various
matrix elements of this covariance matrix must be positive or negative.
Well I guess that both <D^T*D> and <G^T*G> must be positive definite,
but apart from that there are no restrictions.
Cheers.
Zigoteau.
.
|
|
|
|
|
|
|
|
|
Related Articles |
|
|
