| Topic: |
Science > Physics |
| User: |
"Bert" |
| Date: |
15 May 2004 03:22:07 PM |
| Object: |
Esimation of Experimental Errors in FFT |
Hello,
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
Cheers,
Bert
.
|
|
| User: "Old Man" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
15 May 2004 05:38:39 PM |
|
|
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151222.b19866e@posting.google.com...
Hello,
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
Cheers,
Bert
Error analysis is a *****. It takes an order of magnitude more
effort than that of the main analysis, but the added effort is
nothing other than tedious and mundane. it's nothing more
or less than delta_f = [df / dx ] delta_x. For more than one
independent variable, tale the RMS. fore more than one function,
you may have to invert a matrix. Get a subroutine to do that.
[Old Man]
.
|
|
|
| User: "Bert" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
15 May 2004 09:25:50 PM |
|
|
Hello,
Thanks for the quick reply.
Perhaps I don't understand what you mean. I have a set of
experimental errors associated with a set of data points and then I
transform them. Below you describe error propagation. Are you saying
that 'f' is the spectrum?
Cheers,
Bert
"Old Man" <nomail@nomail.net> wrote in message news:<VamdneXRe_FvBjvdRVn-vw@prairiewave.com>...
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151222.b19866e@posting.google.com...
Hello,
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
Cheers,
Bert
Error analysis is a *****. It takes an order of magnitude more
effort than that of the main analysis, but the added effort is
nothing other than tedious and mundane. it's nothing more
or less than delta_f = [df / dx ] delta_x. For more than one
independent variable, tale the RMS. fore more than one function,
you may have to invert a matrix. Get a subroutine to do that.
[Old Man]
.
|
|
|
| User: "Old Man" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
15 May 2004 11:49:20 PM |
|
|
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151825.28665c5e@posting.google.com...
Hello,
Thanks for the quick reply.
Perhaps I don't understand what you mean. I have a set of
experimental errors associated with a set of data points and then I
transform them. Below you describe error propagation. Are you saying
that 'f' is the spectrum?
Cheers,
Bert
Top posting isn't popular in sci.physics
If you don't have an analytical function to estimate f(x),
then the slope, df / dx, is calculated from straight line
fits to small successive intervals of adjacent data points.
Bert is evidently doing Fourier transforms. So, differentiate
the integral WRT to the limits of integration or WRT the
parameters of the function to be transformed.
For instance:
Fit a series of orthogonal functions to the original data,
g = g(y). The fit parameters, a_0, a_1, a_2, ... have
uncertainties, delta_(a_i) that depend upon your
experimental uncertainties. After integration over y, you
have f = f( x, a1, a2, a3, ...). Now, for each parameter,
a_i, and for various values of x, calculate
delta_(f_i) = [df / da_i] delta_(a_i).
delta_f, which is a function of x, is the RMS for all i.
Error analysis is a tedious *****. [Old Man]
<nomail@nomail.net> wrote in message
news:<VamdneXRe_FvBjvdRVn-vw@prairiewave.com>...
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151222.b19866e@posting.google.com...
Hello,
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
Cheers,
Bert
Error analysis is a *****. It takes an order of magnitude more
effort than that of the main analysis, but the added effort is
nothing other than tedious and mundane. it's nothing more
or less than delta_f = [df / dx ] delta_x. For more than one
independent variable, tale the RMS. fore more than one function,
you may have to invert a matrix. Get a subroutine to do that.
[Old Man]
.
|
|
|
| User: "Franz Heymann" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
16 May 2004 06:03:29 AM |
|
|
"Old Man" <nomail@nomail.net> wrote in message
news:qYqdnYQij4lObzvdRVn-gQ@prairiewave.com...
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151825.28665c5e@posting.google.com...
Hello,
Thanks for the quick reply.
Perhaps I don't understand what you mean. I have a set of
experimental errors associated with a set of data points and then
I
transform them. Below you describe error propagation. Are you
saying
that 'f' is the spectrum?
Cheers,
Bert
Top posting isn't popular in sci.physics
If you don't have an analytical function to estimate f(x),
then the slope, df / dx, is calculated from straight line
fits to small successive intervals of adjacent data points.
Bert is evidently doing Fourier transforms. So, differentiate
the integral WRT to the limits of integration or WRT the
parameters of the function to be transformed.
For instance:
Fit a series of orthogonal functions to the original data,
g = g(y). The fit parameters, a_0, a_1, a_2, ... have
uncertainties, delta_(a_i) that depend upon your
experimental uncertainties. After integration over y, you
have f = f( x, a1, a2, a3, ...). Now, for each parameter,
a_i, and for various values of x, calculate
delta_(f_i) = [df / da_i] delta_(a_i).
delta_f, which is a function of x, is the RMS for all i.
Error analysis is a tedious *****. [Old Man]
In the present case, it would probably be easier to just do, say,
twenty or thirty FFT's of the same data, modified by introducing
random variations in the data, commensurate with the error bars in the
data.
Franz
.
|
|
|
| User: "Old Man" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
16 May 2004 08:31:57 PM |
|
|
"Franz Heymann" <notfranz.heymann@btopenworld.com> wrote in message
news:c87hq0$574$6@sparta.btinternet.com...
"Old Man" <nomail@nomail.net> wrote in message
news:qYqdnYQij4lObzvdRVn-gQ@prairiewave.com...
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151825.28665c5e@posting.google.com...
Hello,
Thanks for the quick reply.
Perhaps I don't understand what you mean. I have a set of
experimental errors associated with a set of data points and then
I
transform them. Below you describe error propagation. Are you
saying
that 'f' is the spectrum?
Cheers,
Bert
Top posting isn't popular in sci.physics
If you don't have an analytical function to estimate f(x),
then the slope, df / dx, is calculated from straight line
fits to small successive intervals of adjacent data points.
Bert is evidently doing Fourier transforms. So, differentiate
the integral WRT to the limits of integration or WRT the
parameters of the function to be transformed.
For instance:
Fit a series of orthogonal functions to the original data,
g = g(y). The fit parameters, a_0, a_1, a_2, ... have
uncertainties, delta_(a_i) that depend upon your
experimental uncertainties. After integration over y, you
have f = f( x, a1, a2, a3, ...). Now, for each parameter,
a_i, and for various values of x, calculate
delta_(f_i) = [df / da_i] delta_(a_i).
delta_f, which is a function of x, is the RMS for all i.
Error analysis is a tedious *****. [Old Man]
In the present case, it would probably be easier to just do, say,
twenty or thirty FFT's of the same data, modified by introducing
random variations in the data, commensurate with the error bars in the
data.
Franz
Good. Straight forward, but still tedious. The pain is inevitable.
It must be a law that, no matter the technique, the work performed
in error analysis is conserved across the board. [Old Man]
.
|
|
|
| User: "Franz Heymann" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
17 May 2004 04:17:33 AM |
|
|
"Old Man" <nomail@nomail.net> wrote in message
news:BMednTr15oWSizXdRVn-ug@prairiewave.com...
"Franz Heymann" <notfranz.heymann@btopenworld.com> wrote in message
news:c87hq0$574$6@sparta.btinternet.com...
"Old Man" <nomail@nomail.net> wrote in message
news:qYqdnYQij4lObzvdRVn-gQ@prairiewave.com...
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151825.28665c5e@posting.google.com...
Hello,
Thanks for the quick reply.
Perhaps I don't understand what you mean. I have a set of
experimental errors associated with a set of data points and
then
I
transform them. Below you describe error propagation. Are
you
saying
that 'f' is the spectrum?
Cheers,
Bert
Top posting isn't popular in sci.physics
If you don't have an analytical function to estimate f(x),
then the slope, df / dx, is calculated from straight line
fits to small successive intervals of adjacent data points.
Bert is evidently doing Fourier transforms. So, differentiate
the integral WRT to the limits of integration or WRT the
parameters of the function to be transformed.
For instance:
Fit a series of orthogonal functions to the original data,
g = g(y). The fit parameters, a_0, a_1, a_2, ... have
uncertainties, delta_(a_i) that depend upon your
experimental uncertainties. After integration over y, you
have f = f( x, a1, a2, a3, ...). Now, for each parameter,
a_i, and for various values of x, calculate
delta_(f_i) = [df / da_i] delta_(a_i).
delta_f, which is a function of x, is the RMS for all i.
Error analysis is a tedious *****. [Old Man]
In the present case, it would probably be easier to just do, say,
twenty or thirty FFT's of the same data, modified by introducing
random variations in the data, commensurate with the error bars in
the
data.
Franz
Good. Straight forward, but still tedious.
Only a minute or so if there is a good FFT routine to hand.
The pain is inevitable.
Are you going soft with advancing age, Old Man?
{:-))
It must be a law that, no matter the technique, the work performed
in error analysis is conserved across the board. [Old Man]
Franz
.
|
|
|
|
|
|
|
|
|
| User: "" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
17 May 2004 10:33:02 AM |
|
|
(Bert) wrote in message news:<367911ba.0405151222.b19866e@posting.google.com>...
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
This is not a simple question.
There are a number of ways to try to approach this, but the way I
would suggest is to take advantage of the fact that computer power
is cheap these days. Even within my suggestion there are several
well-known methods. But they all come under the heading "Monte Carlo
Methods." I find such methods to be a lot more straightforward
than the very difficult analytic methods that exist, easier to
understand, and more flexible.
The basic algorithm for MC methods is this. Start with your measured
data. By rolling random numbers, construct artificial datasets
that exhibit some feature of the error in the measured data. Use
these to infer results for your calculated data.
The following makes the assumption that you know how the V's vary
within the deltaV. That is, you know it's a Gaussian or that it's
equally likely over the range or whatever. You can get some idea
of this by either repeating a single measurement many times, or by
by examining how well your V measurements follow whatever theoretical
curve you expect them to follow.
Here is one method:
Start with your measured dataset and crank that through your FT.
Now, get out your random number generator. Suppose the deltaV is
Gaussian. Build a new dataset by doing the following. For each V
in your dataset, randomly roll up a number with Gaussian distribution,
centred on zero, and with one-sigma equal to deltaV. Add that to
the V. This will build a new dataset, with each value shifted by
a small random amount. Feed this new dataset through your FT. You
get a new result. Each time starting with your measured dataset,
do this a bunch of times. This will give you a distribution of
results from which you can infer statistics.
The results of this process will simulate doing your experiment
many times. You can then get some indication of how your results
would appear if you had done the experiment many times. It will
also allow you to do such things as work out the shape of the
distribution of your answers, whether it is skewed or symetric,
Gaussian or otherwise, etc.
You can also fairly easily include other effects. For example,
your detector might automatically remove any V's over a certain
value. (Or under.) You could make the MC method do the same.
There are lots of other methods of Monte Carlo type. One of my
personal favourites is the "bootstrap" method. You can look that
up in the book _Numerical Recipes in C_.
Socks
.
|
|
|
|
| User: "zigoteau" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
16 May 2004 09:19:43 AM |
|
|
(Bert) wrote in message news:<367911ba.0405151222.b19866e@posting.google.com>...
Hi, Bert,
I don't think that either Old Man or John has answered your question,
so here's my attempt.
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
It's not totally clear what you are doing, but I would imagine that
you are taking a sequential set of measurements of the one signal, and
then doing an DFT. It is probably safe to assume that the error in any
one measurement is independent of the error in every other
measurement, but has exactly the same standard deviation sigma (and
mean of zero).
It is then the case that the error of each value of the DFT is
independent of the error in every other value, and has the same
standard deviation. What that value is depends on exactly which form
of the DFT you are using, but if you are using the simplest form:
FV_k = \Sigma_x=0^N-1[exp(2*pi*i*k*x)*V_x]
then the standard deviation of all your DFT values is sigma*sqrt(N),
where N is the number of measurements.
Does this answer your question? If not, perhaps you could provide
details of exactly what it is you are doing.
Cheers,
Zigoteau.
.
|
|
|
|
| User: "John Smith" |
|
| Title: Re: Esimation of Experimental Errors in FFT |
15 May 2004 11:27:23 PM |
|
|
The DFT or FFT will act as a low pass filter and could filter out some of
your"noise" from your data.
But it depends upon the bandwidth of your "noise"(where it is located
spectrally), the DFT/FFT used, and "spectral content" of your data. (they
are commonly used in digital filtering of data to lower the error)
It may be in your case that the errors are preserved intact as part of
"data",
as the DFT/FFT does not know the difference between error and data, and you
are giving it a combination of both.
"Bert" <dab722@mail.usask.ca> wrote in message
news:367911ba.0405151222.b19866e@posting.google.com...
Hello,
I have a question regarding error estimation in transformed data.
Suppose I have an experimental error associated with a measurements
series, V +/- deltaV. How do those errors transform when the data is
transformed using a DFT or an FFT algorithm for computing the DFT.
Cheers,
Bert
.
|
|
|
|
|
Related Articles |
|
|
