1

I'm computing the mean of values obtained from a*cos(ωt)+μ but it doesn't converge as I would expect. There is a residual error in the order of 10^-4 which I find quite big for 64 bit precision computation.

I would expect that the computed mean would converge to μ.

I tried different processing duration and ω values without succeeding in increasing the precision.

I also tried different ways to compute the average. One method is to simply accumulate the values and divide the result. Another method is to compute the mean incrementally with m += (x-m)/n and increasing n.

What worries me is that I also compute the variance with this mean and it also affect it's convergence.

Is there a way to increase the precision of the mean computation ? In the end I will have to sum thousands of cosine and sine values of different phase and same frequency. The result is still a cosine. Thus getting the mean computation right and as precise as possible is important.

Edit: here is the Go code I use for my testings.

You can run and test various parameters directly in the go playground at this location: https://go.dev/play/p/hwbYf3Y5tPO

package main

import (
    "fmt"
    "math"
)

const (
    lambda      = 3005
    amplitude   = 2.
    mean        = 5.
    nIterations = 1000_000
)

func cosFunc(lambda, amplitude, mean float64) func() float64 {
    var t float64
    pulsation := 2 * math.Pi / lambda
    return func() float64 {
        x := amplitude*math.Cos(pulsation*t) + mean
        t++
        return x
    }
}

func meanFunc() func(x float64) float64 {
    var n, m float64
    return func(x float64) float64 {
        n++
        m += (x - m) / n
        return m
    }
}

func main() {
    gen := cosFunc(lambda, amplitude, mean)
    mean := meanFunc()
    for i := 0; i < nIterations; i++ {
        x := gen()
        m := mean(x)
        fmt.Printf("x: %f mean: %g\n", x, m)
    }
}

Increasing the number of iterations or lambda doesn't increase much the precision. I suspect it is because x is a cosine function.

chmike
  • 20,922
  • 21
  • 83
  • 106
  • 2
    The mean is µ. Why do you need to calculate samples to compute that? What are the values of *a*, ω, and *t*? In particular, how big does *t* get and how many values of it are in the sequence? Are they regularly spaced? Is ω*t* a rational multiple of π? Can you show a [mre]? Would it suffice to compute the mean from *a*, ω, and the sequence of values of *t* algebraically instead of numerically? – Eric Postpischil Jul 08 '23 at 12:47
  • 1
    @chmike, "Is there a way to increase the precision of the mean computation " --> yes. Post the code used, not just the algorithm. – chux - Reinstate Monica Jul 08 '23 at 14:46
  • @EricPostpischil I don't have direct access to a, ω and µ. I only have access to x varying over time t where x is a.cos(ωt)+µ. I know that the variance must be (a^2)/2 and the mean must be µ. I have to deduce these values from the sequence of x varying over time t. I don't even know ω. All I know is that it is a constant. The values a and µ may vary over time and I'm looking for a method to find them so that I can compensate their change by normalizing x. I use the algorithm described [here](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm) – chmike Jul 09 '23 at 10:24
  • @chux-ReinstateMonica please see my other comment – chmike Jul 09 '23 at 10:29
  • @chmike Posting code, a [mcve], is more valuable than more comments to truly get at the issues. Your call. – chux - Reinstate Monica Jul 09 '23 at 10:45
  • Your question mentions error on the order of 10^−4 but does not say anything about the magnitudes of *a*, ω, *t*, or μ. Numerical analysis of the calculations is not going to go anywhere without numbers to analyze. Without a [mre] and some information about the numbers involved, we cannot tell to what extent errors are caused by the limitations of the binary64 format, by inadequacies in the algorithm used, or other potential problems. – Eric Postpischil Jul 09 '23 at 10:45
  • @chux-ReinstateMonica I updated my question with the code. It can help to reproduce the problem and test the influence of parameters, but the problem is not in the code. – chmike Jul 09 '23 at 12:01
  • @EricPostpischil I added the code to my question so that the problem can be reproduced. – chmike Jul 09 '23 at 12:03

2 Answers2

3

The calculated result is mathematically correct.

The code does not sample whole circles, and the deviation from mean is almost entirely due to the final fractional circle. The deviation is a mathematical property of the sequence, not an artifact of floating-point arithmetic.

With lambda = 3,005, the summation samples 3,005 points distributed around the circle. These points are repeated as the iterations go on. With nIterations = 1,000,000, the first 2,340 points are sampled ⌈1,000,000 / 3,005⌉ = 333 times, and the remaining 665 points are sampled 332 times. Given this, it is unsurprising the mean of the samples differs from the mean of complete circles. The magnitude of the differences, about 10−3, is proportional for the number of whole circles completed.

We can verify the errors of floating-point arithmetic are slight by setting nIterations to an integer multiple of lambda. Mathematically, this would produce a sequence with mean 5 (from the mean parameter), so any deviation from 5 is due to imperfect arithmetic. With nIterations set to lambda (3,005), I get a final mean of 5.00000000000000710542735760100185871124267578125 (using code from the question reimplemented in C). With nIterations set to 1000•lambda (3,005,000), I get a final mean of 5.0000000000002788880237858393229544162750244140625.

In the latter cases of nIterations, we see tiny differences that are due to errors in floating-point arithmetic. With the original nIterations, the differences is almost entirely due to the incomplete final circle. The difference is mathematically real, not an artifact of floating-point arithmetic.

Note that the code at the Go playground link differs from the code in the question. It prints the final result using an extra mean(gen()) evaluation, so it effectively performs nIterations+1 iterations. To reproduce the whole-circle results, set nIterations to one less than the desired multiple, such as 3,004 or 3,004,999 instead of 3,005 or 3,005,000.

(Bonus note: the 3,005-iteration result is eight ULP from 5, and the 3,005,000-iteration result is 314 ULP from five.)

Finding the desired mean.

Per additional request from OP, here is code (in C) that finds the mean.

#include <math.h>


static double f(void)
{
    static const double π = 0x3.243f6a8885a308d313198a2e03707344ap0;

    static const double
        λ = 3005,
        a = 2,
        µ = 5,
        ω = 2 * π / λ;

    static double t = 1234.5678;

    return a*cos(ω * t++) + µ;
}


#include <stdio.h>


/*  Given a function f that returns a*cos(ω*t)+µ for unknown constants a, ω,
    and µ and a t that starts with an unknown value and increments by one in
    each call, calculate µ.

    We do this by finding extrema in the function to estimate its period then
    sampling points equally spaced around the cycle and applying an algebraic
    solution from Maple.  (Judging from the simplicity of the solution, I
    expect it comes from simple trigonometric properties, but I do not know its
    derivation.)  Maple was asked to solve:

        a*cos(w*(t+0))+u = t1,
        a*cos(w*(t+1))+u = t2,
        a*cos(w*(t+2))+u = t3, and
        a*cos(w*(t+3))+u = t4,
    
    for u, a, w, and t.  Since w is an unknown constant, the scale of 0, 1, 2,
    and 3 are irrelevant to the solution for u, so it will work for any equally
    spaced points in the series produced by the function (barring issues such
    as all the points having the same phase so they represent only one point in
    the cycle instead of four).  Since t is unknown, it will work for any
    starting point.

    Note that if the wavelength, λ (see sample f above), is an even integer, we
    can simply average the two extremes to find the mean.  However, λ is
    unknown, and the solution below is general.
    
    Mathematically, adjacent samples would suffice.  However, testing showed
    artifacts due to floating-point precision.  So spacing that is a
    substantial portion of the period is used.  I have not analyzed the
    numerical behavior in this situation.
*/
int main(void)
{
    //  Look for some change in the function.
    double t0 = f(), t1;
    for (int i = 0; i < 100; ++i)
    {
        t1 = f();
        if (t0 != t1)
            break;
    }

    if (t0 == t1)
        printf("Function appears to be constant, mean is %.99g\n", t0);

    else
    {
        //  Find first change of direction and then distance to second change.
        int count = 0;
        if (t0 < t1)
        {
            do {          t0 = t1; t1 = f(); } while (t0 <= t1);
            do { ++count; t0 = t1; t1 = f(); } while (t0 >= t1);
        }
        else
        {
            do {          t0 = t1; t1 = f(); } while (t0 >= t1);
            do { ++count; t0 = t1; t1 = f(); } while (t0 <= t1);
        }

        //  Sample function at roughly quarter-circle spacings.
        count /= 2;
        if (count < 1) count = 1;

        for (int i = 1; i < count; ++i) f();
        double t2 = f();
        for (int i = 1; i < count; ++i) f();
        double t3 = f();
        for (int i = 1; i < count; ++i) f();
        double t4 = f();

        //  Apply solution from Maple.
        printf("u = %.99g.\n",
            (t3*t3 + t3*t1 - t2*t2 - t2*t4) / (t1 + 3*t3 - 3*t2 - t4));
    }
}

Sample output:

u = 4.99999999999999733546474089962430298328399658203125.
Eric Postpischil
  • 195,579
  • 13
  • 168
  • 312
  • I had the same conclusion. My question still holds. Is there a way I could enhance the precision of my current mean computation ? It seam that I should fit the cosine to determine all parameters. How could I do that ? – chmike Jul 09 '23 at 20:17
  • @chmike: From what information do you want to calculate the mean? If you know μ, a.k.a. `mean`, it is the mean. If you know ω or equivalently `lambda`, you can calculate the mean over `lambda` (2π/ω) samples. If you do not know *a*, ω, or μ but can call the function that generates values, then you can call the function to get a few consecutive values and solve for μ. – Eric Postpischil Jul 09 '23 at 20:23
  • [Wolfram Alpha](https://www.wolframalpha.com/input?i=solve+a+cos%28w*0%29%2Bu%3Dx%2C+a+cos%28w*1%29%2Bu+%3D+y%2C+a+cos%28w*2%29%2Bu%3Dz+for+u%2C+a%2C+and+w) shows µ = (x^2+xz-2y^2)/(3x−4y+z) where x, y, and z are the values produced at t=0, t=1, and t=2. However, this seems to suffer from the closeness of the cosine to 1 at these points, yielding µ=5.00046. You can address this by using the same formula with the values produced at t=0, t=10, and t=20 (which gives 4.9999999907056231762680909014306962490081787109375), or another other regular spacing from t=0. – Eric Postpischil Jul 09 '23 at 20:38
  • Thank you for taking the time to try help me solving the problem. All I have are the current and past values of x[t]. Note that the equation to solve is a*cos(ωt+ϕ)+μ because we don't know where is the t0. – chmike Jul 10 '23 at 07:33
  • @chmike, why don't you just create a simple fit? – kvantour Jul 10 '23 at 09:20
  • @EricPostpischil if you want, you could use a form of Kahan summation to decrease the error. – kvantour Jul 10 '23 at 09:26
  • @kvantour: Numerical precision is not the primary issue here. – Eric Postpischil Jul 10 '23 at 13:46
  • @chmike: Maple says µ = (x2•x2 + x2•x0 - x1•x1- x1•x3) / (x0+3•x2-3•x1-x3) for any equally spaced samples x0, x1, x2, and x3. Testing with samples 100 steps apart (for lambda = 3005) produces 5.00000000000089617202547742635942995548248291015625. – Eric Postpischil Jul 10 '23 at 13:51
  • Thank you very much. This is impressive. I have also posted an answer with another method. I checked its validity with Excel but don't have evaluated the precision yet as you did. I award you the answer for the effort you have put into it. This work was very instructive. – chmike Jul 11 '23 at 11:42
1

Eric Postpischil provides one method. I found another method.

For the equation Acos(wt+p)+m = x[t] with t varying in time, we want to find the parameters A, w, p and m.

The above expression can be rewritten as

Acos(wt+p)+m = a*cos(wt)+b*sin(wt)+m = x[t]

Once w is determined, we have a system of linear equations with 3 unknowns a, b and m that we can solve trivially as we can compute cos(wt) and sin(wt) for some picked t0 value.

The trickiest task is thus to find w, the pulsation.

To determine w we need 4 values with a precise relative distance. We need x[-2k], x[-k], x[k] and x[2k] where k can be any value different from 0. We then have the equivalence

 (x[-2k]-x[2k])/(x[-k]-x[k]) = 2cos(kw)           (1) 

We can pick k = 1, but successive values might be very close. There is a big loss of precision when computing the difference of two close numbers. Ideally, k should be picked to maximize the magnitude of both differences to minimize the loss of precision.

Once we have w, we solve the system of linear equations to find a, b and m.

The parameters A and p are then obtained with the relations:

A = sqrt(a*a+b*b) 

p = -atan(b/a). 

σ = A/sqrt(2)

This works fine when there are no errors (e.g. noise) in x[t].

I still have to find out how to deal with noise in the x values.

Demonstration of (1)

x[-k]-x[k] = Acos(wt-kw)+m - (Acos(wt+kw)+m) 
x[-k]-x[k] = A(cos(wt-kw) - cos(wt+kw)) 

cos(wt-kw) = cos(wt)cos(kw)+sin(wt)sin(kw)
cos(wt+kw) = cos(wt)cos(kw)-sin(wt)sin(kw)
cos(wt-kw)-cos(wt+kw) = 2sin(wt)sin(kw)

cos(wt-2kw)-cos(wt+2kw) = 2sin(wt)sin(2kw)

(x[-2k]-x[2k])/(x[-k]-x[k]) = 2Asin(wt)sin(2kw)/2Asin(wt)sin(kw)
(x[-2k]-x[2k])/(x[-k]-x[k]) = sin(2kw)/sin(kw)

sin(2kw) = 2sin(kw)cos(kw)

(x[-2k]-x[2k])/(x[-k]-x[k]) = 2sin(kw)cos(kw)/sin(kw)

(x[-2k]-x[2k])/(x[-k]-x[k]) = 2cos(kw)

Precision evaluation of w

Running the algorithm with lambda=100 and k=1 lambda times, I get a stdDev of w values around 2e-13.

Computing the mean of the 100 w values gives a residual error around 3e-15. The residual error is the difference between computed mean w value and the w value used to generate the x values with the cosine.

As expected the precision varies with the location we are in the cosine. The difference between two consecutive values is the biggest when x crosses the mean value. The ulp varies around 100.

As expected, when computing with k=2, the residual error drops around 1e-16. The ulp computed on the average varies from 3 to 38.

By increasing the value k to 10, the biggest ulp value is around 5.

With a value k set to 20, the biggest ulp is 1 and mostly 0.

This validates the anticipated degrading effect using small k values. K must remain small compared to lambda which is 101.5 in my tests.

The Go program to play around with it is here https://go.dev/play/p/jHg4AHjDpwI. The code is not intended to be optimized. It's just for testing and measuring.

chmike
  • 20,922
  • 21
  • 83
  • 106