Linearity

Linear equations

In this installment, we will consider the subtle consequences of the assumptions of linearity for a system model intended for Kalman Filtering. You already know what linearity means of course. It is hard to imagine how anything so simple could derail your efforts — until it does.

 

Linear system: an example

Let's start with a simple and familiar system, to show how easy it is to get into trouble. This system consists of a simple "Hooke's Law" spring, which opposes an applied force. As the force increases, the spring length reduces in proportion. So the "input" is considered to be the compression force, and the "output" is the length of the spring.

Table 1 shows value pairs of spring length and compression force, for a typical spring of this sort.

It is not hard to deduce that the characteristic behavior of this system is given by the equation
   y = -0.1 · F + 10.
If we plot the force-compression variable pairs that satisfy this equation, it will be clear that these point all lie along the same line. The relationship is obviously linear — isn't it?

Strictly speaking, this is an affine relationship, not linear. To be linear in a strict mathematical sense^[1], a relationship f must satisfy two properties.

1. Separability: f(x + y) = f(x) + f(y)
2. Homogeneity: f( a x ) = a f(x)

If we let f represent the spring characteristic that yields the spring lengths given two arbitrary levels of applied force x and y, we can plug in the [force, length] data pairs from the tabulation into the two properties for linearity, and find that the conditions are not satisfied. One of the consequences of the linearity conditions is that a linear function f will produce a zero output given a zero input.

That is somewhat along the lines of common sense: "put nothing in, get nothing out." Yet, it clashes with our ordinary sense that "if the data lie along a straight line, then the system is linear."

The dynamic equations for Kalman filters will have a kind of strict linearity property. A strictly stable linear model, given a zero input for a sufficient time, will eventually stabilize to produce a zero output.

 

Equivalent Strictly Linear System

Suppose that we have a system that is intuitively linear, but not strictly so. It is possible to modify its variables to produce an equivalent system that is linear in the strict mathematical sense — by "canceling the offset." The process is completely reversible, so that any results you obtain using the modified variables can be mapped back to the original variables without losing anything.

In fact, there is more than one way. Here are some approaches for doing this, using the "linear spring" as an example.

1. Instead of modeling the complete position of the spring system, establish an initial equilibrium position, and declare this to be the "zero state" for your model. For the spring example, define a new position variable that is the displacement from the equilibrium position. Similarly, define a new force variable that is the displacement from the equilibrium force value.

   You will find that at the equilibrium position, the new force and position variables are both zero, and it is easy to verify that the strict linearity conditions are satisfied.

2. Use a statistical approach. If you have lots of input and output data, you can easily calculate averages, to obtain a good approximation of statistical mean values.

   $$E\left(x\right)=\underset{N\to \infty }{\lim }\frac{1}{N}\stackrel{i=N}{\sum _{i=1}}{x}_i$$

   Now construct modified data sets that are "shifted" by the corresponding mean values of the input and output data sets.

   Let's verify that the relationship between the adjusted input and output variables is strictly linear. Start with the original affine relationship y = a x + b, and substitute the new adjusted values.

   $$y=a·x+b$$

    

   $$Ey=a·Ex+b$$

   Define:

   $$\stackrel{‸}{y}=y-Ey$$ $$\stackrel{‸}{x}=x-Ex$$

   Then:

   $$\left(\left((,\stackrel{‸}{y},+,E,y,)\right)\right)=a·\left(\left((,\stackrel{‸}{x},+,E,x,)\right)\right)+b$$ $$\stackrel{‸}{y}=a·\stackrel{‸}{x}$$

   In these modified variables, the relationship is strictly linear.

3. It is possible to construct new variables directly from the known model parameters a and b, as follows, and verify that it satisfies the strict linearity properties.

   $$y=a·x+b$$

   Define:

   $$\stackrel{‸}{y}=y-a·b-b$$ $$\stackrel{‸}{x}=x-b$$

   Then:

   $$\left(\stackrel{‸}{y}+a·b+b\right)=a·\left(\left((,\stackrel{‸}{x},+,b,)\right)\right)+b$$ $$\stackrel{‸}{y}=a·\stackrel{‸}{x}$$

 

Is there a point to this?

Unfortunately, yes.

We will later see that important special cases of the Least Squared Error (Least Squares) parameter estimation problem can produce results that are strictly linear in the parameters or not, depending on how you define the problem.

 

---