Linearity

Linear equations

In this installment, we will consider the subtle consequences of the assumptions of linearity for a system model intended for Kalman Filtering. You already know what linearity means of course. It is hard to imagine how anything so simple could derail your efforts — until it does.

 

Linear system: an example

Let's start with a simple and familiar system, to show how easy it is to get into trouble. This system consists of a simple "Hooke's Law" spring, which opposes an applied force. As the force increases, the spring length reduces in proportion. So the "input" is considered to be the compression force, and the "output" is the length of the spring.

Table 1 shows value pairs of spring length and compression force, for a typical spring of this sort.

It is not hard to deduce that the characteristic behavior of this system is given by the equation
   y = -0.1 · F + 10.
Clearly the relationship is linear — isn't it?

Strictly speaking, this is an affine relationship, not linear. To be linear in a strict mathematical sense^[1], a relationship f must satisfy two properties.

1. Separability: f(x + y) = f(x) + f(y)
2. Homogeneity: f( a x ) = a f(x)

If we let f represent the spring characteristic that yields the spring lengths given two arbitrary levels of applied force x and y, we can plug in the [force, length] data pairs from the tabulation into the two properties for linearity, and find that the conditions are not satisfied. One of the consequences of the linearity conditions is that a linear function f will produce a zero output given a zero input.

That is somewhat along the lines of common sense: "put nothing in, get nothing out." But on the other hand, it clashes with our ordinary sense that "if the data lie along a straight line, then the system is linear."

The dynamic equations for Kalman filters will have a kind of strict linearity property. A strictly stable linear model, given a zero input for a sufficient time, will eventually stabilize to produce a zero output.

 

Equivalent Strictly Linear System

Suppose that we have a system that is intuitively linear, but not strictly so. It is possible to modify its variables to produce an equivalent system that is linear in the strict mathematical sense — by "canceling the offset." The process is completely reversible, so that any results you obtain using the modified variables can be mapped back to the original variables without losing anything.

In fact, there is more than one way. Here are some approaches for doing this, using the "linear spring" as an example.

1. Instead of modeling the complete position of the spring system, establish an initial equilibrium position, and declare this to be the "zero state" for your model. For the spring example, define a new position variable that is the displacement from the equilibrium position. Similarly, define a new force variable that is the displacement from the equilibrium force value.

   You will find that at the equilibrium position, the new force and position variables are both zero, and it is easy to verify that the strict linearity conditions are satisfied.

2. Use a statistical approach. If you have lots of input and output data, you can easily calculate average values, to obtain a good approximation of statistical mean values.

   $$E\left(v\right)=\underset{N\to \infty }{\lim }\frac{1}{N}\stackrel{i=N}{\sum _{i=1}}{x}_i$$

   Now construct modified data sets that are "shifted" by the corresponding mean values of the input and output data sets.

   Let's verify that the relationship between the adjusted input and output variables is strictly linear. Start with the original affine relationship y = a x + b, and substitute the new adjusted values.

   $$\left[\begin{array}{c}\stackrel{‸}{y}=y-E\left(y\right)E\left(\stackrel{‸}{y}\right)=0\\ \stackrel{‸}{x}=x-E\left(x\right)E\left(\stackrel{‸}{x}\right)=0\end{array}\right]$$ $$\left(\stackrel{‸}{y}+E\left(y\right)\right)=a·\left(\stackrel{‸}{x}+E\left(x\right)\right)+b$$ $$\stackrel{‸}{y}=a·\stackrel{‸}{x}+a·E\left(x\right)+b-E\left(y\right)=a·\stackrel{‸}{x}$$

3. It is possible to construct new variables directly from the known model parameters a and b, as follows, and verify that it satisfies the strict linearity properties.

   $$\left[\begin{array}{c}\stackrel{‸}{y}=y-a·b-b\\ \stackrel{‸}{x}=x-b\end{array}\right]$$ $$\left(\stackrel{‸}{y}+a·b+b\right)=a·\left(\stackrel{‸}{x}+b\right)+b$$ $$\stackrel{‸}{y}=a·\stackrel{‸}{x}$$

 

Is there a point to this?

Unfortunately, yes.

For example, it is typical that when the inputs of a temperature control system are set to zero, the temperature stabilizes at the ambient temperature, not absolute zero. For such systems, you need to define appropriate "shifted" variables that do settle to zero.

We will later see that important special cases of the Least Squared Error (Least Squares) parameter estimation problem produce equations that are strictly linear in the parameters. But it is possible to determine models that are not strictly linear using these methods.

 

---