# Why Model?

Developing Models for Kalman Filters

## Models and Embedded Systems

You have a deterministic processor architecture, predictable task scheduling, a watchdog timer with robust error recovery capabilities, strongly typed variables with carefully-scoped visibility, MISRA-conforming code that compiles clean. Coding passed a detailed review. You have now have an embedded system, right? Or am I missing something important? Like, maybe, the system should attempt to do something useful?

Sometimes "do something" is rather trivial — toggle a bit to actuate a relay, or collect a value by reading data from a hardware register. Other times, things are not so simple.

Perhaps this was why in past years I eagerly awaited each
installment of Jack Crenshaw's "Embedded Toolkit,"
^{[1]} exploring
the art of getting things done when the means are not so obvious.
The kinds of tools that Jack explored include: the "Rosetta Stone"
equation that relates real-world continuous signals to processor-world
discrete sequences; "minimizers" to seek a performance optimum; and
foundational mathematical tools such as matrices and vectors for
representing composite data and operations on them.

## From Models to Kalman Filters

Jack was starting explorations of "least squares" methods for generating "formulas" that compactly represent information from complex data sets, with an eventual goal to cover the classic Kalman Filter. I hope to pick up the story line, though I can't guarantee that the wanderings will take the same track.

Kalman filtering is not as mysterious and complicated as it might first seem. It is actually a combination of several ideas, each of which is relatively simple when taken in isolation. Sometimes all you really need is just a few of these pieces — and you are better off without the rest.

Kalman filtering is the ultimate expression of the idea "don't
control the world, control a *model* of the world." If you can
model the world perfectly, and devise a strategy that produces optimum
performance from the model, that same strategy will work to equal
perfection in the real world. Unfortunately, this is an elaborate
variation on wishful thinking. The world is too complicated and
interconnected to represent perfectly. Still, the more you know about
"the world" of your embedded system, as demonstrated by the success of
your model, the better your chances of obtaining results that are
* good enough* in the real world.

## What is a model?

A model is a kind of mathematical statement about what you know — or think you know — about your system. It distinguishes features deemed important, and disregards aspects that are deemed unimportant or not of immediate direct concern. The better the model behaviors correspond to real-world behaviors, the more confidence you can have that your basic understanding is right.

There are many sorts of things that models can represent. Kalman filters are intended to model the sorts of processes that are continuous in character, though typically computed using computer methods at discrete instants equally distributed through time.

Sometimes the goal is to estimate effects that are extraneous but consistent. For example, one of the most powerful ways to improve sensor readings is to take lots of reading and average them — the "Central Limit Theorem." This effectively removes much of the "pure randomness." It is ineffective, however, at removing systematic errors. If you can use a Kalman Filter to model systematic but extraneous effects accurately, these too can be cancelled out of measurement data.

A useful model will represent critical behaviors *very well*,
represent important behaviors *sufficiently well*, and as for the
other things — forget about them. * The model is not the
system. *

A judgment call determines *importance*, and decides whether the
correspondence to the real world is * close enough.* What should
you expect about the ultimate accuracy of an optimal solution based
on judgment calls? (Models that produce precise results from crude
or even speculative approximations were studied in the late 19th
century.^{[2]}) This topic will arise
again when considering what "optimality" of Kalman Filters means in
practice.

## What data does the model use?

The model will be driven by sequences of *input variables*,
the *independent* values — for example, * advancing time*,
or some *control variables* that you can adjust. The *output variables*
— or at least *measurements* of them — provide
information about what the system does in response to the input variables.
In addition to the input and output variables, there can be internal
variables maintained within the model; these are called *state variables*.

Given values of the inputs as applied to the actual system, the mathematical model should be able to predict what the corresponding output variables should be; and these results can be validated against the observed actual system output values.

## Discrepancies between model and system

Models can show *disturbed results* that do not exactly match
the behavior of the actual system. There are two complementary but opposed
ideas about how this happens.

Assume that the deficiencies arise because the model is imperfect. This is the

*adaptive systems*approach, consistent with the general philosophy of "recursive least squares". The way to correct the deficiencies in the results is to make the model better. Each new measurement provides more information about how the system works, and this new information can be incrementally absorbed to improve the model.Assume that the model is accurate, but deficiencies arise because of disturbances in the values of variables used within the model. Inconsistencies between actual outputs and the outputs predicted by the model arise randomly. This is the

*Kalman Filter*approach. It is presumed that, with perfect knowledge of all the variable values, the model will produce perfect results. Adjustments can be made to values of the model's internal variables in an effort to counteract cumulative random effects.

## Linear models

Again anticipating the Kalman Filter, attention is restricted
(at least initially) to the subset of models that are
*linear*. Informally, system behaviors can be categorized
as linear or nonlinear according to the following criteria:

**The harder you push, the further it leans.**

This is a*linear behavior*.**If you push hard enough, it will fall over.**^{[3]}

This is a*nonlinear behavior*.

Clearly, we will need to formalize this a bit. But many systems show the linear kind of behavior, responding in proportion to the degree that inputs drive them. In reality, nothing is really linear. It is always an approximation. It is just a matter of whether the approximation is good enough. A lot of the time it is. As a practical matter, linear equations have the nice mathematical property that we can solve them. Mostly.

## Parametric models

For purposes of building the model, we can further simplify
by starting from a conventional, well-proven equation form, which is
configured by establishing the values for a finite number of
adjustable variables called * parameters*. Once their values
are established, parameters are treated as constants (though in practice
they might require adjustments to account for changing conditions
or new information).

Parameters are the *bane of politicians*, who seem
unable to distinguish them from *constraints*. There is a
huge difference, and as a practical-minded person, you do not need
to be confused. Parameters organize things in a useful way to
bring clarity. Constraints limit the things that can be done.
Parameters variables are critically important when using Least
Squared Error ("least squares") techniques — because the
direct goal of these techniques is to establish good values
for parameter terms.

## Next step

So far we have qualitatively considered the ideas of

- A
*system*- a portion of the real world relevant to current consideration - A
*model*- a mathematical representation with acceptable correspondence to the real system - The
*input variables*that drive the model - The
*output variables*that indicate the response to input variables - A
*linear model*- a restricted form of model that represents output behaviors in proportion to inputs - A
*parameter variable*- an adjustable variable that, when its value is established, determined how a generic model form is applied in a specific case.

Kalman Filters, at least in their basic and classical form,
require *all of the above.* Extensions that include essential
nonlinearity can be added later. The discussion regarding linearity
has been superficial, but Kalman Filters are very strict about it,
which can pose some challenges for model builders. We will examine
those details more thoroughly in the next installment.

**Full disclosure**: This series will make extensive use of
matrix notations. This is done to make things easier, not harder, but
if you are not completely up to speed on concepts such as vectors,
dimension, rank, singularity, transposition, inversion, and so forth,
you should read through the Matrix
Theory Sidebar note to prepare for the rest of the series.
^{[4]}

Footnotes:

[1] Start at the Embedded Systems site for Jack's articles.

[2] At the time, it was known as
*Marxist Economics.*

[3] Fudd's law, per the Firesign Theater
exposition. *How can you be in two places at once, when you're not
anywhere at all?*

[4] Matrix Theory Sidebar on this site. But it is still recommended to read through Jack's in-depth discussion at Who Needs Matrices? first.