Why Model?

Models and Embedded Systems

You have a deterministic processor architecture, predictable task scheduling, a watchdog timer with robust error recovery capabilities, strongly typed variables with carefully-scoped visibility, MISRA-conforming code that compiles clean. Coding passed a detailed review. You have now have an embedded system, right? Or am I missing something important? Like, maybe, the system should attempt to do something useful?

Sometimes "do something" is rather trivial — toggle a bit to actuate a relay, or collect a value by reading data from a hardware register. Other times, things are not so simple.

Perhaps this was why in past years I eagerly awaited each installment of Jack Crenshaw's "Embedded Toolkit," ^[1] exploring the art of getting things done when the means are not so obvious. The kinds of tools that Jack explored include: the "Rosetta Stone" equation that relates real-world continuous signals to processor-world discrete sequences; "minimizers" to seek a performance optimum; and foundational mathematical tools such as matrices and vectors for representing composite data and operations on them.

 

From Models to Kalman Filters

Jack was starting explorations of "least squares" methods for generating "formulas" that compactly represent information from complex data sets, with an eventual goal to cover the classic Kalman Filter. I hope to pick up the story line, though I can't guarantee that the wanderings will take the same track.

Kalman filtering is not as mysterious and complicated as it might first seem. It is actually a combination of several ideas, each of which is relatively simple when taken in isolation. Sometimes all you really need is just a few of these pieces — and you are better off without the rest.

Kalman filtering is the ultimate expression of the idea "don't control the world, control a model of the world." If you can model the world perfectly, and devise a strategy that produces optimum performance from the model, that same strategy will work to equal perfection in the real world. Unfortunately, this is an elaborate variation on wishful thinking. The world is too complicated and interconnected to represent perfectly. Still, the more you know about "the world" of your embedded system, as demonstrated by the success of your model, the better your chances of obtaining results that are good enough in the real world.

 

What is a model?

A model is a kind of mathematical statement about what you know — or think you know — about your system. It distinguishes features deemed important, and disregards aspects that are deemed unimportant or not of immediate direct concern. The better the model behaviors correspond to real-world behaviors, the more confidence you can have that your basic understanding is right.

There are many sorts of things that models can represent. Kalman filters are intended to model the sorts of processes that are continuous in character, though typically computed using computer methods at discrete instants equally distributed through time.

Sometimes the goal is to estimate effects that are extraneous but consistent. For example, one of the most powerful ways to improve sensor readings is to take lots of reading and average them — the "Central Limit Theorem." This effectively removes much of the "pure randomness." It is ineffective, however, at removing systematic errors. If you can use a Kalman Filter to model systematic but extraneous effects accurately, these too can be cancelled out of measurement data.

A useful model will represent critical behaviors very well, represent important behaviors sufficiently well, and as for the other things — forget about them. The model is not the system.

A judgment call determines importance, and decides whether the correspondence to the real world is close enough. What should you expect about the ultimate accuracy of an optimal solution based on judgment calls? (Models that produce precise results from crude or even speculative approximations were studied in the late 19th century.^[2]) This topic will arise again when considering what "optimality" of Kalman Filters means in practice.

 

What data does the model use?

The model will be driven by sequences of input variables, the independent values — for example, advancing time, or some control variables that you can adjust. The output variables — or at least measurements of them — provide information about what the system does in response to the input variables. In addition to the input and output variables, there can be internal variables maintained within the model; these are called state variables.

Given values of the inputs as applied to the actual system, the mathematical model should be able to predict what the corresponding output variables should be; and these results can be validated against the observed actual system output values.

 

Discrepancies between model and system

Models can show disturbed results that do not exactly match the behavior of the actual system. There are two complementary but opposed ideas about how this happens.

1. Assume that the deficiencies arise because the model is imperfect. This is the adaptive systems approach, consistent with the general philosophy of "recursive least squares". The way to correct the deficiencies in the results is to make the model better. Each new measurement provides more information about how the system works, and this new information can be incrementally absorbed to improve the model.

2. Assume that the model is accurate, but deficiencies arise because of disturbances in the values of variables used within the model. Inconsistencies between actual outputs and the outputs predicted by the model arise randomly. This is the Kalman Filter approach. It is presumed that, with perfect knowledge of all the variable values, the model will produce perfect results. Adjustments can be made to values of the model's internal variables in an effort to counteract cumulative random effects.

 

Linear models

Again anticipating the Kalman Filter, attention is restricted (at least initially) to the subset of models that are linear. Informally, system behaviors can be categorized as linear or nonlinear according to the following criteria:

• The harder you push, the further it leans.
  This is a linear behavior.

• If you push hard enough, it will fall over. ^[3]
  This is a nonlinear behavior.

Clearly, we will need to formalize this a bit. But many systems show the linear kind of behavior, responding in proportion to the degree that inputs drive them. In reality, nothing is really linear. It is always an approximation. It is just a matter of whether the approximation is good enough. A lot of the time it is. As a practical matter, linear equations have the nice mathematical property that we can solve them. Mostly.

 

Parametric models

For purposes of building the model, we can further simplify by starting from a conventional, well-proven equation form, which is configured by establishing the values for a finite number of adjustable variables called parameters. Once their values are established, parameters are treated as constants (though in practice they might require adjustments to account for changing conditions or new information).

Parameters are the bane of politicians, who seem unable to distinguish them from constraints. There is a huge difference, and as a practical-minded person, you do not need to be confused. Parameters organize things in a useful way to bring clarity. Constraints limit the things that can be done. Parameters variables are critically important when using Least Squared Error ("least squares") techniques — because the direct goal of these techniques is to establish good values for parameter terms.

 

Next step

So far we have qualitatively considered the ideas of

• A system - a portion of the real world relevant to current consideration
• A model - a mathematical representation with acceptable correspondence to the real system
• The input variables that drive the model
• The output variables that indicate the response to input variables
• A linear model - a restricted form of model that represents output behaviors in proportion to inputs
• A parameter variable - an adjustable variable that, when its value is established, determined how a generic model form is applied in a specific case.

Kalman Filters, at least in their basic and classical form, require all of the above. Extensions that include essential nonlinearity can be added later. The discussion regarding linearity has been superficial, but Kalman Filters are very strict about it, which can pose some challenges for model builders. We will examine those details more thoroughly in the next installment.

Full disclosure: This series will make extensive use of matrix notations. This is done to make things easier, not harder, but if you are not completely up to speed on concepts such as vectors, dimension, rank, singularity, transposition, inversion, and so forth, you should read through the Matrix Theory Sidebar note to prepare for the rest of the series. ^[4]

 

---