Better Derivative Estimates by Pre-filtering

Numerical Methods for Derivative Estimation

In this section, we want to consider a pre-filtering strategy that eliminates any undesirable frequencies from the input data stream prior to applying the Central Difference derivative estimator. If there is no high frequency disturbance, the estimator will have nothing to amplify. But before committing to this idea fully, let's first test the concept in a relatively simple, brute-force manner. We will construct a lowpass digital filter that clears away high frequencies, but leaves low frequencies intact. Then, apply the Central Difference formula.

 

Selecting a lowpass filter

We want the input data set to be as undamaged as possible in the frequency band below the "20% of Nyquist Limit." Thus, the filter should not attenuate or shift the data in that band. But above that 20% limit, as much response as possible should be attenuated away. The tricky part is getting an appropriate flat lowpass filter characteristic without excessive computational effort.

This is a nontrivial design problem, and this is not a tutorial in digital filter design, so it is not feasible to present all the details here. The filter characteristic is intended to satisfy the following criteria:

  • preserve frequencies below 20% of the Nyquist limit
  • attenuate frequencies above 20% of the Nyquist limit
  • vigorously attenuate frequencies above 50% of the Nyquist limit

Preconditioning filter for derivative estimation (29 terms)


  -0.0000106  -0.0000634   0.0002528   0.0011268   
   0.0000000  -0.0049328  -0.0054297   0.0090374   0.0231928
  -0.0000000  -0.0542194  -0.0518104   0.0862069   0.2966433
   0.4000126   0.2966433   0.0862069  -0.0518104  -0.0542194
  -0.0000000   0.0231928   0.0090374  -0.0054297  -0.0049328
   0.0000000   0.0011268   0.0002528  -0.0000634  -0.0000106  
   

The form of the filter is a "linear convolution filter" very much like the "Central Difference" estimator filter, except with an even symmetry instead of an odd symmetry. To apply this, align the center of the filter to the point in your data set where you want a filtered value. Multiply the aligned terms from your data set with the corresponding filter coefficients, pairwise, and sum these intermediate products to obtain the smoothed data stream value.

 

Results of prefiltering the input stream

The preconditioning filter is first applied to the "noisy" data stream, and then the Central Differences derivative estimator is applied to the cleaned data stream.

Here is signal data before (green) and after the pre-filtering (blue). Notice that the changes are subtle.

effects of prefiltering on original data

Now compare the derivative estimates obtained using the pre-filtered data (in blue) to the derivative estimates obtained using the original unfiltered data (red).

effects of prefiltering on derivative estimates

The results with filtered data show a definite improvement. However, it is unclear how exactly to quantify the benefit, or how to determine necessary filter adjustments to best align with the performance objectives of the derivative analysis.

 

Further improvements?

It seems like we should be able to do better. The cost of the two-stage pre-filtered estimation is rather high. The experimental lowpass filter used 29 filter coefficients, quite a lot compared to the Central Differences estimator which uses 7.

Both filters have a similar "linear convolution kernel" form. Perhaps it is possible to construct a single hybrid filter characteristic that achieves both the derivative estimation and noise rejection objectives simultaneously, and maybe more efficiently as well? We are not done yet.

 


Footnotes:

 

Contents of the "Numerical Estimation of Derivatives" section of the ridgerat-tech.us website, including this page, are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise specifically noted. For complete information about the terms of this license, see http://creativecommons.org/licenses/by-sa/4.0/. The license allows usage and derivative works for individual or commercial purposes under certain restrictions. For permissions beyond the scope of this license, see the contact information page for this site.