Why Parameter Uncertainty Is An Inadequate Modelling Strategy

We live in a world of uncertainty. One strategy used in economics is to incorporate the notion of parameter uncertainty: we have the correct model, but the parameters have some random variation from a baseline value. This strategy is highly inadequate, and has been rejected by robust control theory. The belief that we have the correct model was an underlying premise of optimal control theory, and the weakness of this premise in practice explains why optimal control theory was largely abandoned in controls engineering. (Interestingly enough, it persists in Dynamic Stochastic General Equilibrium (DSGE) models).

In this article, I give an example of an abject failure of parameter uncertainty as a notion of model uncertainty.

Why Care About This Example?

The example I give here is deliberately simple: it is a model with only a single parameter. It is not a recognisable economic model. However, it is easy for the reader to experiment with this model to validate the failure of parameter uncertainty.

Optimal control failed in practice as a result of the general principles illustrated by this example. To be clear, this is not a model system that caused them difficulty in particular. I will return to optimal control after I describe the example.

One could look at the simplicity of the example and argue that modern economists are far too sophisticated mathematically to make such an error. However, such arguments ring hollow when we consider that the optimal control engineers fell into a similar trap.

Firstly, the optimal control engineers were more sophisticated mathematically than modern economists. They developed the mathematics that DSGE macro modellers now use. A major driver in the development of optimal control theory was the path-planning required to get a manned mission to the moon in the 1960s: they were literally rocket scientists.

Secondly, they were working on engineering systems. Macro economists constantly complain that they cannot do experiments on their systems of study (except by using DSGE models, of course!). The optimal control engineers had the luxury of doing almost any tests they wanted on the physical systems they studied to determine the dynamics.

Even with these advantages, optimal control still failed as a design technique (outside of said path-planning problems).

The Example

(This article uses a bit of mathematics. Equation-averse readers may be able to skip most of them. The figures were generated by running simulations that were computed using my sfc_models equation solver. The code is given below, and is available in the development branch of the project on GitHub.) 


We have an extremely simple system, and we have a baseline model for it, denoted $P_0$. (We use $P$ to stand for "plant," which is the traditional control systems name for such a system, and it technically is an operator from a discrete-time single input to single output.)

1. We know that the steady-state gain from the input $u$ to the output $x$ is 1. That is, if we have a constant input of $k \in R$, the output converges to $k$.
2. We believe that the model lies within a class of models defined by a parameter $a$. The baseline model has $a=.05$, and that $a \in [0.005, 0.09]$.
3. The input $u$ is being set by the controller of the system, and hence has access to the data.

The parameterised model is defined by: $x[k] = (1-a) x[k-1] + a u[k-1], \forall k > 0, x(0) = 1.$

(Following electrical engineering tradition, I denote the discrete time index with $[k]$.)

We assume that we have access to a great deal of historical data for this system. Once we have validated that the steady state gain is equal to one, the resulting linear system has to have the above form (or be related to it by a linear scaling of $x$).

For some reason, we have reason to be certain that the parameter $a$ lies in the interval $[0.005, .09].$ We can run simulations of the model with three parameter values (the baseline $a=0.05$, plus the two endpoints of the interval) to get the simulated response of a step rise of the input $u$ from 1 to 2. (Figure below.)

We can then compare the output of the true system (in red) to these simulations. We see that the true step response is quite close to the baseline model. (An eagle-eyed reader might spot the problem here, but this would be difficult if I buried the true system response with random noise.)

As we can see, the system marches at a leisurely pace from 1 to 2, following the change in the input, However, one could imagine that this slow adjustment would be seen as suboptimal, and we can then speed the response up.

We want the output $x$ to track a reference signal $r$. We define the tracking error $e[k]$ as $x[k[ - r[k]$. We would like the tracking error to have the dynamics: $e[k] = \frac{1}{4} e[k-1].$

We can achieve this by setting $u$ so that it cancels out the existing dynamics for the baseline model, and force $x[k]$ to emulate the above behaviour. This is achieved by setting a control law:

$u[k] = \frac{-0.7 x[h] + 0.75 r[k]}{.05}.$

If we simulate the closed loop responses for our baseline system, and the systems at the extremes of the parameter set, we see that behaviour is relatively acceptable for all three models.

However, if we apply the control law to the actual system model, we end up with unstable oscillatory behaviour (in red).

In other words, although parameter uncertainty covered the open loop behaviour nicely, actual closed loop behaviour was nothing close to what was implied by the extremes of the parameter set.

The reason for this failure is quite familiar to anyone with a background in control engineering -- or taking showers -- a lag between input and output. In this case, the model is perturbed from the original model by adding two lags to the input signal. This is enough to make the resulting system unstable. The same effect is felt if one is impatient in setting a shower temperature. If you keep setting the dial temperature based on the current water temperature, you will end  up repeatedly overshooting between too hot and too cold. You need to give time to allow the water to flow through the pipe to see whether the temperature dial needs further adjustment. (An automatic shower temperature control is a standard control systems engineering project.)

In summary, systems can behave in ways quite different than predicted solely by parameter uncertainty. Missing dynamics can be fatal.

So What?

The simplicity of this example might make some readers impatient. "That's a lag. We know how to incorporate them into our estimation strategy by adding parameters."

Not really. Optimal control engineering did not fail because they did not have enough parameters. I had to use a fairly drastic lag to destabilise the system only because the underlying system (a low-pass filter) is remarkably stable. If the system had oscillatory dynamics of its own, much more subtle perturbations to the model would achieve destabilisation.

The optimal control strategy failed because it assumed that the model was known. The methodology was:

1. Take assumed model of the plant.
2. Calculate the optimal trajectory (using some objective function).
3. Force the system trajectory to follow the optimal trajectory by cancelling out the assumed plant dynamics.

This procedure was equivalent to determining the inverse of the mathematical operator of the plant, and using that inverse to calculate the target dynamics. Unfortunately, cancelling out an operator with an inverse creates severe numerical instability unless the matching is perfect. (Your system matrices end up being ill-conditioned.) This numerical instability made optimal control laws useless in practice.

For those interested in the history of optimal control, I want to underline that I do not think that they were misled by parameter uncertainty. When I did my doctorate in the early 1990s, optimal control theory was only studied as a historical curiosity; nobody really cared what their exact thought processes were. Based on my hazy memory of the literature (and the only optimal control textbook I own), there was no formal notion of model uncertainty. Instead, the explanation for mismatches between models and reality was explained as follows.

1. Our measurement of outputs was corrupted by random noise.
2. Model dynamics included additive random disturbances.

These issues were dealt with by using the Kalman filter to estimate the state despite the noise, and the assumed stability of the system would handle (finite energy) external disturbances. (This should sound familiar -- this is exactly the strategy that DSGE modellers inherited.) As can be seen, this is inadequate. Incorrect dynamics can imply a consistent force driving the actual output away from the theoretical trajectory, a possibility that is not included in random disturbance models.

I am unaware of optimal control theory that dealt with parameter uncertainty. However, it seems likely that engineers would like have done basic "sensitivity analyses" where they varied parameters. They would have done so without the aid of any formal theory to guide them, and would likely have ended up with an approach similar to what I sketched out above (what happens if a parameter is near its assumed limit?).

In any event, robust control -- sometimes called $H_\infty$ control -- abandoned the approach that we know the plant model with certainty. We still have a baseline model, but we want to ensure that we stabilise not only it, but a "cloud" of models that are "close" to its behaviour. There are formal definitions of these concepts, but readers would need to be familiar with frequency domain analysis to follow them.

Real World Implications?

In the modern era, we are unlikely to see disasters similar to those created by the application of optimal control in engineering. People are aware of the issue of variable lags, and so policy is unlikely to be as aggressive as they were in engineering. Furthermore, modern mainstream applications of optimal control to policy is in the domain of interest rates. Fortunately, interest rate changes have negligible effect on real economic variables, and so policymakers cannot do a lot of damage.

Instead, the applications of the notion of model uncertainty would be more analytical. We might begin to ask ourselves: although we managed to fit some model to historical data, is that fit spurious? How large is the class of models would yield an equally good fit? We just need to keep in mind that varying a parameter is a much weaker condition that having unknown model dynamics.

Code

(c) Brian Romanchuk 2017