Modeling and Simulation

TechLabs builds and applies mathematical simulation models to complex system and decision problems, often on fixed-price commitments. Modeling and Simulation, properly done, is a major factor in reducing the costs and risks of a complex project. However, if the modeling and simulation becomes an end in itself, it can detract from progress.

Good models are resilient, multi-use, verifiable and long-lived. Our remote-sensor integration model, AIRS, was in use for 25 years over multiple projects and multiple computer hosts.

In over 35 years of building and applying mathematical models to practical solutions of real-world problems, we have learned a few lessons:

1. Computer models built for a specific design (i.e., one solution among many) are not only high-risk, but also cannot fairly consider alternatives and contingencies. One-design simulations beg the questions that were asked. Two one-design simulations, each for a specific solution, often cannot be fairly compared. 2. One ounce of mathematics is better than ten pounds of programming. Our CAINS network simulator sets up and runs thousands of times faster than a conventional network simulator, because it propagates mathematical distributions over the simulated network, rather than individual packets. 3. Brute-force simulations should be avoided. There are often too many interacting variables--many of which have to be guessed anyway--to reliably relate causes and effects. If a simulation has as few as 300 interacting input variables, all of which have to be studied, the number of possible combinations exceeds the number of electrons in the Universe. Makes for long run times. 4.Beware the tendency of complex software to become mathematically chaotic. What's nice about chaos is that you can get any result you want. See Example below. 5. The number of lines of code in a model is not a measure of results; it is a measure of the effort expended. We believe that our clients want results, not running the meter. 6. Three cardinal rules: Test, test, and guess the third rule...

A fashionable method for building simulations of complex systems is to write a simple model for every component, then use Monte Carlo simulation methods to generate "answers." Monte Carlo methods are important, and often needed, but not trivial. They form a two-edged sword: If the model is linear, it rarely behaves like the real-world system, especially at extrema where reliable results become important. If the model is non-linear, it can easily become chaotic in the mathematical sense, and yield surprising--and random--results that cannot be related to the input data.

We have learned how to avoid this two-edged trap.

We have followed these precepts over the years. As a result, we have been instrumental in the success of many major programs, and have been instrumental in the publication of some of the key textbooks in the field of military modeling.

Our models are strongly mathematical and statistical, and only use random simulation where proven valid and required. We do not value "lines of code" as a measure of productivity.

* We built and managed the first simulation (SEE) to be used as a procurement tool for a major communications network; for designing the system, evaluating proposals, measuring contractor progress, and acceptance-testing the final product. This model was subsequently used for many other high-reliability system designs and evaluations, because of its ability to predict failure events as unlikely as 1 part in 10 million&emdash;something an ordinary brute-force simulation could not do.

* Our network simulation model, CAINS, has been used for the design of large commercial communications networks, satellite communications networks, and command-control networks. CAINS has been used successfully, with modifications, for over 35 years, and is still valid.

* We built and delivered a general-purpose surveillance and reconnaissance mathematical model (AIRS) used by the client for over 20 years, encompassing several generations of hardware system design.

* Many of our models have entered into the TechLabs Toolkit, a description of which is available separately.

---

Footnote: Example of How Easy it is for a Model to Become Chaotic.

Let us take the simplest example of the problem, which you can easily reproduce on an ordinary spread-sheet (if interested, we'll be happy to email you an Excel spreadsheet of the simulation).

Let's say we need to study a flock of therbligs, or gerbils, or railroad engines. Assume further that the need to acquire more is proportional to the shortage of things on hand (100% minus the percentage on hand), and that since they cost money or other resources the amount of supply is also proportional to the resources available (100% times the amount on hand. Thus, for each day:

(New Amt. on Hand) = (Former Amt) * (1- Former Amt) * (Constant)

Where Constant is the variable we wish to find. This equation moves the simulation from day to day.

(At this point you astutely have identified our simulation as the Logistic Equation, equally applicable to business and ecology.)

Now try running this iteration for various values of C. Start with 50% or 0.5 on hand. As long as C is less than 3, the simulation behaves innocuously. Now try values of C from 3.1 to 3.9999999, making tiny changes each time. The chaos will appear almost immediately. The smallest change in C will result in a major change in the behavior of the system. Even adjusting C from 3.9998 to 3.9999 will induce major changes.

Here's how our population of elves (or whatever) behaves when C approaches 4. Given any set of data we can find a value of C that gives the desired answer.

<-- Please click here to return to Papers, Papers, Papers.