Predicting best-case or worst-case scenarios — and any scenario in between — for more accurate cost models and decision analysis.
NSWC Dam Neck engineers recognized that making trend predictions for Navy electronics systems maintenance required data collection from a wide variety of sources. So to make tracking easier, they developed a Web-based application that tracks data in a customizable management and predictive tool, called FROST, Future Readiness and Optimized Scheduling Tool.
FROST is a collaborative effort between the Naval Sea Systems Command's warfare centers. FROST was initially created by combat system engineers to forecast inventory shortfalls and to identify requirements before they developed into problems.
"We try to predict ahead of time when we are going to have a problem," said Michael Gaintner, FROST project lead and systems engineer.
Originally, the team entered data in a huge Excel spreadsheet, but changing data was tedious, labor intensive and error prone.
"Excel couldn't handle the calculations anymore, so we made the first version of the Web tool — FROST. We added a few other features, like including 'harvesting' in our predictions. The first time a system would be taken off a ship, we would go out, grab some of the electronic components, and use them to support the systems that are still in the fleet. It cut down the inventory we were using, which means less warehouse fees and less purchases," Gaintner said.
There are several basic data inputs that go into FROST. According to Gaintner, the problem is that the information needed for forecasting is dispersed across different commands.
For example, NSWC Corona, Calif., has a database that provides data about parts failure. At NSWC Crane in Indiana, engineers are in constant contact with vendors to determine how long the vendors can support each part and the cost of new parts. NSWC Port Hueneme, Calif., keeps track of the parts inventory and is the performance-based logistics organization.
"The information needed to sustain Ship Self Defense System (SSDS) is dispersed. When it was a manual process and we would go to a new schedule, it would take us weeks to provide our program office with feedback. The new schedule could affect our ability to support the system. For every part, we had to contact all these organizations to get all the information," Gaintner said.
FROST now automatically collects, tracks and shares relevant information in one tool and automates the process of analyzing system supportability. It automatically alerts engineers and logisticians when a part needs attention.
"You may get an alert that pops up and tells you that five years from now you will have a problem with a particular part. This enables us to look for a solution now. We can be proactive instead of having to react with limited options. The earlier we do it, the more options we have and the more efficient the solutions are," Gaintner said.
Since managing the parts and electronics that make up these systems is captured in FROST, data and forecasts the team collects can be used by managers to help determine the effects of scheduling and budgeting decisions.
"We have managers dealing with entire ships and systems. There is not a lot of information being rolled up from the parts to the entire system to tell them what effects their decisions are going to have on supportability.
"They come out with a new fielding schedule and they need to determine what effect the schedule is going to have on supportability. We are trying to roll up all this information, so we can tell them, 'If you move this ship out two years, we can expect these problems, and we can expect these costs.' We can make more informed decisions not only on scheduling, but also on planning and budgeting," Gaintner said.
FROST is PKI-accessible. It creates reports on-the-fly with the most up-to-date information.
"The moment data changes, we can generate a new report from the system," Gaintner said.
In addition to automated tasking, FROST developers are working on implementing a mathematical model called "Monte Carlo."
"Without going into too much math, using Monte Carlo enables us to apply a complex statistical process to see the most likely scenario. The most likely scenario in this case is that we'll have a problem in two years. We'll also see that there is a 10 percent chance that we will have a problem this year and a 30 percent chance of a problem next year.
"We can run through every possible scenario to make sure that nothing happens that catches us off guard. Standard systems that don't use this tool have to make a lot of assumptions that may not end up being true. Most of these systems assume that parts fail at steady rates: three parts fail this year, three parts fail next year, but it doesn't happen like that," Gaintner said.
The Monte Carlo model also allows analysts to treat inputs as variables and generate options or uncertainties (randomness). This flexibility can be combined with information, such as scheduling changes, budget constraints and parts availability and obsolescence, for more accurate and meaningful forecasting.
"You may be lucky for four years, nothing breaks, and suddenly five or six [parts] break at the same time. Part of the statistical process is that we can account for that. We also do a lot of statistics with vendor data. They say they are going to produce this part for the next three years and support it for four years after that."
The actual results often differ.
"When you base all of your projections on assumptions that vendor data is correct, you can get in hot water and make bad projections and bad decisions based on that. We have analyzed how often they [vendors] are accurate and how much variance there is, and we are building it into the new system," Gaintner said.
According to Gaintner, best-case or worst-case scenarios — and any scenario in between — can be supported with more accurate cost models.
"One of the things we can offer management that they have never had before is projection of sustainment costs based on individual electronics. If we are going to run out of parts for this processor in five years, we can predict what it is going to cost to solve that problem," he said.
In this regard, FROST connects the life cycle sustainment process together to increase overall efficiency. It allows engineers and logisticians to discover supportability problems at the earliest possible time, increasing their options, thus allowing high-quality, more affordable solutions. It also provides previously unavailable data to management for more cost-effective decisions.
A ship's availability is a huge factor in scheduling changes. A lot of planning goes into this part of the forecasting, according to Gaintner.
"The next goal is to use Monte Carlo to find the optimal fielding schedule for systems. Management can tell us all the constraints, whatever you need to consider, put it into the system, determine that there are 30 or 40 possible schedules that fit this, and let the system tell you which one is going to save us the most money," he said.
"It has been a great tool. Program managers have been able to use it to support COTS obsolescence initiatives," said Pamela Schools, former SSDS commercial off-the-shelf (COTS) obsolescence lead, who is now on the Strike Force Interoperability team.
Gaintner said he would like to add to FROST's capabilities.
"We would like to go further and have the system compute optimal refresh cycles."
FROST was developed through Program Executive Office Integrated Warfare Systems (PEO IWS) sponsorship and is used primarily for SSDS. But it is being extended to be able to handle other Navy systems.
The software, servers and hardware behind FROST are maintained at NSWC Dam Neck. It is designed to be used on a daily basis by the engineers responsible for sustaining SSDS.
"I taught myself to program and wrote the first version which is specific to SSDS. Now we are contracting out to build the next version. As soon as we created this basic picture, we realized that we could build on it and do other things," Gaintner said.
There are about 225 parts tracked for SSDS ranging in cost from $200 to $40,000. The team performs COTS obsolescence management for SSDS and the Advanced Combat Direction System, which is a major undertaking when dealing with COTS products, according to Gaintner.
A working prototype of FROST is in use. Version 1.0 is expected to be deployed in June 2008.