## Reliability Growth Analysis and Plots

*[Please note that the following article — while it has been updated from our newsletter archives — may not reflect the latest software interface and plot graphics, but the original methodology and analysis steps remain applicable.]*

During the first
phases of a product's development, the estimate of the product's final
reliability is called the reliability goal. However, the first prototypes
produced will almost certainly contain design, manufacturing and/or
engineering deficiencies that prevent the product from reaching that goal. In
order to identify and correct these deficiencies, prototypes are usually
subjected to a rigorous testing program and appropriate corrective actions are
implemented to improve the design. This structured process of finding
reliability problems and monitoring the increase of the product's reliability
through successive phases is called *reliability growth*.

Until now, the software available for analyzing reliability growth data has been fairly limited. However, ReliaSoft is currently working in cooperation with Dr. Larry Crow, the premier expert in the field of reliability growth, to develop the next generation of reliability growth analysis software, RGA. This article presents a brief overview of the capabilities of RGA and an introduction to some of the results and plots that are available for your reliability growth and related analyses.

**RGA
6 Overview **

The RGA software includes a wide variety of features to aid you in your
reliability growth analyses so that you not only can obtain the results, but
also understand the results. Features include:

- Data entry spreadsheets support continuous (time-to-failure), discrete (success/failure) and reliability data.
- Data analysis with the major reliability growth models: Crow (AMSAA), Gompertz, Modified Gompertz, Logistic and Lloyd-Lipow. Maximum Likelihood Estimation (MLE) is used for parameter estimation.
- Projections analysis using A, B and C failure mode classifications and the definition of effectiveness factors for use in the Crow (AMSAA) Projection model. This model can be used to estimate the number of unseen failure modes, the maximum achievable reliability and other important metrics.
- Repairable systems (overhauls) analysis using the Crow (AMSAA) model and Dr. Crow's analysis methodology.
- Chi-Squared and CVM methods for goodness-of-fit testing (depending on the data type) as well as the Statistical Test for Growth.
- Expanded plotting capabilities and automated custom reports created in Microsoft Word and Excel.
- The ability to attach any type of file to the analysis. For example, you can attach an Excel file that has the original data or a Word document that contains the report based on the data analysis.

**Basic
Analysis Results and Plots **

Although the results and plots that can be generated for your analysis will
depend on the type of data that you have collected and the reliability growth
model selected for analysis, some basic plots and results can be generated for
all analyses. Figures 1 and 2 demonstrate two plots that present reliability
growth results over time. Figure 1 presents the expected number of failures
and Figure 2 presents the instantaneous MTBF.

**
Figure 1: Expected Number of Failures vs. Time**

**Figure
2: Instantaneous MTBF vs. Time**

RGA’s QCP also provides point estimates for these metrics given time. In addition, you can generate charts and results for the cumulative MTBF and similar output for instantaneous and cumulative failure rates.

**Reliability
Growth Projections **

RGA will of course support the analysis of data from test-fix-test
reliability growth tests, where the fixes are applied as the problems are
discovered during the test. In addition, the software will also support the
incorporation of test-find-test data, where the fixes are delayed until after
the completion of the test. You can use the Crow (AMSAA) Projection model,
which utilizes A, B and C failure mode classifications, to analyze this type
of data. Using this terminology, you can specify which failure modes you are
not going to fix (A), which failure modes will be corrected at the end of the
test (B) and which failures modes will be fixed before the test has been
completed (C). In addition, you can assign a factor to each B mode that
estimates the effectiveness of the correction that will be implemented after
the test. There is no reliability growth for A modes and the effectiveness of
the corrective actions for C modes is assumed to be demonstrated during the
test. Analysis with Crow’s projection model then allows you to
consider different management strategies to see if you will still reach your
goal for reliability growth.

A variety of charts and results are available to support this effort. For example, Figure 3 shows the Growth Potential MTBF plot, which presents the reliability achieved during the test, the reliability that is projected after the implementation of delayed fixes and the maximum achievable reliability, given the current management strategy. If you determine that you will not meet your reliability goal, then you can re-evaluate your failure modes and change some A modes to B modes. In other words, you can decide to correct more failure modes. While doing projections, the assumption is that Beta is equal to one. Figure 4 shows one of several methods to check whether this assumption is valid, the Beta Bounds plot, which displays the confidence bounds on Beta at different confidence levels and demonstrates how these compare to the line where Beta equals one.

**Figure 3: Growth
Potential MTBF plot**

**Figure 4: Beta
Bounds plot to confirm assumption**

RGA also provides pie charts and bar charts for this type of analysis. For example, the bar chart in Figure 5 displays the actual (current) failure rate with the predicted failure rate for all the B modes in the analysis. The chart can also be generated for each individual failure mode. In these charts, the red bar (left) represents the actual failure rate and the green bar (right) represents the failure rate after the fixes have been implemented. From the chart in Figure 5, you can see how each failure mode is contributing to the failure rate of the system. In addition, you can also see how the failure rate for each failure mode is decreasing after the implementation of the fix.

**Figure 5: Before
and after failure rates for B modes**

Figure 6 displays a pie chart of failure modes with categories to represent the steps taken to address the modes: "Type A" will not be addressed. "Type B-Unseen" have not yet appeared in testing but are estimated from the analysis. "Type B-Remain" are failure modes that remain in the system since the corrective actions were not 100% effective and "Type B-Removed" are failure modes that have been removed through corrective actions and their effectiveness. For example, according to this pie chart, almost 29% of the failure modes have not even been observed yet through testing.

**Figure 6: Failure
modes pie chart**

**Repairable
Systems Analysis **

RGA will also facilitate the analysis of repairable systems data using the
Crow (AMSAA) model. For example, you may have a fleet of systems (e.g., a
population of cars, motorcycles or ships) such that each of these systems can
undergo an overhaul or a repair and be placed back into the field. Analysis of
a repairable system using RGA allows you to get an overview of the system
without having the large data requirements that would normally be required for
system reliability analysis, as in the BlockSim
software. You may want to
use RGA to track the progress of the system during development and then use
BlockSim in accordance with the already known results to gain more detailed
information.

In RGA's repairable systems interface, you can enter a start and end time for each system, along with any failure data that you may have for the system. You also have the ability to remove individual systems from consideration in a particular analysis if, for example, the data is not representative of the rest of the population. You can then analyze the data to combine each of these individual systems into a single "superposition" system. The parameters Beta and Lambda for that system, along with the results of the Laplace Trend Test and the Cramer Von Mises goodness-of-fit test, are also displayed for each system individually and for the combined "superposition" system.

Figure 7 displays one of the plots available for repairable systems analysis in RGA. This is the System Operation plot, which displays a timeline of the failures for each of the individual systems, along with the failures for the combined superposition system. You can also generate plots of reliability and unreliability vs. time for the extrapolated superposition system, as shown in Figure 8. Other plots, such as the Cumulative Number of Failures vs. Time plot with either linear or logarithmic axes, are also available.

**Figure 7: Failures
for individual, superposition systems**

**Figure 8:
Reliability vs. Time for superposition system**