Ongoing Comprehensive Warranty Analysis for Repairable Systems

Ongoing Comprehensive Warranty Analysis for Repairable Systems

The costs and liabilities that companies incur in supporting their warranty policies could consume staggering percentages of their budgets. Therefore, warranty analysis is an important activity for a manufacturing company's financial planning. Continuous monitoring for emerging negative reliability issues is also extremely important. An ongoing comprehensive warranty analysis program helps in revealing the truth in warranty data and provides much useful and insightful information, such as warranty returns and related cost forecasts, optimum schedules of spares shipments and deviation between the actual and predicted warranty returns. In addition, a comprehensive warranty analysis program will look at warranty data from several different angles to help identify trends, warranty issues and possible corrective actions. These analyses should look at the big picture, or global view, and should also look at the customer’s perspective during the warranty period.

Some warranty analyses that can be performed on non-repairable systems (or at the component level if a failed component is replaced by a new one or repaired to perfect "as good as new" condition) involve using distribution analysis methods (commonly known as Weibull analysis). For repairable systems, however, distribution analysis is not valid as systems are typically not put back into a like new condition after repair. In general, the intervals between failures of a complex system do not follow the same distribution. Therefore, for systems that are repaired and not replaced when they fail during warranty, the analysis methods provided in ReliaSoft’s RGA 6 software are generally appropriate. Some of the RGA 6 methods, e.g. the Global Fleet Analysis, will also apply to systems that are replaced when they fail.

This article presents different types of warranty analysis for repairable systems that address different sets of warranty data and related questions. The total package of analyses gives an ongoing assessment of the reliability during the warranty period from several viewpoints. Warranty analysis can help in forecasting future return numbers and costs of fulfilling warranty claims, observing MTBF change over time and predicting changes in the future, verifying constancy in warranty performance and checking for wearout or infant mortality problems (quality problems) during warranty.

Global Fleet Analysis
This analysis tracks the number of failures and warranty age mix of systems in an operating fleet (where systems can have different ages and can age at different rates) over different time increments during a warranty period. The fleet is defined as all units that are under warranty during the time periods of interest. As noted earlier, the Global Fleet Analysis can be applied to both repairable and non-repairable systems.

The Global Fleet Analysis uses the fleet warranty failures and total fleet operating hours over successive periods of calendar time to:

Determine if a trend exists.
Calculate the as-is average fleet MTBF.
Determine the chief failure modes influencing any negative trends and low fleet MTBF.
Project new fleet MTBF if certain problem warranty failure modes receive corrective action.
Address associated cost avoidance.

Sample Data and RGA 6 Instructions

The data set for this type of analysis is to be grouped by a certain increment (e.g. months, quarters), which shows the accumulated hours (or other time unit such as mileage) of all the systems in the field and the number of fleet failures. This type of analysis uses the following data type and model in RGA 6:

Grouped Failure Times (under Developmental Testing)
Crow-AMSAA (NHPP) Model

In Table 1, the warranty data are grouped according to quarters. In each quarter we note the total number of fleet operating hours for all systems that are under warranty and the number of failures during the quarter. The results from this analysis were discussed in a recent Reliability HotWire article, see http://www.weibull.com/hotwire/issue65/relbasics65.htm.

Table 1: Global Fleet Data

Quarter	Fleet Accumulated Hours per Quarter	Number of Failures
Q1	125000	992
Q2	267000	1190
Q3	386000	981
Q4	524000	1096

Global Fleet Analysis with Failure Modes
The next type of analysis is the same data setup as the previous type (Global Fleet Analysis), but with the addition of failure modes and classification on fleet failures. If certain failure modes of the system received corrective action, the Crow Extended model can be used to understand the influence of repairs targeted toward certain failure modes on the global fleet's warranty returns. Failure modes are to be identified and classified according to the type of corrective action they received, if any.

A: No corrective action will be applied
BC: Corrective actions have been applied to all systems during the operating period being analyzed
BD: Corrective actions were delayed and have not been applied during the operating period being analyzed
Sample Data and RGA 6 Instructions

As mentioned above, the data set is to be grouped by a certain increment (e.g. months, quarters), which shows the accumulated hours of all the systems in the field and the number of fleet failures. Failure modes are named and classified depending on the repair action type. This type of analysis uses the following data type and model in RGA 6:

Grouped Failure Times (under Developmental Testing)
Crow Extended Model

In Table 2, the warranty data are grouped by months. In each month we note the total number of fleet operating hours for all systems that are under warranty, the number of failures during the month and the type of repair action received by each failure mode.

Table 2: Global Fleet with Failure Modes Data

Month	Number of Failures	Fleet Accumulated Hours	Classification	Mode
Month 1	3 7	50 50	BD BD	FM1 FM2
Month 2	1 1 1 2	100 100 100 100	BD BC BD A	FM4 FM21 FM5 FM3
Month 3	1 1 2 1 1 3 1	150 150 150 150 150 150 150	BD A BD BD BC BD BD	FM7 FM16 FM8 FM5 FM22 FM9 FM10
Month 4	1 1 1 1	200 200 200 200	BC BC BD BD	FM8 FM19 FM10 FM11
Month 5	2 1 1 1 1 1 1	300 300 300 300 300 300 300	A BD A BD A BD BD	FM17 FM12 FM3 FM1 FM18 FM6 FM13
Month 6	1 1 1 1 2 1 2 1 1	400 400 400 400 400 400 400 400 400	BD A A BD BD BD A BD BD	FM4 FM3 FM17 FM12 FM10 FM5 FM18 FM14 FM15

Figure 1 shows the estimated effectiveness factors for the failure modes with delayed fixes (i.e. the BD modes). The effectiveness factor estimates the percent decrease in the item’s failure rate after the corrective actions are applied. The figure also shows a chart with the demonstrated MTBF of the population at the end of the 6th month, the Projected MTBF (after including the delayed repairs) and the Growth Potential MTBF (i.e. the maximum MTBF that can be attained with the current management strategy when all BD modes have been observed and fixed with an effectiveness equal to the average of the effectiveness factors that have been estimated for the observed BD modes).

Effectiveness Factors for the BD Failure Modes and Some Results from the Global Fleet Analysis

Figure 1: Effectiveness Factors for the BD Failure Modes and Some Results from the Global Fleet Analysis

Within Warranty Cycle Analysis
The next type of analysis addresses the serialized system's reliability within the warranty period. This is also a customer view of the system reliability during the warranty period. The analysis requires the failure times for each system in the field (if some systems never failed, these systems should also be included in the data set). If collecting data for all systems is not possible or too difficult, a random sample of serialized systems can be used (the sample might include systems that never failed). This type of warranty analysis uses the Power Law model, which is a generalization of the homogeneous Poisson process that allows for changes in the intensity function as the repairable system ages.

Sample Data and RGA 6 Instructions

This type of analysis uses the following data type and model in RGA 6:

Repairable (under Fielded Systems)
Power Law Model

Table 3 shows the failure times for each unit in a sample of 11 fleet systems; the end time is the last recorded known age when the analysis was performed. Figure 2 shows the probability (with 90% 2-sided confidence bounds) that a system that accumulated 2,000 hours of operation will operate for another 200 hours.

Table 3: Within Warranty Data

System ID	End Time	Failure Time
System 1	1268	68, 1137, 1167
System 2	1300	682, 744, 831
System 3	1593	845
System 4	1421	263, 399
System 5	1574	No Failures
System 6	1415	No Failures
System 7	1290	598
System 8	1556	No Failures
System 9	1426	No Failures
System 10	1124	730
System 11	1568	No Failures

RGA QCP with Conditional Reliability Results

Figure 2: RGA QCP with Conditional Reliability

Across Warranty Cycle Analysis (Fleet Analysis)
The next type of analysis addresses serialized system data across warranty periods. For this analysis, the systems are sorted from the oldest configuration to the most recent configuration (or any other ordering of interest). As in the previous analysis types, the analysis requires the failure times for each system in the field, even if some systems never failed. If collecting data for all systems is not possible or too difficult, a random sample of serialized systems can be used (the sample might include systems that never failed). The data should reflect the complete history of the systems during the entire warranty period. For the most recent systems in the analysis we should still try to use systems with completed warranty cycles in order for the trend analysis (b estimate) to be valid and reflect whether there are changes in warranty failures as a function of configuration build dates. For stable configurations, we would expect no trend (i.e. b estimate close to 1).

Sample Data and RGA 6 Instructions

After entering the failure data, this analysis requires converting the timeline into grouped data. To accomplish this, a group interval is required. The group interval length should be chosen so that it is representative of the data and is sufficiently large to ensure that there are failures within each interval (this can also be determined automatically by RGA 6 using a proprietary algorithm). Also note that the intervals do not have to be of equal length. This type of analysis uses the following data type and model in RGA 6:

Fleet (under Fielded Systems)
Crow-AMSAA (NHPP) Model

Table 4 is a data sample for an automotive product that has a 10,000 mile warranty policy. The systems are sorted from the oldest configuration to the most recent one. The data set shows the entire failure record of each system during the warranty period. The group interval is defined to be an increment of 1,000 miles.

Table 4: Across Warranty Data

Build	System ID	End Time	Failure Time
Build 1	1 2 3 4 5 6	10,000 10,000 10,000 10,000 10,000 10,000	No Failures 3160, 9430 8869 227, 1289, 3987, 4589 No Failures 5678, 8987
Build 2	7 8 9 10 11 12 13 14	10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000	2024 8460 6000, 7897 525, 2345, 6789, 8989 2074, 9105 5079 No Failures 498, 1232, 2345, 7899
Build 3	15 16 17 18 19	10,000 10,000 10,000 10,000 10,000	9161 348, 1200, 4497, 7900, 9939 9830 4233, 8234 6877, 8527
Build 4	20 21 22 23 24 25 26 27 28 29 30	10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000	No Failures 9556 8085 No Failures 135, 568, 1239, 6789 No Failures 6228, 9795 No Failures 8120 7023 128, 1396, 5600, 8796

First, we verify the assumption that b is close to 1. As shown in Figure 3, using the RGA QCP, we find that we can say with 90% confidence that the interval on b includes 1. Therefore, the assumption is valid. Figure 3 also shows the estimated MTBF at 12,000 miles for this analysis.

Parameter Bounds and MTBF calculated by RGA for the Across Warranty Cycle Analysis

Figure 3: Parameter Bounds and MTBF calculated by RGA for the Across Warranty Cycle Analysis

Across Warranty Cycle Analysis (Fleet Analysis) With Failure Modes
The next type of analysis uses the same data as in the previous analysis type but with failure mode identification and classification added into the analysis. This model addresses the projected average configuration reliability if certain failure modes receive corrective action in the future. This analysis addresses mainly the inherent average reliability of a serial number configuration during the entire warranty period. That is, the warranty age mix of the fleet is not a factor. For projections, the application of this model requires the configurations to be consistent, as verified by a b value close to 1.

Sample Data and RGA 6 Instructions

Since this analysis studies the effect of future reliability improvements on the fleet, the only logical failure mode classifications are A (no action) and BD (delayed fixes); no BC failure mode classification should be used for this analysis since it represents failure modes that receive improvements during the observed operating period. Similarly to the previous analysis type, this analysis requires grouping the data by a certain interval increment. This type of analysis uses the following data type and model in RGA 6:

Fleet (under Fielded Systems)
Crow Extended Model

The data set in Table 5 was gathered from the failure logs of 27 military systems during their warranty period (5,000 miles). The manufacturer tracks the failures by mileage of occurrence and records the failure cause based on failure analysis performed when the manufacturer is requested to repair failures and decide whether the failure mode needs to be addressed in a future planned improvement (BD modes) or can be ignored (A modes).

Table 5: Across Warranty with Failure Modes Data

System ID	End Time	Failure Time(s)	Class.	Mode
1	5,000	1396	BD	FM1
2	5,000	4497	BD	FM2
3	5,000	525	A	FM14
4	5,000	1232	BD	FM3
5	5,000	227	BD	FM4
6	5,000	135	BD	FM2
7	5,000	19	BD	FM2
8	5,000	812	BD	FM1
9	5,000	2024	BD	FM1
10	5,000	316, 943	BD, A	FM5, FM14
11	5,000	60	BD	FM1
12	5,000	4233, 4234	BD, BD	FM2, FM6
13	5,000	1877, 2527	BD, BD	FM7, FM2
14	5,000	2074, 2105	BD, BD	FM4, FM2
15	5,000	4079	BD	FM1
16	5,000	546, 577	BD, A	FM1, FM14
17	5,000	453, 4085	BD, BD	FM8, FM1
18	5,000	1023	A	FM14
19	5,000	161	BD	FM3
20	5,000	36, 4767	BD, BD	FM2, FM1
21	5,000	2795, 3375, 4228	BD, BD, BD	FM1, FM9, FM1
22	5,000	68	BD	FM10
23	5,000	1830	BD	FM1
24	5,000	1241	BD	FM11
25	5,000	871, 2573	BD, BD	FM12, FM1
26	5,000	3556	BD	FM13
27	5,000	186	BD	FM2

As shown in Figure 4, we first verify the assumption that b is close to 1; it is. Figure 4 also shows the accumulated number of failures per system at 5,500 miles.

Parameter Bounds and Number of Failures calculated for the Across Warranty with Modes Analysis

Figure 4: Parameter Bounds and Number of Failures calculated for the Across Warranty with Modes Analysis

Note: If b < 1, this could be a sign that the environment in which the systems are operated is changing over time (e.g. less harsh use of the systems). In other words, even if no corrective actions were implemented, it is possible to see “improvement” in the systems. This, however, should not be interpreted as actual growth or configuration improvement. Rather, the failure times may be extended because the use conditions are becoming less stressful. If there are actual engineering changes that caused b < 1, the use of the Crow Extended model's BC modes classifications may be justified and the model may be used to evaluate the reliability improvement. If b > 1, the use of the Crow Extended model is not recommended. This is a negative trend, which means that the average warranty reliability is getting worse with each configuration build. In such cases, this issue should be corrected and future system configurations should be stabilized (b close to 1) before direct projection analysis may be applied using the Crow Extended model.

Conclusion
Warranty analysis and forecasting is an important activity for manufacturing companies. It plays a central role in financial planning for warranty repair cost estimates and in detecting alarming reliability issues. This article presented different types of practical warranty analysis for repairable systems that can be performed in RGA 6 to address various data types and questions. These types of methodologies can be utilized at different stages of the warranty period and used to look at the warranty data from different vantage points. For more information, go to http://RGA.ReliaSoft.com and look for additional articles in future issues of the Reliability Edge and Reliability HotWire. The RGA software and this article were developed in cooperation with Larry H. Crow of Crow Reliability Resources.