Reliability Program Plan Survey Results

ReliaSoft’s global survey on reliability program planning has been completed. The objective of the survey was to reach out to the reliability community to poll experiences and current practices in reliability program planning and implementation. The survey obtained responses from 542 participants from various industries who have a common interest in either developing reliability program plans (RPPs) for their organizations or providing reliability program planning as a consulting service. In this article, we will present the results of the survey together with further analysis and interpretation based on ReliaSoft's collective experience with assisting organizations in their reliability program planning activities.

Participation Analysis

The survey was launched on August 4, 2009 and closed at the end of September 2009. The intent of this survey was to collect the participants’ opinions on issues related to reliability program planning without burdening the respondent for more than approximately 5 minutes of time to fill out the questionnaire. Figure 1 shows the participation breakdown by region.

Figure 1: Participation by region

72% of the participants identified the organizations for which they work, while 28% preferred not to provide the names of their organizations. Among the participants who identified their affiliation, a total of 311 different organizations are represented in the survey. The highest participation number from a single company is 11. About 90% of the participants are from different organizations and 10% represent 2 or more individuals from the same organization. Figure 2 shows a high level categorization based on industry sectors.

Figure 2: Industry sectors represented in the survey

Assumptions and Considerations

The results are based on the aggregation of responses from various industries and regions. There is no claim that these results represent the de facto state of the art today and they should be considered to be just a particular snapshot within the range of current practice for reliability program planning. Furthermore, ReliaSoft solicited participation for this survey from within its network of subscribed newsletter recipients who have already declared an interest in reliability engineering in general. This fact skews the results because the participants in this survey may be among the more active and involved practitioners. They are only a subset of the total population that is active in reliability program planning and this subset could be considered to be slightly more informed and involved.

Development Stage When Reliability Program Plans Are Created

Question:

Within your organization, at what product development stage do you typically create your Reliability Program Plan?

Figure 3 shows the results. As observed, a little less than 30% of reliability program plans are developed during the concept stage. Most reliability program plans (almost 42%) are developed during the design stage and a significant percentage of them (about 28%) are developed at even later stages during product development.

Figure 3: Stage at which reliability program plans are created

This is clearly an area in which many organizations can improve in order to achieve optimum results from their product development activities. The ideal reliability program plan should be created as early as possible in order to allow for adequate planning, assurance of resources, integration with the product schedule and clear definition of the reliability objectives of the program.

Current Activities vs. Activities That Should Be Done

Questions:

Within your organization, what are the top 5 most important tasks in your Reliability Program Plan (i.e. what do you do currently)?

In your opinion, what are the top 5 most important tasks associated with a Reliability Program Plan that should be implemented (i.e. what should you be doing)?

The intent of these two questions was to capture the areas in which most of the effort is currently invested during a reliability program plan and show a comparison with the areas where participants think they should ideally be focusing their efforts.

Figure 4 shows the results and enables a visual comparison. This figure can be considered as a reflection of what respondents consider as the relative priorities when executing reliability program plans but the mix of various industries should be kept in mind when reading the chart. It is notable that there seems to be a general consistency between what is currently done and what practitioners believe they should be focusing on, which can be considered as an encouraging sign. The chart also shows that FMEA is at the top of the list of priorities, while participants report that they feel there should be more focus on generating and confirming operating use profiles and conducting reliability demonstration tests.

Figure 4: Comparison of what we are currently doing vs. what we think we should be doing in reliability program plans

Rating of Effectiveness, Execution, Integration and Overall Value

Questions:

Within your organization, how would you rate the overall effectiveness for the top 5 Reliability Program Plan tasks that you selected in the previous question?

Within your organization, please rate the following as they apply to the Reliability Program Plan: 1) Reliability program plan execution, 2) Reliability program plan integration with product development and 3) Overall value.

This series of questions was intended to capture the participants’ personal assessments of reliability program planning in their organizations. The results are graphically presented in Figure 5.

Figure 5: Effectiveness, execution, integration and overall value of the reliability program plan

If we use a ratio scale from 0 to 100, where 0 = poor, 25 = below average, 50 = average, 75 = good and 100 = very good, Table 1 shows the weighted average scores.

Table 1: Weighted average scores

The scores for all of the questions exhibit a consistent "normal-like" pattern with a central tendency a little above average, so all four areas are, essentially, rated as mediocre. Integration with product development activities appears to be a slightly more problematic area so our analysis concerning integration with product development is described next. [1]

The fundamental questions of project management – "who does what? by when? and how?" – need to be applied to the identified reliability tasks. A work breakdown structure (WBS) of reliability activities needs to be clearly defined. Based on this structure, a reliability program schedule should be developed where each activity is linked with resources: engineers, technicians, facilities, equipment, testbeds, failure analysis labs, data capturing and analysis tools. Reliability tasks then can become part of the "critical path" for the project and reliability can be managed and tracked at the highest level.

A practical guideline to identify whether a reliability program plan is properly integrated with an organization’s product development plan is to see if the reliability organization within the company is empowered to "raise the red flag" and delay the overall project schedule or negotiate more resources in order to meet the project objectives. If the reliability program plan is flowing in parallel with the product schedule but does not influence overall decisions, then the integration has not been designed properly or there are cultural and management structure issues that need to be identified and resolved.

Top-Down or Bottom-Up?

Question:

Within your organization, how does the support and development of the Reliability Program Plan typically flow?

Figure 6 shows that 57% of the program plans are supported bottom-up, meaning that engineers feel that they have to persuade management regarding the value and necessity of the reliability program in order to assure commitment and support. This is another area in which the survey results suggest that many organizations can make adjustments in order to achieve optimum results. Like any project initiative, the best approach to assure smooth flow is for top level management to provide up-front support of the reliability program planning initiatives. Since serious reliability programs require an up-front investment (which eventually pays off through lower warranty costs, lower life cycle costs and better brand name), top level management needs to "buy in" to the idea of systematically approaching reliability. This step will assure that efforts are supported throughout the program.

Figure 6: Typical flow of support and development of the reliability program plan

W. Edwards Deming’s second of fourteen points was to "adopt the new philosophy," suggesting that any quality initiative would require management leadership for change and successful implementation. [2] The same still applies today, and also applies for reliability program planning initiatives.

Formality of Process

Question:

Please rate how formally your Reliability Program Plan processes are defined, documented, reviewed and updated within your organization.

This question captures the level of formality of the reliability program planning processes. Figure 7 shows the distribution of responses as they relate to a brief definition of different levels of process formality. About 51% of the participants say that their organizations have either "formal" or "highly formal" processes for reliability program planning, while the rest have only some formal processes, very informal processes or no processes at all related to reliability program planning.

Figure 7: Formality of the reliability program plan processes

Processes that are clearly defined, documented and frequently reviewed and updated can go a long way toward creating a solid reliability management culture. This is especially important as projects get larger and in larger organizations that need to disseminate knowledge throughout different sites, product lines, R&D labs, countries, etc.

Data Analysis Tools

Question:

Within your organization, what data analysis tools do you use? (check all that apply)

Figure 8 shows an aggregation of reliability-related data analysis tools used within industry today. FMEA shows up again in the lead, followed by reliability life data analysis (aka Weibull analysis) and FRACAS.

Figure 8: Reliability-related data analysis tools most commonly used within organizations

Pitfalls and Barriers

Question:

In your opinion, what are the greatest pitfalls or barriers during the development and execution of a Reliability Program Plan?

A tremendous wealth of knowledge was provided through this open-ended question. The most common themes were:

Lack of test time and resources (engineers, infrastructure, tools) to execute a proper reliability program plan.
Lack of management support and commitment for the reliability program plan.
Lack of understanding of reliability engineering in the organization.
Not learning from field data.
Not embracing test failures as opportunities for improvement.
Difficulty in managing reliability through vendor and contract relations.

Even though this list is an accurate overall summary of the responses, it is also useful to provide a few specific "in their own words" replies. Some notable and characteristic comments are given next:

"Reliability, manufacturing and support engineering engage late (after design is 90% done)."
"The number one issue by far is an accelerated development schedule. With shortened schedules, it is difficult if not impossible to verify reliability requirements within the development cycle."
"We can never test adequately so we make simplifying assumptions and execute on accelerated testing. Both the assumptions and accelerated testing are opportunities for errors."
"[Not] getting the risk assessments such as DFMEA done at appropriate times to make them meaningful analysis tools."
"Lack of understanding of end use environment."
"Too much focus on characterizing reliability without solving known issues."
"[Lack of] true integration into the design process."
"Not understanding the system interactions, waiting too long to begin the system testing in the design cycle."
"[Not] convincing a client of the need to budget for a reliability program."
"The distance between the system reliability model and the actual production processes."
"[Inadequate] quality of data capturing."
"[Lack of] proper integration of Design for Reliability (DFR) into all work processes."
"Time is often the worst enemy, with insufficient time allocated in the development of a project plan to reliability testing. Great pressure is brought upon the concept of getting a product to market quickly. Cost would be next in importance… Communication between project members is the last major hurdle, with different goals for different areas of the company, which can create significant confusion and inefficiency."

Conclusions

These are some of the conclusions from the survey, as we interpreted the results:

Most respondents develop the reliability program plan during the design stage of the product development process. Ideally, this would be done during the concept stage.
Performing good FMEAs and defining good reliability requirements are at the top of the list of current and desired best practice tasks for reliability program plans.
More than half of respondents say that support and development of reliability program plans flow from the bottom-up in their companies. Ideally, the reliability program plan support and development flows from the top-down with management providing up-front investment and support for reliability activities.
Integration of reliability programs with product development is a major concern. There is a great deal of room for improvement in the effectiveness, value and execution of reliability program plan tasks.

Acknowledgements

ReliaSoft would like to thank all of the respondents who took the time to participate in this survey. Your input is invaluable for understanding the needs and challenges that the industry faces.

We hope that this two-way interaction will trigger the right questions within organizations to explore more effective ways of approaching reliability, not just as a data analysis and testing methodology, but from a systematic, program-wide view.

References

[1] Carlson, C., Sarakakis, G., Groebel, D. and Mettas, A., "Best Practices for Effective Reliability Program Plans," Proceedings of the Annual Reliability and Maintainability Symposium, 2010.

[2] W. Edwards Deming, Out of the Crisis. Cambridge, Massachusetts: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1986.