Senior Fellow (Quality and Reliability)
University of Warwick (United Kingdom)
The University of Warwick (United Kingdom) is currently conducting a detailed and far-reaching survey of reliability practices, the "Benchmark Survey of Timely and Cost- Effective Application of Reliability Techniques." Companies world-wide and across all industries are called upon to participate. The survey has been designed to investigate the contextual factors that lead a project to choose specific reliability techniques. It attempts to benchmark reliability activity according to the speed that reliability techniques deliver useful output, the quantity and cost-effectiveness of that output, its authority and accuracy and the uses to which the output contributes.
This article presents a discussion of some of the issues that were considered during the development of the University of Warwick’s survey and a general overview of the final structure and implementation strategy. Readers may wish to incorporate this information into a framework for examining their own reliability practices and are also encouraged to participate in the survey.
Product reliability, be it expressed in terms such as durability, endurance or absence of warranty claims and in-service faults, is not achieved by conduct of a deterministic set of techniques. Indeed, over recent years, prescriptive standards have been viewed by many as being obsolete. Therefore, companies are much more required to use judgment and experience to choose the likely, most cost-effective techniques to be their reliability programme plan.
A reliability programme plan is influenced by many factors. Simple benchmark comparison of reliability techniques is unlikely to yield meaningful results because projects are so diverse. Of course, benchmark between comparable industries, companies and projects may be possible where there are sufficient similarities, but a universal benchmark requires an understanding of internal and external influences as well as measuring the performance achieved.
The model chosen for this benchmark survey focuses on benefits achievable from reliability effort, namely to:
- Assess risk or
- Contribute to
- Provide improved
- Facilitate sustainable process improvement.
Each reliability technique derives knowledge and applies it to current and future projects (as demonstrated in the benchmark model diagram displayed in Figure 1). The cost-effectiveness of any particular benefit is measured in terms of:
- Resources, skill
and cost requirements.
accuracy and validity of contribution.
Figure 1: Benchmark Model [Click to Enlarge]
These are the primary benchmark metrics. This model has been validated through workshop critique and individual review by several UK companies.
The survey aims to do more than measure benchmark performance. Through the process of workshop and individual company discussions, a number of additional objectives have been identified. These objectives come from the perspective of wishing to introduce greater authority to the process of selecting and scoping an optimally cost-effective reliability project plan. Such activity is usually undertaken early in a project and has a major impact on costs and risk. Accordingly, the survey additionally aims to understand how the following factors influence the benchmark metrics:
- Culture and
- R&D finance
- Project finance
- Organisation (reliability as a separate or organic function)
Influences deriving from written requirements
stability (frequency and scope of change)
- Justification required
- Type of customer
- Data requirements
Project related factors
- Industry and
(experience and learning curve)
- Lead times
In order to incorporate suitable survey questions covering the primary benchmark metrics and all associated influencing factors, it was immediately apparent that careful layout design would be necessary. The layout should:
easy to understand.
- Minimise textual answers.
- Avoid ambiguity.
- Avoid repetition.
Accordingly, the benchmark survey is split into four sections, grouping common data and reducing repetition:
- Part 1: Corporate information
Part 2: Project information
- Part 3: Project phase information
- Part 4: Reliability technique information
Each part maximises the use of check boxes
and all questions have help reference to both clarify and avoid ambiguity. The
linkage between sections is illustrated by Figure 2.
Figure 2: Benchmark Survey Structure [Click to Enlarge]
The result is a powerful survey requiring minimal time to complete. Provided that appropriate fact-holders are available, the survey may be completed accurately (with two reliability techniques) in less than two hours. All completed surveys will be treated confidentially.
Analysis and Distribution of Results
The statistical analysis objectives to be achieved through the survey include:
- Identify the
strength of influencing factors.
- Benchmark each reliability technique.
- Benchmark each company in relation to comparable returns, using the influencing factors to extend the benchmark.
In addition, however, many of the influencing factors are controllable, and hence specific guidance will be available to respondents, highlighting those factors currently limiting the impact of reliability techniques and their benchmark performance.
All respondents will receive aggregate and analysed results, together with a report highlighting the manner in which corporate, customer, project and requirements factors hinder or enhance the cost-effectiveness of reliability techniques.
This survey is open-ended. However, an initial "critical mass" of returns is hoped to be received by mid-2003. Returns will continue to be encouraged and will lead to analysis updates to all participants.
Summarised results will be published in journals and on the Web. However, only participants will receive aggregate results that will allow them to conduct more detailed analysis pertinent to their own projects and planning processes.
Participate in the Survey
is pleased to cooperate in this valuable endeavor as a sponsoring
partner. We encourage you to participate in the survey in order to
share your experience and take advantage of the analysis results
provided for your reliability practices and those in other companies.
The required materials may be obtained from the author or downloaded
directly from the University of Warwick website.
Download Survey Form and Explanatory Cover Folder http://www.wmg.org.uk/Benchmarksurveyforms-finalversion.pdf http://www.wmg.org.uk/BenchmarkFolde-finalversion.pdf