For Additional Information Contact the Authors:

Lessons Learned Database

• Concept: (What is it and what does it do for you?)

Lessons Learned Database is a method for capturing, storing and retrieving important information as it would relate to future activities in the same or similar fields of endeavor. It would provide guidance in product improvement, process improvement or any cyclical activity. This methodology is also a good way to capture "tribal" knowledge.

• Mechanics: (How does it work?)

Each lesson is captured and stored in a relational database. The lessons are then coded to allow easy retrieval when similar sets of circumstances are encountered in the future. Similar lessons will share similar codes so they would all appear in the answer when the proper "question/query" is asked of the database. A consistent and logical set of codes is necessary to provide the most useful groupings in answers. In a Customer Lessons Learned Database we could capture data items which change our relationship with the customer. We would then code them as either good or bad, what area of the relationship they effect, and when they should be used or avoided.

• Application: (Where should it be used and what are some things to watch out for?)

Life Cycle Cost (Maintenance)

• Concept: (What is it and what does it do for you?)

A methodology for determining how much a component or system will cost over its entire useful life of operation. In many instances, customers only look at the initial acquisition cost of a system. This can be very misleading when trying do determine the "actual" cost of the equipment over time.

• Mechanics: (How does it work?)

Life Cycle Cost is the combination of acquisition cost, installation cost, maintenance costs and any other anticipated costs associated with the equipment over its lifetime of operation.

• Application: (Where should it be used and what are some things to watch out for?)

LCC predicts a more meaningful picture of the cost of a product. It should be calculated/estimated during the initial phases of the concept and design of new products. Trades-offs can be made in the initial cost of the unit vs. costs after it leaves the factory. Customers are willing to pay higher acquisition costs, if it can be demonstrated that this will result in better field reliability and lower maintenance costs. It should not be used as a justification for high initial costs if there are not real benefits to the customer later in the products life cycle.

Marketing MTBF / MTBUR Inputs

• Concept: (What is it and what does it do for you?)

Obtaining Marketing MTBF and MTBUR inputs can be very valuable during the initial stages of a Project. Many times estimates of the actual MTBF and MTBUR will not be available, but Marketing inputs can tell you what reliability levels you must attain to be to be competitive in the market place. These inputs should be compared to current capabilities and used in the early stages of the design process as guides or targets.

• Mechanics: (How does it work?)

Before many of the Design Constraints can be formalized, an estimate of the reliability must be determined. If details are not available on the design and similar designs are not available for comparison, Marketing inputs can be used to set the hurdle value on MTBF and MTBUR. As the design progresses, these hurdle values can be refined based on other sources of information.

• Application: (Where should it be used and what are some things to watch out for?)

Marketing MTBF and MTBUR inputs can be derived from two main sources. The first is from customer requests or desires. In this case, the customer is setting the expectation for the reliability of the final product. The second can be based on competitor's products. In this case, the product must perform at least as well as the competitor's product. In this second case, be sure that the competitors can actually achieve the hurdle values, since the values can be inflated to win contracts, with no intention of actually meeting the numbers when the product is ready for market.

Use Environment

• Concept: (What is it and what does it do for you?)

The environment in which a product will be used directly affects the reliability performance of the developed product. The product design must be accomplished with an understanding of this use environment in order to ensure reliable product performance.

• Mechanics: (How does it work?)

Anticipation of adverse use environment characteristics can be used to influence product design concepts and establish product requirements that result in a more robust product design. Environmental characteristics include, but are not limited to: type of vehicle/installation location, expected life cycle events, ambient operating temperature range, maximum thermal rate of change, vibration characteristics, storage conditions, pressure range, electrical supply characteristics, etc.

• Application: (Where should it be used and what are some things to watch out for?)

The environment in which a product will be used must be defined and understood in the earliest product development phases. Understanding of a product use environment is critical when a legacy product will be used in a new application. Differences in the use environments may affect the reliability performance that can be achieved by the legacy product in the new application. Therefore, appropriate requirements must be established for the product in the new application based upon the new use environment.

DFR � Technology Selection

• Concept: (What is it and what does it do for you?)

*MIL-HDBK-217

• Mechanics: (How does it work?)

The reliability design requirement is typically expressed in terms of product Probability-of-Success (Ps), Mean Time Between Failure (MTBF), Failure per Interval of Time (FITs), etc. To ensure that the product design meets the reliability design requirement, a reliability design analysis (e.g., Ps, MTBF, or FITs analyses) is performed.

• Application: (Where should it be used and what are some things to watch out for?)

DFR technology selection should be applied on products where high reliability is essential for mission success (e.g., medical equipment, military equipment, commercial aircraft, spacecraft, etc.).

Reliability vs. Safety

• Concept: (What is it and what does it do for you?)

The relationship between Reliability and Safety must be recognized and coordinated throughout the product development process.

• Mechanics: (How does it work?)

Application of Reliability design principles must be coordinated with Safety design principles to accomplish desired performance. Dependencies exist between the two disciplines, as well as areas of commonality. Activities must be harmonized to support and utilize these interrelationships.

• Application: (Where should it be used and what are some things to watch out for?)

Reliability Allocation

• Concept: (What is it and what does it do for you?)

Reliability allocation divides the total / system reliability into a number of subsystem reliability numbers. This approach provides a systematic way to find the reliability of a complex system. In this way the reliability of a one-of-a-kind or hard to test system might be understood by the analysis of the reliability of the components of that system.

• Mechanics: (How does it work?)

Component and subsystem reliabilities are assigned such that the overall system reliability goal is met. The subsystems should present a logical break point where the specific reliability can be more easily analyzed. Some iteration may be required to finalize the allocation.

• Application: (Where should it be used and what are some things to watch out for?)

Reliability allocation should be used for complex systems or for a system which is difficult to test by itself (it may be too expensive). Ideally, the parties responsible for the different component reliabilities can agree on the allocation. If there is one particularly problematic component, it may be possible to allow a higher failure rate for one component when other components can be determined to have lower failure rates than their allocation.

Reliability Prediction: MIL-HDBK-217

• Concept: (What is it and what does it do for you?)

MIL-HDBK-217 provides a method of calculating reliability of electronic piece parts, subassemblies, systems, etc. It is primarily intended to provide a common basis for assessing reliability, which permits reliability comparison of differing electronic hardware.

• Mechanics: (How does it work?)

MIL-HDBK-217 defines two reliability prediction techniques and provides the piece part level data required for performing reliability predictions based on either methodology. The first technique is referred to as the "parts count" method. In this approach, failure rates for electronic piece parts are defined based on the use environment. 14 environments are defined and the user selects the one most closely matching their hardware application. The piece part failure rates are summed to establish the failure rate of the higher level assembly/system. Using this technique, thermal and electrical component stresses are assumed to be nominal values. The second technique is the "parts stress" method. This method also establishes failure rates for individual components, which are summed to establish the failure rate for higher level assemblies/systems. However, with this technique, more sophisticated models are employed which take into account things such as thermal stress, electrical stress, device complexity, device package type, manufacturing process controls, and the use environment.

• Application: (Where should it be used and what are some things to watch out for?)

The use of MIL-HDBK-217 for reliability predictions has been a highly debated topic throughout the reliability industry for many years. Due to disagreement over its accuracy, further updates to the document are not being performed. However, it is still the most commonly used technique for assessing reliability of military/space electronics. The "parts count" methodology is intended for quick assessments and for evaluating designs in their early infancy. The "parts stress" method is much more robust and should be used whenever more accurate results are required and designs have been defined to the point where specific parts have been selected and the thermal characteristics have been determined. Either prediction technique is intended to provide a failure rate for single string hardware. It will not establish the reliability of redundant systems. However, the results of these predictions for the individual components of a system are used in the system reliability models. The value of MIL-HDBK-217 based predictions is greatest as a comparison tool. It is useful in comparing components against each other, more than providing an absolute value of component reliability. Unfortunately, the military/space industry has generally placed a high significance on the raw resulting values. Results from MIL-HDBK-217 predictions are most accurate for high reliability military/space grade components. Commercial grade component models, and particularly plastic encapsulated device models are considered highly inaccurate. Alternate sources for piece part failure rates are generally used for these types of devices.

Reliability Prediction: Ps Assessment

• Concept: (What is it and what does it do for you?)

Probability of Success (P_s) assessment is a method of identifying the reliability of redundant and single string systems. It is used to indicate overall reliability of the complete system in its intended application as opposed to just the static reliability of individual components. Ps indicates a probability, expressed as a decimal, that a component/system will complete its intended mission. Ps is the assessment tool for determining if the failure rate of a device is acceptable for use in a particular application. It is used primarily as a system level evaluation tool.

• Mechanics: (How does it work?)

There are many ways to determine the Ps. The methodology employed is dependent on the system being evaluated. In all cases, the failure rate of individual components, and the duration the system is required to operate must be known. If repair and/or preventative maintenance are to be used, these can be factored in as well. The simplest system to calculate is a single string system. In this case, where l is the failure rate and t is time. More complex systems can generally be solved with redundancy equations, which are just statistical combinations of the individual probabilities. Other common methods of determining Ps are Monte Carlo simulation and Markov analysis.

• Application: (Where should it be used and what are some things to watch out for?)

Ps assessment should be used whenever the inherent reliability of a component/system needs to be assessed for suitability in a system application. Reliability failure rates tell the relative frequency of failure. Ps determines whether that rate is acceptable in an application. The most critical aspect in determining Ps is to assure the system redundancy and any component dependencies on other components are properly modeled. Oversimplification of models is a common source of error.

(P_S) / MTBF / Failure Analysis

• Concept: (What is it and what does it do for you?)

It is helpful to know the Probability of Success (P_S) for a one of a kind system (i.e. space vehicle) or the MTBF of a system with multiple units available. Often there is a customer requirement (based on their needs) or an internal requirement for sparing / maintenance, etc. Knowledge of the Ps will determine when the system is ready to be deployed, and knowledge of the MTBF can help provide an optimal inventory program. Various tests can be performed to assess the Ps or MTBF; failure analysis of the faulty units from these tests can provide information about weaknesses so that corrective action can be taken.

• Mechanics: (How does it work?)

The Ps or MTBF can be determined through various tests including reliability allocation, life testing, HALT, simulations, current system field analysis, etc. Failure analysis is performed on failed units or subunits. With this information specific failure modes can be designed out or minimized.

• Application: (Where should it be used and what are some things to watch out for?)

Some variation of the Ps or MTBF should always be available to help determine when a system is ready to be deployed. Use an appropriate set of analysis tools to measure the reliability in the real word. Failure analysis resources are necessary to determine the root cause of any failure.

Dispatchability / Availability

• Concept: (What is it and what does it do for you?)

Air Transport Customers are focusing more and more on higher level metrics such as dispatchability and availability. Each time an aircraft is delayed, it translates to lost dollars to the carrier, so they are becoming very interested in minimizing these events. As an aviation equipment supplier, we should be analyzing how we contribute to the occurrence of these events and minimizing our products' contributions.

• Mechanics: (How does it work?)

ATA Spec 2000 Chapter 11 now contains information on delays in aircraft dispatch. This information will become increasingly important in the future and more readily available to suppliers. As this happens, we will be required to act on the data and provide improvements in system reliability.

• Application: (Where should it be used and what are some things to watch out for?)

The improved information provided by the adoption of Chapter 11 will allow correlation of aircraft removals and shop actions. This provides a much better picture of how and why a component failed and the consequences of that failure. In terms of pain for a customer, it may not be the high-ticket items that hurt the most, but many smaller items stacking together and introducing many small delays in dispatch of aircraft. Our future design efforts should be focused on solving problems hurting the customer the most.

Life Testing

• Concept: (What is it and what does it do for you?)

A component, subsystem, or system will often be tested over its expected / specified life conditions. These conditions may include combinations of time, temperature, vibration, shock, voltages, etc. The testing results can be used to demonstrate the reliability of an item and to correct systematic problems.

• Mechanics: (How does it work?)

The item to be tested is generally stressed at the extremes of its life conditions for the specified time (different from HALT which goes beyond the specified conditions). Failures from this testing can be analyzed to determine the likely field failure modes. The problems identified can be addressed to improve the field failure rate.

• Application: (Where should it be used and what are some things to watch out for?)

Life testing is the best way of determining the field failure modes and the reliability of a product, if there are sufficient units, means for performing the test, time, and funding. It is important to analyze the failures to determine what corrective actions can be taken. It is best to try to account for the worst case conditions and to make the conditions as real as possible.

HALT

• Concept: (What is it and what does it do for you?)

Highly Accelerated Life Testing (HALT) is an accelerated environmental testing process to evaluate and improve the robustness of the product design and component/materials. The product robustness is a direct indicator of the eventual reliability performance. The results of the HALT effort are used to establish the HASS profile for production.

• Mechanics: (How does it work?)

The product is exposed in a systematic manner to a series of specific environmental conditions applied at increasing stress levels for specified periods of time. The combination of high stress and exposure time accelerates the aging of the hardware and precipitating failures. The exposure continues and evidence of failure is monitored. When a failure occurs, the failure is evaluated, the cause identified, and corrective action implemented. The product is then returned to the HALT environment and the testing continues. This procedure is repeated until it is no longer practical to improve the product further.

• Application: (Where should it be used and what are some things to watch out for?)

HALT should be applied as early as possible in the new product development process. It should also be applied to major changes made to a product, or after a number of smaller changes have accumulated to a significant amount in a product. It can also be applied to evaluation of new technologies, new components/materials and new processes.

FRACAS

• Concept: (What is it and what does it do for you?)

Failure Reporting, Analysis and Corrective Action System is a systemic approach to the collection of failure data from one or more sources, the compilation and analysis of the data for root cause, and the identification of corrective actions. A FRACAS system is used to assure that problem identification and resolution with corrective action is done efficiently and effectively.

• Mechanics: (How does it work?)

Typically the failure data is collected and compiled into a database and made available for analysis. Analysis of the data may be done using a broad variety of tools to identify specific failure modes enable the identification of the root cause. Corrective actions are developed and deployed and their effectiveness assured through monitoring of the data.

• Application: (Where should it be used and what are some things to watch out for?)

A FRACAS process should be applied in the development, production, delivery and in-service phases of the product life cycle. Each process may be independent, but the resulting data is more useful if it can be linked/correlated across processes. Typical applications for reliability are on Red Label Hardware under simulation and test, and on in-service hardware in use by the OEMs, Owners and Operators.

Weibull Plot Analysis

• Concept: (What is it and what does it do for you?)

Weibull Plot Analysis is a method to analyze time to failure data. The analysis can predict when a given proportion of failures will occur, the nature of the failure mode or mechanism, i.e., infant mortality, constant failure rate, or wear out.

• Mechanics: (How does it work?)

Construct a plot of times to failure on specially scaled paper. The fitted line through the points gives the predictions needed. Many software packages, e.g., Minitab, can provide more sophisticated calculations and facilitate application on a broader range of problems.

• Application: (Where should it be used and what are some things to watch out for?)

These results are used to assess component and product reliability for suitability in a design, establish burn-in requirements, estimate a component�s field reliability, predict warranty & maintenance costs, and, as input to system models, predict system reliability. Be sure to have good definitions for �failure�, �time�, and �use environment�.

Reliability Demonstration

• Concept: (What is it and what does it do for you?)

A Reliability Demonstration Test (RDT) is an "accounting" test conducted to obtain measured proof of the level of reliability achieved by a product design.

• Mechanics: (How does it work?)

During an RDT, product items are placed in a test chamber and exposed to thermal and vibration cycles for an extended period of test time. The RDT environmental test parameters should be planned to closely represent the intended use environment in terms of total temperature range, relative exposure duration for temperatures within the range, vibration levels, and cumulative vibration exposure. The relationship between total test time and total number of failures determines the demonstrated MTBF of the product design.

• Application: (Where should it be used and what are some things to watch out for?)

The RDT should be conducted after the product design is mature using production-representative product samples. An RDT is not intended to benefit the product development process, it is intended to provide verification that the specified MTBF has been achieved by a product. Vibration should be applied concurrently (continuously, in chamber) with thermal cycling to avoid skewing the reliability performance measurement due to disproportionate acceleration of individual test conditions or falsely concentrated exposure to a single environmental parameter.

BIT / Demonstration

• Concept: (What is it and what does it do for you?)

A system with Built-in-Test (BIT) can test itself to determine if faults have occurred or will occur. The information provided may be useful for maintenance purposes � allowing a problem to be corrected before any real errors are encountered. It may also identify what the problems are. A "low battery" indicator will encourage maintenance before errors are occurring or information is lost. A common example of BIT is the booting up of the personal computer � when diagnostics are performed to determine the system configuration and its correct operation.

• Mechanics: (How does it work?)

The system is configured such that diagnostic (BIT) tests are performed on a predetermined basis (usually on start up or after a set amount of time). One approach is to have ROM code which interrogates the system in a fashion where actual responses are compared to known responses. Improper responses may generate error messages which inform the user of the problems identified.

• Application: (Where should it be used and what are some things to watch out for?)

Built-in-Test is often used in complex systems where microprocessors can perform diagnostics. It is also possible to have simple tests that provide important information on specific components � low battery, sensor not functioning, seat belts not fastened, etc. One must be careful that the overhead hardware/software for the BIT does not make the system less reliable or too costly. Also, the occurrence of false error messages needs to be minimized.

Radiation Analysis

• Concept: (What is it and what does it do for you?)

Designing to mitigate radiation effects will result in increased product reliability. The effects of radiation are sometimes factored into Probability-of-Success (Ps) calculations (particularly single-event effects). This provides the direct measure of impact of the radiation environment.

• Mechanics: (How does it work?)

• Application: (Where should it be used and what are some things to watch out for?)

Field Reliability Performance Assessment/Measurement

• Concept: (What is it and what does it do for you?)

Field Reliability Performance Assessment/Measurement is used to determine how well a product is performing in the field. The information collected can be used to determine current performance levels, possible enhancements to future products, improve current product designs, component/material applications, production processes and predict warranty costs.

• Mechanics: (How does it work?)

Failure data is collected on units in operation in the field. Information such as Unit Part Number, Serial Number, Aircraft, Customer, Time of Operation and Failure mechanism are collected for the fleet and product of interest. The failure mechanisms are grouped, dated, coded and counted resulting in summary data that can be used for assessment. The data is then applied to calculations of various "Time to Failure" reliability measurements useful in assessing product and fleet performance as well as for planning and conducting business. The data is also used for Weibull analysis to classify the failure modes.

• Application: (Where should it be used and what are some things to watch out for?)

This summary information is used to do fleet assessments such as product performance against Customer commitments, determining warranty costs, predicting future failures, establishing failure trends, calculating Mean Time Between Failure (MTBF) and Mean Time Between Unscheduled Removal (MTBUR). To assure a high confidence in the analysis, the failure mechanisms must be grouped carefully and removal counts and failure times must be matched for the periods of interest. Careful "coding" of the data is required to assure correct MTBF/MTBUR calculations and Weibull Analysis.

Trend Analysis at the Sub-assembly level

• Concept: (What is it and what does it do for you?)

Sub-Assembly Trend Analysis is a method of determining the reliability of sub-assemblies from the field removal data for existing product. It provides insight into which sub-assemblies should be grand fathered into the next generation of product, which ones need design improvements, and which ones need to be completely re-designed for use future product.

• Mechanics: (How does it work?)

Service Shop Records on the Product capture information about the reason for removal and the shop action used to repair the product. This includes information on which sub-assemblies and components failed. By totaling each of the types of failures over a period of time, the relative reliability of each sub-assembly or component can be determined. The ratio of sub-assembly failures to Product failures provides great insight into which sub-assemblies fail most often and which ones are suitable to keep as they are for the next generation product.

• Application: (Where should it be used and what are some things to watch out for?)

When designing the next generation of a product which is to be based on the previous generation, the results provided by Trend Analysis at the Sub-assembly level can tell you which sub-assemblies should be retained in the new design. This method should not be used as the sole indicator to determine if a sub-assembly is suitable for future product, because there may be other external influences reflected in the data. Externally induced failures in the field, system type problems and variations in troubleshooting procedures can be perceived as sub-assembly problems. These types of problems may be misinterpreted as failures associated with a sub-assembly when they may in fact have a very different cause.

Data Collection Systems (QDS / Field Data)

• Concept: (What is it and what does it do for you?)

WSC and QDS are data collection systems used to capture failure and repair data on our products. When analyzed, this data can yield valuable information on how our products are performing in the field. Strengths and weaknesses can be identified and carried into or eliminated from future designs.

• Mechanics: (How does it work?)

The large quantity of data available allows uncovering trends in component and subassembly failures as well as design, installation, maintenance and reliability problems. Data is aggregated over time, aircraft fleets, customers, and products to identify trends in failures. Once the failure is identified, the shop data is further analyzed to uncover possible failure modes that can be avoided or fixed permanently and in future designs. Strengths can be determined in the same way and carried forward into future designs.

• Application: (Where should it be used and what are some things to watch out for?)

When considering a new design, the strengths and weaknesses similar existing products should be studied. The results of these studies can provide guidance in how much of an existing design can be carried forward into the new design without modification. Entire subassemblies may be carried forward with very minor modifications, if they have proven to be very reliable, allowing the concentration of resources on "problem" subassemblies. The concepts may also be applied at the component level to a lesser extent. Accuracy of shop data becomes more of an issue the lower we go down the System-LRU-SRU-Component pathway.

Risk Assessment Tool � Failure Modes and Effects Analysis (FMEA)

• Concept: (What is it and what does it do for you?)

The Failure Modes and Effects Analysis (FMEA) is a risk assessment tool used to design reliability into the product by assessing the risk of potential failures. The FMEA is a "what-if" analysis that looks at possible failure modes and determines their effects on product operation. All effects are then evaluated for product impact (e.g., probability-of-failure, severity-of-failure, etc.). High-risk failure modes or unacceptable failure effects (e.g., failure resulting in single-point failure with high probability-of-occurrence) are then flagged for further design evaluation to either eliminate or mitigate the failure mode. In some cases, a redundant element may be required for a failure mode that cannot be mitigated enough.

• Mechanics: (How does it work?)

• Application: (Where should it be used and what are some things to watch out for?)

FMEA should be performed on products that require a high degree of failure tolerance (e.g., medical equipment, military equipment, commercial aircraft, spacecraft, etc.). FMEA can also be applied to manufacturing processes to determine where to apply resources.

Worst Case Circuit Analysis (WCCA)

• Concept: (What is it and what does it do for you?)

Worst Case Circuit Analysis (WCCA) is evaluation of circuits to ensure that the performance requirements will be met in spite of simultaneous circuit part parameter tolerance drifts (e.g., part parameter variations due to initial, temperature, aging, radiation, etc.) under worst-case operating conditions (environmental, input power, load variations, output power, etc.). WCCA can assure that the circuit meets the design requirement under worst-case conditions; thus, the risk of the circuits not performing to mission requirements is mitigated and product reliability is enhanced.

• Mechanics: (How does it work?)

• Application: (Where should it be used and what are some things to watch out for?)

WCCA should be performed on circuits which control critical product functions (circuits which an affect human safety, circuits which can result in the loss of product or mission, etc.).

Monte Carlo Simulation - System Level Reliability

• Concept: (What is it and what does it do for you?)

Monte Carlo Simulation Analysis is a methodology for determining system level reliability. It is a useful technique for determining reliability of complex systems, and is the only accurate method of determining system reliability when time dependent failure rates (e.g., normal or Weibull distributions, characteristic of wearout or infant mortality) are present in the system.

• Mechanics: (How does it work?)

A software model of the system operation is developed. Individual component failure rates, distribution types, and dependencies are modeled. The system is simulated using Monte Carlo techniques. This means that component/system failures are introduced randomly at a frequency defined by the failure rate/distribution being applied. These failure times are compared to system operational requirements to determine if the system met operational requirements throughout its required operational life. The complete required system life is repeatedly simulated to compile operational/non-operational statistics. The percentage of times the system operates successfully is defined as the system Probability of Success (P_s).

• Application: (Where should it be used and what are some things to watch out for?)

Monte Carlo simulation can be used to determine system reliability for essentially any system. However, it is not recommended for use when exact solutions can be readily calculated. It is best used for very complex systems and time dependent systems (whose failure rates vary over time). As with any Monte Carlo simulation, exact results are not repeatable (unless the same random number string is used) since the result is based on the generation of random numbers per a defined frequency distribution. In general, the greater number of data points generated, the more accurate the resulting statistics. Use caution when defining the number of significant digits in the result. The number of data points required will be dependent on the number of significant digits required in the result and the system complexity. Greater system complexity requires a larger number of data points since there are a greater number of operational states for the system. Caution should also be used when generating random numbers. Random number generators are actually pseudo-random. Depending on randomness of the number string(s) used, results can be highly skewed. This is particularly critical to simulating highly reliable systems.

Monte Carlo Analysis - Design Robustness

• Concept: (What is it and what does it do for you?)

Monte Carlo Analysis is a simulation method used to gage the relative robustness of a design. It provides information on how well the design behaves as component tolerances are varied on each of the critical components.

• Mechanics: (How does it work?)

Typically a model is constructed of the design and typical tolerance limits are included on each of the components in the design. A simulation is then ran monitoring the necessary outputs and varying the tolerances on each of the components randomly between the tolerance limits. The simulation is ran for many iterations accumulating results for each iteration. The applied tolerance is recorded for each component along with the associated outputs and the results are analyzed to indicate where the design is most susceptible to component tolerance build up.

• Application: (Where should it be used and what are some things to watch out for?)

This methodology is best used after an initial paper design has been completed, but before any prototype hardware has been constructed. Based on the results, circuit component values are adjusted, the simulation is ran again. Improvements are noted and retained and degradation is noted and removed. The simulation is continued until suitable component values and tolerance limits have been established. When this happens, the prototype hardware may be constructed. The main area to watch out for is that actual circuits may not always behave the same as the model.

HIRAP

• Concept: (What is it and what does it do for you?)

The Honeywell In-Service Reliability Assessment Program (HIRAP) method reliability prediction process. It was developed as a viable alternative to MIL-HDBK-217 (Military Handbook, Reliability Prediction of Electronic Equipment) for MTBF predictions of new systems, end item, or LRU designs. HIRAP was developed for the purpose of predicting the "in-use" reliability of an item as well as a circuit design comparison measure in design trade studies.

• Mechanics: (How does it work?)

This methodology reduces uncertainty in the reliability predictions by relying primarily on recent Line Replaceable Units (LRUs) field failure data history of in-service LRU designs with verified field performance that are similar in hardware and function to new LRU designs. In addition, weighting factors are applied to adjust for maturity of technology, processes and materials.

• Application: (Where should it be used and what are some things to watch out for?)

Experience demonstrates that this new method as applied to hardware used in the commercial aviation industry reflects a high degree of accuracy on the prediction of the reliability performance of new LRU designs. Benefits that result from applying HIRAP include an accurate assessment of in-service repair costs during warranty and useful life periods, improved spares provisioning, enhanced life cycle cost analysis, and higher confidence in the safety analysis process when Failure Modes and Effects Analysis (FMEA) and Fault Tree Analysis (FTA) are performed.

Parts Derating Analysis

• Concept: (What is it and what does it do for you?)

Derating criteria should be used as a trigger for further evaluation of an application, not necessarily an absolute limit. There are component applications, which necessitate component stresses in excess of normal derating margins. This may be acceptable. Consider derating as a flag for potential component application issues.

• Mechanics: (How does it work?)

The criteria used for derating come from many different sources. Some of the more common military based derating standards are MIL-STD-975, MIL-STD-1547, and AFSC pamphlet 800-27. Additionally, many companies have internally developed standards based on one or more of the generic standards.

• Application: (Where should it be used and what are some things to watch out for?)

Derating should be applied to all electronic designs where reliability/life is a concern. Derated limits should be used as indicators of potential design weaknesses. They do not imply a definite design flaw. When derating violations occur, further research into the part application is needed. This additional review will establish whether a design modification is required. Over-reacting to derating violations can result in unnecessary increases in size, weight, power, product cost, and schedule. Under-reacting to violations can cause premature failure and increased warranty costs.

FMEA and Criticality Analysis

• Concept: (What is it and what does it do for you?)

FMEA is the systematic evaluation of failure modes to determine their effect on system operation. A criticality analysis is an additional step to an FMEA, which assesses the criticality of the failures to the system�s application. Probabilities of occurrence are calculated for failure modes, as determined by the failure rate of the failing component, the frequency the part fails in that particular failure mode (? factor), and in some cases, weighting factors to assess specific operating conditions.

• Mechanics: (How does it work?)

An FMEA can be performed at various depths of analysis. The two most common depths are referred to as "piece part" and "functional". Higher indenture levels can also be assessed, though the significance of the results becomes less valuable. Lower levels of indenture are rarely considered due to the volume of effort (and cost) involved. The first step of the analysis is to identify the failure modes to be assessed. This is established from knowledge of the failure mechanisms of the individual components. This is true independent of the indenture level. Failure modes addressed in the analysis must generally be limited to a select subset, commonly hard failures. Intermittent and drift failures generally result in unmanageable analysis complexity. The second step is to systematically assess each failure mode. Documentation typically includes the failure mode, its effect on the local circuit/system, and its effect on the next higher indenture level. If a criticality analysis is required, then failure frequency data, based on the failure rate data would be included. The objective of this analysis is to identify undesirable operational conditions, which could result from the identified failure modes. Undesirable conditions are defined uniquely for each system. For example, a failure mode causing inadvertent operation of a critical function might not be acceptable.

• Application: (Where should it be used and what are some things to watch out for?)

FMEA/FMECA should be used whenever the effects of component/system failures must be considered. Whenever a system is expected to fail such that it doesn�t cause critical effects, an FMEA is essential. The most common weakness in FMEA�s is too high an indenture level. Performing the analysis at too high a level increases the risk of missing critical failure modes. Many internal failure modes can have effects on combinations of outputs. These combinations can be missed if the system is assessed only at a high level.

Fault Tree Analysis (FTA)

• Concept: (What is it and what does it do for you?)

* System or sub-system failure occurs only if Sub-Failure Cause 1 and Sub-Failure Cause 2 both occur, or if Sub-failure 3 occurs.

• Mechanics: (How does it work?)

• A system or subsystem may have many critical functions any of which may cause system or subsystem failure.
• Define these top-level product failure modes (functions) and develop a fault tree such as is shown in the above diagram. The fault tree, as above, captures the general sequence in doing a FTA.
• Note: There will often be cases where a sub-failure contributes to the occurrence of another sub-failure. These cross relationships can also be shown in the FTA using the basic rules illustrated above.
• Upon completion of the fault tree, design re-evaluation may be directed to determine the feasibility of mitigating high-risk high- probability sub-failure causes (e.g., causes resulting in mission loss, causes resulting in unacceptable safety hazards, etc.) resulting in increased product failure tolerance.

• Application: (Where should it be used and what are some things to watch out for?)

FTA should be performed on products that require a high degree of failure tolerance (e.g., medical equipment, military equipment, commercial aircraft, spacecraft, etc.). FTA can also be applied to manufacturing processes to minimize downtime, and for logistics considerations such as sparing. FTA is useful in the assessment of complex mechanical and electrical systems where operator or user safety is critical.

Manufacturing Process Capability Flow-up (DFMA)

• Concept: (What is it and what does it do for you?)

Manufacturing Process Capability Flow-up is essential in producing better products in the future. It is a process of matching design technology decisions to the capabilities of the manufacturing facility. The design process must take into account the processes that are available to produce the product and how the strengths and limitations of those processes affect the quality and reliability of the product produced.

• Mechanics: (How does it work?)

New production processes being considered in a design need to be assessed by Manufacturing to insure that they have or can acquire processes that are capable of producing the product with minimum variation in the finished product. Additionally, Manufacturing provides inputs back into the product design on where they have areas of expertise. These areas of expertise will help make decisions in the design choice of various technologies as to whether they are suitable for the new product.

• Application: (Where should it be used and what are some things to watch out for?)

As component and subassembly geometries shrink, it is extremely important that the product design capitalize on the strengths of the manufacturing facility. For instance, designs should not specify the use of ball grid arrays if the equipment to be used to manufacture the sub-assemblies is not capable of placing and soldering the devices with minimum variation. If advanced or new processes are required to produce a product, assessments must be made to see how that capability can be acquired in the manufacturing process and what the cost impact will be to the entire program.

Pre-Conditioning Program - Developmental (HASS / ESS)

• Concept: (What is it and what does it do for you?)

Highly Accelerated Stress Screening and Environmental Stress Screening apply a specific combination of environmental stresses over a prescribed period of time to newly manufactured products. The primary goal is to precipitate infant mortality defects contained in the product.

• Mechanics: (How does it work?)

Based upon the results of any development stress testing such as HALT, a profile is established which combines the specific environmental conditions such as temperature extremes and rates of change, vibration, powers level, etc. The stress levels and the specific time of exposure of the profile are optimized for efficiency and effectiveness.

• Application: (Where should it be used and what are some things to watch out for?)

HASS/ESS should always be applied to newly manufactured product. The optimum points in the manufacturing process to conduct HASS/ESS must be determined. HASS/ESS may also be used on product repaired for return to service, or to precipitate failure modes in products where the problem is intermittent or cannot be duplicated.

NEM / SPC

• Concept: (What is it and what does it do for you?)

Statistical Process Control (SPC) is a systematic way of controlling a process based on statistics. Usually the process has a targeted output with some tolerance. The use of SPC will help control the process output in the acceptable area and help determine when process improvements or equipment qualifications are called for. The proper use of SPC will provide more process outputs that meet the requirements more often. Numerical Evaluation of Metrics (NEM) uses SPC control charts to understand variation and isolate sources of variation.

• Mechanics: (How does it work?)

The output of a process is charted and must be demonstrated to be in control. This may require much experimentation (DOEs) such that the input variables that influence the output are well understood. By comparing the process output to previous outputs, one can determine when special events have occurred or when action is required. Generally one follows a set of rules (the most common set is the Western Electric group) to make these judgments. SPC tools help determine when a process needs to be "tweaked" to keep it under control. Many software tools are available to help such a process, including EXCEL and Minitab.

• Application: (Where should it be used and what are some things to watch out for?)

In order to use SPC a process must have been thoroughly studied such that the output follows normal statistical patterns. This work may involve fishbone diagrams which show how the different inputs affect the outputs and some mathematical models. The data will show when scrutiny or action is appropriate, but an understanding of the process is required to know what to do when a rule is violated.

Test Coverage & Analysis

• Concept: (What is it and what does it do for you?)

Test Coverage must be designed into modern products. The complexity of units has grown to the point that a unit must be able to determine its own health or allow easy determination using external equipment.

• Mechanics: (How does it work?)

Simple yet effective test methods using boundary scan, ATE, BITE, fault annunciation etc. provide the capability to analyze and determine where a defect or fault has occurred. This allows fixing the problem in much less time and easily storing the data for later aggregation into summary trend reports.

• Application: (Where should it be used and what are some things to watch out for?)

If a unit can determine and annunciate a failure clearly, it is much less likely to be removed from an aircraft when it is not at fault. In manufacturing, boundary scan is a very useful tool for determining connection integrity which, accounts for nearly 80% of all manufacturing defects. Automatic Test Equipment and Built In Test Equipment provide a wealth of information for analysis and feedback for improved future designs.

Reliability Scorecard

• Concept: (What is it and what does it do for you?)

• Mechanics: (How does it work?)

For the Scorecard to be of value, the user must first have a clear picture of what goals and objectives are to be accomplished. This view may be a list of Reliability MTBF Goals for a Business Market or a Customer Aircraft, or it may be a set of Reliability Process/Activity Goals on a Project, Program of for a Department or Location. These Goals are then captured on the scorecard and a measurement method established for each. Criteria are defined for each measured item to act as an indicator of the health of that item. A common criteria method is to apply "Green", Yellow" and "Red" colors to the measurement to indicate the amount of variation or the trend of the measurement. The measurements are then made on a defined period and reported together on one "scorecard" page. The scorecard, when viewed as a whole, provides a comprehensive summary of how the key goals and performance look together. The Scorecard then can provide the right attention and focus to areas needing improvement.

• Application: (Where should it be used and what are some things to watch out for?)

The Scorecard does not provide the level of detail to do detailed data analysis and problem solving.

Design of Experiments (DOE)

• Concept: (What is it and what does it do for you?)

A systematic way to design experiments to gain the most information about what factors or combinations of factors influence a process result using the least resources. Using the results of the DOE, multiple process factors can be adjusted at the same time to optimize the process outputs.

• Mechanics: (How does it work?)

With the knowledge of what factors may affect a process result, one may run experiments with all possible combinations of factors or an optimized subset of these combinations. The results are then analyzed to determine which factors or combination of factors provide the most / least effect so that the process can be optimized to produce the best results � generally a specific result in a stable manner (i.e. modest deviation in the controlling parameters has little effect on the result). There are a number of commercial software packages which will aid this process, including JMP and Minitab.

• Application: (Where should it be used and what are some things to watch out for?)

Designed experiments are often used in manufacturing processes to optimize the control of an output. Most semiconductor wafer fabrication processes are set up through the use of a DOE. For example temperature, gas mixture, and sputtering voltage may be varied to optimize the deposition of a dielectric film for a specific dielectric constant. The process helps to determine combined effects � e.g. sometimes the gas mixture is more important at a high temperature than at a low temperature. One should be careful to test the process over a set of conditions where the results will change, so that the experimental results for a lot of different conditions are not identical.