QA6640 Chat Room Transcript February 14, 2007

QA6640 Chat Room Transcript February 14, 2007

Prof. Atkins 20:22:23
The chat room is closed for this evening. It will reopen next Wednesday at 7:35 pm.

Prof. Atkins 20:21:24
If there are no more questions, we can close the chat room for this evening.

J Scott 20:18:23
Have a nice evening everyone.

Rich Weaver 20:16:55
Good night!

gstevens 20:16:32
Thanks and have a good night.

Rich Weaver 20:14:12
It's more difficult because of time differences, distances, language. In our assembly plant, when we get a bad part made in Alabama, it's no big deal to get them on the phone, or to have them fly in. For taillamps made in Germany, it's more difficult.

gstevens 20:12:31
I will but wanted to know in the area of supplier management if there are anyone who have had first hand experience in the quality of delivered goods.

Rich Weaver 20:12:08
A flippant answer would be "with difficulty". GM is getting more and more components from overseas, and it makes the SQ function quite difficult.

Prof. Atkins 20:10:05
Gstevens,

Your question is very interesting and it is too complex for me to even begin to answer. May I suggest that you try a Google search to find the information you are looking for.

gstevens 20:08:10
I am interested to know in light of globalization how companies today are ensuring the quality of products developed overseas for the US market place.

alan dial 20:06:48
No questions.

Prof. Atkins 20:06:19
The thirty-minutes are now up. Responses posted after this message appears will not be evaluated for chat room participation credit.

If you have a question, you may ask it at this time.

el hamraoui hanane 20:04:34
Hanane El Hamraoui
helhamra@spsu.edu
Systematic measurement error. (d) (or bias error): Average deviation of the measurement result from the true value by which the measurement result increases, when repeating the measurement a large number of times.
Systematic errors which change during an experiment (drift) are easier to detect. Measurements show trends with time rather than varying randomly about a mean.
As example, Systematic errors characteristic of a GPS receiver include: inaccuracy resulting from satellite ephemeris, and partial availability of satellites. For GPS receivers, systematic errors include inaccurate modeling of the atmospheric effects. Digital maps exhibit systematic error with partial availability and precision of position coordinates. Generalized systematic errors from a sensory system include sensor synchronization problems, resulting from the asynchronous availability of information from distinct sensory devices. For instance, in a GPS sensor the available position information is updated asynchronously and relatively infrequently compared to the differential odometer, requiring interpolation of absolute location systematically adding error.
http://www.patentstorm.us/patents/5434788-description.html

Michael Ginn 20:03:18
Michael Ginn
Email: ginnmichael@hotmail.com

1. Systematic error

The company I will reference to is a tier two automotive supplier (CNC Machining). I worked there as a quality manager. This company is a smaller batch/job shop (35-40 employees). It had a couple of jobs that ran every day, but the rest of the parts were job shop stuff. At this plant we would run into Systematic error on some of our Gauge R&R studies.
Systematic error is variation in measurement which can lead to measured values to be systematically too low or too high. The causes of this variation can be from differences in technique, equipment, environment, or technician. Systematic error is also called bias. It is called this because the systematic error has a biasing effect due to the cause(s) of variation (As mentioned above) For example, a quality technician might cause Systematic error when they do not properly zero a gauge. This is an example, of Systematic error in equipment or instrument. If Systematic error is constant it can be hard to control, because its effect is only noticeable if it can be alleviated. One way to alleviate Systematic error is to take the measurement with a more accurate/sophisticated piece of measuring equipment. Systematic errors that change during an experiment are called drift. This form of Systematic error is much easier to detect. A good example of a way to check for Systematic error is to measure a known valued specimen (Ex: Master Ring). This known value on the master ring could be used to demonstrate to a technician performing a Gauge R&R that the measurement in question was not correct. This error might have been created because the gauge was not initial zeroed at the start of the gauge R&R.

Brad McGuire 20:02:30
Brad McGuire
bmcguire@spsu.edu
QA6640

1) I will be using the joint venture of Lockheed Martin and the Defense Contractors Management Agency for my example. These two organizations come together in the assembly and delivery of products to the U.S. Air Force.
2) I will be discussing the concept of Systematic Error in aircraft Repeatability and Interchangeability inspections, specifically the differences in equipment, inspectors, and techniques.
3) In the F-22 final assembly line at Lockheed Martin, a series of inspections must take place (per the DCMA contract specifications) on a regular basis on previously identified aircraft panels/doors that clearly demonstrate that the panels/doors are interchangeable between difference aircraft. In order to accomplish this, Lockheed Manufacturing and QA Engineering developed a system of measuring these specific panels that meet the integrity of the product and satisfy the customer. And, depending on the availability of a QE, a QA inspector will demonstrate the panel/door measurements to the customer. However, as we’ve come to find out, due to the highly idiosyncratic nature of some of these aircraft doors/panels, different people have different measuring techniques that they prefer. On a recent I&R demo preparation, I was training one of the QA inspectors by demonstrating the customer-approved method of measurement. Seeing this, the inspector then showed me his own personal method using the same measuring tools. This inspector had a particular bias toward this panel seam due to the odd nature of the installation and the illusion of a large step, which in fact was much smaller than it seemed. The fact that this inspector was not in full agreement with Engineering caused some minor concern in that he would “buy” the operation with our measuring methods until he fully understood that it was customer approved. This type of systematic error could also be described as a culture-based bias in that it had to do with the mindset, background, and training of the individual. The individual was convinced of the correct method but had he not, he would have continued to produce much different data from the exact same panel seam using identical tools. Lastly this systematic error could have lead to a failed requisite I&R demonstration to the customer.

Komic 20:02:25
The company I choose is Cardinal Health. The company develops, manufactures, packages and markets products for patient care, develops drug delivery technologies, manufacture and distribute pharmaceutical medical/surgical and laboratory supplies.

It is impossible to perform a chemical analysis that is error free or without uncertainty. Random errors are the statistical fluctuations (in either direction) in the measured data due to the precision limitations of measurement device. Random errors usually result from the experimenter's inability to take the same measurement in exactly the same way to get exact the same number.

It is important in the Cardinal Health pharmaceutical microbiology laboratories where the variations in the mass of the ingredients are important. For instance if the operator measure the mass of an pharmaceutical ingredients three times using the same balance and get slightly different values there is measurement performance issue or random error. To minimize the random error more data should be taken. Random errors can be evaluated through statistical analysis and can be reduced by averaging over a large number of observations.

Random errors are usually related to insufficiently controlled variations in experimental conditions. They affect precision, but not accuracy. Random errors cannot be eliminated, but can be treated (statistically) and are related to the small, random errors in an experiment that combine to give an overall error.

gstevens 20:02:13
Glenn Stevens
Email: anandale@comcast.net

1. Briefly describe the company you have chosen.
2. Briefly describe the concept you will discuss.
3. Use a specific example to help explain the concept, and include as much detail as possible without exceeding the thirty-minute time limit for your response.

Company Selected: LHS was in the business of developing software solutions for the telecommunications industry. The product was developed and tested in house before release to the general market place. The company’s product was widely used internationally and had dominant market share in the US for wireless companies. The software product was released twice per year with new feature upgrades.

Concept: The concept being discussed is that of Random error. Random error is a measure of the variation observed when an entity is tested for the same characteristics or property using the same tools and methods, that is the test environment is stable for each test. Any variation detected is caused by imperfections (small) in the tools or apparatus used for the testing. The causes of these variations are considered common, that is no extraneous agents is responsible for them, and such variations tend to be small.

Example: This example to demonstrates how variation is tested for a software product. The software product developed by LHS had certain features and functionalities that must remain stable for each release unless the functions have been changed. The combination of these features and the supporting workflow used to exercise them is known as a critical path or the kernel path. The functionality and resultant behavior of the application supporting these features must be the same, for release after release, using the same test data, same test tools, and same test methodology.

The following steps outline the process.
Step 1: the critical functionality or workflow is tested, and the results stores. Test scripts (automated programs) are developed and stored with a version number; test data created and used in the test are developed and stored with a matching version number. A set of operational test procedures are also created that a test engineer uses to run the test. Let’s call this version release 1

Step 2. The first test is done for the critical path and the test results stored for release 1. The application is enhanced, with no changes to the core feature set, this is release 2. The test is run again (called a regression test) where the same test data, the same test scripts and the same test procedures are used. The new test results are generated and compared with the last set of test results from the first test run. This comparison of the test data is done automatically via the automated test tool. The results comparison will demonstrate any varations between the two versions of the release. The comparison between the expected test results (oracle) and the actual test results for release 2 will show any variations.

Step3. Comparison of the results have shown variations which have been discovered to be due to the time clock of the computer that runs the software application. In some cases one goes back and “fix” the clock time to what it was when the test was executed in result in release 1. In this case the comparison between the two releases should show 0 differences. If a variation is discovered, then comparisons are made with test data from release 1 to the test data to release 2, the test scripts from release 1 with the test scripts from release 2. Under normal circumstances the results should always be the same.

This regression test is automated and can be run multiple times on demand, whenever there is reason to believe that any code change may have unduly affected the kernel functionality.

gleiter 20:01:43
Kimberly Gleiter
kgleiter@hedonline.com

SC Packaging produces foam molded, protective packaging parts. For example, when a consumer purchases a DVD player it is typically surrounded by molded foam to protect it during shipment and storage.

Random error as defined by Bossert, is the variability due to assignable causes. They are variations in the measured data due to limitations in the precision of the measurement device and are typically the result of the Inspectors inability to take the same measurement in exactly the same way to get exact the same number. In the case of SC Packaging, typically a caliper is used to conduct dimensional checks on foam parts. Since foam is designed to be flexible, the inspector needs to be aware of the amount of pressure exerted when measuring a part. There could be variation in the results when the same Inspector, tests the same part, in the same location depending on how much they tighten the caliper. To minimize this, additional inspections are taken since random errors can be evaluated through statistical analysis and reduced by averaging additional inspections.

Rich Weaver 20:00:37
Company: General Motors
Application: measurement of fit between adjacent panels on an automobile
Type of error: Systematic

“Fit-and-finish” is one aspect of perceived quality that has received a tremendous amount of attention in recent years. The common perception is that Japanese vehicles have excellent fit-and-finish, and American cars do not. (Without a doubt, this was true in the past, but the gap has closed considerably in the past several years.) Since proper fit of the body panels is a critical characteristic, and since there are a great number of areas in which different panels meet up, a large amount of measuring is done in this area in assembly plants. And there are several ways in which systematic error distorts the data.

The standard specification for many body panel gaps is 5 millimeters, +/- 2 millimeters. (And parallel within 2mm; you can’t have a 7mm gap at the top and a 3mm gap at the bottom.) To take an example that’s easy to visualize, imagine the gap between the hood and the LH fender. Nominal is a 5mm gap, with no area below 3mm. Inspectors are provided with plastic go / no-go gauges, with are inserted between the fender and the hood. (Metal gauges are not used, because they could scratch the paint.) The 3mm end of the gauge must fit along the entire length of the gap. We started noticing an increasing number of vehicles in our completed vehicle audit with gaps less than 3mm. Checking the inspection records, all of the discrepant vehicles were good at the fit verification station. The first assumption was that the inspector was not doing this portion of his check properly, or not doing it at all. However, the operator was observed using the gauge on all jobs. Embarrassingly, it took quite a while before someone thought to examine the gauge. His technique for checking for tight gaps was to insert the 3mm end of the gauge between the hood and the fender, and slide it the length of the interface. This repeated sliding of the gauge was OK as long as the fits were close to nominal; but when the inspector checked vehicles with tight fits, the friction of dragging the gauge wore the 3mm end down. It was measured, and found to be closer to 2mm than 3mm. With the skinnier gauge, he was passing vehicles with a 2.5mm gap.

The wear and tear on the gauge is an example of systematic (rather than random) error. We’ve instituted a regular verification on plastic gauges for wear, and they are now replaced when they start to degrade.

Chris Wellman 20:00:21
Christopher J. Wellman

Company: ARC

ARC produces composite materials for our company. They are a small company of less than 50 people. They produce various pressure proof assemblies for military applications.

Concept: Random Error

Random error exists in all processes and materials. As Bosert explains on page 48, the variations tend to be small and normally expected. These are called common causes.

Example:

I required that ARC supply test data as part of a first article delivery. I reviewed the data and I noticed that although all the measurements recorded were in specification, some of the characteristics were identical and at the midpoint of the specification. I knew based on random error I should see some slight variation between the measurements, so I questioned the results.

It turned out that ARC had used fixtures for some of the measurements. I told them that we would then need to verify the accuracy of their fixture in order to qualify it for the purpose of making those measurements. They explained that the fixtures were the property of a mutual customer and they were allowed to use the fixture to accept those characteristics by that mutual customer.

I pointed out that my Company now had the responsibility to verify the process and after some discussion we agreed to qualify the fixture. All worked out, but there was no documentation to qualify their fixture for measurement purposes, so the use of the Random Error concept allowed me to find a potential hole in the process. We were then able to implement appropriate controls to ensure control of the fixtures for future production measurements.

J Scott 19:59:43
The company I chose to discuss is Harris, Inc. We manufacture heavy industrial equipment for the scrap and recycling industry.

In the discussion of Systematic error, the process I chose concerns the sampling and testing of hydraulic fluid. Systematic error is described in the text as variability due to assignable causes resulting from inconsistencies in technicians, materials or the environment. We use up to 1,500 gallons of hydraulic fluid in our machines so the cleanliness of the fluid is of the utmost importance. The cleanliness of the fluid is vital in maintaining hydraulic pump, valve, and manifold integrity. In the beginning of our testing program, we found that the test results were often sporadically high. After reviewing these results with the lab, we decided to focus on our initial fluid collection process. I found that samples could have been taken from any 1 of 3 potential technicians. To reduce variability in the testing process, a simple collection procedure was implemented. The procedure focused on cleaning and purging the drain valve before obtaining the sample. We also required the pumps to be operating for a predetermined amount of time. When this procedure was fully implemented, our test results became much more stable. It appears that particles were being inadvertently introduced (by some technicians) into the collection bottle via dirty valves or particles that had collected in the valve bodies.

For the discussion of Random cause, this is described as the variability of independent test by the same laboratory on the same material for the same property. Once we felt our assignable cause error had been eliminated, we focused on the common cause error. Using control charts for each tank, we tracked sample results weekly. We’ve had only minor out of limit deviations with this process. However, we’ve implemented a split sampling procedure, in which the split sample is sent to another lab for testing. This helps validate our collection and testing procedures.

Since starting this collection and testing procedure, we have virtually eliminated any pump failures due to fluid contamination on our machines and are able to better define our filtering capabilities.

Joel 19:58:10
Joel Centeno
jcenteno@twt.com
QA6640

The company chosen is Third Wave Technologies. This company manufactures molecular diagnostics for the detection of infectious diseases or genetic predispositions of patients to drugs. The company manufactures class I, II, and III medical devices regulated by the FDA.
Systematic Error: As Bossert explains, this is the variability due to assignable causes. During testing, this is equivalent to the source of variation coming from different machines, operators, lots, etc. Systematic error is also refer to as “bias”.
Currently, the company is implementing a test method to inspect incoming material from a vendor. The test is very complex is currently being validated. The test method validation is composed of the test method characterization (TMC) and the analytical performance characteristics (APC). The method is used to understand specifications but also to ensure that the measurement tool to judge incoming inspection is robust and data can be trusted. For instance, in the TMC portion, the tests are done varying the operator, machine, days, lots, etc. Statistical tools such as ANOVA (Analysis of Variance) is used to determine which buckets are the source of variation. The test method is used at the supplier and our facility as well. The first task was to verify that the test used at the supplier was the same or equivalent to our test so that we can compare material test results. Initially, we had seen discrepancies between their data and ours.

Testing at Third Wave:
During testing, four lots, 3 operators, 3 machines were used in the study over a period of three days. After the data was generated and analyzed with a statistical software package, we found that one of the biggest sources of variability was the instrument (at our end). While drilling down on the cause of this variation, we found out that the supplier was using a different machine brand for testing. When we swapped the equipment, the source of variation was eliminated and the results from the supplier and our data matched. We continue to implement these studies across other incoming materials and we want to ensure that suppliers tests align with this initiative. We understand the value added of trusting test results. By identifying sources of variation we have collaborated with suppliers to ensure that their QC tests (final release mostly) and our incoming test are well understood and characterized so that we can reduce the probability of accepting a bad part or rejecting a good part.

Natasha_Romero 19:57:00
Natasha Romero
nromero@spsu.edu

Systematic Error

1. I currently work for Sandia National Laboratory. We are a contractor to the Department of Energy and Department of Defense. Our mission is national security. We design both embedded and non-embedded software systems.

2. Systematic Error is defined as an error due to a variation in the process.

3. Recently a systematic error was discovered in one of our systems at work. The system is a 24-7 Command and Control Center. The system monitors a fleet of vehicles while they are in route to their destination. Recently there was a change in procedure and the vehicles began to be used on the weekends. During one trip there was large problem with the date/time stamp of messages being transmitted between the Command and Control Center and the vehicle. There time stamping of messages was inaccurate. This was a problem that had never been encountered before and the system has been in production for several years. Previously the operation hours for trips were usually Mon – Fri but due to change in schedule or process the operation hours for trips began to run from Mon – Sun. On this particular trip it was the weekend of day light savings time. During the trip the time changed and due to this shift in time the date/time stamp within the system was inaccurate for the duration of the trip. The software program was not equipped to handle day light savings time changes. Because of the change in process of times in which trips were scheduled this systematic problem occurred in the system.

aalan dial 19:53:40
Briefly describe the company you have chosen.

I have chosen to discuss GE Contractual Services (GECS). This is the business that I work for presently. GECS is a sub-business of its parent GE Energy. GECS provides long-term contractual services for its customers to provide the planned maintenance of the Gas Turbines it manufactures and sells to the Customer.

Briefly describe the concept you will discuss.

I will discuss “systematic error” in the calibration of plant equipment and instrumentation. Systematic error shows up in the performance of calibrations by different operators using various test equipment.

Use a specific example to help explain the concept, and include as much detail as possible without exceeding the thirty-minute time limit for your response.

Electronic maintenance technicians use devices such as micrometers, calipers, meters, etc. when performing calibrations on plant instrumentation. The plant instrumentation that affects plant or personnel safety is highly controlled. This control includes but is not limited to record keeping, annual calibrations, local device identification and plant management review. In an effort to reduce systematic error from operator variation, the calibrations are performed verbatim to a specific procedure written specifically for that device. As well, the maintenance record documents the historical performance of the equipment during the calibration. Many times certified calibration gases are used for the emissions monitoring systems called CEMS (Continuous Emissions Monitoring System). This system is used for regulatory reporting; therefore to eliminate systematic error, the calibration gas is certified for content by a certifying agency that uses special equipment. The gas content, specifically Oxygen, Nitrous Oxide and Carbon Monoxide, is passed through the CEMS unit to monitor performance against a known concentration. The testing agent calculates the monitoring bias by running the gases a series of 3 times on different concentration levels for the same gas (low, mid, high) and the tests are run 3 times on each. The bias is accounted for in a final calculation across the entire range of the monitoring equipment. This rigor is intended to put controls around the procedure and testing performance to ensure that personnel and equipment do not intrude systematic error into the results of the plant testing and calibration.

Prof. Atkins 19:34:18
CHAT ROOM DISCUSSION QUESTION:

You have thirty-minutes to answer the following chat room question:

In the sixth chapter of “The Supplier Management Handbook” on page 48 the author briefly discusses the following two concepts:

1. Random Error.
2. Systematic Error.

Select any ONE of the above two concepts and discuss it in detail as it applies to a specific example that you are familiar with. You may discuss your example in context to any ONE of the following:

The company you now work for, or
a company you have worked for before, or
any company you are familiar with.

Your discussion should cover the following three issues: