Defending Your Data

Jay Arthur
888-468-1537
303-756-9144
KnowWare International, Inc.
DBA LifeStar
2253 S. Oneida
Ste 3D
Denver, CO 80224

We work with companies
that want to fire up their profits using
Lean Six Sigma

Copyright © 2011

Everybody likes to feel good about their job and themselves, and nobody likes to feel bad. This is one of the major challenges of quality improvement. Most people would prefer to focus on what's going well rather than fixing what isn't quite working.

Sadly, when it comes to using facts and figures to improve the business, most people get busy trying to cast a shroud of suspicion over the data to discredit it and avoid doing anything.

Almost daily we get calls from QI Macros users who are trying to prepare for the onslaught of criticism they're sure they'll face when presenting their data, charts, and graphs to a "higher power." Nurses tremble when facing doctors. Employees worry when presenting to the boss. Most employees aren't statisticians, just people trying to do a good job for a customer, but they worry that someone will challenge their lack of understanding of math and statistics.

Here are some of the common issues we hear. Let us know about yours.

The Data's Not Perfect
And it can happen to anyone. In March 2004, a report by the U.S. Centers for Disease Control (CDC) concluded that poor diet and lack of exercise were responsible for 400,000 deaths in 2000, up 33 per cent from 10 years earlier.

In November 2004, the Wall Street Journal reported the number may have been overstated by 80,000 because of mathematical errors such as including total deaths from the wrong year.

The CDC, acknowledged there may have been statistical miscalculations in the report. The agency plans to submit a correction to the Journal of the American Medical Association, which published the original study.

Even with the corrections, obesity remains the second leading cause of preventable death.

All data is imperfect. Get over it. You can make a lot of progress using imperfect data.

The Data is Not Valid
This is the easiest way to throw the hounds off the scent.

Ask: Do you have better data? Show me. (Most of the time they won't.)
Say: Until you bring us better data, we'll have to move forward with what we have.

Why Don't We Measure Our Successes Rather Than Our Failures?
People want to feel good about what's going right, but improvement is about reducing mistakes, defects, and errors. So you have to focus on the failures. Prevent the failures and success will improve automatically.

I Don't Like The Answer
When you start showing people pareto charts, control charts, and other documents that actually reveal the extent of a problem, they won't feel good about it. The fastest way out of feeling bad is to discredit the data or the person who brought it up. I've even heard managers say the phrase: "Wrong answer."

When people use our GageR&R template, they often find that their measurement system needs improvement. Either the gage or the process for measuring has too much variation.
"There must be something wrong with the analysis," they cry.

When people use a control chart they find that the process is unstable and needs improvement.
"You must be using the wrong chart," they proclaim.

Many of these nay-sayers know how to sound confident and competent enough to make the presenter doubt their data. Don't buy it.

Ask: Show me what's wrong with it. What chart would you recommend? Let's draw it now! (And you can using the QI Macros.)

I Don't Get The Same Answer - The Formulas Aren't Right
Some bosses want their people to verify the QI Macros by creating their own formulas and spreadsheets and then they wonder why their 15-minute effort doesn't correspond with software we've been developing for a decade.

Just because the QI Macros aren't the most expensive piece of SPC software in the world, some people think they're cheap (i.e., poorly constructed, badly made, inaccurate). "Wrong answer!" The formulas in the QI Macros have been endlessly tested and come from the most up-to-date statistical references (like Juran's Quality Control Handbook) and standards groups (like the AIAG).

More often than not, the user just misinterpreted the formula. I had one customer fiddling with the formulas for Cp and Cpk. He got the formulas off a website (which were correct), but he missed the little bar over the R for range that means the average of the ranges. So he used the maximum minus the minimum to get a range and then choose the wrong value for n to calculate sigma estimator.

Ask: What formulas are you using? What reference book are you using?
Say: The formulas are fine. If you want to know more about the formulas, buy a copy of a good SPC book. Meanwhile, what is the data telling us?

The QI Macros have already been independently verified by some very stringent customers in healthcare. The test data provided in c:\qimacros\testdata gives the references we've used to verify the results.

Why are there so many control charts?
Why isn't there just one? Why don't you just use standard deviation? Aren't the UCL and LCL supposed to be +/- 3 standard deviations away from the mean?
This is another example of people not understanding some basic stuff.

The answer: You could use standard deviation if you have all of the data and it's normally distributed, but when you use samples or have different kinds of distributions (e.g., defects) the formulas vary to account for the differences. Download our SPC Quick Reference Card to figure out which chart to choose.

My Statistics Book Doesn't Match Your Statistics Book
One customer asked why Breyfogle's GageR&R example came up with different results than the QI Macros. On investigation, Breyfogle clearly got his information from the AIAG Second Edition, 1995, while we're using AIAG 3rd Edition.

Another thing I've noticed is that every author has to change the symbols or the layout or something to avoid looking like they copied the stuff from another source. The same customer asked us why the formulas in the GOAL/QPC GageR&R book weren't in the macros. Would we consider adding them. On further investigation we found that the formulas are there in the format specified by the AIAG. No wonder it gets so confusing.

One Bad Apple
Many customers have created a histogram and then wondered why they have one big bar on the left side and a small bar way out on the right (or vice versa). A lot of data gets entered manually. We usually find that one data point is entered with the decimal point in the wrong place. For example, we may see data in the form of: 0.01, 0.03, 3.0, 0.02.

Ask: Have you checked your data?

Dummy Data
There's an old saying in information technologies: GIGO (garbage in, garbage out). Several customers have put "dummy data" into tools like the GageR&R template, and then been caught off guard because the template tells them their gage system needs improvement. Dummy data can lead to dummy results.

Ask: Where did this data come from?

Preprocessing the Raw Data
Several users have sorted the data before drawing a histogram, which affects how the ranges are calculated and really screws up the Cp and Cpk calculations.

Other customers turn their raw data into ratios or averages, but then try to use the ratio in a chart that needs the raw data. Many healthcare clients take ratios like falls per 1000 patient days, but then try to use the ratio in a p chart that needs the raw falls and patient days. Another person tried to use two averages to do a statistical t-test.

Ask: Have you done anything to the raw data?

Not Preprocessing the Raw Data
Several users have tried to get the QI Macros to make a chart out of a bunch of text fields like "order error" and "billing error". We can plot your numbers, but you first have to use Excel's Pivot Table function to count the occurrences of these errors.

Here's my point: Focus on the Goal, Not Methods or Tools or Data
You can make a lot of progress with imperfect data. Stop using your data as a crutch to avoid fixing important problems. Stop using your charts as an excuse to argue about statistics and tools. Instead, ask yourself: "What can we learn from this chart or graph? What's the data telling us? Is there a problem worth solving? Where should we focus our improvement effort?"

Want to feel good again? Improve some mission critical process by making it far better, faster, and cheaper than ever before. That will make you feel good. Stop haggling about data and formulas. Start making some progress on real business goals.

© 2008 Jay Arthur, the KnowWare® Man, works with managers who want to plug the leaks in their cash flow.

Hire Jay Arthur to train your staff in his one-day Lean Six Sigma Workshop! Contact Jay at (888) 468-1537, support@qimacros.com.

Rights to reprint this article in company periodicals is freely given with the inclusion of the following tag line: "© 2008 Jay Arthur, the KnowWare® Man, (888) 468-1537, support@qimacros.com."

Want to increase your profits and slash costs by $250,000? Consider our risk-free trial of the Complete Lean Six Sigma System (#490)

To subscribe to this Ezine, go to http://www.qimacros.com/freestuff.html

Try QI Macros
FREE
For 30-Days

Or Buy It Now!