|Lab and POCT Quality Control|
Dr. Sharon Ehrmeyer
Professor of Pathology and Laboratory Medicine
Dr. Sharon Ehrmeyer is a Professor of Pathology and Laboratory Medicine and the Director of the Clinical Laboratory Science Program at the University of Wisconsin School of Medicine and Public Health in Madison, WI. Dr. Ehrmeyer has served on CLSI’s Board of Directors and Joint Commission’s Technical Advisory Committee, chaired AACC’s Government Relations Committee, and is currently the Regulatory Affairs section editor for Point of Care - the journal of near patient testing and technology. She has authored numerous journal and web-based articles and traveled the world speaking on the topics of laboratory and POCT quality issues. Dr. Ehrmeyer is a globally recognized laboratory quality expert and her extensive experience and contacts make her a natural to answer your QC questions.
Transferring the references ranges from an established method to a new method/instrument yielding similar patient results (determined through a comparison study) is a legitimate approach provided it is acceptable to the director.
According to the January 24, 2003 CLIA regulations, laboratories using FDA-approved, unmodified, nonwaived tests introduced on or after April 24, 2003 must verify the method performance specifications (below) before placing into routine use and these characteristics include: accuracy, precision, reportable range of test results for the test system, and identification of reference intervals (normal values) appropriate for the laboratory's patient population.
Note: CMS' Appendix C (SOM), Survey Procedures and Interpretive Guidelines for Laboratories and Laboratory Services, under §493.1253 (b)(1)(ii), states: The laboratory may use the manufacturer's reference range provided it is appropriate for the laboratory's patient population. Practically, this means that the laboratory director conscientiously selects the reference range(s) to use whether they be from the manufacturer, literature sources, or making the selection through evaluating patients' results as to whether they "make sense" in the context of patients' diagnoses. The decision must be documented.
Note that CAP, The Joint Commission, and COLA requirements essentially parallel these approaches. Of course, an actual reference range study also could be conducted. Estimation of reference intervals is time-consuming and costly. Typically, it is recommended to have 120 healthy individuals for each group, and, if there are male and female subgroups, then 240 specimens are needed. Guidelines for reference value studies can be found in Tietz's Textbook of Clinical Chemistry and in CLSI's guideline, C28-A3, Defining Laboratory, Establishing, and Verifying Reference Intervals in the Clinical Laboratory.
However, if the FDA-approved tests are modified and in-house developed tests, the reference range must be established.
Remember that coefficient of variation (CV) reflects the imprecision or random error of a method. When the method’s error distribution is symmetrical or bell-shaped gaussian, 68% of the observed QC values for a particular control fall within + 1 SD, 95% for + 2 SD, and 99.7% for + 3 SD. This means that depending on the SD rule selected to evaluate the QC results, by definition 32% of QC values will fall outside + 1 SD, 5% outside of + 2 SD, and 0.3% or 3 values in 1,000 when + 3 SD limits are used. All QC regulations mandate that test sites respond to situations where QC value(s) fall outside established limits. Yet depending on the QC rule(s) used, many of the results outside these limits will be false alarms – signaling a problem when none exists.
By setting “tighter” limits, more QC values will fall outside that limit and each out-of-control result requires investigation. In reality, when a test site determines the CVs are too large for a methodology to deliver medical useful information, another (accurate) method with less random error should be selected and then the right QC rule(s) applied based on quality goals and method performance to minimize false rejection.
To assure proper organism growth and identification, the media needs to be right. If pH is an important factor in assuring "right" media, then the measurement of pH must be accurate. At first glance, the difference in pH from 6.6 to 7.6 may seem relatively small. However, the pH scale is logarithmic, making this difference much greater in terms of hydrogen ion concentration. Each one-unit change in the pH scale corresponds to a 10-fold change in H ion concentration.
As for everything that we do in the laboratory, following manufacturer's instructions is rule number 1. So you want to follow the specific calibration instructions for your pH meter. Typically, pH meters are balanced with 7.0 pH buffer and the balance or intercept control shifts the entire slope of the calibration line. Another buffer of a different pH (e.g., 4.0) is used to adjust the slope of this line to match what is predicted by the Nernst equation.
Realize that the purpose of the values listed on the product insert for assayed QC materials is really only for guidance, since these values generally are determined from several instruments across several laboratories. Laboratories use these values to assess the values obtained in their settings. Quality control materials that do not include expected mean and ranges of values for particular analytes are termed unassayed QC products.
Establishing the means and SDs for sodium and potassium using unassayed materials is no different than for what is done with assayed materials. However you must know that the method is generating correct (accurate) patient results. Accuracy assessment determines if a method yields the "right" answers and is evaluated by at least one of the following: (a) assaying materials with assigned or known values, (b) comparing patient specimen results with a method in long standing use and known to be accurate, (c) verifying results from interlaboratory (voluntary or regulatory) survey data, or (d) splitting specimens with another laboratory. CLIA and all professional accreditation organizations require test sites to evaluate accuracy prior to implementing a method for routine use as part of the performance specification evaluation for nonwaived test methods.
Analyzing 20 QC samples over a period of time to include variations in analyts, instrument maintenance, reagent, and calibrator changes, etc. is the recommended practice. The mean and SD are then used to establish performance limits. In selecting QC rules, the laboratory director is expected to understand the implications of rule selection in terms of medical significance and probabilities for error detection and false rejections. For example, in selecting a + n SD rule, remember that with two controls per run, the false rejection rate for the + 2.0 SD rule is approximately 10% and for the + 3.0 SD rule about 1%. The false rejection increases when the number of controls increases. The false rejection rate for + 1.0 SD rule is approximately 32% when only one QC is analyzed. Therefore, pick rules carefully to maximize error detection and minimize false rejection, since all out of control situations need to be investigated. The best approach to selecting QC rules is to base the decision on an identified quality goal and the method's (in)accuracy and (im)precision.
CLIA (and for that matter CAP, TJC, COLA) require new shipments of current lot numbers be evaluated (verified) before placing into use. This assures that nothing has happened to the product in shipping. So the check consists of taking a sampling of the new shipment, assaying the material like patients, and then checking the results. Since the controls are the same lot numbers as that now in use, the results should fall within the current and predetermined range of QC acceptability.
If a control lot number differs from what is currently being used, the test site will need to assay the new lot number(s) along with the current control materials. The current lot number is used to verify the acceptability of the method. The values obtained for the new lot number(s) would be used to determine mean and SD. As discussed in an earlier question, statisticians recommend at least 20 values collected over 20 days for these determinations. (see earlier question regarding modifications for lots with limited stability / infrequent testing)
The mean and SD calculated from monthly control data do vary (the SD more so than the mean value). A better estimation and representation of the mean and SD for a control lot number is obtained by using more data points collected over a longer period of time. For stable methods, the cumulative value approach, which includes control measurements for a longer, specified period of time eg, 2, 3, 6 months, can be used. Many instruments and/or QC programs automatically perform these calculations. For manual calculations, make sure to use the correct calculations. (See Chapter 13, Calculations, Basic QC Practices, by J.O. Westgard)
The purpose of QC is to assess the performance (variation) of testing conducted in your laboratory. The QC limits for acceptability are based on the method's past performance and established by using the observed means and standard deviations (SDs) obtained by your laboratory's methods through repeated measurements of the controls.
The means and SDs identified in the product insert reflect the variation observed across several different laboratories. These "group" variations typically are larger than what are observed by an individual laboratory. These cause larger SDs and wider control limits, and in your laboratory, will reduce error detection. So the bottom line, is use these values as a guide, but don't use the means or SDs from the control's product insert to calculate the control limits for your methodology. If necessary, it is acceptable to use bottle values (as a guide) until sufficient in-house data (at least 20 values) are collected and means, SDs, and control limits can be established.
A crossover study is performed before the current lot expires or test sites run out of QC material. The new lot number is run in parallel with the current lot to establish the new control limits. So that the control values reflect changes in personnel, reagents, calibrators, testing conditions, maintenance, etc, that influence QC results, statisticians typically state that at least 20 values should be collected over 10 or 20 days to estimate the means and SDs of the new lots of control.
When this is not possible, particularly for materials with short outdates or current lots are in short supply, test sites can evaluate QC materials over a shorter period of time. CLSI's C24-A3, Statistical Quality Control for Quantitative Measurement Procedures: Principles and Definitions, suggests one provisional approach for establishing the mean value by making no more than four control measurements per day for 5 days. The CV from the old lot number having a mean similar to the mean of the new material could then be used to establish the control limits.
However, what many forget is that once sufficient control data are collected over a longer period of time, a new mean and SD for the new control materials should be determined and control limits reevaluated and adjusted accordingly.
Stay tuned! It's coming.
In March 2012, CMS announced in a memo to its surveyors Individualized Quality Control Plans (IQCPs), based on the concepts presented in CLSI's guidance document, EP23-A: Laboratory Quality Control Based on Risk Management. IQCPs will become a new CLIA QC policy and replace the current EQC policy once CMS releases its revised Survey Procedures and Interpretive Guidelines for Laboratories and Laboratory Services and announces the implementation date for the new QC policy.
Typically, CMS allows for a 2-year education and phase-in period for new policies. If CMS releases these Guidelines in fall of 2013, the effective date will most likely be fall 2015. In the meantime, CLIA-inspected test sites performing nonwaived testing and using manufacturers' built-in quality assessments to meet CLIA's daily QC requirements should begin to learn about risk management and the many concepts to develop IQCPs covering the entire testing process – preanalytical, analytical, and post-analytical. Those test sites not choosing to develop IQCPs will need to run at least two levels of external QC per test per day a test is performed. CMS terms this "default" QC. At this time, it is not known whether or not CMS-approved accrediting bodies such as CAP, TJC, and COLA will follow CLIA's lead.
The purpose of QC is to evaluate the quality of testing conducted in your laboratory. The QC performance limits for acceptability are based on the method's past performance and established by using the observed means and standard deviations (SDs) obtained by your laboratory's methods through repeated measurements of the controls.
The control limits identified in the product insert reflect the variation observed across several different laboratories. This "group" variation typically is larger than what is observed by an individual laboratory causing larger SDs and wider control limits, which reduce error detection.
Since the goal of QC is to evaluate the quality of testing in your laboratory, control limits should be established based on the QC data (means and SDs) collected in your laboratory. To begin, it is acceptable to use bottle values until sufficient in-house data (10 – 20 values) are collected and means, SDs and control limits are established.