Science.qxp

Received 11.18.05 Revisions Received 1.26.06 Accepted 2.2.06 Validating New Reagents:
Roadmaps Through the Wilderness
Roberta A. Martindale, BSc (MLS), MT(ASCP),1 George S. Cembrowski MD, PhD,1,2 Lucille J. Journault,1,2 Jennifer L. Crawford,
BSc (MLS),1,2 Chi Tran,1,2 Tammy L. Hofer, BSc (MLS), MBA,1,2 Bev J. Rintoul, BSc (MLS)1,2 Jean S. Der,1,2 Cathy W. Revers,
BSc (MLS),1,2 Cheryl A. Vesso,1,2 Carol E. Shalapay,1,2 Connie I. Prosser, PhD, FCACB,1,2 Donald F. LeGatt, PhD, FCACB1,2
( 1Department of Laboratory Medicine and Pathology, University of Alberta Hospitals, 2Capital Health Authority, Edmonton, Canada)
simply ignoring the QC shift. We offer a variation. Depending on the test type, specific One of the most frequent quality control issues systematic approach to shifted quality control information is gathered about the magnitude of faced by laboratory professionals is how to and/or patient data following a reagent lot the shifts in either the QC and/or the patient respond appropriately to a shift in quality change. We divide laboratory tests into 3 data. The control mean is reset following an control (QC) following a reagent lot change.
types, (1) tests for which the analysis of QC isolated quality control shift. Evaluation of the Possible actions include adjusting the control specimens is sufficient, (2) tests which shift in patient data is initiated by the range, checking for shifts in patient data, or demonstrate between reagent lot shifts laboratory director when the shift exceeds a infrequently, and (3) tests with between lot multiple of the allowable error.
It is well known that reagent lot changes can cause significant whenever a complete change of reagents occurs, "the laboratory shifts in patient results. Mueller-Hinton agar lot changes can affect . . demonstrate that changing reagent lot numbers does not drug susceptibility results.1 In hematology, new lots of Simplate II affect the range used to report patient test results, and control have caused incorrect bleeding times,2 and new lots of hematology values are not adversely affected by reagent lot number reagent have affected automated complete blood counts.3 In the changes."12 Soon after the enactment of CLIA ‘88, the require- realm of clinical chemistry, reagent lot changes have affected im- ment for lot validation was promulgated in the accreditation munoassays more than general chemistry tests. Lot-to-lot variation checklists of various professional organizations including that of has been frequently implicated in shifts of patient and quality con- the College of American Pathologists (CAP).
trol prostate specific antigen (PSA) values,4,5 probably because of The rigor in fulfilling lot validation depends on the knowl- the extensive application of PSA testing for screening and moni- edge, confidence, and prior practices of the laboratory directors toring of prostate cancer. Variation in reagent lots has resulted in and the supervisory technologists. In preparing for our first CAP significant variation in the testing of patients' hepatitis C accreditation inspection last winter, our senior biochemistry staff antibody,6 patients' human chorionic gonadotropin (hCG),7 and and medical biochemistry supervisors spent well over 30 hours theophylline quality control.8 Even point-of-care cholesterol synthesizing 3 different, test-dependent lot validations. Being tests9,10 are affected by reagent lot variation. heavily influenced by our quality system, we present these vali- In some analytical systems, reagent lot changes seem to dations as flow charts. For one group of tests, we determined primarily affect quality control measurements. The influence of that the analysis of quality control materials was sufficient for between lot variation on control product analysis is often as- the workup of new reagent lots. For the other tests, the investi- cribed to "matrix effects"; differences in constituent concentra- gation depended on the history of QC and patient shifts with tion and constituents in the control product and the plasma or new reagent lots. With a stable method, quality control analysis serum specimen that the control product is imitating. This was usually adequate; with a method that frequently phenomenon has been frequently observed in dry reagent sys- demonstrated patient shifts with new reagent lots, we required tems like that of the Vitros (Ortho-Clinical Diagnostics, the analysis of retained patient specimens as well as quality con- Rochester NY). Hill and colleagues have shown that slide gen- trol specimens. This general approach is summarized in Table 1.
eration changes in the Vitros can cause significant excursion in Multiple quality control specimens must be run in order to quality control measurements but little change in re-analyzed detect the presence of a shift. To quantify a quality control shift, patient specimens.11 we suggest that 4 measurements be made at each control level.
Prior to the 1992 enactment of CLIA ‘88,12 reagent lot val- The shift is calculated by subtracting the usual control mean idation practices were highly variable. General guidelines8 for from the average of the controls. We suggest that the magnitude measuring and responding to lot-related shifts in control and of the QC shift be assessed by comparing the control shift to the patient data were not widely applied and very few laboratories method's stable standard deviation. If the magnitude of the QC validated new reagent lots by re-analyzing retained patient speci- shift exceeds 1 SD, we require recalculation and resetting of the mens. Section 493.1255 of the CLIA ‘88 Standard specifies that June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE

To quantify the shift in patient data, we suggest that 10 different previously analyzed patient specimens be selected and
re-analyzed with the new reagents. If possible, the specimens
should be chosen to represent the usually measured concentra-
tion range. The shift is calculated by subtracting the average of
the initial results from the average of the new results. The
reanalysis of 10 specimens will result in greater than 60% of sig-
nificant shifts being discovered and corrected. (Personal commu-
nication, November 17, 2005, A. Srinivasan, LifeScan, Milpitas,
CA). The magnitude of the patient shift is assessed by compar-
ing the shift to a multiple of the allowable error (AE) of the
method. This allowable error may be the CLIA proficiency test
(PT) limit. As CLIA proficiency test limits are not available for
all analytes and are sometimes regarded as too broad, alternate
allowable errors based on physiologic variation are available.13
Table 2 shows the allowable errors that we use in our medical
biochemistry laboratory. We suggest that if the patient shift ex-
ceeds 0.5 x AE, then the method is judged as unacceptable and a
new reagent lot be obtained. Some laboratory professionals may
elect to use more stringent criteria for maximum allowable pa-
tient shift (eg, a maximum bias of 0.33 x AE). Some clinical lab-
oratories may require even tighter limits (eg, clinical trial
laboratories). These laboratories might attempt to use allowable
error limits derived from state of the art analytical performance.
Test-Specific Algorithms for Reagent
Validation
I. Tests where only QC is evaluated (Unstable analyte/
Figure 1_Tests where only QC is evaluated: Unstable analyte/Unstable
Unstable reagents/Insufficient material for retesting/Extremely
reagents/Insufficient material for retesting/Extremely tedious or tedious or time-consuming)
There are at least 4 types of tests where we assess only the shifts in quality control data, even when between lot variation inreagents may cause shifts in patient data. Some tests measurevery unstable analytes such as ACTH, insulin, vitamin A, and the discovery of QC shifts exceeding 1 standard deviation (SD).
vitamin E. For these analytes, it would be misleading to reana- The 4 replicate quality control measurements are averaged with lyze previously analyzed specimens with alternate set of reagents.
their average compared to the usual QC average. If the reagents Some tests use highly unstable reagents (eg, bile acids and free are deemed unacceptable, reagents are remade if prepared in- fatty acids). These reagents are so unstable that they need to be house. Otherwise, manufacturer controls may be analyzed before reconstituted daily. In order to validate such reagents with pa- the manufacturer is notified. tient specimens, we would need to measure the retained speci-mens daily. For other tests, there may be little or no specimen II. QC-Based Reagent Lot Validation for Methods with
remaining (eg, tissue iron, tissue copper, etc). Finally, some tests Usually Clinically Unimportant Lot-to-Lot Variation
are so laborious or time-consuming that we rarely would repeat Some chemistry methods rarely demonstrate between lot retained specimens with new reagent lots (eg, fecal fats). variation for either control or patient results. We propose the Figure 1 shows the algorithm for validating new reagent
following criterion for defining a method to be independent of lots with only quality control analysis. This validation can occur lot variation: a method is stable if the QC bias is less than 1 SD during patient testing. Briefly, quality control specimens are run for 3 successive reagent lot changes. The history of lot stability in duplicate until there are 4 control observations at each level.
must be documented. Examples of stable methods include elec- Trouble-shooting and/or recalibration follow out of range QC or trolytes, calcium, phosphorous, total protein, albumin, urea, and Table 1_General Approach to the Investigation of New Reagent Lots
Retained Patient Specimen Analysis Laborious/Infrequently performed/Unstable analyte 4 control replicates, each level Stable, generally no QC or patient shift with new reagent lots 4 control replicates, each level 10 retained patient specimens if control shift discovered Sometimes demonstrates clinically important lot to lot variation in 4 control replicates, each level 10 retained patient specimens patient specimens with new reagent lot LABMEDICINE 䊏 Volume 37 Number 6 䊏 June 2006
Table 2_Allowable Error (AE) Criteria
Alternate allowable Alternate allowable error limit based on error limit based on physiologic variation physiologic variation Alpha-1 Antitrypsin Human chorionic gonadotropin Alanine aminotransferase Alkaline phosphatase Angiotensin converting enzyme Lactate dehydrogenase Apolipoprotein A1 Luteinizing hormone Beta 2 microglobulin Beta hydroxybutyrate Bilirubin, direct Carcinoembryonic antigen Procainamide /N-acetyl Cholesterol, total Parathyroid hormone Rheumatoid factor Creatine kinase, MB Sex hormone binding globulin Creatinine, urine C-reactive protein Thyroid binding globulin Thyroxine, free (FT4) Triiodothyronine (T3) Free testosterone Thyroid stimulating hormone Follicle stimulating hormone June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE

creatinine measured by the Beckman LX-20 and TSH measured with alternate reagent lots. Some of our core chemistry labo- by the Bayer ADVIA Centaur. ratory assays demonstrate this variation: troponin, hCG, For these reagent lot-independent methods, we recommend and folate on the Bayer ADVIA Centaur and the enzyme that only the quality control results be evaluated with the new and turbidimetric tests on the Beckman LX-20. Each sec- reagent before a new reagent lot is placed into service. For these tion in medical biochemistry maintains a list of tests that tests, it is acceptable to analyze quality control material just after usually demonstrate large variation with alternate lots of reagent lot change and then in the usual schedule. If a persistent reagents. For these methods, before being placed into serv- QC shift is detected in the 48 hours following reagent lot ice, new reagent lots must evaluated for significant shifts in change we then recommend that the magnitude of the patient patient and quality control results. bias also be evaluated. Figure 3 shows the algorithm used to validate such un-
Figure 2 shows the algorithm used to validate such stable
stable tests. After the reagent lot change, quality control tests. Because of the infrequent occurrence of significant lot de- specimens are run in duplicate to more rapidly obtain 4 pendent variation, we suggest that this validation occur during replicate observations at each level. If the QC has not patient testing. Briefly, quality control specimens are run in the shifted, 10 retained patient specimens are analyzed with the usual manner after reagent lot change. Trouble-shooting and/or new reagent lot and the size of the patient bias assessed. If recalibration follows the violation of any quality control rule. In the QC has shifted, trouble-shooting and/or recalibration the case of unsuccessful trouble-shooting, 10 retained patient are attempted. If these procedures do not restore the origi- specimens or previously analyzed retained PT materials (in the nal QC means (indicating a QC shift), the retained patient absence of retained patient samples) are reanalyzed. If the patient specimens are analyzed with the new reagent lot. If the pa- or PT bias is too large, the clinical biochemist or pathologist is tient bias is too large, the clinical biochemist or pathologist consulted. Otherwise, the QC mean is reset if the QC has shifted is consulted. Otherwise, the QC mean is reset if the QC by more than 1 SD. has shifted by more than 1 SD. III. Patient-Based Reagent Lot Validation for Methods
With Significant Lot to Lot Variation
Some chemistry methods can demonstrate large variations in quality control and/or patient specimen testing Figure 2_QC-based reagent lot validation for methods with usually
Figure 3_Patient-based reagent lot validation for methods with signif-
clinically unimportant lot to lot variation.
icant lot to lot variation.
LABMEDICINE 䊏 Volume 37 Number 6 䊏 June 2006

Calculations and Record-Keeping
We designed a Microsoft Access program for entering, ana- lyzing, and retrieving the required quality control and patient
replicate values. Figure 4 shows an output screen from this pro-
gram that summarizes the reagent information as well as the req-
uisite QC replicate data. Figure 5 shows an example of an input
screen for the entry of CK patient data (current and new lot).
The "drop-down" menu of analyte names is linked with the al-
lowable errors shown in Table 2 and permits easy evaluation of
the magnitude of the patient shift.
We have been using this new lot qualification system for the last 16 months. For the LX-20 systems, we evaluated 103new reagent lots. Two lots of acetaminophen were judged unac-ceptable; the manufacturer has just narrowed its acceptance cri-teria for acetaminophen variation. With respect to the otherLX-20 analytes, all were acceptable; the following tests requiredquality control adjustments: alkaline phosphatase (2 lots), ala-nine transaminase (1), amylase (2), C-reactive protein (1), CSFprotein (4), gamma glutamyl transferase (1), lipase (1), microal- Figure 4_Output screen showing evaluation of the control data.
bumin (2), salicylates (1), theophylline (2), and triglycerides (1).
Until recently, only troponin and vancomycin were tested withthe lot qualification system; troponin required quality controladjustments with 2 lots. We are adding more Centaurimmunoassay tests to our reagent qualification system. Despite these efforts, we were unable to detect successive increases in gamma glutamyl transferase patient results. Anothercity laboratory discovered our higher results with a patient com-parison study. We are now scrutinizing our gamma glutamyltransferase lot validations.
The system has provided us with defensible and logical cri- teria for remaking new reagents and even returning "defective"
reagent lots. We have found that our QC adjustments are made
with increased confidence. As our chemistry laboratory is large
and run by 4 laboratory scientists, the flowcharts presented here
have standardized the evaluation of new reagents. The Access
data base program has greatly simplified our record keeping of
validation of new reagent lots. LM
Acknowledgement: We wish to acknowledge the initial
Figure 5_Input screen showing results of reagent lot comparisons using
patient specimens.
guidance that was provided by Ms. Elsa Quam of the Universityof Wisconsin Hospital Clinical Laboratories, Madison, WI.
1. Fuchs PC, Barry AL, Brown SD, et al. Intrepretive criteria and quality control 8. Cembrowski GS, Carey NR. Laboratory Quality Management. Chicago: ASCP Press, parameters for testing of susceptibilities of Haemophilus influenzae and Streptococcus pneumoniae to trimethoprim and trimethoprim-sulfamethoxazole. J Clin Microbiol.
9. Gottschling HD, Reuter W, Ronquist G, et al. Multicentre evaluation of a non-wipe system for the rapid determination of total cholesterol in capillary blood, Accutrend 2. Houdijk WP. Warning: Simplate II, lack of standardization in standardized bleeding cholesterol on Accutrend GC. Eur J Clin Chem Clin Biochem. 1995;33:373-381. time devices—reply: Lot-to-lot variation in ejection force has been rectified. Thromb 10. Boerma GJ, Gelderland J, van Gorp I, et al. Use of the Reflotron system for Haemost. 1991;66:625-626. cholesterol assay in capillary blood, venous blood, and serum—evaluation of 3. Anonymous. COULTER lyse S III diff: Reagent lot gives erroneous results. Health accuracy and lot-to-lot reagent comparability. Clin Chem. 1988;34:2117-2119.
11. Hill SA, Heathcote JC, McQueen MJ. The significance of matrix effects on the 4. Carey RN, Frye RM, Cook JD, et al. Between-lot/between-instrument variations of measurement of lactate dehydrogenase (LD) activity using Kodak dry slide the Abbott IMx method for prostate-specific antigen. Clin Chem. 1992;38:2341- technology in the Ontario Laboratory Proficiency Testing Program. Clin Biochem.
5. Wener MH, Daum PR, Brawer MK. Variation in measurement of prostate-specific 12. Department of Health & Human Services Health Care Financing antigen: importance of method and lot variability. Clin Chem. 1995;41:1730-1737.
Administration Public Health Service 42 CFR Part 405, et al. Clinical 6. Dufour DR. Lot-to-lot variation in anti-hepatitis C signal-to-cutoff ratio. Clin Laboratory Improvement Amendments of 1988; final rule. Federal Register Chem. 2004;50:958-960. Friday February 28, 1992.
7. Goodman DB, Bulley M, Hendricks M, et al. Assessment of the Abbott IMx assay 13. Ricos C, Alvarez V, Cava F, et al. Current databases on biologic variation: pros, system for the measurement of human chorionic gonadotropin levels in the cons and progress. Scand J Clin Lab Invest. 1999;59:491-500. (updated in treatment of ectopic pregnancy. Arch Pathol Lab Med. 1993;117:701-703. June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE

Source: http://eqas.ir/pdf/validating%20new%20reagent.pdf