Quick™ ARSENIC TEST KIT: Verification Report

July 2002Environmental TechnologyVerification Report QUICK™ ARSENIC TEST KITPrepared by BattelleUnder a cooperative agreement with U.S. Environmental Protection AgencyJuly 2002Environmental Technology VerificationReportETV Advanced Monitoring Systems CenterQuick™ Arsenic Test KitbyAdam AbbgyThomas KellyCharles LawrieKaren RiggsBattelleColumbus, Ohio 43201iiNoticeThe U.S. Environmental Protection Agency (EPA), through its Office of Research andDevelopment, has financially supported and collaborated in the extramural program describedhere. This document has been peer reviewed by the Agency and recommended for public release.Mention of trade names or commercial products does not constitute endorsement orrecommendation by the EPA for use.iiiForewordThe U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting thenation’s air, water, and land resources. Under a mandate of national environmental laws, theAgency strives to formulate and implement actions leading to a compatible balance betweenhuman activities and the ability of natural systems to support and nurture life. To meet thismandate, the EPA’s Office of Research and Development provides data and science support thatcan be used to solve environmental problems and to build the scientific knowledge base neededto manage our ecological resources wisely, to understand how pollutants affect our health, and toprevent or reduce environmental risks. The Environmental Technology Verification (ETV) Program has been established by the EPA toverify the performance characteristics of innovative environmental technology across all mediaand to report this objective information to permitters, buyers, and users of the technology, thussubstantially accelerating the entrance of new environmental technologies into the marketplace.Verification organizations oversee and report verification activities based on testing and qualityassurance protocols developed with input from major stakeholders and customer groupsassociated with the technology area. ETV consists of six environmental technology centers.Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/. Effective verifications of monitoring technologies are needed to assess environmental quality andto supply cost and performance data to select the most appropriate technology for that assess-ment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA fundingand support to plan, coordinate, and conduct such verification tests for “Advanced MonitoringSystems for Air, Water, and Soil” and report the results to the community at large. Informationconcerning this specific environmental technology area can be found on the Internet athttp://www.epa.gov/etv/centers/center1.html.ivAcknowledgmentsThe authors wish to acknowledge the support of all those who helped plan and conduct theverification test, analyze the data, and prepare this report. In particular we would like to thankA. J. Savage, Raj Mangaraj, Daniel Turner, and Bea Weaver of Battelle. We also acknowledge theassistance of AMS Center stakeholders Vito Minei, Dennis Goldman, Geoff Dates, and MartyLink, who reviewed the test/QA plan and verification reports.vContentsNotice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiForeword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii1. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. Technology Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23. Test Design and Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.2 Test Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.3 Test Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.3.1 QC Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.3.2 PT Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.3.3 Environmental Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.4 Reference Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.5 Verification Schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74. Quality Assurance/Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1 QC for Reference Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 Audits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2.1 Performance Evaluation Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2.2 Technical Systems Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2.3 Audit of Data Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.3 QA/QC Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.4 Data Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125. Statistical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.2 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.3 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.4 Method Detection Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.5 Matrix Interference Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16vi5.6 Operator Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165.7 Rate of False Positives/False Negatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166. Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.2 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.3 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.4 Method Detection Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.5 Matrix Interference Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.6 Operator Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.7 Rate of False Positives/False Negatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.8 Other Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.8.1 Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.8.2 Data Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317. Performance Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34FiguresFigure 2-1. Industrial Test Systems, Inc., Quick™ Arsenic Test Kit . . . . . . . . . . . . . . . . . . . . . . 2Figure 6-1. Comparison of Quick™ Test Kit to Reference Method Results from PT Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26TablesTable 3-1. Test Samples for Verification of the Quick™ Test Kit . . . . . . . . . . . . . . . . . . . . . . . . 5Table 3-2. Schedule of Verification Test Days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Table 4-1. Reference Method QCS Analysis Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Table 4-2. Reference Method LFML Analysis Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Table 4-3. Reference Method Duplicate Analysis Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Table 4-4. Reference Method PE Audit Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Table 4-5. Summary of Data Recording Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Table 6-1a. Results from Laboratory Performance Test Sample Analyses . . . . . . . . . . . . . . . . 18viiTable 6-1b. Results from Drinking Water Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Table 6-1c. Results from Freshwater Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Table 6-2a. Accuracy of the Quick™ Test Kit with Laboratory Performance Test Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Table 6-2b. Accuracy of the Quick™ Test Kit with Drinking Water Samples . . . . . . . . . . . . . . 22Table 6-2c. Accuracy of the Quick™ Test Kit with Freshwater Samples . . . . . . . . . . . . . . . . . . 23Table 6-3. Summary of Qualitative Accuracy Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Table 6-4a. Precision Results for Quick™ Test Kit from Laboratory Performance Test Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Table 6-4b. Precision Results for Quick™ Test Kit from Drinking Water Samples . . . . . . . . . . 26Table 6-5a. Results from Laboratory Performance Test Sampleswith Low-Level Interferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Table 6-5b. Results from Laboratory Performance Test Samples withHigh-Level Interferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Table 6-6. Rate of False Positives from Quick™ Test Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Table 6-7. Rate of False Negatives from Quick™ Test Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30viiiList of AbbreviationsAMS Advanced Monitoring SystemsASTM American Society for Testing and MaterialsDW drinking fountain waterEPA U.S. Environmental Protection AgencyETV Environmental Technology VerificationFW freshwaterHDPE high-density polyethyleneHI high interferenceICPMS inductively coupled plasma mass spectrometryLBC Little Beaver CreekLC Lytle CreekLFM laboratory-fortified matrixLI low interferenceMDL method detection limitNIST National Institute of Standards and Technologyppb parts per billionppm parts per millionPE performance evaluationPT performance testQA quality assuranceQA/QC quality assurance/quality controlQC quality controlQCS quality control standardQMP Quality Management PlanRB reagent blankRPD relative percent differenceRSD relative standard deviationSR Stillwater RiverTSA technical systems auditTW treated well waterWW well water1Chapter 1BackgroundThe U.S. Environmental Protection Agency (EPA) has created the Environmental TechnologyVerification (ETV) Program to facilitate the deployment of innovative environmental tech-nologies through performance verification and dissemination of information. The goal of the ETVProgram is to further environmental protection by substantially accelerating the acceptance anduse of improved and cost-effective technologies. ETV seeks to achieve this goal by providinghigh-quality, peer-reviewed data on technology performance to those involved in the design,distribution, financing, permitting, purchase, and use of environmental technologies.ETV works in partnership with recognized testing organizations; with stakeholder groupsconsisting of buyers, vendor organizations, and permitters; and with the full participation ofindividual technology developers. The program evaluates the performance of innovative tech-nologies by developing test plans that are responsive to the needs of stakeholders, conductingfield or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance(QA) protocols to ensure that data of known and adequate quality are generated and that theresults are defensible. The EPA’s National Exposure Research Laboratory and its verification organization partner,Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Centerrecently evaluated the performance of four portable analyzers for arsenic in water. Thisverification report presents the procedures and results of the verification test for Industrial TestSystems, Inc., Quick™ test kit arsenic analysis systems. The Quick™ test kit is an inexpensive,portable, rapid device designed for on-site analysis of arsenic in water.2Figure 2-1. Industrial Test Systems, Inc.,Quick™ Arsenic Test KitChapter 2Technology DescriptionThe objective of the ETV AMS Center is to verify the performance characteristics ofenvironmental monitoring technologies for air, water, and soil. This verification report providesresults for the verification testing of the Quick™ test kit for arsenic in water. Following is adescription of the test kit, based on information provided by the vendor. The informationprovided below was not verified in this test.The Quick™ test kit can be used to test for totalarsenic in water. Up to 2.0 mg/L of hydrogensulfide is tolerated without test result interference,and up to 5 parts per million (ppm) of antimony istolerated. The Quick™ test kit consists primarily oftwo reaction bottles, two caps for holding the teststrip, three spoons, three bottles of reagent, andone bottle of arsenic test strips in a waterproof,plastic case. The three reagents are addedsequentially to the water sample and shaken. A teststrip is placed into the turret of the cap. The teststrip is exposed to arsine gas evolved from thesample solution, resulting in a color change in thetest strip. When the reaction is complete, the teststrip is compared with a color chart provided with the kit. The intensity of the yellow/brown colordeveloped on the test strip relative to the color chart is proportional to the arsenic concentrationin the sample and, therefore, provides a semi-quantitative analysis of the arsenic concentration.The color chart consists of the gradations: 0, 5, 10, 20, 40, 60, 100, 200, 300, and 500 parts perbillion (ppb). The kits are available in three sizes: for two tests, 50 tests, or 100 tests.3Chapter 3Test Design and Procedures3.1 IntroductionThis verification test was conducted according to procedures specified in the Test/QA Plan forVerification of Portable Analyzers.(1) The verification was based on comparing the arsenic resultsfrom the Quick™ test kit to those from a laboratory-based reference method. The referencemethod for arsenic analysis was inductively coupled plasma mass spectrometry (ICPMS),performed according to EPA Method 200.8(2) The Quick™ test kit does not require calibration,but relies on comparisons to a color chart provided with the test kit to allow semi-quantitativemeasurements of arsenic concentrations. The test kit was verified by analyzing laboratory-prepared performance test samples, treated and untreated drinking water, and fresh surface water,with both the test kit and the reference method.3.2 Test DesignThe Quick™ test kit was verified in terms of its performance on the following parameters:# Accuracy# Precision# Linearity# Method detection limit (MDL)# Matrix interference effects# Operator bias# Rate of false positives/false negatives.All preparation and analyses were performed according to the manufacturer’s recommended pro-cedures. Results from the Quick™ test kit were recorded manually. The results from the Quick™test kits were compared to those from the reference method to assess accuracy, linearity, anddetection limit. Multiple aliquots of performance test samples and drinking water samples wereanalyzed to assess precision. Identical sets of samples were analyzed independently by two separate operators (a technical anda non-technical Battelle staff member). The technical operator was a research technician atBattelle with three years of laboratory experience and a B.S degree. The non-technical operator4was a part-time temporary helper at Battelle with a general education development certificate.Because the reagents of the Quick™ test kits are consumed in use, it was not feasible for the twooperators to switch kits as a means of quantitatively assessing operator bias. However, eachoperator used multiple kits in order to analyze all the samples, so it is assumed that kit-to-kitvariability was similar for both operators. Consequently, qualitative observations could be madeon operator bias.Matrix interference effects were assessed by challenging the test kit with performance testsamples of known arsenic concentrations containing both low-level and high-level interferences.False positives and negatives were evaluated relative to the recently established 10-ppb maximumcontaminant level for arsenic in drinking water. In addition to the analytical results, the timerequired for sample analysis and operator observations concerning the use of the test kit (e.g.,frequency of calibration, ease of use, maintenance) were recorded.In a few instances, the test kit operator interpolated between the test kit gradations in reporting anarsenic value. This is not unusual in use of such kits, and typically resulted in an arsenic readingmidway between two gradation values (e.g., 30 ppb, between gradations of 20 and 40 ppb).3.3 Test SamplesThree types of samples were used in the verification test, as shown in Table 3-1: quality control(QC) samples, performance test (PT) samples, and environmental water samples.The QC and PT samples were prepared from National Institute of Standards and Technology(NIST) traceable purchased standards. Under the Safe Drinking Water Act, the EPA lowered themaximum contaminant level for arsenic from 50 ppb to 10 ppb, effective in January 2006.Therefore, the QC sample concentrations for arsenic were targeted at that 10-ppb level. The PTsamples were targeted to range from 10% to 1,000% of that level, i.e., from 1 to 100 ppb. Theenvironmental water samples were collected from various drinking water and surface watersources. All samples were analyzed using the Quick™ test kits and a reference method. Everytenth sample was analyzed twice by the reference method to document the reference method’sprecision.3.3.1 QC SamplesAs Table 3-1 indicates, prepared QC samples included laboratory reagent blanks (RB),laboratory-fortified matrix (LFM) samples, and quality control samples. The RB samplesconsisted of water collected from the same tap and were exposed to handling and analysisprocedures identical to the other prepared samples. These samples were used to help ensure thatno sources of contamination were introduced during sample handling and analysis. Two types ofLFMs were prepared. The LFMF samples consisted of aliquots of environmental samples thatwere spiked in the field to increase the analyte concentration by 10 ppb of arsenic. These sampleswere analyzed by the test kits in the field both before and after spiking. The spike solution used5for the LFMF samples was prepared in the laboratory and brought to the field site. The LFMLsamples were aliquots of environmental samples that were spiked in the laboratory to increase theanalyte concentration by 25 ppb of arsenic. These samples were used to help identify whethermatrix effects influenced the reference method results. At least 10% of all the prepared samplesanalyzed were RBs, and at least one sample taken from each sampling site was an LFMF.Table 3-1. Test Samplesa for Verification of the Quick™ Test KitType of Sample Sample Characteristics Concentration No. ofSamples Quality ControlReagent Blank (RB)b ~ 0 10% of allLaboratory Fortified Mixture (LFMF)b 10 ppb above native level 1 per siteLFMLb 25 ppb above native level 6Quality Control Sample (QCS)b 10 ppb 10% of all Performance TestPrepared arsenic solution (PT6) 25 ppb 7Prepared arsenic solution (PT1) 1 ppb 4Prepared arsenic solution (PT2) 3 ppb 4Prepared arsenic solution (PT3) 10 ppb 4Prepared arsenic solution (PT4) 30 ppb 4Prepared arsenic solution (PT5) 100 ppb 4Prepared arsenic solution spiked with interference (LI)10 ppb with lowinterference 8Prepared arsenic solution spiked with interference (HI)10 ppb with highinterference8Environmental Columbus municipal drinking water(DW) Unknown4Well water (WW) Unknown 4Treated well water (TW) Unknown 4Stillwater River (SR) Unknown 4Lytle Creek (LC) Unknown 4Little Beaver Creek (LBC) Unknown 4a Listing is for clarity; samples were analyzed in random order for the verification testing.b See Section 3.3.1 for descriptions of these samples. 6Quality control standards (QCS) were used as calibration checks to verify that the Quick™ testkit reference instrument was properly calibrated and reading within defined control limits. Thesestandards were purchased from a commercial supplier and were subject only to dilution asappropriate. Calibration of the test kit and the reference instrument was verified using a QCSbefore and after the testing period, as well as after every tenth sample. An additional independentQCS was used in a performance evaluation (PE) audit of the reference method.3.3.2 PT SamplesThe two types of PT samples used in this verification test (Table 3-1) were prepared in thelaboratory using tap water as the water source. One type of PT solution contained arsenic atvarious concentrations and was prepared specifically to determine Quick™ test kit accuracy,linearity, and detection limit. To determine the detection limit of the Quick™, a solution with aconcentration five times the vendor’s estimated detection limit was used. Seven non-consecutivereplicate analyses of this 25-ppb arsenic solution were made to obtain precision data with whichto estimate the MDL. Five other solutions were prepared to assess the linearity over a 1- to100-ppb range of arsenic concentrations. Four aliquots of each of these solutions were pre-pared and analyzed separately to assess the precision of the test kit, as well as the linearity. The second type of PT sample was used to assess the effects of matrix interferences on the per-formance of the Quick™ test kit. These samples were solutions with known concentrations ofarsenic spiked with potentially interfering species likely to be found in typical water samples. Onesample (designated LI) contained low levels of interferences that consisted of 1 ppm of iron,3 ppm of sodium chloride, and 0.1 ppm of sulfide per liter at a pH of 6. The second sample(designated HI) contained high levels of interferences that consisted of 10 ppm of iron, 30 ppm ofsodium chloride, and 1.0 ppm of sulfide per liter at a pH of 3. Eight replicate samples of each ofthese solutions were analyzed.3.3.3 Environmental SamplesDrinking water samples listed in Table 3-1 include Columbus municipal water collected from aBattelle drinking fountain (DW), well water (WW), and treated well water (TW) from a schoolnear Columbus, Ohio. The WW was pumped from a 250-foot well and collected directly from anexisting spigot with no purging. The TW was treated by running the WW through a Greensandfiltration system in the basement of the school. These samples were collected directly from thetap into 2-L high-density polyethylene (HDPE) containers. Four aliquots of each sample wereanalyzed in the field at the time of collection by each set of the test kits being verified. Onealiquot of each sample was preserved with nitric acid and returned to Battelle for referenceanalysis. The remaining collected sample was stored at 4°C for later use, if necessary.Freshwater (FW) samples from the Stillwater River (SR), Lytle Creek (LC), and the Little BeaverCreek (LBC) (in Ohio) were collected in 2-L HDPE containers. The samples were collected nearthe shoreline by submerging the containers no more than one inch below the surface of the water.Each body of water was sampled at four distinct locations. An aliquot of each sample was7analyzed in the field at the time of collection by each set of each test kit being verified. Onealiquot of each sample was preserved with nitric acid and returned to Battelle for referenceanalysis. The remaining collected sample was preserved and stored at 4°C for later use, ifnecessary.3.4 Reference AnalysisThe reference arsenic analysis was performed using a Perkin Elmer Sciex Elan 6000 ICPMSaccording to EPA Method 200.8, Revision 5.5.(2) The sample was introduced through a peristalticpump by pneumatic nebulization into a radiofrequency plasma where energy transfer processescause desolvation, atomization, and ionization. The ions were extracted from the plasma througha pumped vacuum interface and separated on the basis of their mass-to-charge ratio by aquadrupole mass spectrometer. The ions transmitted through the quadrupole were registered by acontinuous dynode electron multiplier, and the ion information was processed by a data handlingsystem.The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using aminimum of five calibration standards at concentrations ranging between 0.1 and 250 ppb, and arequired correlation coefficient of a minimum of 0.999. Internal standards were used to correctfor instrument drift and physical interferences. These standards were introduced in line throughthe peristaltic pump and analyzed with all blanks, standards, and samples. 3.5 Verification ScheduleThe verification test took place over a 19-day period from October 25 to November 12, 2001. Theenvironmental samples were collected and analyzed over the seven-day period from November 2through November 8, 2001. Table 3-2 shows the daily testing activities that were conductedduring these periods. In all field locations, the samples were analyzed shortly after collectionusing the Quick™ test kit by both the technical and the non-technical Battelle staff member. Thereference analyses on all samples were performed on December 21, 2001, approximately sixweeks after sample collection.8Table 3-2. Schedule of Verification Test DaysTest Day Testing Location Activity10/25-11/12/01 Battelle Preparation and analysis of PT and associated QCsamples.10/25/01 Battelle Collection and analysis of DW and associated QCsamples within Battelle.11/02/01 Ohio Field Location Collection and analysis of WW samples, TW samples,and associated QC samples at Licking Valley MiddleSchool.11/06/01 Ohio Field Location Collection and analysis of environmental andassociated QC samples at four locations on LittleBeaver Creek.11/07/01 Ohio Field Location Collection and analysis of environmental andassociated QC samples at four locations on LytleCreek.11/08/01 Ohio Field Location Collection and analysis of environmental andassociated QC samples at four locations on theStillwater River.9Chapter 4Quality Assurance/Quality ControlQuality assurance/quality control (QA/QC) procedures were performed in accordance with thequality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verificationtest.(1) 4.1 QC for Reference MethodField and laboratory RB samples were analyzed to ensure that no sources of contamination werepresent. The test/QA plan stated that if the analysis of an RB sample indicated a concentrationabove the MDL for the reference instrument, any contamination source was to be corrected andproper blank readings achieved before proceeding with the verification test. A total of three fieldRB and one laboratory RB were analyzed. All of the blanks analyzed were below the 0.1-ppbreference MDL for arsenic. The instrument used for the reference method was initially calibrated using 11 calibrationstandards, with concentrations ranging between 0.1 and 250 ppb of arsenic. The accuracy of thecalibration also was verified after the analysis of every 10 samples by analyzing a 25-ppb QCS. Ifthe QCS analysis differed by more than ±10% from the true value of the standard, the instrumentwas recalibrated before continuing the test. As shown in Table 4-1, the QCS analyses werealways within this required range. The maximum bias from the standard in any QCs analysis was6.04%.LFML samples were analyzed to assess whether matrix effects influenced the results of thereference method. The percent recovery (R) of these LFML samples was calculated from thefollowing equation:(1)RC Css=-´ 100where Cs is the analyzed concentration of the spiked sample, C is the analyzed concentration ofthe unspiked sample, and s is the concentration equivalent of the analyte spike. If the percentrecovery of an LFML fell outside the range from 85 to 115%, a matrix effect was suspected. Asshown in Table 4-2, all of the LFML sample results were well within this range, so no matrixeffect on the reference analyses is inferred.10RPDC CDC CD=-+´( )( ) / 2100 (2)Table 4-1. Reference Method QCS Analysis ResultsSample ID Date of AnalysisMeasuredArsenic (ppb)ActualArsenic (ppb) Percent BiasQCS 12/21/2001 24.1 25.0 3.56%QCS 12/21/2001 23.5 25.0 6.04%QCS 12/21/2001 23.8 25.0 4.64%QCS 12/21/2001 23.9 25.0 4.32%QCS 12/21/2001 24.4 25.0 2.52%Table 4-2. Reference Method LFML Analysis ResultsLFMLSample IDDate of AnalysisUnspiked SampleArsenic(ppb)Spiked SampleArsenic (ppb)Spiked AmountArsenic (ppb)PercentRecoveryLaboratory RB 12/21/01 <0.1 23.8 25.0 95.3%Field QCS 12/21/01 10.9 35.7 25.0 99.0%DW LFMF 12/21/01 10.6a 34.6 25.0 96.2%LBC 3 Duplicate 12/21/01 2.26 26.6 25.0 97.5%LC 4 12/21/01 1.37 26.3 25.0 99.7%SR 4 12/21/01 1.88 26.4 25.0 98.0%a Amount of arsenic in the sample after it was spiked in the field.Duplicate samples were analyzed to assess the precision of the reference analysis. The relativepercent difference (RPD) of the duplicate sample analysis was calculated from the followingequation:Where C is the concentration of the sample analysis, and CD is the concentration of the sampleduplicate analysis. If the RPD was greater than 10%, the instrument was recalibrated beforecontinuing the test. As shown in Table 4-3, the RPDs for the duplicate analysis were all less than10%. The maximum RPD in any duplicate analysis was 4%.11Table 4-3. Reference Method Duplicate Analysis ResultsSample ID Date of AnalysisSample Arsenic(ppb)DuplicateSample Arsenic(ppb) RPDPT QCS 12/21/2001 9.80 9.81 0%PT1 (tap) 12/21/2001 1.76 1.76 0%WW 1 12/21/2001 86.6 86.1 1%LBC 4 12/21/2001 2.54 2.44 4%SR QCS 12/21/2001 9.33 9.37 0%4.2 Audits4.2.1 Performance Evaluation AuditA PE audit was conducted to assess the quality of the reference measurements made in thisverification test. For the PE audit, an independent, NIST-traceable, reference material wasobtained from a different commercial supplier than the calibration standards and the field QCS.The PE standard was prepared from Claritas PPT™ Grade Standard purchased through SPEXCertiPrep. Accuracy of the reference method was determined by comparing the measured arsenicconcentration using the verification test standards to those obtained using the independentlycertified PE standard. Percent difference was used to quantify the accuracy of the results.Agreement of the standard within 10% was required for the measurements to be consideredacceptable. Failure to achieve this agreement would have triggered recalibration of the referenceinstrument with the original QC standards and a repeat of the PE comparison. As shown in Table4-4, the PE sample analysis was well within this required range.Table 4-4. Reference Method PE Audit ResultsSample IDDate of AnalysisMeasuredArsenic(ppb)Actual ConcentrationArsenic(ppb)Percent AgreementPE-1 12/21/01 23.7 25.0 5.2%4.2.2 Technical Systems AuditThe Battelle Quality Manager conducted a technical systems audit (TSA) between October 22and December 21, 2001, to ensure that the verification test was being performed in accordancewith the test/QA plan(1) and the AMS Center QMP.(3) The standard solution preparation and PTsample preparation were observed on October 22, the environmental testing (drinking water) on12October 25, the testing with PT samples on October 26, and the reference method performanceon December 21. As part of the audit, the reference standards and method used were reviewed,actual test procedures were compared to those specified in the test/QA plan, and data acquisitionand handling procedures were reviewed. Observations and findings from this audit were docu-mented and submitted to the Verification Test Coordinator for response. No findings weredocumented that required any corrective action. The records concerning the TSA arepermanently stored with the Battelle Quality Manager.4.2.3 Audit of Data QualityAt least 10% of the data acquired during the verification test was audited. Battelle’s QualityManager traced the data from the initial acquisition, through reduction and statistical analysis, tofinal reporting, to ensure the integrity of the reported results. All calculations performed on thedata undergoing the audit were checked. 4.3 QA/QC ReportingEach assessment and audit was documented in accordance with Sections 3.3.4 and 3.3.5 of theQMP for the ETV AMS Center.(3) Once the assessment report was prepared, the Verification TestCoordinator ensured that a response was provided for each adverse finding or potential problemand implemented any necessary follow-up corrective action. The Battelle Quality Managerensured that follow-up corrective action was taken. The results of the TSA and the audit of dataquality were sent to the EPA.4.4 Data ReviewRecords generated in the verification test received a one-over-one review within two weeks ofgeneration before these records were used to calculate, evaluate, or report verification results.Table 4-5 summarizes the types of data recorded. The review was performed by a Battelletechnical staff member involved in the verification test, but not the staff member that originallygenerated the record. The person performing the review added his/her initials and the date to ahard copy of the record being reviewed.13Table 4-5. Summary of Data Recording ProcessData to beRecordedResponsiblePartyWhereRecordedHow OftenRecordedDisposition of DataaDates, times oftest eventsBattelle Laboratoryrecord booksor ETV fielddata sheetsStart/end of testeventUsed toorganize/check testresults; manuallyincorporated in dataspreadsheets asnecessaryTest parameters(temperature,analyte/interferantidentities, andQuick™ test kitresults)Battelle Laboratoryrecord booksor ETV fielddata sheetsWhen set orchanged, or asneeded to documenttestUsed toorganize/check testresults, manuallyincorporated in dataspreadsheets asnecessaryReference methodsample analysis,chain of custody,and resultsBattelle Laboratoryrecord books,data sheets, ordataacquisitionsystem, asappropriateThroughout samplehandling and analysisprocessTransferred tospreadsheetsa All activities subsequent to data recording are carried out by Battelle.14Chapter 5Statistical MethodsThe statistical methods presented in this chapter were planned for verifying the performancefactors listed in Section 3.2. In a few cases, qualitative comparisons are reported.5.1 AccuracyWhen possible, accuracy was assessed relative to the results obtained from the referenceanalyses. Samples were analyzed by both the reference method and the test kit being verified. Foreach sample, accuracy was expressed in terms of a relative bias (B) as calculated from thefollowing equation:(3)BdCR= ´ 100where d is the difference between the reading from the Quick™ test kit and that from thereference method, and CR is the reference measurement. Because of the semi-quantitative nature of the visual test kit results, it was not possible to makethis determination for many of the results. For this reason, all of the data also were judged by aqualitative measure that was not specified in the test/QA plan. If the result from the test kit agreedwithin 25% of the reference result, the measurement was considered accurate; if it did not, themeasurement was considered not to be accurate. The percentage of accurate measurements wasdetermined for each of the three types of water samples as calculated from the followingequation:AYT= ´ 100 (4)where A is the percent of accurate measurements, Y is the number of measurements within the±25% criterion, and T is the total number of measurements. The criterion of 25% for agreementwas based on the measurement resolution of the several portable arsenic analyzers tested and onscientific judgment of the required degree of accuracy for these analyzers. Readings below the15detection limit (e.g., <10 ppb) were judged to be in agreement with the reference result if thereference value was in the specified “less than” range.5.2 PrecisionWhen possible, the standard deviation (S) of the results for the replicate samples was calculatedand used as a measure of Quick™ test kit precision at each concentration. ( )SnC Ckkn=--éëêêùûúú=å1 1211 2/(5)where n is the number of replicate samples, Ck is the concentration measured for the kth sample,and is the average concentration of the replicate samples. The instrumental precision at eachCconcentration was reported in terms of the relative standard deviation (RSD), e.g.,(6)RSDSC= ´1005.3 LinearityLinearity was assessed by linear regression of Quick™ test kit results against the referenceresults, with linearity characterized by the slope, intercept, and correlation coefficient (r).Linearity was tested using PT samples over the range 1 to 100 ppb of arsenic. The color chart forthe Quick™ test kit has a range of concentration from 5 to 500 ppb. If the concentration ofarsenic for any sample is greater than 500 ppb, a smaller sample size can be used to extend thelinearity beyond 500 ppb.5.4 Method Detection LimitThe MDL for the Quick™ test kit was assessed from the seven replicate analyses of a fortifiedsample with an analyte concentration of 25 ppb, i.e., five times the manufacturer’s estimateddetection limit of 5 ppb. The MDL was calculated from the following equation:(7)MDL t S= ´where t (= 3.14) is the Student’s t value for a 99% confidence level with n = 7, and S is thestandard deviation of the replicate samples.(4)165.5 Matrix Interference EffectsThe effect of interfering matrix species on the response of the Quick™ test kit to arsenic istypically calculated as the ratio of the difference in analytical response to the concentration ofinterfering species. For example, if adding 500 ppb of an interfering species results in a differenceof 10 ppb in the analytical result, the relative sensitivity of the test kit to that interferant would becalculated as 10 ppb/500 ppb = 2%. In this test, three interfering species were added to thesamples, all at either low or high concentrations (Section 3.3.2). Thus, it is not possible todetermine which of these compounds would be responsible for any observed interferences. Onlyqualitative observations could be made assessing whether there was a positive or negative effectdue to matrix interferences.5.6 Operator BiasTo assess operator bias for the Quick™ test kit, in all tests the results obtained from each operatorwere compiled independently and subsequently compared. However, because of the semi-quantitative nature of the test kit data and the inability of the operators to independently use thesame test kits, quantitative assessments of operator bias could not be made. Qualitativeobservations were made concerning the results from the two operators.5.7 Rate of False Positives/False NegativesThe rates of false positives and false negatives of the Quick™ test kit were assessed relative to the10-ppb target arsenic level. A false positive result is defined as any result reported to be equal toor greater than the guidance level (10 ppb) and greater than 125% of the reference value, whenthe reference value is less than that guidance level. Similarly, a false negative result is defined asany result reported below the guidance level and less than 75% of the reference value, when thereference value is greater than that guidance level. The rates of false positives and false negativeswere expressed as a percentage of total samples analyzed for each type of sample.17Chapter 6Test ResultsThe results of the verification test of the Quick™ test kits are presented in this section. 6.1 AccuracyTables 6-1a-c present the measured arsenic results from analysis of the PT, drinking water, andFW samples, respectively. Both reference analyses and Quick™ test kit results are shown in thetables, and Quick™ test kit results are shown for both the technical and non-technical operators.Some Quick™ test kit results could not be distinguished from blank sample results and wereassigned a value of <5 ppb.The field spike results indicate apparent inconsistencies in some of the spike concentrations. TheWW LFMF and LBC-4 LFMF samples apparently were not spiked in the field and the TW LFMFsample may have been spiked twice. However, these spiking errors have no effect on theusefulness of the data.Tables 6-2a-c show the percent accuracy of the Quick™ test kit results. Shown in the second andthird columns in each of Tables 6-2a-c are the percent bias values determined according toEquation 3, in Section 5.1. Bias was not calculated for values reported as <5 ppb. The percentbias values that are shown in Tables 6-2a-c range from 8 to 83% for the non-technical operatorand 8 to 84% for the technical operator for the PT samples, 8 to 92% for the non-technicaloperator and 8 to 54% for the technical operator for the drinking water samples, and 2 to 320%for both the non-technical operator and for the technical operator for the FW samples. In general,the larger bias values were associated with lower arsenic concentrations.In addition to the quantitative bias results, the qualitative accuracy was compared usingEquation 4 in Section 5.1. The fourth and fifth columns in Tables 6-2a-c show the assignment ofeach Quick™ test kit result, in terms of whether that result fell within 25% of the reference value,or within a corresponding “less-than” range. The results of this qualitative evaluation of accuracyare shown in Table 6-3, which lists the overall percent of results meeting the qualitative accuracycriteria for each operator and sample type. Table 6-3 shows that the qualitative accuracy for theQuick™ test kit for the PT samples was 71% for the non-technical operator and 55% for thetechnical operator. The qualitative accuracy for the drinking water samples was 57% for the non-technical operator and 52% for the technical operator. The qualitative accuracy for the FW18Table 6-1a. Results from Laboratory Performance Test Sample AnalysesSampleNon-Technical Arsenic (ppb)Technical Arsenic (ppb)Reference MethodaArsenic (ppb)Laboratory RB <5 <5 <0.1Laboratory RB NA <5 <0.1Laboratory RB NA <5 <0.1Laboratory RB NA <5 <0.1Laboratory RB NA <5 <0.1Laboratory RB NA <5 <0.1QCS 10 20 10.9QCS 10 20 10.9QCS 10 20 10.9QCS NA 20 10.9QCS NA 20 10.9PT1-1 <5 <5 1.76PT1-2 <5 <5 1.76PT1-3 <5 <5 1.76PT1-4 <5 <5 1.76PT2-1 <5 5 3.97PT2-2 <5 5 3.97PT2-3 <5 5 3.97PT2-4 <5 5 3.97PT3-1 5 10 10.9PT3-2 10 10 10.9PT3-3 10 10 10.9PT3-4 10 10 10.9PT4-1 30 40 34.8PT4-2 30 40 34.8PT4-3 20 40 34.8PT4-4 30 20 34.8PT5-1 100 100 113PT5-2 100 100 113PT5-3 100 100 113PT5-4 100 100 113PT6-1 5 20 29.6PT6-2 5 20 29.6PT6-3 10 20 29.6PT6-4 10 20 29.6PT6-5 10 20 29.6PT6-6 10 20 29.6PT6-7 20 20 29.6a Only one aliquot of each sample was analyzed by the reference method (except for the laboratory RB). Multiplealiquots of each sample were analyzed by Quick™ test kit.NA: Not analyzed.19Table 6-1b. Results from Drinking Water AnalysesSampleNon-Technical Arsenic (ppb)Technical Arsenic (ppb)Reference MethodaArsenic (ppb)Laboratory RB <5 <5 <0.1QCS 10 10 10.9DW-1 <5 <5 0.87DW-2 <5 <5 0.87DW-3 <5 <5 0.87DW-4 <5 <5 0.87DW LFMF 5 5 10.6Laboratory RB <5 <5 <0.1QCS 10 10 10.9WW-1 100 60 86.6WW-2 60 60 86.6WW-3 60 40 86.6WW-4 60 60 86.6WW LFMF 70 60 82.1Laboratory RB <5 <5 <0.1QCS 5 10 10.9TW-1 10 40 26.0TW-2 10 40 26.0TW-3 10 40 26.0TW-4 50 40 26.0TW LFMF 40 60 50.8a Only one aliquot of each sample was analyzed by the reference method. Multiple aliquots of each sample were analyzedby Quick™ test kit.20Table 6-1c. Results from Freshwater AnalysesSampleNon-Technical Arsenic (ppb)Technical Arsenic (ppb)Reference Method aArsenic (ppb)Laboratory RB <5 <5 <0.1QCS 10 10 9.33SR-1 <5 <5 1.73SR-2 <5 5 1.72SR-2 Duplicate <5 5 1.71SR-3 <5 <5 2.03SR-4 <5 5 1.88SR-1 LFMF 10 20 11.6Laboratory RB <5 <5 <0.1QCS 10 10 9.43LC-1 <5 5 2.13LC-2 <5 5 1.30LC-3 <5 5 1.44LC-4 <5 <5 1.37LC-4 Duplicate <5 5 1.36LC-3 LFMF 10 10 12.0Laboratory RB <5 <5 <0.1QCS 10 10 9.81LBC-1 <5 5 2.48LBC-2 <5 5 2.60LBC-3 <5 <5 2.14LBC-3 Duplicate <5 <5 2.26LBC-4 <5 <5 2.54LBC-4 LFMF 10 10 2.3821Table 6-2a. Accuracy of the Quick™ Test Kit with Laboratory Performance Test SamplesSampleBiasa Non-TechnicalBiasaTechnicalWithin Range (Y/N)bNon-Technical Within Range (Y/N)bTechnical Laboratory RB -c - Y YLaboratory RB NA - YLaboratory RB NA - YLaboratory RB NA - YLaboratory RB NA - YLaboratory RB NA - YQCS 8% 84% Y NQCS 8% 84% Y NQCS 8% 84% Y NQCS NA 84% NQCS NA 84% NPT1-1 - - Y YPT1-2 - - Y YPT1-3 - - Y YPT1-4 - - Y YPT2-1 - 26% Y NPT2-2 - 26% Y NPT2-3 - 26% Y NPT2-4 - 26% Y NPT3-1 54% 8% N YPT3-2 8% 8% Y YPT3-3 8% 8% Y YPT3-4 8% 8% Y YPT4-1 14% 15% Y YPT4-2 14% 15% Y YPT4-3 43% 15% N YPT4-4 14% 43% Y NPT5-1 12% 12% Y YPT5-2 12% 12% Y YPT5-3 12% 12% Y YPT5-4 12% 12% Y YPT6-1 83% 32% N NPT6-2 83% 32% N NPT6-3 66% 32% N NPT6-4 66% 32% N NPT6-5 66% 32% N NPT6-6 66% 32% N NPT6-7 32% 32% N Na Percent bias calculated according to Equation 3, Section 5.1. b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or referencevalue not within < range. c Non-detect, no calculation of bias can be made.NA: not analyzed.22Table 6-2b. Accuracy of the Quick™ Test Kit with Drinking Water SamplesSampleBiasa Non-TechnicalBiasaTechnicalWithin Range(Y/N)bNon-TechnicalWithin Range(Y/N)bTechnicalLaboratory RB -c - Y YQCS 8% 8% Y YDW-1 - - Y YDW-2 - - Y YDW-3 - - Y YDW-4 - - Y YDW LFMF 53% 53% N NLaboratory RB - - Y YQCS 8% 8% Y YWW-1 15% 31% Y NWW-2 31% 31% N NWW-3 31% 54% N NWW-4 31% 31% N NWW LFMF 15% 27% Y NLaboratory RB - - Y YQCS 54% 8% N YTW-1 62% 54% N NTW-2 62% 54% N NTW-3 62% 54% N NTW-4 92% 54% N NTW LFMF 21% 18% Y Ya Percent bias calculated according to Equation 3, Section 5.1.b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or referencevalue not within < range. c Non-detect, no calculation of bias can be made.23Table 6-2c. Accuracy of the Quick™ Test Kit with Freshwater SamplesSampleBiasa Non-TechnicalBiasaTechnicalWithin Range(Y/N)bNon-TechnicalWithin Range(Y/N)bTechnicalLaboratory RB -c - Y YQCS 7% 7% Y YSR-1 - - Y YSR-2 - 191% Y NSR-2 Duplicate - 192% Y NSR-3 - - Y YSR-4 - 166% Y NSR-1 LFMF 14% 72% Y NLaboratory RB - - Y YQCS 6% 6% Y YLC-1 - 135% Y NLC-2 - 285% Y NLC-3 - 247% Y NLC-4 - - Y YLC-4 Duplicate - 268% Y NLC3 LFMF 17% 17% Y YLaboratory RB - - Y YQCS 2% 2% Y YLBC-1 - 102% Y NLBC-2 - 92% Y NLBC-3 - - Y YLBC-3 Duplicate - - Y YLBC-4 - - Y YLBC-4 LFMF 320% 320% N Na Percent bias calculated according to Equation 3, Section 5.1.b Y = result within ±25% of reference, or reference value within < range; N = result not within ±25% of reference, or referencevalue not within < range. c No calculation of bias can be made.Table 6-3. Summary of Qualitative Accuracy ResultsPercent Accurate Within 25%(Non-Technical Operator)Percent AccurateWithin 25%(Technical Operator)Laboratory performance test samples 71% 55%Drinking water samples 57% 52%Freshwater samples 96% 54%24samples was 96% for the non-technical operator and 54% for the technical operator. Many of theQuick™ results judged qualitatively accurate were the result of sample concentrations below themanufacturer’s estimated detection limit of 5 ppb. For the 25 samples in Tables 6-1a and b withreference arsenic values between 26 and 113 ppb, the qualitative accuracy was 40% or less withboth operators.6.2 PrecisionTables 6-4a and b, respectively, show the data used to evaluate the RSD of the Quick™ test kitresults for the replicate laboratory PT and drinking water samples, along with the percent RSDvalue for each set of replicate analysis. The percent RSD was determined according to Equation 6in Section 5.2. Percent RSD was not calculated if all of the results for a set of replicates were <5ppb. These data sets illustrate the consistency in the Quick™ test kit replicate analyses. Seven ofthe 14 replicate sets for the PT and QCS samples (Table 6-4a) showed an RSD of 0%. The resultsfor three of the replicate sets were <5 ppb. The remaining replicate sets for the non-technicaloperator had an RSD ranging from 29 to 50%, and the remaining replicate set for the technicaloperator had an RSD of 29%. For the drinking water samples (Table 6-4b), all results for two ofthe replicate sets were <5 ppb. The remaining sets had an RSD of 29 to 100% for the non-technical operator and 0 to 18% for the technical operator.6.3 LinearityThe linearity of the Quick™ test kit readings was assessed by means of a linear regression of theQuick™ test kit results against the reference method results, using the 27 data points from the PTsamples (Table 6-1a). In this regression, results reported as below detection limit by the Quick™test kit were assigned a value of half the detection limit (2.5 ppb). Figure 6-1 shows a scatter plotof the Quick™ test kit data from both non-technical and the technical operators versus thereference method results. The one-to-one line is also shown in Figure 6-1.A linear regression of the data in Figure 6-1 gives the following regression equations:with the Quick™ test kit for the non-technical operator,ppb = 0.90 (±0.086) x (reference, ppb) - 5.2 (±4.1) ppb, with r = 0.974, andwith the Quick™ test kit for the technical operator,ppb = 0.88 (±0.056) x (reference, ppb) - 0.45 (±2.7) ppb, with r = 0.988where the values in parentheses represent the 95% confidence interval of the slope and intercept.Both regressions show slopes that are significantly different from 1.0.25Table 6-4a. Precision Results for Quick™ Test Kit from Laboratory Performance TestSamples ReferenceConcentration (ppb)Non-TechnicalaArsenic (ppb)TechnicalArsenic (ppb)QCS 10.9 10 20QCS 10 20QCS 10 20QCS 20QCS 20%RSD 0 0PT1-1 1.76 <5 <5PT1-2 <5 <5PT1-3 <5 <5PT1-4 <5 <5%RSD -b -bPT2-1 3.97 <5 5PT2-2 <5 5PT2-3 <5 5PT2-4 <5 5%RSD -b 0PT3-1 10.9 5 10PT3-2 10 10PT3-3 10 10PT3-4 10 10%RSD 29 0PT4-1 34.8 30 40PT4-2 30 40PT4-3 20 40PT4-4 30 20%RSD 29 29PT5-1 113 100 100PT5-2 100 100PT5-3 100 100PT5-4 100 100%RSD 0 0PT6-1 29.6 5 20PT6-2 5 20PT6-3 10 20PT6-4 10 20PT6-5 10 20PT6-6 10 20PT6-7 20 20%RSD 50 0a For the purpose of calculating %RSD, all “less than” values are considered zero.b No %RSD could be calculated.26Table 6-4b. Precision Results for Quick™ Test Kit from Drinking Water SamplesReferenceConcentration (ppb)Non-TechnicalaArsenic (ppb)TechnicalaArsenic (ppb)DW-1 0.87 <5 <5DW-2 <5 <5DW-3 <5 <5DW-4 <5 <5% RSD -b -bWW-1 86.6 100 60WW-2 60 60WW-3 60 40WW-4 60 60% RSD 29 18TW-1 26.0 10 40TW-2 10 40TW-3 10 40TW-4 50 40%RSD 100 0a For the purpose of calculating %RSD, all “less than” values are considered zero.b No %RSD could be calculated.0204060801001201400 20 40 60 80 100 120 140Reference Arsenic, ppbQuick, ppbNon-Technical Arsenic, ppbTechnical Arsenic, ppbLinear (Non-Technical Arsenic, ppb)Linear (Technical Arsenic, ppb)Linear (1:1 Line)Figure 6-1. Comparison of Quick™ Test Kit to Reference Method Results fromPT Samples276.4 Method Detection LimitThe manufacturer’s estimated detection limit for the Quick™ test kit is 5 ppb. An attempt wasmade to determine the MDL by analyzing seven replicate samples at approximately 25 ppb (PT6samples, Table 6-1a). The Quick™ results for both operators were all less than the referencevalue, but in particular the technical operator’s results were all identical (20 ppb), providing novariation with which to quantitatively assess the MDL.(4) The non-technical operator reportedarsenic between 5 and 20 ppb. Since the Quick ™ test kit is only semi-quantitative, no MDL wascalculated from these data. Qualitative indication of the Quick™ test kit MDL can be obtainedfrom the results of the PT2 and PT3 samples (Table 6-1a) of concentrations 3.97 and 10.9 ppb,respectively. With the 3.97-ppb samples, the non-technical operator reported results of <5 ppb,whereas the technical operator reported results of 5 ppb. With the 10.9-ppb samples, all Quick™results were 10 ppb except for one result of 5 ppb with the non-technical operator.6.5 Matrix Interference EffectsTables 6-5a and b show the analytical results from laboratory performance test samplescontaining about 10.5 ppb arsenic, with low and high levels of interference, respectively. TheQuick™ test kit produced positive readings on all the matrix interference samples with bothoperators, with a small increase in readings with the higher interference levels. For example, thenon-technical operator reported 10 ppb in five of eight analyses of the LI samples, with threereadings of 5 ppb, and reported 10 ppb in seven of eight analyses, with only one reading of 5 ppb. for the HI samples. Similarly, the technical operator reported six of eight values at 10 ppb andtwo at 5 ppb with the LI samples, but five of eight at 10 ppb and three at 20 ppb with the HIsamples. These results indicate a minor tendency toward higher readings (3 ppb on average) fromthe Quick™ test kit at the higher interference levels. Because of the study design, it was notpossible to determine which ion was responsible for the observed result. Table 6-5a. Results from Laboratory Performance Test Samples with Low-LevelInterferencesNon-Technical Arsenic (ppb)Technical Arsenic (ppb)LI-1 10 5LI-2 5 10LI-3 5 10LI-4 10 10LI-5 10 10LI-6 10 5LI-7 5 10LI-8 10 10a Only one aliquot of LI solution was analyzed by the reference method. Eight aliquots of LI solution were analyzed byQuick™ test kits.28Table 6-5b. Results from Laboratory Performance Test Samples with High-LevelInterferencesNon-Technical Arsenic (ppb)Technical Arsenic (ppb)HI-1 10 10HI-2 5 10HI-3 10 10HI-4 10 20HI-5 10 10HI-6 10 20HI-7 10 10HI-8 10 20a Only one aliquot of HI solution was analyzed by the reference method. Eight aliquots of HI solution were analyzedby Quick™ test kits.6.6 Operator BiasThe effect of operator skill level does not appear to be a major factor with the Quick™ test kit.The non-technical operator had a higher percentage of accurate results, although the greaterfrequency of non-detects with the non-technical operator played a part in that outcome. On theother hand, the technical operator had fewer false positive and negative results (see Section 6.7).6.7 Rate of False Positives/False NegativesTables 6-6 and 6-7, respectively, show the data and results for the rates of false positives and falsenegatives obtained from the Quick™ test kit. All PT and environmental samples (Table 3-1) wereconsidered for this evaluation.Table 6-6 shows that 24 samples had reference arsenic concentrations less than the target decisionlevel of 10 ppb. Of the samples tested by the non-technical operator, in only one sample did theQuick™ test kit results indicate a concentration of 10 ppb or higher. The result was a falsepositive rate of 4% relative to the 10 ppb value. The samples tested by the technical operator hada false positive rate of 0%, with no Quick™ test kit results at or above the 10-ppb decision level.Table 6-7 shows that 43 samples had reference arsenic concentrations greater than the targetdecision level of 10 ppb. In seven of the 43 samples, the analyte was detected at a level less than10 ppb by the non-technical operator (i.e., a false negative rate of 16%). The technical operatorreported only two such results, for a false negative rate of 5%.29Table 6-6 Rate of False Positives from Quick™ Test KitNon-TechnicalArsenic (ppb)TechnicalArsenic (ppb)ReferenceMethod Arsenic (ppb)Non-TechnicalFalse Positive(Y/N)TechnicalFalse Positive(Y/N)PT1-1 <5 <5 1.76 N NPT1-2 <5 <5 1.76 N NPT1-3 <5 <5 1.76 N NPT1-4 <5 <5 1.76 N NPT2-1 <5 5 3.97 N NPT2-2 <5 5 3.97 N NPT2-3 <5 5 3.97 N NPT2-4 <5 5 3.97 N NDW-1 <5 <5 0.87 N NDW-2 <5 <5 0.87 N NDW-3 <5 <5 0.87 N NDW-4 <5 <5 0.87 N NSR-1 <5 <5 1.73 N NSR-2 <5 5 1.72 N NSR-3 <5 <5 2.03 N NSR-4 <5 5 1.88 N NLC-1 <5 5 2.13 N NLC-2 <5 5 1.3 N NLC-3 <5 5 1.44 N NLC-4 <5 <5 1.37 N NLBC-1 <5 <5 2.48 N NLBC-2 <5 <5 2.6 N NLBC-3 <5 <5 2.14 N NLBC-4 10 <5 2.54 Y NTotal number of applicable samples 24 24Total false positive 1 0Percent false positive 4% 0%Y = yesN = no30Table 6-7 Rate of False Negatives from Quick™ Test KitNon-TechnicalArsenic (ppb)TechnicalArsenic (ppb)Reference MethodArsenic (ppb)Non-Technical False Negative (Y/N)Technical False Negative (Y/N)PT3-1 5 10 10.9 Y NPT3-2 10 10 10.9 N NPT3-3 10 10 10.9 N NPT3-4 10 10 10.9 N NPT4-1 30 40 34.8 N NPT4-2 30 40 34.8 N NPT4-3 20 40 34.8 N NPT4-4 30 20 34.8 N NPT5-1 100 100 113 N NPT5-2 100 100 113 N NPT5-3 100 100 113 N NPT5-4 100 100 113 N NPT6-1 5 20 29.6 Y NPT6-2 5 20 29.6 Y NPT6-3 10 20 29.6 N NPT6-4 10 20 29.6 N NPT6-5 10 20 29.6 N NPT6-6 10 20 29.6 N NPT6-7 20 20 29.6 N NLI-1 10 5 10.6 N YLI-2 5 10 10.6 Y NLI-3 5 10 10.6 Y NLI-4 10 10 10.6 N NLI-5 10 10 10.6 N NLI-6 10 5 10.6 N YLI-7 5 10 10.6 Y NLI-8 10 10 10.6 N NHI-1 10 10 10.4 N NHI-2 5 10 10.4 Y NHI-3 10 10 10.4 N NHI-4 10 20 10.4 N NHI-5 10 10 10.4 N NHI-6 10 20 10.4 N NHI-7 10 10 10.4 N NHI-8 10 20 10.4 N NWW-1 100 60 86.6 N NWW-2 60 60 86.6 N NWW-3 60 40 86.6 N NWW-4 60 60 86.6 N NTW-1 10 40 26.0 N NTW-2 10 40 26.0 N NTW-3 10 40 26.0 N NTW-4 50 40 26.0 N NTotal number of applicable samples 43 43Total false negative 7 2Percent false negative 16% 5%Y = yesN = no316.8 Other FactorsThe operators felt that the Quick™ test kit was easy to use and free of maintenance. The kit islightweight, easy to transport by car, and can be carried through fields and wooded areas. Thereaction bottles, however, are tall, narrow, and lightweight, making them susceptible to fallingover with a moderate breeze. The Quick™ test kit allows two samples to be analyzed simultaneously. The total reaction time isless than 15 minutes. The reagents are ready to use and do not require preparation. Three sizes ofscoops are included in the Quick™ test kit, making it easy to add the three reagents to thesample. However, the narrow top of the reaction bottles makes it difficult to add the reagents. Thereagent bottles can be cleaned and reused. However, the operators experienced some difficultywith the reagents sticking to the reaction vessel. This can be remedied by washing in a dilute acidsolution. This kit requires no liquids or concentrated acids, making it safe and easy to carry in the field. Thesolid reagents contain no toxic materials.6.8.1 CostsThe Quick™ test kit is available in three sizes. The smallest kit costs $12.99 and is capable ofanalyzing two samples. The 50-sample test kit costs $79.99. The large kit, capable of analyzing100 samples, sells for $139.99.6.8.2 Data CompletenessAll portions of the verification test were completed, and all data that were to be recorded weresuccessfully acquired. The non-technical operator analyzed only one of the three requiredlaboratory reagent blanks, otherwise data completeness was 100%.32Chapter 7Performance SummaryAn assessment of quantitative accuracy showed that percent bias values ranged from 8 to 83%for the non-technical operator and 8 to 84% for the technical operator for the PT samples. Thepercent bias ranged from 8 to 92% for the non-technical operator and 8 to 54% for the technicaloperator for the drinking water samples. For the FW samples, the percent bias ranged from 2 to320% for both the non-technical and technical operators. An additional qualitative criterion foraccuracy was the percentage of samples for which the Quick™ test kit result was within 25% ofthe reference result or within a corresponding “less than” range. By this criterion, the Quick™test kit yielded a qualitative accuracy for the PT samples of 71% for the non-technical operatorand 55% for the technical operator. The qualitative accuracy for the drinking water samples was57% for the non-technical operator and 52% for the technical operator. The qualitative accuracyfor the freshwater samples was 96% for the non-technical operator and 54% for the technicaloperator.Percent RSD data illustrate consistency in the Quick™ test kit replicate analyses. Seven of the14 replicate sets for the PT samples showed an RSD of 0% (i.e., all replicate results wereidentical). The remaining replicate sets for the non-technical operator had an RSD ranging from29 to 50%, and the remaining replicate set for the technical operator had an RSD of 29%. For thedrinking water samples, the RSDs for the non-technical operator ranged from 29 to 100%, andthe RSDs for the technical operator ranged from 0 to 18%.The linearity of response of the Quick™ test kit was assessed using the PT samples containing2 to 112 ppb arsenic. The linear regression for the Quick™ test kit results for the non-technicaloperator was ppb = 0.90 (±0.086) x (reference, ppb) - 5.2 (±4.1) ppb, with a correlation coefficient(r) of 0.974. The corresponding equation for the results for the technical operator was ppb = 0.88(±0.056) x (reference, ppb) - 0.45 (±2.7) ppb, with a correlation coefficient (r) of 0.988.The manufacturer’s estimated detection limit for the Quick™ test kit is 5 ppb. Seven replicatesamples of 25-ppb arsenic produced Quick™ readings of 5 to 20 ppb with the non-technicaloperator and seven identical readings of 20 ppb with the technical operator. No MDL wascalculated quantitatively from these data.The Quick™ test kit showed a minor tendency toward higher readings (3 ppb on average) withhigher levels of sodium chloride, iron, sulfide, and acidity. Because of the study design, it was notpossible to determine which ion was responsible for the observed result. 33The operator skill level does not appear to be a major factor determining Quick™ test kit results.The rates of false positives and false negatives for the Quick™ test kit were assessed relative to thereference method using 10 ppb of arsenic as the decision level. The rate of false positives for theQuick™ test kit was 4% for the non-technical operator and 0% for the technical operator. The rateof false negatives was 16% for the non-technical operator and 5% for the technical operator.The Quick™ test kit is available in three sizes. The smallest is capable of analyzing two samplesand costs $12.99. The 50-sample test kit costs $79.99. The large kit, capable of analyzing100 samples, sells for $139.99. The test kit allows two samples to be analyzed simultaneously. Thetotal reaction time is less than 15 minutes. The reagents are ready to use and do not requirepreparation. Three scoop sizes are included in the Quick™ test kit, making addition of the reagentssimple, but the size and shape of the reaction vessels limit the ease of use of the test kit.34Chapter 8References1. Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2.0.2. U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes byInductively Coupled Plasma Mass Spectrometry, Revision 5.5, April 1991.3. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot,Version 2.0, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus,Ohio, October 2000.4. U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B.