WSP | Parsons Brinckerhoff

Performance-Based Measurement Systems

- By:

Courtesy of WSP | Parsons Brinckerhoff

Many of the Environmental Protection Agency's (EPA's) methods used to monitor air, water and soil for environmental contaminants prescribe step-by-step details on how the chemical analysis must be conducted and prohibit any deviations or variations. The EPA has recognized that monitoring equipment and technology have improved tremendously over the last few years; that environmental media do not always act the same way for each analysis; and that the matrix may affect the analysis and results. The EPA has been working to implement a new program, called 'Performance Based Measurement Systems (PBMS),' to allow more flexibility in its monitoring programs.

PBMS are a set of processes specifying the data quality needs, mandates, or limitations of a program or project and which serve as the criteria for selecting appropriate methods to meet those needs in a cost-effective manner. On October 6, 1997, the EPA published a Notice of Intent in the Federal Register to implement PBMS in all of its monitoring programs. When implemented, PBMS will allow the regulated community to select any appropriate analytical test method for use in meeting regulatory compliance, as long as the goals of the monitoring are met. PBMS represent a major policy shift from EPA's strict regulation of the use of analytical methods. PBMS will apply to each media monitored for compliance: drinking water, wastewater, landfills, hazardous waste sites, and air. The EPA intends that implementation of PMBS will improve data quality while advancing new monitoring technologies.

Currently, some of the EPA’s monitoring programs require the regulated community to use specific analytical methods for determining compliance. For example, many National Pollutant Discharge Elimination System permits require that EPA Method 624 (40 CFR 136 Appendix I) be used to monitor for volatile organic compounds. Under PMBS, the regulated entity would have the opportunity to choose one of the several methods that meet the goal of the testing as long as appropriate quality controls can be demonstrated.

Under PBMS, the EPA will identify appropriate performance characteristics of testing methods and will specify the quantitative criteria, such as detection limits and calibration specifications, that must be achieved, for each of those characteristics. It will not prescribe the specific procedures, techniques, methods, or instrumentation to be used. Performance criteria would be established based on the decision to be supported by the measurement process, the acceptable level of uncertainty, and the ease with which method performance can be verified. These criteria may be published in regulations or guidance documents.

Using PBMS, the regulated community will be required to demonstrate that the testing methods they use meet the criteria of the monitoring program by documenting the performance of the method. The regulated entity will also be required to maintain records of method performance and written certification that the laboratory used appropriate quality assurance/quality control procedures. PBMS will apply to most physical, chemical, and biological testing conducted in a laboratory or in the field. It will not initially apply to methods stipulated in regulations, such as the toxicity characteristic leaching procedure (TCLP) under the Resource Conservation and Recovery Act (RCRA), the five-day biological oxygen demand test under the Clean Water Act, or airborne or stationary particulate matter monitoring under the Clean Air Act. Even when PBMS are fully implemented, the EPA expects that established methods such as those found in SW-846 (EPA 1986 and subsequent) would still be widely used in Agency monitoring programs.

Several goals have been established for the use of PBMS. Such systems would emphasize the appropriate analytical chemistry needs of the monitoring project rather than requiring the laboratory to use a specific analytical instrument. The analytical chemist would be able to use professional judgment to develop and modify methods to get the best result in the most cost effective manner, rather than follow step-by-step instructions in a prescribed method. Use of PMBS relies heavily on using data quality objectives (DQOs) and method quality objectives (MQOs) to achieve method performance criteria. Use of DQOs and MQOs involves stating a set of goals for a project and then developing the appropriate sampling and monitoring systems to meet those objectives. Under the PMBS, if a regulated entity is monitoring lead in groundwater, it will not matter whether the laboratory analyzed the sample by inductively coupled plasma spectroscopy (ICP); ICP-mass spectrometry; atomic absorption spectroscopy; or ion chromatography, as long as the appropriate quality control (QC) criteria are demonstrated. These QC criteria include the relative standard deviation of the initial calibration of the analytical system; the amount the calibration is allowed to vary during use; the range of recoveries for compounds being monitored; the relative precision of duplicate samples; and the amount of contamination found in field and laboratory blanks.

PBMS will increase the flexibility that analysts need to deal with difficult matrices. Laboratories often receive samples that contain parts per billion concentrations of toxic chlorinated compounds such as vinyl chloride. The same samples may also contain part per million levels of toluene. Using approved EPA methods, under the current system, the laboratory will prescreen the sample and analyze it at a dilution appropriate to get the toluene concentration into the calibration range of the instrument. However, the vinyl chloride will not be detected at such a dilution. The detection limit for vinyl chloride winds up being above the specified cleanup standard or maximum contaminant level. This creates uncertainty whether vinyl chloride is present at a level exceeding the standard of concern. PBMS will allow chemists to develop and use appropriate analytical techniques to measure small concentrations of one compound in the presence of large concentration of another or in samples affected by matrix interference.

Implementation of PBMS

The Environmental Laboratory Advisory Board (ELAB), a federally chartered advisory committee, provides advice and counsel to the EPA, the Environmental Monitoring Management Council and the National Environmental Laboratory Accreditation Conference concerning systems and standards for accreditation of environmental laboratories. The ELAB formed a workgroup to address the issues involved with the implementation of PBMS and presented their recommendations to the EPA in January 1999. The ELAB’s recommendations on implementation of PBMS included:

The EPA should establish a consistent approach for PBMS incorporating the 'essential elements of successful PBMS.'
Each EPA program office should prepare a public report of its plans to incorporate the 'essential elements' into their PBMS implementation plans.
Critical elements of PBMS should be specifically addressed in guidance, plans, policies, and regulations.
The ELAB supports the National Environmental Laboratory Accreditation Conference incorporating PBMS into its standards for laboratory accreditation, consistent with the EPA's use of PBMS into its monitoring programs.
The ELAB (1999) divided the elements deemed essential to implementing PBMS into those considered 'critical' and those merely 'important.' They believed it was critical that data generated using PBMS have same legal standing as data generated using promulgated EPA methods. The requirements for validating PBMS methods should be consistent across all EPA programs. These should be cost effective for small laboratories, large laboratories, and instrument manufacturers. The method validation processes and supporting documentation should be based on widely accepted, sound scientific principles and on the intended use of the data.

The EPA should develop and publish PBMS performance criteria appropriate to the anticipated use of the data. These would include criteria covering precision, accuracy, selectivity, and sensitivity. The EPA should develop methods that are capable of achieving the regulatory monitoring requirements and should submit the technical studies used to develop these methods to peer review. In addition, the documentation required under PBMS must be sufficient so it can be independently verified by third party auditors.

Several elements were deemed important, but not critical, by the ELAB (1999). The implementation of PBMS should be made as simple as possible. Regulated entities needed to have the flexibility to modify methods or use new methods, as long as PBMS requirements are met. The EPA must approve proposed new analytical methods quickly and efficiently. This can only be achieved if the Agency uses consistent definitions, objectives, and criteria for aspects of PBMS among EPA program offices, EPA regions, and the states. The performance criteria must be represented by clear objectives, which are easily understood, applied, and demonstrated. The EPA must also evaluate how existing regulations and monitoring requirements will be affected.

EPA's Office of Air and Radiation (OAR) and the Office of Solid Waste and Emergency Response (OSWER) have developed and released their PBMS implementation plans. OAR programs that will be affected by PBMS include ambient air, stationary sources, fuels, engines and vehicles, acid rain, stratospheric ozone, indoor air, and radiation. The OAR plans to apply PBMS initially to the analytical sections of its methods because the sampling sections are defined by the methods, sampling affects the results, and insufficient reference materials exist to evaluate sampling. The ambient air, acid rain, and engines and vehicles programs already incorporate PBMS. The OAR expects to implement PBMS in all its monitoring programs by May of 2000.

The OSWER is responsible for monitoring under the RCRA and Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) programs. The OSWER’s implementation of PBMS will affect use of promulgated methods from SW-846 (1986 and subsequent) for RCRA and the Contract Laboratory Program (CLP) under CERCLA. The OSWER believes that obstacles to the use of PBMS must be removed. For example, a common misconception among regulators and the regulated community is that RCRA programs require SW-846 methods and CERCLA programs require CLP methods. The OSWER views SW-846 as a guidance document (except for characterizing hazardous waste). CLP methods are very rigid and do not allow any adaptability.

To further the use of PBMS in RCRA and CERCLA monitoring programs, the OSWER is working to change regulations that require the use of SW-846 methods. The OSWER is also working to change the sampling procedures in SW-846 to make them more flexible and cost effective. They also plan to revise Appendix VIII of 40 CFR 261, which lists over 480 hazardous chemicals regulated under Subpart C of RCRA. Revision of Appendix VIII will eliminate many of the inappropriate uses of this list by regulators, which have lead to unnecessary and costly analytical testing.

The OSWER plans to encourage wider use of field screening methods in the CERCLA program, including immunoassay tests (SW-846 Method 4000 series) and the use of portable instrumentation such as X-ray fluorescence and portable gas chromatographs. Additionally, emphasis will be placed on using the DQO/MQO process to develop site specific monitoring programs. The OSWER plans to implement PBMS by October 1999.

The EPA, in conjunction with independent groups such as NELAC, has been working hard to develop standards for laboratories including auditing programs, performance evaluation programs and performance criteria checklists for the implementation of PBMS. These systems will benefit environmental regulators and the regulated community. Implementing PBMS will permit flexibility in selecting testing methods; greater use of new and emerging technologies for monitoring programs; and development and use of cost-effective methods that meet regulatory requirements and method performance criteria.

Customer comments

No comments were found for Performance-Based Measurement Systems. Be the first to comment!