Traditional Software Testing
Traditionally, the following tests are made for each new version of the software:
- Situations that can be calculated by hand covering all separate attenuation terms
- Situations for regression and stability tests covering small and large models
- Situations for checking solved errors
- Situations for new modelling functionality
This procedure has always been followed for each product release of all Brüel & Kjær software.
The Nordtest Method
A software certification process is described in Nordtest Method Framework for the Verification of Environmental Noise Calculation Software ACOU107 (2001). First, the calculation method is developed along with a set of test cases, with certified results and accepted tolerances. Once the new method has been implemented in software, the manufacturer fills out the Result Comparison Form and a Declaration of Conformity and submits the forms to the certifying authority (for example, the National Environmental Protection Agency in the country of the software developer). The manufacturer encloses documentation of the calculations performed or states that the authority will receive such documentation upon request. This documentation shall ensure that the calculation results are fully traceable. The manufacturer declares that unless explicitly specified in the Result Comparison Form, the software is applicable to any situation covered by the calculation method. If the authorities find the results acceptable, the software is certified for use (in this case, in all Nordic countries), the software manufacturer is permitted to label the software as being compliant with the method and the Result Comparison Form is then made public. The procedure is repeated for every new release of the software.
The Nordtest Method is an important development in ensuring the quality of independent implementations of methods by verifying that environmental noise calculation software conforms to calculation methods. However, there are difficulties in implementing the method for the quality assurance of software, making it difficult to determine the quality of the implementation of the method used in the software. Even so, the Nordtest Method, together with traditional software testing, helps to assure the quality of software calculations, and forms the backbone of quality assurance of Brüel & Kjær environmental noise calculation software.
Diagram of the Nordtest Method framework for the Verification of Environmental Noise Calculation Software ACOU107 (2001).
Click on image for a larger diagram