Most utilities carry out regular surveys of subsets of their assets, and this task is never more important than for water utilities. Knowledge of the condition and current performance of water distribution system and collection system assets is essential to maintaining service levels.
However, surveys can be problematic. They are expensive, because the activity is very people-intensive. They are prone to error, because of the processes involved in surveys and the wide range of skill levels usually involved. They are also very difficult to manage, because they are spread out geographically on site, the activity is temporary and there is little time to iron out teething problems. The key to managing surveys is asset definition and locating the parameters of the assets that are to be surveyed. The early detection and correction of problems is crucial, before the staffing has changed, or the teams have moved on to another geographic area, or even to another contract. In this document we will be looking at the ability of InfoNet to reduce the occurrence of errors in data collection.
There is, as many engineers know to their cost, almost infinite scope for errors in survey results, but here are some of the commonest:
· Simply entering the wrong number – 200 instead of 20 for example
· Entering with the wrong units – metric instead of imperial, or even centimetres instead of metres.
· A special case of this is using the wrong qualitative parameters. For example a weather condition that should be entered “very wet” may be entered “storm”, thus preventing subsequent analysis because it is not a parameter recognized by the system or does not comply with company standards.
· Missing data – not recording all data required and specified in the contract when surveying a single manhole, for example
· Entering the wrong asset ID or location of the asset being surveyed
· Missing out an asset or a group of assets completely, either because the team miss one specified asset in error, or the survey specification has failed to include all assets in the sub network defined for the survey
· Problems of skill level, where a specific team or individual lacks the skills for the task.
The process by which this data usually reaches a database is usually a combination of paper and keying into a system. Sometimes the surveyor enters data into forms, for subsequent keying or OCR (optical character recognition) for entry to the computer database. Sometimes the process is paperless and the surveyor will key the data into anything from a small intelligent device such as a PDA (personal digital assistant) or a data terminal, to a full-function laptop computer. The process of entering the data into the database may take place either quickly from the field, or after the data has been collated and sent to the central offices.
Clearly, in the above processes, there is a need for rigorous error checking at all stages. The benefit that primary error removal ensures is that the responsibility to provide accurate data is lodged with the data creators rather than the data keepers. Data keepers may not be present at the survey site and therefore will never know the correct reading.
The use of InfoNet
Increasing numbers of utilities and survey companies are using InfoNet, Wallingford Software's asset and data management solution to manage data collection survey. InfoNet's ability to monitor all data as it is input is a powerful tool in the timely identification of survey errors.
There are two levels of monitoring that can be used. The first can be called field checks.
· This is where an item of data being input into a particular field is automatically checked against a value, user-defined and contractually specified, for that particular field - at the time of data entry. The value that underlies that field, such as a range check on a number or a textual check against a pre-defined list of values are predefined prior to the start of the survey.
The second level can be called validation.
· This involves validating a data item against other data items in the dataset, or in another dataset, according to user-defined rules. An example of rule-based validation is checking that a specific dataset is a connected network rather than two or more independent networks.
The first level of checking takes place at the moment of data entry. A visual indication is given: changing the colour of the entry box either pink, yellow or purple depending on the type of error highlights a possible error for closer scrutiny. Validation is a separate exercise that takes place when a data series or dataset has been entered. This may be at the end of the shift, or the end of the day, or at any time thereafter. Sometimes different levels of validation are appropriate, with some rules being run at an early stage, and others later across bigger datasets. Validation can create a report, which can then be studied and the errors corrected immediately, or the report can be returned to the survey crew for them to clarify the situation by a further visit to the asset.
It is also important to remember that InfoNet does not overwrite existing survey data, as is the case with most asset databases. InfoNet permits multiple survey data sets for the same asset, so the survey figures are simply an addition, not a replacement, and are defined by parameters of date, who carried the survey, and any other parameters required – the format and content is user-defined. As the survey data sets are signed off as correct, validated and an audit trail created, the data sets can be uploaded to the network asset and overwrite fields as necessary. This ensures that the master asset network available to all users is always correct. The benefit is the ability to analyse the condition, performance etc. of the network based on values collected over a number of surveys throughout the lifetime of the network.
In the light of the different levels of error monitoring set out above, the error types identified earlier can be addressed as follows:
Wrong numbers, including wrong units – range checks behind every field, reporting any data item that lies outside this range for further scrutiny, can largely address this error. This can be made more sophisticated at the validation stage by having user definable rules that use another item of data for the check – the diameter of a pipe is unlikely to be smaller than that of an upstream pipe in the network, for example.
Wrong parameters – text entry that is outside the permitted parameter list can be checked against a look-up table and reported on.
Missing data – fields can be set as either mandatory or optional. Thus if the mandatory check is set on a field, the fact that data is missing can be reported on when the data entry for that asset is finished.
Entering the wrong asset ID or asset location – simple errors that try to introduce an invalid new asset ID can be picked up with field checks. Entering an ID or location that is correct but attributed to the wrong asset can be identified at the validation stage, where rules can ensure that all the survey data refers to a sub network.
Missing out an asset or a group of assets completely - either because the team miss one specified asset in error, or the survey specification has failed to include all assets in the sub network defined for the survey. Like the error above, this can be picked up at the first validation.
Problems of skill level, where a specific team or individual lacks the skills for the task – statistics of error rates and volumes of data entered can be produced from InfoNet, producing what are in effect quality parameters of a survey team. These parameters can be used, either on their own in comparison with the performance of other survey teams, to identify management or performance problems.
With the power that InfoNet provides for survey management, it makes sense to get it out into the field on a laptop for swift response. Teams can key in directly, or InfoNet can take digital data from PDAs and other survey systems. Both field checks and validation checks can be run in the field across all the data in the laptop, and then the field dataset can be sent to the central InfoNet database by any standard communications method for second level validation against the existing data.