Data management is a critical component of an integrity management plan. Current products and services in the integrity management sector can generate an enormous amount of uncontrolled and disjointed data, housed on multiple platforms. This text examines methods of mapping and linking multiple data sources to achieve optimal data usefulness, while reducing redundant data, through use of spatial and relational techniques. By defining relationships between fixed points, linear values can be generated from calibrated routes. Developing methods to introduce new data, standardized from spatial data, serves to maintain data quality. Recurring data transfer logistics, using relational keys in conjunction with ETL procedures, serve to link databases. Value is achieved on a large-scale using girth welds to automate the process of generating mile post values for point features. Data generated at remote sensors are aggregated from multiple vendors and populated using an exchange governed by universally unique identifiers (UUID).


Regulation and business needs of an organization are primary drivers for the decisions made in a pipeline integrity management data plan. How data is acquired and stored can vary greatly as determined by those factors. Two production databases and multiple remote sensor vendor databases are referenced in this text, each with a business case to operate independently.

In this example, pipeline attributes contained within the organization's corrosion control database can weigh heavily on decisions made by users of the pipe book database and those made by users of the corrosion control database. By implementing a system to quickly and accurately query attributes from both databases, generating reports with the information required to make those business decisions becomes less complicated.

Technologies in remote sensors have an established backbone in the pipeline integrity management industry. Each remote sensor vendor holds proprietary technology and a unique data structure. Over time, advancements in technology, market trends, and product application requirements are likely to result in diversification of vendors. Regardless of vendor loyalties, technological advancements cause older models of hardware to become obsolete and newer technology to take its place. The resulting variations require special attention to ensure reporting is accurate across all vendors and models.

This content is only available via PDF.
You can access this article if you purchase or spend a download.