Data Engineering

Definition:

The execution of the non-physical components of the data infrastructure to assure the data is fit-for-purpose.

Commentary:

Data engineering is a subcomponent of the data architecture execution plan.

Data engineering is the function of managing meaning. This is about harmonization of data across multiple repositories to ensure that there is a unified view of the meaning of the data. Some data (e.g., reference data) is aligned to precise contractual meaning. Some data (e.g., customer data) is specific to the individual enterprise. The EDM Council is supporting this with the development of FIBO (Financial Industry Business Ontology) for the financial service industry.

A best practice is to manage the meaning of data to ensure it is precise, unambiguous, consistent and transparent.

The non-physical components of the data infrastructure includes, but is not limited to:
– Data domains
– Data glossary
– Logical and physical data requirements
– Metadata
– Models
– Ontologies
– Taxonomies