Data mapping

Data transformation/Source transformation
Concepts
metadata · data mapping
data transformation · model transf.
Languages
ATL · AWK · MOFM2T · QVT · TXL
XML languages
Techniques and transforms
identity · synthesis · refinement
Applications
data migration · data conversion
ETL · program transformation
Application fields
Data warehouse
Software engineering
Software languages: macro, preprocessing, template
v · This article includes a list of references, related reading or external links, but its sources remain unclear because it lacks inline citations. Please improve this article by introducing more precise citations. (June 2010)

Data mapping is the process of creating data element mappings between two distinct data models. Data mapping is used as a first step for a wide variety of data integration tasks including:

  • Data transformation or data mediation between a data source and a destination
  • Identification of data relationships as part of data lineage analysis
  • Discovery of hidden sensitive data such as the last four digits social security number hidden in another user id as part of a data masking or de-identification project
  • Consolidation of multiple databases into a single data base and identifying redundant columns of data for consolidation or elimination

For example, a company that would like to transmit and receive purchases and invoices with other companies might use data mapping to create data maps from a company's data to standardized ANSI ASC X12 messages for items such as purchase orders and invoices.

Contents

Standards

X12 standards are generic Electronic Data Interchange (EDI) standards designed to allow a company to exchange data with any other company, regardless of industry. The standards are maintained by the Accredited Standards Committee X12 (ASC X12), with the American National Standards Institute (ANSI) accredited to set standards for EDI. The X12 standards are often called ANSI ASC X12 standards.

In the future, tools based on semantic web languages such as Resource Description Framework (RDF), the Web Ontology Language (OWL) and standardized metadata registry will make data mapping a more automatic process. This process will be accelerated if each application performed metadata publishing. Full automated data mapping is a very difficult problem (see Semantic translation).

Hand-coded, graphical manual

Data mappings can be done in a variety of ways using procedural code, creating XSLT transforms or by using graphical mapping tools that automatically generate executable transformation programs. These are graphical tools that allow a user to "draw" lines from fields in one set of data to fields in another. Some graphical data mapping tools allow users to "Auto-connect" a source and a destination. This feature is dependent on the source and destination data element name being the same. Transformation programs are automatically created in SQL, XSLT, Java programming language or C++. These kinds of graphical tools are found in most ETL Tools (Extract, Transform, Load Tools) as the primary means of entering data maps to support data movement.

Data-driven mapping

This is the newest approach in data mapping and involves simultaneously evaluating actual data values in two data sources using heuristics and statistics to automatically discover complex mappings between two data sets. This approach is used to find transformations between two data sets and will discover substrings, concatenations, arithmetic, case statements as well as other kinds of transformation logic. This approach also discovers data exceptions that do not follow the discovered transformation logic.

Semantic mapping

Semantic mapping is similar to the auto-connect feature of data mappers with the exception that a metadata registry can be consulted to look up data element synonyms. For example, if the source system lists FirstName but the destination lists PersonGivenName, the mappings will still be made if these data elements are listed as synonyms in the metadata registry. Semantic mapping is only able to discover exact matches between columns of data and will not discover any transformation logic or exceptions between columns.

See also

References


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Data Mapping — Le data mapping est un procédé permettant de définir au niveau d un langage de programmation la correspondance entre deux modèles de données. L accès aux données se fait habituellement à travers des requêtes SQL fortement typées selon la… …   Wikipédia en Français

  • Data mapping — Le data mapping est un procédé permettant de définir au niveau d un langage de programmation la correspondance entre deux modèles de données. L Office québécois de la langue française propose comme équivalent en français mise en cohérence de… …   Wikipédia en Français

  • Data integration — involves combining data residing in different sources and providing users with a unified view of these data.[1] This process becomes significant in a variety of situations, which include both commercial (when two similar companies need to merge… …   Wikipedia

  • Data transformation — Data transformation/Source transformation Concepts metadata · data mapping data transformation · model transf …   Wikipedia

  • Data governance — is an emerging discipline with an evolving definition. The discipline embodies a convergence of data quality, data management, data policies, business process management, and risk management surrounding the handling of data in an organization.… …   Wikipedia

  • Mapping — In geomatics or geospatial science and technology: Mapping usually refers to map making and often used instead of cartography. Mapping term is also sometimes used for geospatial data collection (e.g. LIDAR mapping) but in fact it is not mapping… …   Wikipedia

  • Mapping — Cette page d’homonymie répertorie les différents sujets et articles partageant un même nom. Sur les autres projets Wikimedia : « Mapping », sur le Wiktionnaire (dictionnaire universel) Mapping est un mot anglais possédant de… …   Wikipédia en Français

  • Data structure alignment — is the way data is arranged and accessed in computer memory. It consists of two separate but related issues: data alignment and data structure padding. When a modern computer reads from or writes to a memory address, it will do this in word sized …   Wikipedia

  • Data Web — refers to a government open source project that was started in 1995 to develop open source framework that networks distributed statistical databases together into a seamless unified virtual data warehouse. Originally funded by the U.S. Census… …   Wikipedia

  • Mapping+objet-relationnel — Mapping objet relationnel Un mapping objet relationnel (en anglais object relational mapping ou ORM) est une technique de programmation informatique qui crée l illusion d une base de données orientée objet à partir d une base de données… …   Wikipédia en Français


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.