News
 Where
Does Poor Data Originate?
Ever wondered where poor quality data comes from?
And most importantly, what often contributes to data
quality problems? A report from The Data Warehousing
Institute describes in detail the origins of scrappy
data quality.
According to the research, poor quality data
originates in both IT and the business. Problems
arise from technical issues (conversion projects 46
percent, system errors 25 percent), business
processes (employee data entry 75 percent, user
expectations 40 percent) and a mix of both
(inconsistent terms 75 percent).
The report also reveals that problems come from
outside sources (customer data entry 26 percent,
external data 38 percent). So basically – data
quality is assaulted from all quarters, requiring
great diligence from both IT and the business to
keep its problems at bay, with both internal
processes and external interactions, the report
notes.
Another originator of data quality problems is
inconsistent data definition. According to the
report, the data itself isn’t wrong – it’s just used
incorrectly. For example, multiple systems may each
have a unique way of representing a customer.
Application developers, integration specialists and
knowledge workers regularly struggle to learn which
representation is best for a given use.
When good data is referenced incorrectly, it can
mislead business processes and corrupt databases
downstream, the report reveals. With 75 percent of
survey respondents pointing to this problem, it ties
with data entry as the most common origin of data
quality problems.
Who is the biggest offender of sparkling data
quality? It’s data about customers that’s the
leading offender (74 percent), while product data
came in at a distant second (43 percent). It’s
because customer data changes frequently as
customers pay bills, move to new locations, change
their names, get new phone numbers, change jobs,
etc. As a result, every change is an opportunity for
data to be entered incorrectly or to become out of
date. Since customer data is often strewn across
multiple systems, synchronizing it and resolving
conflicting values are common data quality tasks.
The Origins of Poor Quality Data
Which of the following most often contribute to data
quality problems in your organization? Source: TDWI.
* Inconsistent definitions for common terms 75%
* Data entry by employees 75%
* Data migration or conversion projects 46%
* Mixed expectations by users 40%
* External data 38%
* Data entry by customers 26%
* System errors 25%
* Changes to source systems 20%
* Other 7%
|
|
|
