Dealing with Defective Data
Data quality isn't a terribly riveting concept on its face. But poor data is a problem that can contribute to many other corporate problems, particularly as data and analytics become increasingly critical to strategy.
"Bad data remains a major cause of botched marketing campaigns, failed CRM and data warehouse projects, angry customers, and lunkhead decisions," according to InformationWeek. "Despite all we know about the importance of data scrubbing and quality management, many companies are still using data that's redundant, incomplete, conflicting, outdated, and just plain wrong."
Recognizing the risks they run and the costs they bear, many companies are now investing big money in efforts to get a better view of their data, integrate it and ensure it is scrubbed for quality. They recognize the interconnectedness of data quality and corporate performance. "Our marketing effectiveness leads to our sales effectiveness, which leads to our service effectiveness. Data quality is key to the success of that," says Chuck Scoggins, VP of customer solutions at Hilton Hotels. "If you don't have quality data, that whole chain breaks down."
As corporate managers become ever more dependent on performance management scorecards and dashboards, they start to see the critical importance of the data underneath. As the article suggests, the greatest hurdle associated with addressing the problem is the tendency to point fingers of blame. Business managers see data quality as an IT problem -- even though IT doesn't control the business processes that generate bad data in the first place. "Business has to accept the fact that it has primary responsibility for data quality. Data is a business asset," says Nigel Turner, a manager for data quality programs at BT Group (formerly British Telecom) in the late '90s.
How big is the problem? Stamford, CT-based Gartner contends that more than 25% of important data within large enterprises is inaccurate or incomplete in some way. Business result? One survey of 750 IT managers and business executives by the Data Warehousing Institute in 2005 found that 53% had experienced losses or increased costs due to data quality problems.
But there's no reason to throw up one's hands. BT found a way to address the challenge:
Rather than create a top-down, companywide program, Turner targeted line-of-business operations and identified a data quality "champion" in each to lead an information management forum. The groups targeted specific projects with demonstrable returns on investment, such as improving names and addresses in marketing data to reduce the number of letters sent to the wrong people and improving private-line inventory record keeping to increase the number of disconnected circuits returned to stock for reuse."We had to prove to BT that these things were worth doing," Turner says. "Data quality isn't very sexy." The original budget for the data quality efforts was a measly $30,000. As the project expanded, Turner's group developed a data quality methodology incorporating best practices gleaned from inside the company and from outside experts, and centralized data quality management. Recognizing that errors will creep into databases despite its best efforts, BT uses data profiling and cleansing tools from Trillium to identify and remove errant data.
The efforts have paid off: BT has realized as much as $800 million in aggregate savings by improving inventory management, boosting productivity through improved automated interactions with suppliers and customers, and reducing revenue leakage through more accurate billing. BT has parlayed its data quality know-how into a consulting business headed by Turner.
Other companies are putting data quality under broad "data governance" programs. The objective is to establish best practices managing, securing and using data. "It requires establishing a formal set of business processes and policies to ensure that data is handled in a prescribed fashion," according to the magazine. "Data governance includes standard definitions for data elements to be used throughout a company--just what a 'lost customer' is, for example--and metrics for measuring data quality, says Terry Haas, director of the enterprise data management practice at PricewaterhouseCoopers. Data governance also defines the data management roles and responsibilities of managers and employees and limits the ability to change data to designated 'data stewards.'"