Sunday, November 1, 2009

A well-oiled machine is critical to Data Quality

The beauty of human beings is that people will look for creative ways to solve their problems. That means, when users have technical problems or run into business limitations during data entry, they will find ways to do it, even if it means breaking business rules or overriding well defined processes. From a Data Quality perspective, that is not a good thing, but who is to blame the users? After all, they may be facing a particular customer need that doesn't fit an existing business process, or a system bug that is delaying a high profit transaction.

Let's assume you organization does have all elements in place, such as Data Governance, Data Stewardship, Data Quality, IT support, etc. Users are less likely to engage the proper teams if their confidence in the support process is low. They may think: “oh boy, by the time I get this problem resolved through the proper mechanisms, it will be too long and I'll have a customer satisfaction issue beyond repair.” Therefore, for the “benefit” of the organization, they act with imagination and solve the immediate problem with non-approved solutions. Making matters worse, detecting these out-of-spec practices and associated data issues are sometimes difficult to monitor and correct.

With that said, your goal as an organization should be not only to have the proper elements of a well governed organization, but have them working effectively as well. That comes with maturity, and a constant focus on process improvement. Simply improving your data entry process alone is not enough. You have to improve the support process around it. Just about everything is constantly changing: business needs, business landscape, technology, people, etc. Your only hope is to have an efficiently adaptive model that in spite of all these changes, can continue to deliver results quickly. Let's focus our creativity on this problem, and be really dull when it comes down to being creative breaking business rules!


  1. Dalton. I love this statement 'Your only hope is to have an efficiently adaptive model that in spite of all these changes, can continue to deliver results quickly'. So very true in todays business world, and is something that I keep on pushing at all levels.

    Adaptive business focused processes and IT capabilities are critical to business success.

  2. Dalton,

    Excellent post. Over the years I have encountered many situations in which the creativity of business users has resulted in data quality issues, some more serious than others.

    On Euro Changeover programmes, we used the term 'extended semantics' to describe instances in which data fields were used by business end users to record information not originally catered for in the system design.

    More recently, I have observed instances that adversely affect Anti Money Laundering systems.
    Let me cite one example. When a customer performs a Money Transfer, it can take a number of working days for the funds to be cleared. By contrast, when a customer lodges cash, the value is immediately available on the customers account.

    To give their customers a 'better service', and provide them with immediate access to funds, some bank assistants have been known to process Money Transfers as Cash. One can understand the good intent of the bank assistant.

    The above well intentioned action can result in many unforseen consequences. One of these may be an Anti Money Laundering alert on the customers account, since significant cash lodgements are one of the indicators of Money Laundering activity.

    As you can see from the above, non-cash transactions processed as cash can cause chaos for an AML system.

    It may be impossible to design a data entry validation process to prevent the above occurring. Educating front line staff on the importance of correctly processing transactions, combined with your recommendation for an 'efficiently adaptive model' is the best route forward.

    Rgds Ken

  3. Charles, thanks a lot for your comments. Ken, really REALLY appreciate your example! Very well put it, and indeed illustrates my point very well.

    Thanks both!!