Data Profiling

Data profiling is essential to managing change in business systems...

​Where change to business systems is anticipated & commissioned its successful implementation can, in theory, be verified by traditional reporting. Typically, change driven by mergers & acquisitions, new products or services, technology upgrades, removal of ‘key man’ dependencies, new regulations or legal requirements, new corporate branding or policies etc. is validated by targeted querying of data.​

However, on its own this approach to change management is too simplistic, because it limits you to finding issues only where you decide to look for them (as opposed to all issues which may result from proposed change, either directly or indirectly). Therefore, it won’t reveal issues which result from unforeseen change – i.e. change which just ‘happens’ during ‘business as usual’. Change caused by abnormal market conditions, errors or omissions, misunderstandings, data validation failures, system outages, viruses or malicious actions etc. is highly unlikely to be captured by traditional reporting.

So far safer than validating proposed change is to detect all change, whether anticipated & commissioned or completely unforeseen, with regular data profiling. By performing statistical analyses of data values & formats, data profiling tools are extremely good at identifying anomalies and disparities which could be harmful if left unaddressed. Repeated with a frequency appropriate to the volatility of the relevant data, data profiling allows all change which manifests itself in data to be detected before it can have any significant, negative impact.​

Data profiling is also very useful for identifying the logic implicit in physical data – e.g. logical entities and attributes, primary and foreign keys, relationships between entities, logical domain values – and therefore expedites the task of establishing the definition of business activities, which is a fundamental pre-requisite of any effective change management approach.