Businesses that understand how to navigate the data landscape can use their knowledge to put insights into action. Businesses can use data to inform every aspect of their operations, from customer relations to product development. Data is great, but problems develop when they are not accurate. Business that do not understand the data landscape are vulnerable to misinterpreting data, which can lead to big problems later down the line.
The Value of Data
If you are working in business it is likely that you have heard the terms metadata, big data and data quality quite frequently. Data truly is the new oil; it can impact everything from finding new areas for growth to strengthening relationships with customers. However, in order to fulfil the potential of data use, businesses first need to understand it. In fact, poor data quality typically costs a company a sizeable chunk of its revenue and productivity. So, by not clearly understanding the data landscape, businesses are not only losing out on potential profits but damaging their operations in the long-term.
Metadata is fundamental to ensuring the quality of your data, as it is the role of metadata to provide information about other data. Metadata categorizes data – it provides the documentation necessary for it to be interpreted and understood. Metadata is necessary in order for data to be placed into the correct categories for determining controls that are needed for quality and data protection. There are many distinct types of metadata that exist, such as:
- Descriptive Metadata
This type of metadata is descriptive information about a resource and is used for discovery and identification. Examples of descriptive metadata include elements like title, author and keyword.
- Structural Metadata
The term structural metadata refers to the way in which data is formatted and assembled. It is the job of structural metadata to indicate how one datum relates to another.
- Administrative Metadata
Administrative metadata is used to manage resources, offering information such as who created the file, who can access it and where it is stored.
- Reference Metadata
Reference metadata provides information about the contents and quality of statistical data.
- Statistical Metadata
This type of data can describe the processes that collect, process or produce statistical data.
Businesses that manage any data assets at all should use metadata to cope with the task of data management. Metadata makes any data analysis infinitely easier as it allows for data to be found retrospectively in good time. Good use of metadata will also help to make any data audits quick and painless, as well as clarify why data is being recorded and used, which can encourage compliance with data governance and regulations.
When metadata isn’t conducted properly, it can result in major inefficiencies as anybody who needs a certain datapoint or asset will have to scour through the system, lost in the folders and files that data management requires.
Once data has been accurately defined it must then undergo quality controls to check its accuracy. Data quality involves the planning, implementation and control of activities that apply quality management techniques to data. These processes are used to ensure that data is fit for consumption and that it fulfils the needs of data consumers. Without knowing data quality, businesses leave themselves vulnerable to a whole host of potential problems.
One example of data quality creating issues is apparent when you consider marketing. If you have poor quality data, you are more likely to overspend your marketing budget or irritate your potential customers by sending them the same piece of information multiple times due to inconsistencies with contact data.
There are many different methods that can be used to ensure data quality:
- Data quality management is commonly used by companies to get rid of data that does not meet their key performance indicators.
- Referential integrity can be defined into the database.
- Data profiling can be used – this is the process of looking through existing data in a database and simply looking over them with mathematical formulae to uncover their quality, as well as characteristics and possible errors. The statistics that are obtained by data profiling can greatly aid the clean-up of problematic data.
Poor quality data is responsible for an average loss of $15 million per year, something that will likely increase in future years as big data gets bigger and bigger. Poor data quality can be caused by human error, data decay (e.g. a customer moves address) or changes to systems (take the 2000 switch as an example of what can happen when a system change causes massive data decay).
When data quality is poor, it can impact decision-making, as your decisions will only ever be as good as the data that they’re based on. It can also reduce productivity, as inefficiencies are created which result in significant operational costs in some instances. Poor quality data will increase the known issues that your employees have to navigate every day.
In a world where not adhering to GDPR can result in mammoth fines, data governance plays a larger and larger role in the data landscape. Countries all across the world are starting to construct their own data compliance regulations and if any company is reported or discovered to not adhere to those regulations the consequences can be anything from fines to actual prison time for those responsible.
Medium to large companies often need to construct a data governance department, which should develop procedures that keep the rest of the company in line. This serves the additional benefit of contributing to data quality management, as well as metadata construction as in order to monitor data for governance purposes, data quality and metadata often have to be involved in some way.
Data governance departments should assign data stewards, who create data models as well as define and manage metadata. Archival and effective archival are vital for data regulation, as clear records of data should be kept so that in the instance of a data audit, the company can fully comply with the auditing body.