With 32% of respondents to our Autumn 2017 Risk and Confidence survey investing in R&D and 42% investing in tech according, it is likely the trend for businesses gathering big data will accelerate even further in 2018. Big data – data sets that are so voluminous and complex that traditional data processing application software is inadequate to deal with them – is already changing the playing field when it comes to business innovation and efficiency.
If done well big data has the potential to revolutionise a business, but if not it could potentially be a waste of time, money and increase security and compliance risks, especially with the implementation of General Data Protection Regulation (GDPR).
Implementing the gathering of big datasets as part of a business is inevitably going to be a complex process, with a heavy time and financial cost, but the masses of data it can generate is an exciting prospect that can bring real insight. However in many of these projects it can be the case for the vast amount of data gathered to not even be used, making it a waste of resources.
Importing, profiling, contextualising, integrating and enriching the data are long and intricate procedures, and the introduction of GDPR in May 2018 will complicate this further with the addition of extra security and privacy concerns to managing this data.
Potential methods for ensuring the successful implementation of Big Data gathering could include:
- Understanding what you wish to use the data you are gathering for, and then from that working out what data is actually needed. Focused data sets will be of more use and easier to gain insights out of than general ones, as well as requiring less processing power.
- Ensuring the data gathered is high quality and easily consumable to provide insights. Eliminate old or duplicate datasets, anomalies or mistakes before analysing the data.
- Mapping out the process of gathering the data and ensure the extraction costs are of less value than the data you are gathering. Big data initiatives can prove more costly than expected so it is good to know both the expense of extracting the data and what that data is then worth to your business.
- Having an application ready that can arrange the data into a usable format. The issue with big datasets is they are often so huge as to prove unquantifiable, so having this process in place ensures that you can use the data you are gathering.
- Data being siloed to individual teams or departments can often inhibit success and innovation, but with the implementation of GDPR there must be a business purpose to store the data. Even then data must be properly secured; a leak, especially of personal information, could have massive repercussions in terms of fines and reputational damage.
Big Data has the potential to be an amazing tool for progression and innovation for businesses, and it is important to map out the processes and potential risks ahead of time as well as to learn lessons from previous failures. Successful management of the data and the process from start to finish is vital for enabling data gathering projects to thrive, and only then will the true capabilities of connectivity be realised by businesses.
Matt Sumpter, UK Underwriting Director - Technology & Cyber Risks
Download our Risk and Confidence Survey at http://www.cnahardy.com/pulse