Big data and the profession - what are the practical challenges?

While there are lots of ways in which accountants are making use of new capabilities in data and analytics, we need to recognise a number of practical barriers and challenges, many of which were highlighted in our annual lecture on the topic of big data.

First is the question of business case and justifying investment in new tools and technologies. Where tangible benefits are not immediately obvious – and businesses often don’t have a clear idea of the specific insights they are going to get – investments can become rather a leap of faith. While this is not unusual in IT investments, lack of business case, or lack of ownership for investing in cross-departmental capabilities, can slow adoption.

Second is a lack of skills and knowledge. Without the right skills and experience, it is difficult to know how new tools and techniques can give different insights. It is also difficult to interpret results correctly, and there is a shortage of talent everywhere for good people in this space.  

The third difficulty is in identifying what you want to know from the data. Most businesses have no shortage of data, and the question usually is – out of all of that data, what is actually useful? Of course, it depends on what you want to know. But having clarity on what you want to know can be a time-consuming thought process.

Finally the biggest part of any data analytics project concerns data. Most organisations have poorly managed data – there may be multiple, unintegrated systems and spreadsheets, and the quality may be poor with duplication, errors, gaps and inconsistencies. Therefore, most projects start with a lot of work to extract and cleanse the data. Without that, meaningful analysis is impossible.

It’s no surprise, then, that the biggest users of these technologies are either larger companies with lots of resources to spend, or tech-savvy, newer companies who have built their infrastructure and business models around data.

So, our eight lesson is that businesses need a realistic strategy for reaping the benefits of big data and analytics. In most cases, they should start small, in areas with clear benefits, build knowledge and confidence and then expand into new areas or more sophisticated analysis.