Why companies struggle with implementing AI and How to overcome it.
If you have become a Data Scientist in the last three or four years, and you haven’t experienced the 1990’s or the 2000’s or even a large part of the 2010’s in the workforce, it is sometimes hard to imagine, how much things have changed. Nowadays we use GPU-Powered Databases, to query billions of rows, whereas we used to be lucky if we were able to generate daily aggregated reports.
But as we have become accustomed to having data and business intelligence/analytics, a new problem is stopping eager Data Scientists from putting the algorithms they were using on Toy Problems, and applying them on actual real-life business problems. Other wise known as the Cold Start Problem with Artificial Intelligence. In this post, I discuss why companies struggle with implementing AI and how they can overcome it.
Start with Data
Any company either startup or enterprise, who wants to take advantage of AI, needs to ensure that they have actual useful data to start with. Where some companies might suffice with simple log data that is generated by their application or website, a company that wants to be able to use AI to enhance their business/products/services, should ensure that the data that they are collecting is the right type of data. Dependent on the industry and business you are in, the right type of data can be log data, transactional data, either numerical or categorial, it is up to the person working with the data to decide what that needs to be.
Besides collecting the right data, another big step is ensuring that the data that you work with is correct. Meaning that the data is an actual representative of what happend. If I want a count of all the Payment Transactions, I need to know what is the definition of a Transaction, is it an Initiated Transaction or a Processed Transaction? Once I have answered that question and ensured that the organization agrees on it, can I use it to work with.
With the wide adoption of SCRUM and frequent releases, companies have to devote resources to ensure that the data is correct. Companies could, add new sources of data, changes in the code that can have an impact on the logged data or even outside influences like GDPR or PSD2, that can cause the data to be altered because it needs to be more secured or stored in a different way. By ensuring that during each process the correctness of the data is ensured, only then can you move on to the next phase of analytics.