By Dr. Mehrzad Mahdavi, Executive Director, CEO Financial Data Professionals Institute
According to a recent WSJ article, (subscription required) many companies have just discovered that AI is not some kind of magic sauce that could be poured over any business process to transform it into a moneymaking machin. The article also points out that “the corona virus has helped to erode the hype around AI”.
Common complaints about AI deployments include: a) can’t find robust use cases for AI; b) AI needs large pools of data—Big Data! and c) many pre-pandemic models for business functions are no longer useful or did not perform as expected during the pandemic.
The article then goes on to point out that some businesses for which AI is more of an add-on, as well as struggling startups and smaller firms, are furloughing data scientists previously thought of as untouchable. They reported a survey that found 6% of data scientists respondents had been affected by furloughs, pay cuts or layoffs. They also add on that hiring for such business-critical roles [data scientists] requiring high-demand specialized skill sets, has slowed significantly, down by 50% since before the pandemic. They point out, on the other hand, that there’s still demand, though it’s diminished.
By contrast, big tech companies clearly see AI core to their businesses, and plan to keep hiring at a fast pace. The Big Five have said they’ll continue to add to their engineering ranks (WSJ subscription required) during this downturn, including data scientists and AI experts.
A just-released survey of nearly 1,400 AI professionals, conducted by O’Reilly Media, found the two biggest barriers to the use of AI in businesses are leaders who don’t appreciate its value, and the difficulty of finding business problems in these firms for which AI might be useful.
To separate hype vs reality, let’s synthesize these [conflicting] reports and provide some key takeaways:
- AI is an umbrella term covering many technologies applied in countless use cases in multiple industries. So, when we talk about AI trends, success or failures, we need to know the specifics – use case, application, industry, company, etc.
- Hype Curve – Every major technology innovation goes through some kind of a “hype curve.” Below, is a version of the hype curve developed by Gartner for AI. Pick an AI-based technology and find [vs time] if you are in the “peak of inflated expectations” or “trough of disillusionment” or on the “plateau of productivity.” Any consequential technology will go through some form of hype curve as part of their maturation process. Any success or failure needs to be in the context of this hype curve.
- key success factors for AI are: 1) relevant business case; 2) patient investment and management support; 3) human intelligence + machine intelligence to assist in the data-driven decision making.
In summary, the AI perceived hype is really a “hype curve” as is for every major game changing technology. As the AI technology moves through these gating steps, it will mature and becomes more robust – a multi-year road map.
There are many successful use cases of AI in the financial services where machine-learning methods provide the technical means to conduct more nuanced and complete investment analysis that were not possible with traditional analytical tools. To take advantage of these opportunities, we emphasize the following:
- Training and skills development – Many of these problems with models arise because more businesses are buying machine-learning systems but lack the in-house know-how needed to operate and maintain them. Retraining a model can require expert human intervention. All of this is possible only with a dedicated team including data scientists, data engineers, data translators, etc. We have also learned, due to the pandemic, that AIs should be trained not just on the ups and downs of the last few years, but also on events like the Great Depression of the 1930s, the Black Monday stock market crash in 1987, and the 2007-2008 financial crisis.
- It is all about data - “Know your data” should be an important guiding principal for a successful AI and machine-learning project.
Connect with Dr. Mehrzad Mahdavi on LinkedIn or follow on Twitter @MehrzadMahdavi