Reality check: How helpful is AI in product development really?

Ever since ChatGPT appeared on the screen, we have been experiencing enormous hype around the topic of AI. The fact that AI has been in use in all areas of our lives for decades is irrelevant. We are already feeling the effects in systems engineering, for example in programming. But how helpful is AI really in product development? A tip: less in the short term, more in the long term than you think.

On our own behalf: 4-week MVP for your systems engineering use case

I am currently working on a concept to implement AI-supported experiments on a monthly basis: Define use case, set KPIs, build MVP, conduct experiment, evaluate results. Do you have a use case from product development with AI in your company for which this would be an option? Then let's arrange a meeting >>

gitHub Copilot

One of the first ChatGPT-based products for development was probably the gitHub Copilot. This short video shows in an impressive way how Copilot can support programmers. I tried Copilot myself, of course, but was a little disappointed: many suggestions were not quite right or consisted more or less of code that I had already written elsewhere. For inexperienced developers, this could lead to copy-and-paste code instead of proper reuse.

But that doesn't stop many developers from using gitHub Copilot. Microsoft now has 1.3 million subscribers (as of February 2024). However, the productivity gains appear to be in the single-digit percentage range, claims Chirantan Desai, COO of ServiceNow. But users are enthusiastic: a Microsoft survey showed that few people voluntarily stop using Copilot.

The productivity gain of gitHub Copilot seems to be in the single-digit percentage range

Chirantan Desai, COO of ServiceNow

It is still too early

Even though there is a lot of talk about generative AI, its use by companies is still limited. For example, only 20% of Microsoft Azure's revenue growth attributable to AI (let alone the base turnover, figures from February 2024). Companies are experimenting with AI, but few are using it on a broad front.

There is little doubt that we are close to the peak of the Gartner hype cycle are. The question is whether generative AI will be dragged down in the coming months or whether it will be so successful that its use will continue to grow despite disappointments.

The Boston Consulting Group (BCG) conducted a survey on this topic. The majority of managers stated that it will take at least two years before the hype surrounding AI is over.

According to a BCG survey, the AI hype will continue for at least another two years.

Economist

Even today, it is impressive how quickly AI is developing, as can be seen in the example here. In this respect, I can well imagine that the "valley of disappointment" will be comparatively mild. Because in two years' time, generative AI will be even better.

It all depends on the application

Hardly any companies are currently using generative AI systematically; there is a lot of experimentation going on. However, some forward-thinking companies are already preparing to use AI to solve customer problems. A few examples (also from Economist):

• JPMorgan Chase already has over 300 AI use cases in productive use.
• Capgemini is working on a library with over 500 industrial use cases.
• Bayer has a collection of over 700 use cases of generative AI, although it is unclear whether these are already in use.

Investors are also looking closely at the use case. John Lange explains in Business Insider, What investors look for in AI startups. The first point he calls for is AI at the core of value creation. This means either the development of AI infrastructure components or the solution of specific problems, i.e. use cases.

Use cases can be divided into three categories: Whitewashing, tools for less experienced employees and tools for the most valuable employees. Currently, whitewashing is of course the most widespread. Here, existing use cases are "decorated" with AI without creating any real added value.

"Whitewashing" with AI can also backfire. For example, Instacart used AI-generated images, some of which were rejected by customers as unappetizing.

Business Insider

However, use cases for less experienced employees can be very effective. These include simple things such as extracting data from Excel (Bayer).

There are not yet so many use cases for "valuable employees", but there will be more. The target group here are lawyers or analysts, for example. One example would be an AI system that creates a draft report overnight from large amounts of financial data, which the banker then has to revise. Starting with an AI-generated draft can save several hours of working time.

No job losses in sight

At the moment at least, it doesn't look like AI will destroy jobs. AI can make employees more productive, but rarely redundant. Even worse: Generative AI is called "generative" because it generates. However, the generated content usually also needs to be checked and processed by humans. In other words, we need people to help us process the AI-generated data. Brave new world!

AI in product development

The trends described here are just as relevant for systems engineering as they are for many other areas of application in companies. We are already seeing the first use cases sprouting up. The gitHub Copilot example shows how AI can be used selectively. Small use cases are also being covered effectively in CAD. As was to be expected, we are seeing more and more use cases from the three categories mentioned above.

Microsoft is not the only company trying to reopen the market here. With Simcenter Studio, Siemens is pursuing the "generative engineering" approach with the aim of breaking down silos. In a "computational notebook", reminiscent of Jupyter Notebook, stakeholders can collaborate across the board with the help of AI. This is in the video on the Simcenter Studio page where it is also shown how the generation of architectures takes place.

Conclusion

It will be years before we really feel the effect of the new AI technologies - but then the real storm will break. AI will transform the way we work, not just in product development, but in many other areas too.

I don't believe that there will be a major loss of jobs. However, I believe that we can't start rethinking soon enough.

Image source: FreePik