Debate on GenAI in requirements engineering: We are only at the beginning!

I recently took part in an online panel discussion on generative AI (GenAI) in requirements engineering with Chris Rupp, Jens Kawelke and Henning Femmer. The recording of this discussion is now also available as a video.

Below is the video as well as a summary, which was not created with AI, but the old-fashioned way, by hand. Whether this is better than an AI-generated one is up to you to decide and you are welcome to write this in the comments.

Diana Hauser moderated the panel extremely professionally, giving the recording an enormously high density of information. The event was structured in several parts. Below is a concise summary for each block, followed by the contributions of the individual participants.

Status quo on GenAI in requirements engineering

Euphoria, quick tactical gains, but also uncertainty regarding data confidentiality and lack of integration. Researchers did not see the possibilities coming either.

All participants reported success stories, especially quick, tactical wins (Michael). Chris spoke of the euphoria in many teams when they get to know the possibilities of tools like ChatGPT for the first time.

However, uncertainty quickly crept in (Jens and Chris): on the one hand regarding data confidentiality, and on the other regarding the lack of integration. Michael pointed out that some users have a knack for prompt engineering, but by no means all of them. Better integrations, like Microsoft them, for example, with Copilot for gitHub has realized, make this much easier.

Henning pointed out the relevance of the topic in research. The quality of the result has a lot of variance, which is of course exactly what research actually wants to avoid. Nevertheless, from a research perspective, what we see today was unimaginable just a few years ago. Henning cited the recognition of contradictions in content as an example.

Effective GenAI platforms

Always separate developers and users. It is best to buy in technology, not develop it yourself. Do not confuse GenAI with traditional AI.

There were definitely different opinions on the question of which platforms are the most effective. Michael said that users want results, not technology. If AI helps, then AI will help. Chris confirmed that in companies that want to implement AI themselves, the roles of user and developer must be clearly separated.

Jens added the topic in the context of requirements management, where a lot of work was initially done with chatbots - if only because there was nothing else. He took a rather negative and opportunistic view of the introduction of AI components. He found chatbots that have access to the company network and certain projects much more interesting: That's where he sees the future.

If AI is used in companies, then in a container, i.e. without the risk of data escaping from the company. Henning confirmed this, but pointed out that this discussion had already taken place ten years ago with regard to other cloud technologies. Back then, there was a difference between what companies said and what they did: If it's useful, it will be done. And it will be the same with AI technologies.

Specific questions about the reQLab and Samantha tools came from the chat. Michael pointed out that at least reQlab is not a GenAI. As traditional AI systems are currently being completely left behind, this differentiation is important.

What is the next step for GenAI in requirements engineering?

Wherever we have data volumes and translation is required, AI will automate. Where there is a lack of data and creativity is required, not for the time being. AI systems are becoming more powerful, but they also need more and more contextual information.

Jens pointed out that there is so much movement at the moment that the question can only be answered from year to year. He also pointed out that some things have already been successfully automated, such as the extraction of requirements.

Henning and Chris find a good guideline: Where enough data is available and a transformation (translation) is required, GenAI can automate well. An example of this is test case generation, whereas an example would be questions for which there is not enough knowledge to make a decision. And Chris rightly pointed out that the human factor is often also important in business: ChatGPT can't have a cup of coffee with me.

Jens and Michael disagreed: Both pointed out that the models are getting better and better. However, access to company-specific data is important for this. A larger context is required for complex automation.

Motivating requirements engineers for GenAI

Requirements engineers are easy to motivate, demos and training help. A big demotivator are bad tools.

Motivating requirements engineers to use these technologies is easy, says Chris. After all, requirements engineers can be convinced if they see the benefits. And that's easy, because this target group knows many of the nerve-wracking activities and would like to have them automated by AI.

Jens pointed out that the skills in RE are shifting. Curiosity and flexibility are still needed, some traditional skills are becoming less important. But that has always been the case. It is easy to motivate requirements engineers with demos and training.

Michael thought the question was a little too narrow, because we basically have to motivate employees to keep up with the latest technology, now more than ever. One problem he sees in RE in this respect is the quality of the tools. Many are grotty and old-fashioned, whereas good tools are motivating. He sees a need for action here, regardless of the AI reference.

What does the future look like?

Digital Twins and SW-Companions will be drastically accelerated by GenAI. Breaks with suppliers and work relating to functional safety will probably not be solved by AI in the short term.

Michael has 4 theses on requirements engineering in the age of AI set up. In the panel, Diana posed the question of how we can unlock the full potential of GenAI. Michael is convinced that this will only work with a holistic view, which is already being used in parts today under the name of Digital Twin is a reality. However, today these are generally limited to certain areas. Start-ups that are working to solve these problems include Flow engineering and spread AI.

Chris pointed out the major problem of communication between manufacturers and suppliers. In this context, however, requirements have a high value as an important intermediate product for contract fulfillment. Michael confirmed this and pointed out that the problem in software development is largely solved by semantically versioned software components.

Jens referred to his idea of a SW Companion with product knowledge. With the help of such a person, the product owner alone could manage the product life cycle of software products in 2-3 years, a job that currently requires half a dozen employees.

Henning differentiated: he sees enormous potential, even among his students. However, for the time being, he sees the need for people in development and review when it comes to safety-critical products.

Can AI be innovative?

Whether AI can be creative depends on the definition of creativity. But AI can definitely support the creative process.

Michael sees an AI as a valuable sparring partner in brainstorming, which can help with variants of ideas and lists, but is not really creative itself.

Jens saw things differently: he asked what creativity actually is. And he defined creativity as a new combination of existing inventions. And GenAI can do that and will continue to do it better and better.

Chris supplemented this with her personal experience from an experiment in which she controlled the AI's creativity and taught it creativity techniques. It came up with some weird stuff, but at least it had aspects of creativity. She emphasized that prompting is the new gold. Which of course raises the question of whether creativity is shifting from content to prompting.

Conclusion

Instead of writing a conclusion, here is Diana's AI-generated summary at the end, which the AI generated based on the transcript of the session:

The meeting focused on the future of requirements engineering in the age of AI, in particular generative AI (GenAI) and its impact on engineering processes and tools. Experiences and insights from the application of AI in projects were shared, with the discussion also addressing the challenges and potentials of AI-supported tools for requirements engineering. Finally, the importance of a holistic view of the product lifecycle was emphasized, with AI being discussed as a key technology for increasing efficiency across the entire development process.

AI