Aligning requirements and system model with AI

In a month's time, ReConf, an established conference focusing on requirements management, will take place. I will be there again this year with a presentation entitled "Extracting models from texts: Why AI chat is not enough". As can be seen from the title, this involves, among other things, comparing a system model with texts, such as requirements.

I won't get ahead of myself, of course, but I would like to provide a bit of context below. In particular, I will look at how we in systems engineering can sensibly escape the current hype and really put AI to good use.

I would like to exchange ideas with people who see concrete areas of application for AI in systems engineering in their own company. If you are interested, please reply directly to this newsletter, preferably directly for an appointment at the ReConf.

The hype around AI chat

The hype around ChatGPT and other AI chatbots is entirely justified, because two things are really new: firstly, for the first time it is possible to formulate queries to a system using free text, to which the system returns an answer that matches the question. At best, conventional systems have a powerful search function that returns suitable text snippets.

Secondly, we have a context, which makes it possible to ask questions. This has never existed in this form before. In addition, the systems are universal, i.e. they are not tailored to a specific domain.

AI chatbots do two things that are really new: (1) input and output in natural language and (2) dialog in context.

But once the initial enthusiasm has subsided, two questions arise. Firstly, how does such a system help me in my daily work, i.e. in systems engineering? Secondly, how can I incorporate AI into my own products?

Building AI into products

For the second question - building AI into products - we should not start with the solution (AI chat), but with the customer problem. But experienced systems engineers have long known that we should always start with the stakeholder's problem.

To decide whether and how an AI could work in a product, I have in this article on the Reward function described.

The next question is how we could use AI in our daily work as systems engineers.

Using AI for systems engineering yourself

I have now come across a number of suggestions as to how we could immediately use an AI chat system in our work:

We could have the AI complete requirements...	...that actually works very well! But where do the requirements come from? From the AI's training data, of course, which means that they are unlikely to be innovative. There is also the risk that our requirements will be used again for AI training and at some point will be suggested to competitors.
We could ask AI to evaluate and improve requirements...	...the result is impressive the first time! However, we cannot be sure that all quality criteria are taken into account.
We can ask the AI to derive a system model from requirements...	...the challenge here is that the system model is not written in natural language. However, the chatbot was able to create a PlantUML-model. This could be useful for creating an initial model, but it is difficult to maintain or analyze.

In short, we can already use AI chat selectively. However, the benefits and efficiency gains depend heavily on the creativity of the user and the expected efficiency gains are difficult to predict.

Automation or augmentation?

There are tasks that people would like to have done by AI. But people also like to be in control and want - or need - to do many tasks themselves. In these cases, AI can help to complete tasks faster, more efficiently and sometimes even more creatively.

At the moment at least, I clearly see AI in augmentation: for example, I wouldn't trust AI to maintain traceability automatically. However, an AI assistant that makes suggestions for adapting the traceability would be useful, like this:

Here, an AI assistant suggests creating two missing traces and has also identified a (SysML) block to which this request could be assigned. These are typical activities that a user would not want to automate.

The system model and the reference to requirements and other texts

Every text is a model, even if it is often not a particularly precise one. However, it is the kind of model that we humans understand very well and that we will not be able to do without in the foreseeable future. However, with the slow but steady increase in the use of models, the comparison between system models and texts of all kinds is becoming increasingly important.

However, this is where today's AI chat systems do not do a good job, for two reasons. Firstly, the language models have only a rudimentary understanding of models, and this is based on a textual representation such as the PlantUML-syntax or XMI. This problem could be solved by specifically training the AI as a specialized translator.

Furthermore, they have no understanding of the "system of interest". In contrast to generic generative AI, we are only interested in general knowledge to a limited extent. The context is usually clearly defined, our system of interest or the company's internal knowledge of our domain.

Adjust system model interactively

In a certain sense, this limited scope of an assistance system even makes it easier to develop one. In principle, we want a "specialist idiot" who knows everything about our system (but also the domain) and has no interests beyond that.

In systems engineering, we want AI to be a "specialist idiot"!

With such a paradigm, the system model suddenly takes on a new significance, namely the "state of truth" with regard to our "system of interest". When we interact with the AI, we expect it to understand the system model and regard it as the "truth". If it recognizes a conflict through an interaction with a user, it must point this out to the user.

Such an assistance system now has the following capabilities:

• It can answer users' questions about the system without them having to read or understand the model themselves.
• It can recognize contradictions between texts (requirements, descriptions, manuals, etc.) and ask users to resolve the contradictions and clarify ambiguities and the basic comparison of system model and other elements without having to edit the model directly.
• It can recognize gaps or missing information and make suggestions as to how these should be filled.
• It can recognize similarities and thus not only improve requirement texts, but also integrate components from a corresponding library into the system model

What else is possible?

There are certainly many other possible applications. Do you have any ideas? Then please reply directly to this newsletter or use the comments function. We can also arrange to meet at the ReConf in Munich.