RE4AI: 6 recommendations for the development of AI systems (Part 2)

Last week I reported on RE4AI, Requirements Engineering for Artificial Intelligence. Among other things, it dealt with the limitations and challenges of RE4AI. All well and good, but how do we approach the whole thing in practice? One possible approach is Google PAIR, which is presented below.

Here it goes to the first part and here is the paper What's up with Requirements Engineering for Artificial Intelligence Systems? by Ahmad et. al.

AI in products: The new challenges

To better understand the challenges of built-in AI, let's take a look at a prominent example: autonomous driving. A sensible requirement could be:

But the devil is in the detail. Is it always clear what a traffic light is? What does "run over" mean? How do we deal with accuracy and precision? What about special cases, such as an approaching vehicle with flashing blue lights? In this case, can the vehicle drive through the red light to make way for the ambulance? How can we tackle these new problems when existing technologies are no longer sufficient?

Google has published a new set of guidelines for developing AI applications with a human-centered approach called PAIR. The authors examine Google PAIR in terms of their findings from the literature review (see part 1). Based on this, the authors make recommendations for future RE4AI research.

Google PAIR

PAIR stands for "People + AI Research", as on the corresponding Google project page can be read.

The central part of PAIR is the People + AI Guidebookwhich consists of a collection of methods, best practices and examples for RE4AI. The guidebook consists of 6 chapters, which the authors of the research paper have summarized very nicely as an activity diagram (Figure 7).

Recommendation: Figure 7 in the paper is an extremely useful activity diagram that shows the overall process in a compact form.

Download paper with Figure 7 now

The chapter structure also describes the step-by-step procedure, with the authors of the paper focusing on the following four.

• User needs - As in traditional RE, the guide begins with the users and the problem description. But then comes an examination of whether AI is a useful solution at all. Is AI a viable solution, e.g. if there is only limited information or data? If so, the use of AI should be classified: Is it about automation or generation? In particular: Will the result be good enough (precision vs. recall)? The literature review revealed that many teams had not investigated the latter thoroughly enough at this stage.
• Data acquisition - The issues to be considered here are known, but a structured approach is important. These include issues such as data protection, data security and completeness, particularly in order to avoid distortions. Also important: if problems arise, it should be possible to trace them back to data, characteristics or designations. Checklists also help here, for example the "five essential characteristics" of accuracy, completeness, consistency, credibility and timeliness.
• Explainable and trustworthy AI - User expectations must be managed. Too much trust in AI can lead to problems, which the press a lot at the moment in relation to chatbots like ChatGPT discussed. An important tool for trust is explanation: AI systems should be able to explain to users how they arrived at the result. The paper cites a lot of existing research and concrete recommendations for the development of AI systems. However, these guidelines are still often ignored.

Google PAIR steps not examined by the authors, but nevertheless central to RE4AI:

• Mental models - A mental model describes our understanding of how something works. With AI, it is important that these change over time. Users are often unaware of this and are irritated when the system suddenly produces a different result a week later. If, on the other hand, users understand this, they can actively help to improve the system.
• Feedback and control - This aspect goes hand in hand with the previous one: If we give users of AI systems the opportunity to provide feedback, we may be able to drastically improve the results. We therefore need to think about appropriate mechanisms.
• Mistakes and benevolent failure - What exactly is an error? In AI systems, an answer with low confidence can quickly be perceived as an error. It is therefore important to first define what an error is in the context of AI systems. It should also be decided how to proceed after an error has occurred.

Recommendations for RE4AI

The authors found many shortcomings in the development of systems that use AI. By combining these shortcomings with the identified "best practices", they make six specific recommendations:

Many problems only benefit from AI to a limited extent. The authors recommend planning a checkpoint during development at which the following questions must be answered: What problem does the system solve? Why is it needed? How will it be used? Does AI add value to the previous points? Does the organization have the resources to develop an AI product?

Many AI systems have problems because the specifications simply do not cover the aspects relevant to AI, such as data collection, data protection, etc. The authors recommend developing a reference system for AI systems that brings together all the requirements relevant to AI systems in a matrix. Until we have such a reference system, guidelines such as Google PAIR in RE4AI can serve us well.

The authors found that a suitable modeling language can be very effective for the development of AI systems. Two of the studies examined suggested developing a separate modeling language for AI4RE. Alternatives already exist today, but classic modeling languages such as SysML only suitable to a limited extent.

In practice, communication between these three roles does not work well, partly because the responsibilities are not clearly delineated. The authors recommend structuring this accordingly and working with a collaboration platform.

With AI applications, there are rarely yes/no decisions. Many factors flow into the overall result. We need to take a systematic approach here. This usually boils down to a quantitative evaluation with corresponding weightings, as recommended by PAIR. There are countless criteria. Which is better: a more accurate algorithm or one that explains the results better? Are the disadvantages of false negatives significantly higher than false positives, or are they the same? The trade-off between Precision and recall is particularly important for AI applications.

Most AI projects start with existing systems and functioning development processes. Many things work. Therefore, we need to be systematic and only start where RE4KI requires new approaches, for example because we can collect data in new ways. To do this, the authors suggest creating a taxonomy that lists all the different techniques and methods required when building AI systems. Until we have this, we should proceed systematically and ensure that our adaptations to methods and techniques achieve positive results.

Conclusion: a lot to do

Today, we still have many gaps in the development of AI-based products. These will certainly be eliminated in the coming years. To develop such products today, it helps to be aware of these gaps. The authors have also pointed to Google PAIR as a powerful tool for RE4AI. The recommendations will help you to safely cross this minefield in your own environment - and to inspire the market with impressive, AI-based products.

Photo by KOBU Agency on Unsplash