Paths to AI in systems engineering: an INCOSE Mini-Event 2021
An excellent INCOSE event on Artificial Intelligence (AI) in Systems Engineering took place this week. In four presentations, several important topics were addressed that we systems engineers should have on our radar. A few samples: Should training data be treated like software source code? Do we need "MLOps", similar to how DevOps has now become established? And should we introduce "data requirements" as a new category?
On the INCOSE website are all lecture descriptions to find. Here are the documents for the presentation:
Understanding what makes AI tick
The data doesn't know what cause and effect are, we do. You are smarter than your data.
Terril Hurst
Terril Hurst is working on making the decisions of an AI comprehensible. In the lecture "Causal Inference: Key for Opening the AI Black Box for Systems Engineering", he describes a formal approach, Pearl's Causal-Model Inference Engine. His employer Raytheon is a defense contractor - an industry where mistakes can be fatal.
One problem with classic machine learning is that the system only performs pattern recognition. The data does not contain any information on cause and effect. This is why it is so important to involve people with their experience (background knowledge in the image). AI experts always want "more data", but more data is not always the solution.
Bad decisions pose a great danger. This danger can be mitigated, but at a price. A complete, transparent probability model makes it possible to justify decisions. In armaments, the cost of creating such a model is justified.
Treat training data like source code
Barclay Brown posed the following question to the 350 participants: What is the closest thing to training data in development?
- Test data
- Configuration
- Compiled object code
- Source code
- Database
Well, the answer is already in the title. His presentation was entitled "Data Requirements and the Green School Bus Problem". The title refers to the following example:
Whether the system recognizes the green school bus correctly depends to a large extent on the training data. And we can certainly imagine what the headline in the BILD newspaper would look like if a school bus was hit by a AI weapon system would be attacked.
Who is to blame if something goes wrong? The systems engineer!
Barclay Brown
Barclay argues that part of the solution should be a new type of requirement: Data Requirements. He proposes the abbreviation DADS for these, an acronym for:
- D - Diversity: Data diversity
- A - Augmentation: Data diversity can be increased through overlays
- D - Distribution: Should the distribution of the training data in the various categories correspond to the expected distribution? What should be considered?
- S - Synthesis: Virtual test data can effectively supplement or even replace natural data.
There is so much to say about data requirements that I will soon be writing a separate article on the subject in my SEcKI Column will publish.
AI success with MLOps
There is a gold-rush atmosphere in AI! But just like in the gold rush, many organizations are missing out. Mehran Irdmousa addresses this topic in his presentation "Increasing the Success Rate of AI and ML Systems Deployment at the Enterprise Level". First, a few depressing statistics:
There are, of course, many reasons for these figures. One is the inability to operationalize AI. The motivation and effects are similar to DevOps.
I've seen waaaayyy too many AI projects that neglected this, and failed.
Philip Hallenbeck in the chat during the lecture
The author presented possible processes that are based on DevOps but also highlight some important differences. For example, "data drift", an AI-specific phenomenon, must be taken into account.
A new class of systems
In his presentation "Challenges to the Verification and Validation of Intelligent Systems", Alejandro Salado argued that traditional approaches are no longer fit for purpose for AI systems. He posed this question in the context of V&V activities.
We need "Adaptive V&V" for intelligent systems
Alejandro Salado
Current V&V approaches are based on the assumption that system behavior is maintained during the lifetime of a system. However, intelligent systems are designed to evolve their own behavior during their lifetime; this is the purpose of AI. Due to this discrepancy, existing approaches to developing and implementing V&V strategies for systems containing AI are ineffective.
INCOSE working groups on AI
Whether at INCOSE or in Germany at the GfSE - AI is being eagerly discussed, if not hyped, among systems engineers. But hype or not, we need to be aware of the opportunities and risks. If you want to get involved, you will find the right working group at both INCOSE and GfSE.
Image source: FreePik
