Measuring quality requires many questions to be answered. The most obvious ones may be: “What is quality?”, but also “How can we measure it?”, “Which metrics are most accurate?”, “Which are most practical?”.

In this talk, I share some general motivations for measuring quality. I review commonly used metrics that claim to measure quality, I rate them with regards to how they may be helpful or harmful to achieve actual goals. I give some examples how the weaknesses of one metric might be countered by another one to create a beneficial system.

Target Audience: Developers, Project Leader, Manager, Decision Makers, Quality Engineers, Testers, Product Owners
Prerequisites: Basic Software Project Experience, Rough Understanding of Software Development
Level: Advanced

Extended Abstract:
Measuring quality requires many questions to be answered. The most obvious ones may be: “What is quality?”, but also “How can we measure it?”, “Which metrics are most accurate?”, “Which are most practical?”.

In my experience, one question is often not answered or postponed until it is too late: “Why do we want to measure quality?” Is it because we want to control how well our developers are performing? Is it to detect problems early? Is it to measure the impact of changes? Is it the product or the process we care about? Is it to improve locally in a single team or globally across the company? Is there a specific problem that we are trying to solve, and if so, which one?

Instead of trying to define what software quality is – which is hard and depends on a lot of factors – we should first focus on the impact of our measuring. Some metrics may work great for one team, but not for the company as a whole. Some will help to reach your team or organizational goal, some will not help at all, and some will even have terrible side effects by setting unintended incentives. Some can be gamed, others might be harmful to motivation. Consider an overemphasis on lead time, which can lead to cutting corners. Or measuring the number of bugs found, which can cause a testers versus developers situation.

In this talk, I share some general motivations for measuring quality. I review various commonly used metrics that claim to measure quality. Based on my experience, I rate them with regards to how they may be helpful or harmful to achieve actual goals and which side effects are to be expected. I give some examples how the weaknesses of one metric might be countered by another one to create a beneficial system.

UI testing is an essential part of software development. But the automation of UI tests is still considered too complex and flaky.
This talk will cover the "state of the art" of UI testing with an overview of tools and techniques. It will be shown which kind of representations are used by today's test tools and how the addressing of elements in the UI is done.
In addition, the role of artificial intelligence in the different approaches is shown and a prediction of testing tools of the future is presented.

Target Audience: Developers, Testers
Prerequisites: Basic Knowledge of UI-Testing
Level: Advanced

Extended Abstract:
UI testing is an essential part of software development. Despite technological progress, the automation of UI tests is still considered too complex to function completely without manual intervention.
In addition to classical selector-based approaches, more and more image-based methods are being pursued.
This talk will cover the "state of the art" of UI testing with an overview of tools and techniques. In particular, current problems and future developments will be discussed. Furthermore, it will be shown which kind of UI representations are used by today's test tools and how the addressing of elements in the user interface is done.
In addition, the role of artificial intelligence in the different approaches is shown and a prediction of testing tools of the future is presented on the basis of current research.

At first glance, testing AI systems seems very different from testing “conventional” systems. However, many standard testing activities can be preserved as they are or only need small extensions.

In this talk, we give an overview of topics that will help you test AI systems: Attributes of training/testing/validation data, model performance metrics, and the statistical nature of AI systems. We will then relate these to testing tasks and show you how to integrate them.

Target Audience: Developers, Testers, Architects
Prerequisites: Basic knowledge of testing
Level: Basic

Extended Abstract:
From a testing perspective, systems that include AI components seem like a nightmare at first glance. How can you test a system that contains enough math to fill several textbooks and changes its behavior on the whims of its input data? How can you test what even its creators don’t fully understand?

Keep calm, grab a towel and carry on - what you have already been doing is still applicable, and most of the new things you should know are not as arcane as they might seem. Granted, some dimensions of AI systems like bias or explainability will likely not be able to be tested for in all cases. However, this complexity has been around for decades even in systems without any AI whatsoever. Additionally, you will have allies: Data scientists love talking about testing data.

In this talk, we give an overview of topics that will help you test AI systems: Attributes of training/testing/validation data, model performance metrics, and the statistical nature of AI systems. We will then relate these to testing tasks and show you how to integrate them.

Erklärbare KI (Explainable AI, kurz XAI) ermöglicht es, automatisiert situations- und zielgruppenspezifische Begründungen für die Empfehlungen, Prognosen und Entscheidungen von KI-Systemen zu erzeugen.

Hierdurch lassen sich nicht nur Nachvollziehbarkeit und Akzeptanz automatisierter Entscheidungen bei Endanwender:innen steigern, sondern auch Reaktions- und Handlungsmöglichkeiten aufzeigen. Anhand praxisnaher Beispiele demonstrieren wir, wie die Methodenvielfalt Erklärbarer KI für das UX-Design genutzt werden kann.

Zielpublikum: Produktdesigner, UX-Designer, Projektverantwortliche, Datenwissenschaftler, Entscheidungsträger
Voraussetzungen: Grundkenntnisse im UI/UX-Design
Schwierigkeitsgrad: Anfänger

Extended Abstract:
Methoden der Erklärbaren KI (Explainable AI, kurz XAI) werden vor allem zur Analyse und Kontrolle von komplexen KI-Systemen eingesetzt. So nutzen Data Scientists Ansätze wie Partial Dependence Plots oder SHAP, um zu ergründen, welchem Teil der Eingabedaten ein Machine-Learning-Modell besondere Bedeutung zugemessen hat.

Doch Erklärbare KI lässt sich auch zur Gestaltung der User Experience für Endanwender:innen nutzen. “Warum erhalte ich ausgerechnet diese Empfehlung?” “Woran wurde bei dieser E-Mail ein Phishing-Verdacht erkannt?” “Warum ist der prognostizierte Verkaufspreis meiner Immobilie so niedrig?” Fragen dieser Art stellen sich häufig, wenn Nutzer:innen mit Anwendungen und Prozessen in Kontakt kommen, bei denen im Hintergrund ein KI-System am Werk ist, das auch für seine Entwickler:innen eine “Black Box” darstellt.

Erklärbare KI ermöglicht es, automatisiert situations- und zielgruppenspezifische Begründungen für KI-Entscheidungen zu erzeugen. Hierdurch lassen sich nicht nur Nachvollziehbarkeit und Akzeptanz automatisierter Entscheidungen steigern, sondern auch Reaktions- und Handlungsmöglichkeiten aufzeigen.

Neben einer allgemeinen Einführung in das Thema “User-Centric Explainable AI” demonstrieren wir anhand von praxisnahen Beispielen, wie die mittlerweile verfügbare Methodenvielfalt Erklärbarer KI ausgeschöpft und zur Gestaltung der Interaktion von Endanwender:innen genutzt werden kann.