Evals: Unit tests for AI applications

Unit tests are an important concept in software development: they give confidence in the quality of the software and make changes safe. However, unit tests are based on deterministic results, which is not the case with AI applications. Unit tests are therefore not easily transferable to AI applications.

This is exactly where evals come in. They fill this gap and create trust in the quality of the system. But what are evals and how do they work?

Why unit tests are important

Unit tests link requirements and implementation. Used correctly, they create a stable triangular relationship. If a unit test fails, either the implementation, the test code or the requirement (on which the test should be based) is faulty. We are particularly aware of this in the Behavior Driven Development (BDD) to take advantage of this.

These tests are run frequently, ideally during development. But the tests are also carried out later as part of Continuous Integration Pipelines (in addition to other integration tests). Unit tests provide security when changes are made to the code or libraries are replaced. If an automatic unit test fails, we are warned in good time and can quickly localize and rectify the problem. Only when all tests are green again do we continue.

Unit tests are not the only foundation for functioning CI, but they are extremely important. They help companies such as Shopify, Netflix and AmazonDozens of deploys every day without customers noticing.

Why unit tests don't work with AI

First of all: Of course unit tests also work for AI applications. Just not for the parts of the application that generate or process the AI results. In this respect, evals do not replace unit tests, but complement them.

Evals are a supplement to unit tests, not a replacement.

Most AI applications do not work deterministically (remember, LLMs are just one class of AI). We have all seen that the same input can and does produce different outputs. Even worse, there are no binary true/false answers, which is the foundation of unit testing.

Evals solve this problem

Let's start with the Definition of Evalshere the one from OpenAI:

Definition Eval
Evals are structured tests to measure the performance of a model. They help to ensure accuracy, performance and reliability despite the non-deterministic nature of AI systems. They are also one of the few ways to improve the performance of an LLM-based application. (Source: OpenAI)

Evals therefore define what is considered "good" and measure this reproducibly, even if the system is not deterministic. They can be executed in three different ways:

• Human evalsExperts or users (e.g. via thumbs-up/down) evaluate results according to predefined criteria. This approach is suitable for complex, subjective evaluations, but is time-consuming and expensive, and often not particularly meaningful for users.
• Code-based evalsWe write automated checks for formal properties. For example, we can programmatically determine whether generated code can be compiled and executed. This method is cheap and fast, but is not suitable for soft criteria.
• LLM-based evalsWe use a separate model to evaluate the outputs of the system under review based on clearly formulated criteria. This approach works well if the evaluation model has been properly validated.

Here are a few examples of evaluation criteria:

• CompletenessAre all required aspects covered?
• CorrectnessIs the answer consistent with the facts available?
• RelevanceDoes the answer match the question and does it contain the essential points?
• TonalityDoes the linguistic style meet the requirements (e.g. polite, factual)?
• Security: No harmful or inadmissible content?

A good article on Evals can be found at Lenny's Newslettter. If you have some time, I recommend the Interview with Nick Turleyresponsible for ChatGPT at Open AI.

Who writes Evals and when do they run?

The teams that also develop the application, i.e. product managers, AI developers and data scientists, are usually responsible for creating the evals. The QA and DevOps teams must of course also be involved.

Just like unit tests, evals should run in several phases:

• During development for quick feedback on changes
• Before the release as part of the release process
• Continuously in operation to detect quality drift when input data or model versions change

The use of evals during operation is particularly important for several reasons: firstly, it allows us to cover unexpected cases that we had not previously planned for. We can also request user feedback. And very importantly, especially with cloud-based models that we use via an API, their behavior can change without notice!

Since AIs integrated via API can change without notice, monitoring during operation with evals is important.

Introduce evals

As with many practices in development, we should start small with evals and always ensure that there is a clear benefit for development. This means starting where misbehavior causes high costs or risks. For example, when the AI has the competence to quote prices or discounts.

In order to use evals sensibly, we need to define clear, measurable eval criteria for the relevant cases. Initially, code- or LLM-based evals are a sensible choice. Once this approach has been proven to work, we can expand the use of evals further.

In the long term, evals should become an integral part of development processes and DevOps, similar to how unit tests are now a standard part of CI/CD pipelines.

Conclusion

Evals close an important gap in the quality assurance of AI systems. They transfer the basic idea of unit tests to a context in which deterministic expectations do not really work. When implemented correctly, they enable a more objective evaluation, reduce risks and create confidence in the reliability of complex AI applications.

Just as unit tests are an important part of modern software development, evals are becoming an essential concept for the development of robust AI products.