Requirements with Vale "linten"

Programmers know what "lintering" is: to put it bluntly, it is the tidying up and maintenance of software code. More precisely, it is static code analysis: a linter recommends deleting unused variables, for example. Vale is a linter that helps authors of texts to maintain them. And Vale can also be used to line requirements.

My interest in Vale is no coincidence: my goal is to have integrated Vale into Semiant, the quality assistant I developed, by the end of June 2022 and to release the free version. If you want to know more, you can go to the Register for the Semiant webinar on June 29 (English) or 30 (German) >>

Linten for software code

Linten goes to a Tool with the name "Lint" which Bell Labs developed for the C programming language back in 1979. It is now so widespread that it is available as a standard feature in almost all software development tools for almost all programming languages.

A linter performs a static code analysis. The linter finds certain errors that the compiler would wave through: These include uninitialized or unused variables, resources that have not been released again, etc. Linters often also take care of non-functional issues such as compliance with formatting guidelines.

Linten for natural language

A natural language linter basically does the same thing: it finds certain errors in style. For example, a linter can find grammatical errors. However, as with the lintering of software code, natural language is usually more about writing style and not correctness. For example, a linter might mark passive constructions as errors.

Lintering for natural language is now widespread. Many writing tools can be provided with a linter, for example via plug-ins. I am currently writing this article in WordPress with Linter (Yoast) installed, as you can see in the screenshot below:

Linten for requirements

Writing requirements is particularly useful for the following two reasons: Firstly, patterns for requirements can be easily recorded in rules. A good example of this is the Sequences of text templates. Ideally, if the linter works directly while writing and suggests improvements in real time, the chance that employees will adhere to the templates increases dramatically. Other "best practices" for writing natural language requirements can also be called for quite easily. For example, "weak words" can be recognized, such as: "some", "slow", "sometimes", etc.

Secondly, a linter can ensure a uniform style: requirements are often written by many authors. A consistent style ensures that the specification feels "of a piece". This may not make for exciting reading, but it helps readers to understand and recognize problems.

Why Vale?

The Homepage of Vale shows, among other things, how popular the project is. Its users include companies such as Microsoft, Spotify and many more. It can be found as an open source project on github. Vale is also very performant. On the developer side Vale has also listed several "competitors" and benchmarks. These are all important criteria for utilization in Semiant.

Use Vale yourself

Next, let's take a look at what Vale can do. The Installation is simple on all common operating systems. The next step is to configure Vale. A good starting point is the Boilerplate repository. A Vale configuration is made up of many rules. Here is a simple rule from the repository:

OxfordComma.yml
extends: existence
message: "Use the Oxford comma in '%s'."
link: https://docs.microsoft.com/en-us/style-guide/punctuation/commas
scope: sentence
level: warning
tokens:
  - '(?:[^,]+,){1,}\s\w+\sand'

At the Oxford comma the comma is placed before the last "and" in an enumeration, unlike in German. An example:

Remember to check your grammar, spelling, and punctuation.
Oxford-Komma ---------------------------^

If I now use the above sentence - without the comma - as input, I get the following output from Vale:

$ vale oxford.md
 oxford.md
 1:1 warning Use the Oxford comma in OxfordComma
               'Remember to check your
               grammar, spelling and'.                               
✖ 0 errors, 1 warning and 0 suggestions in 1 file.

So what about AI and NLP?

Anyone who has followed me this far will be wondering whether this is about Natural Language Processing (NLP) has been used. After all, the rule above is more reminiscent of Regular Expressions (which is true). The NLP aspect comes from the preliminary work, recognizable by "scope: sentence" or "tokens:". The NLP engine has already broken down the text into these elements to be processed.

The sad truth: Despite all the hype, natural language processing relies heavily on rules rather than AI.

Nevertheless, Vale can also access token metadata, in particular the POS tags. A Example from the Vale website:

extends: sequence
message: Did you mean "%[1]ss" instead of "%[1]s's"?
tokens:
  - tag: NN
  - pattern: "'s"
  - pattern: are

NN" stands for "noun, singular or mass", i.e. a noun that is either singular or uncountable. The Tags come from the library prose and currently only work for English. This pattern recognizes the error of using an apostrophe before the "s" in the plural of a word.

Vale for Linten from requirements

My goal is to help Webinar on June 29/30, 2022 have at least completed a proof of concept. Style rules in particular are very easy to implement, for example the recognition of weak words:

tokens:
  - much(?:|e|es)

The rule shown finds the German Weak Words "viel", "viele" and "vieles".

It will be exciting to see whether Vale is powerful enough to check more complex rules. I will present what is possible in the webinar.

What do you need...?

Finally: I am of course very interested in what Lint activities you, my readers, would like to see automated. I cannot guarantee that I will implement your suggestions. But no one is helped by a Linter that applies rules that no one is interested in. So: I look forward to your feedback. Feel free to do so anonymously in the comments below, or also in a personal conversation.