Monday, May 20, 2024

German Grammar Checkers

I can speak some German. I'll never be fluent, but I can usually get by. Sadly, I make a lot of grammatical errors. It'd be nice to have a tool that could help me find and eliminate them. Syntax and grammar are structured things, seemingly tailor-made for algorithmic analysis. Surely there is software that can analyze my sentences, point out places where I've broken the rules, and tell me how to fix things!

There is. I recently tested more than a dozen programs and web sites that offer this service. The results were less impressive than I'd expected. On my (tiny and unrepresentative) set of sentences containing errors, most tools failed to find most of them. For errors that were found, it was common for the suggested fixes to be wrong. 

I found these sites to offer the most useful results:

  • LanguageTool describes itself as an AI-based spelling, style, and grammar checker. My sense is that the focus is on spelling and grammar, not style. I've found it to do a pretty good job, though there are errors it misses.
  • Scribbr bills itself simply as a grammar checker. It also produces good results, though a hair below those of LanguageTool.
  • DeepL Write claims that its AI approach yields "perfect spelling, grammar, and punctuation" and provides alternative phrasings that "sound fluent, professional, and natural." This means it may rewrite your text to not just eliminate mistakes, but also to make it sound different (presumably better). In my experience, it does a very good job of finding and eliminating errors, but it's sometimes difficult to determine whether it changed something because it's incorrect or because it just felt like rewording it.

In daily use, I generally feed my writing to both LanguageTool and Scribbr, because they're fast, and each sometimes finds mistakes the other misses. If I'm extra-motivated, I also turn to DeepL Write. I've found it to identify mistakes the others miss. I don't use DeepL Write all the time, because I find it annoying to have to tease out whether it changed something on the grounds of correctness or stylistic whim.

In addition to these sites, I also (very cursorily) tested the following systems. I found them to produce notably inferior results. I've listed them in order of decreasing performance, based on my (really limited) tests:

  • QuillBot is a sister company to Scribbr that presumably uses the same underlying technology. I found that the two systems generally give identical results. There are exceptions, however, and in those cases, I found that Scribbr did a better job.
  • Google Docs can be configured to check spelling and grammar as you type. In my testing, it delivered mediocre results.
  • Sapling also produces mediocre results, but it often says "Sign in to see premium edit." I didn't do that, so I can't comment on its premium edits.
  • Microsoft Word, like Google Docs, can be configured to check for spelling and grammar errors as you type. On my tests using Word from Microsoft 365, its coverage was inferior to Google's.
  • Rechtschreibprüfung24 and Korrekturen produced the same results in my testing, so it's possible that they use the same underlying (and unimpressive) checking engine.
  • TextGears and GermanCorrector also produced the same results on my tests, so it's possible that they share a checking engine. The results are similar enough to those from Rechtschreibprüfung24 and Korrekturen that it's conceivable that all four use the same underlying technology. In addition, OnlineKorrektor.de looks and acts identically to GermanCorrector, so it could be that there are two URLs for a single underlying checker.
  • Duden Mentor is the only system I tested that flags the errors it finds, but doesn't offer suggestions on how to fix them.
  • Online-Spellcheck couples its poor ability to find mistakes with a checking speed that is notably worse than its competitors. In addition, it replaces its input window with an output window, so you can't just paste new text in to check something different.
  • Studi-Kompass found none of the errors in my tests. That suggests that it wasn't working or that I was doing something wrong.

I must reiterate that my testing was very limited, so my conclusions are tenuous. If you know of more comprehensive comparisons of German grammar checkers, please share what you know in the comments!

My testing focused on incorrect articles, because that's a problem area for me. I used the following test sentences, where I've boldfaced the part of each sentence that's wrong. I realize that if you know German, you will recognize what's wrong without my help, and if you don't know German, you'll just see randomly boldfaced text, but I can't resist the Siren's call of the boldface error indicator.

  1. Das Tisch sieht gut aus.
  2. Ich gehe im Küche.
  3. Ich bin in die Küche.
  4. Ich will einen Ort finden, die schön aussieht.
  5. Beim Check-in haben wir die Größe des Lobbys bewundert.
  6. Schließlich habe ich mich entschlossen, dass ich einen Ort finden musste, der zwischen Singapur und den USA liegt (d.h., der auf dem Heimweg ist), und die gute Flugverbindungen hat.

I invented sentences 1-3 as representing common simple errors. Sentences 4-6 are from or are variations on things I've actually written.

I scored the systems' results as follows:

  • 2 points if the error was found.
  • 2 more points if only one fix was suggested and it was correct; 1 more point if more than one fix was suggested, but the correct one was among them.
  • -1 point if only incorrect fixes were suggested.
  • -1 point if rewrites were suggested beyond what was in error.  (This is designed to penalize DeepL Write for mixing error corrections and stylistic rewrites.)

If a system found the error in a test sentence and suggested the proper fix (and it didn't suggest anything else), it got the full four points. If it found the error, but it didn't suggest the proper fix, or if it muddied the water with rewrites unrelated to the error, it got between one and three points, depending on the details of what it did.

A perfect score for the set of six sentences would be 24 points. The best any system did was LanguageTool, which got 21. Scribbr was close behind at 20 points. DeepL Write got 19. Then there was a gap until QuillBot's 16 points. Google Docs scored 14, Sapling 13, and Microsoft Word 10. Rechtschreibprüfung24, Korrekturen, TextGears, and GermanCorrector/OnlineKorrektor.de clumped together with 6 points, which is one reason I suspect they may all be using the same checking technology. Duden Mentor also got 6, but its behavior is quite different from the other systems with that score. Online-Spellcheck got 5 points. Studi-Kompass got none, but, as I noted above, my guess is that either the system wasn't working or I was doing something wrong.

 

2 comments:

jbridge said...

ChatGPT 4o corrected all errors and did not modify the rest because I asked it to correct only the syntax.

My post:
Korrigiere folgende Sätze nur syntaktisch (orthografisch und grammatikalisch), jedoch nicht stilistisch:
...

Response of ChatGPT 4o:
Hier sind die Sätze korrigiert:

1. Der Tisch sieht gut aus.
2. Ich gehe in die Küche.
3. Ich bin in der Küche.
4. Ich will einen Ort finden, der schön aussieht.
5. Beim Check-in haben wir die Größe der Lobby bewundert.
6. Schließlich habe ich mich entschlossen, dass ich einen Ort finden musste, der zwischen Singapur und den USA liegt (d.h., der auf dem Heimweg ist), und der gute Flugverbindungen hat.


I use also https://www.epc.de/duden-korrektor-14-fuer-microsoft-office which is an add-in for Microsoft Word and Microsoft Outlook. Disappointingly it missed the errors in the sentences 3 in 4. Furthermore it does not suggest corrections for the found grammatical errors.

Scott Meyers said...

jbridge, this is really useful information, thank you! I'm too cheap to pay for access to ChatGPT 4.0, but I got the same results with Microsoft Copilot (which licenses ChatGPT 4.0). My prompt was

Check the following German sentences for grammatical and spelling correctness. Don't change anything that's not an error.

Copilot's response was:

I’ve found a few errors in your sentences. Here are the corrected versions:

Der Tisch sieht gut aus. (Use “der” instead of “das” for masculine nouns in nominative case)
Ich gehe in die Küche. (Use “in die” instead of “im” when indicating motion towards a place)
Ich bin in der Küche. (Use “in der” instead of “in die” when indicating location)
Ich will einen Ort finden, der schön aussieht. (Use “der” instead of “die” for relative pronouns referring to masculine or neuter nouns)
Beim Check-in haben wir die Größe der Lobby bewundert. (Use “der” instead of “des” for feminine nouns in genitive case)
Schließlich habe ich mich entschlossen, dass ich einen Ort finden musste, der zwischen Singapur und den USA liegt (d.h., der auf dem Heimweg ist), und der gute Flugverbindungen hat. (Use “der” instead of “die” for relative pronouns referring to masculine or neuter nouns)