I’m becoming obsessed with error messages. Working with web development frameworks and related tooling, I bump into lots of these, and as I’ve written before, it’s easy for them to go wrong. It’s also easy to brush off their imperfections and simply continue on my way. By arming myself with this checklist, I’m less likely to do that. Here’s my new error message testing checklist:Does the error message point to the right problem?
Is the error message out-of-date?
We often re-use error messages across multiple versions of a product, but it can become out-of-date. For example, we might have an ASP.NET error message might reference a version of IIS, but that version of IIS might differ than the one shipped with a newer OS.
Is the error message easy to read?
A smaller than usual font might be used to pack a lot of information in a small amount of space. Or maybe there’s a poor choice of formatting. For example, you have a single large paragraph that could easily be broken up into easy to read bullet points.
Is the error message content appropriate for the setting in which it’s encountered?
Stack traces are often helpful, but only in the right context. An ASP.NET error stack trace is often valuable since it tells the user how *their* code triggered an error (e.g., passing a null value). This stack trace, presented in Visual Studio, is a counterexample. Maybe it’s an exception that should have been caught and re-thrown with a user friendly error message. Anyway, you get the idea.
Should the error message include a link to more information?
If you need to display a lot of content and have limited space, you could include a link to a web page that has more details. This has an additional benefit since you can update the information web page without changing any product code. Here’s an example.
Can the error message be easily copied?
In the dialog above, it’s not easy to highlight the link for copying. The trick is to select the dialog and press CTRL+C to get the contents. Probably not obvious to users. Making the link clickable directly from the dialog would have been event better.
Does the error message provide overwhelming detail?
Instead or providing individual error messages for individual error conditions, sometimes a single error message is used for a collection of error conditions. Here’s the error message when an ASP.NET automatic database creation step fails. It covers a number of possible reasons for the failure. I’m not saying this is a bad error message, but it’s a possible example of overwhelming detail.
Is the error message ambiguous?
Is the error message localizable?
Showing the same string regardless of OS language would be a mistake. It needs to be translated. With the .NET framework, we pull the text of error messages from resource files that are localized into different languages.
Does the error message require additional context to be understood?
Showing an error message to a co-worker unfamiliar with a feature is a great way to catch this.
So there you have it. A set of some additional checks you can use to better asses the quality of your error messages. Now if we could only prevent users from causing errors in the first place…