Lets clarify and fix what should happen when states change:

States in increasing order of severity are:

1. Normal
2. Warning
3. Critical
4. Unknown

Notifications

You should get a notification any time:

1. A new incident is opened. This is the first non-normal incident on a closed alert.
2. The status increases from the maximum we have seen this incident.

for the following state transitions:

1. Normal
2. Warning
3. Critical
4. Warning
5. Critical

I would expect notifications only on the bold states. This is not currently the behaviour, because of a bug. We are currently also notifying on 5, which should not be.

Dashboard

The dashboard is fairly complicated at this point. We show a variety of things:

• Current status for the alert key is indicated by the small indented bar on the edge of the panel header.
• Last Abnormal status and template are represented by the color of the panel header and the text of the subject. We want this to show the info from the most recent non-normal check we have.
• Worst status this incident is not shown, but should probably be made availible in the expanded info panel for an alert key.

I broke this particular functionality in an attempt to only show the template from the maximum status. It has caused problems and should be fixed. It causes the empty emails among other things.