Alarms¶

Overview¶

Alarms configuration is global setting that allow organization to subscribe to various Alerts and/or Notifications and forward those to supported notification channels. Notification settings can be modified on UI at "Manage Account" -> "Alarms Settings". Each channel offers 4 types of filters:

• Alert type - allows filtering based on type of alert. Example: Alert / Notification
• Alert priority - allows filtering based on priority of alert. Example: Low / Medium / High / Critical
• Event category - allows filtering based on category. Example: InstanceEvent / UserEvent / ServiceEvent, etc.
• Event type - allows filtering based on specific event type. Example: UnhealthyInstance / LowDiskSpace / FailedUpdate, etc.
• Environment type - allows filtering based on environment type. Example: PROD / QA / DEV
• Payload - a json structure with properties of the event. Example: instance_id, subscription_id

Each organization can configure multiple channels with different event filters.

Supported notification channels¶

Email¶

Email channel sends notification as email message to email address provided in email channel configuration.

PagerDuty¶

PagerDuty channels sends notification as PagerDuty alert. Recipient is identified by PagerDuty integration key that needs to be provided during configuration.

For more details on how to generate integration key, refer to this PagerDuty documentation.

Webhook¶

Webhook calls HTTP-based callback function with configurable payload. By default, Omnistrate will include following details as request body:

{
    "eventID": "{{ $var.id }}",
    "serviceID": "{{ $var.ServiceID }}",
    "eventName": "{{ $var.Name }}",
    "eventDescription": "{{ $var.Description }}",
    "eventType": "{{ $var.Type }}", 
    "payload": "{{ $var.Payload }}"
}

Webhook channel allows method, endpoint and payload to be provided. If webhook action fails, we will retry few times before failing to notify this channel of the event.

Channels configuration¶

Omnistrate generates variety of Notifications and Alerts whenever a corresponding event occurs. Each notification channel can be configured to only receive specific types (based on event type, environment, priority, etc.).

To add new channel, open "Manage account" -> "Notifications" page on UI where you will be able to add new channel. There are 2 configuration options available when adding a channel:

1. Basic - subscription is created based on environment type, alert type and priority. This option offers less control, but makes it easier to create a channel based on fewer inputs. All of basic dimensions (environment, type and priority) have limited set of options that are unlikely to change as we add new types of alerts. An example of a basic notification rule: "High and critical priority Alerts from prod environments".

2. Advanced - subscription is created based on environment type and specific category and/or event type. This option offers more control, but is based on dimensions such as category and type which are often being expanded. An examples of an advanced notification rule: "All instance and service notifications from QA and canary environments" or "HighCPUUsage event from staging environment".