--- /dev/null
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+============================
+Notification Alarm Evaluator
+============================
+
+.. NOTE::
+ This is spec draft of brlueprint for OpenStack Ceilomter Liberty.
+ To see current version: https://review.openstack.org/172893
+ To track development activity:
+ https://blueprints.launchpad.net/ceilometer/+spec/notification-alarm-evaluator
+
+https://blueprints.launchpad.net/ceilometer/+spec/notification-alarm-evaluator
+
+This blueprint proposes to add a new alarm evaluator for handling alarms on
+events passed from other OpenStack services, that provides event-driven alarm
+evaluation which makes new sequence in Ceilometer instead of the polling-based
+approach of the existing Alarm Evaluator, and realizes immediate alarm
+notification to end users.
+
+Problem description
+===================
+
+As an end user, I need to receive alarm notification immediately once
+Ceilometer captured an event which would make alarm fired, so that I can
+perform recovery actions promptly to shorten downtime of my service.
+The typical use case is that an end user set alarm on "compute.instance.update"
+in order to trigger recovery actions once the instance status has changed to
+'shutdown' or 'error'. It should be nice that an end user can receive
+notification within 1 second after fault observed as the same as other helth-
+check mechanisms can do in some cases.
+
+The existing Alarm Evaluator is periodically querying/polling the databases
+in order to check all alarms independently from other processes. This is good
+approach for evaluating an alarm on samples stored in a certain period.
+However, this is not efficient to evaluate an alarm on events which are emitted
+by other OpenStack servers once in a while.
+
+The periodical evaluation leads delay on sending alarm notification to users.
+The default period of evaluation cycle is 60 seconds. It is recommended that
+an operator set longer interval than configured pipeline interval for
+underlying metrics, and also longer enough to evaluate all defined alarms
+in certain period while taking into account the number of resources, users and
+alarms.
+
+Proposed change
+===============
+
+The proposal is to add a new event-driven alarm evaluator which receives
+messages from Notification Agent and finds related Alarms, then evaluates each
+alarms;
+
+* New alarm evaluator could receive event notification from Notification Agent
+ by which adding a dedicated notifier as a publisher in pipeline.yaml
+ (e.g. notifier://?topic=event_eval).
+
+* When new alarm evaluator received event notification, it queries alarm
+ database by Project ID and Resource ID written in the event notification.
+
+* Found alarms are evaluated by referring event notification.
+
+* Depending on the result of evaluation, those alarms would be fired through
+ Alarm Notifier as the same as existing Alarm Evaluator does.
+
+This proposal also adds new alarm type "notification" and "notification_rule".
+This enables users to create alarms on events. The separation from other alarm
+types (such as "threshold" type) is intended to show different timing of
+evaluation and different format of condition, since the new evaluator will
+check each event notification once it received whereas "threshold" alarm can
+evaluate average of values in certain period calculated from multiple samples.
+
+The new alarm evaluator handles Notification type alarms, so we have to change
+existing alarm evaluator to exclude "notification" type alarms from evaluation
+targets.
+
+Alternatives
+------------
+
+There was similar blueprint proposal "Alarm type based on notification", but
+the approach is different. The old proposal was to adding new step (alarm
+evaluations) in Notification Agent every time it received event from other
+OpenStack services, whereas this proposal intends to execute alarm evaluation
+in another component which can minimize impact to existing pipeline processing.
+
+Another approach is enhancement of existing alarm evaluator by adding
+notification listener. However, there are two issues; 1) this approach could
+cause stall of periodical evaluations when it receives bulk of notifications,
+and 2) this could break the alarm portioning i.e. when alarm evaluator received
+notification, it might have to evaluate some alarms which are not assign to it.
+
+Data model impact
+-----------------
+
+Resource ID will be added to Alarm model as an optional attribute.
+This would help the new alarm evaluator to filter out non-related alarms
+while querying alarms, otherwise it have to evaluate all alarms in the project.
+
+REST API impact
+---------------
+
+Alarm API will be extended as follows;
+
+* Add "notification" type into alarm type list
+* Add "resource_id" to "alarm"
+* Add "notification_rule" to "alarm"
+
+Sample data of Notification-type alarm::
+
+ {
+ "alarm_actions": [
+ "http://site:8000/alarm"
+ ],
+ "alarm_id": null,
+ "description": "An alarm",
+ "enabled": true,
+ "insufficient_data_actions": [
+ "http://site:8000/nodata"
+ ],
+ "name": "InstanceStatusAlarm",
+ "notification_rule": {
+ "event_type": "compute.instance.update",
+ "query" : [
+ {
+ "field" : "traits.state",
+ "type" : "string",
+ "value" : "error",
+ "op" : "eq",
+ },
+ ]
+ },
+ "ok_actions": [],
+ "project_id": "c96c887c216949acbdfbd8b494863567",
+ "repeat_actions": false,
+ "resource_id": "153462d0-a9b8-4b5b-8175-9e4b05e9b856",
+ "severity": "moderate",
+ "state": "ok",
+ "state_timestamp": "2015-04-03T17:49:38.406845",
+ "timestamp": "2015-04-03T17:49:38.406839",
+ "type": "notification",
+ "user_id": "c96c887c216949acbdfbd8b494863567"
+ }
+
+"resource_id" will be refered to query alarm and will not be check permission
+and belonging of project.
+
+Security impact
+---------------
+
+None
+
+Pipeline impact
+---------------
+
+None
+
+Other end user impact
+---------------------
+
+None
+
+Performance/Scalability Impacts
+-------------------------------
+
+When Ceilomter received a number of events from other OpenStack services in
+short period, this alarm evaluator can keep working since events are queued in
+a messaging queue system, but it can cause delay of alarm notification to users
+and increase the number of read and write access to alarm database.
+
+"resource_id" can be optional, but restricting it to mandatory could be reduce
+performance impact. If user create "notification" alarm without "resource_id",
+those alarms will be evaluated every time event occurred in the project.
+That may lead new evaluator heavy.
+
+Other deployer impact
+---------------------
+
+New service process have to be run.
+
+Developer impact
+----------------
+
+Developers should be aware that events could be notified to end users and avoid
+passing raw infra information to end users, while defining events and traits.
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Primary assignee:
+ r-mibu
+
+Other contributors:
+ None
+
+Ongoing maintainer:
+ None
+
+Work Items
+----------
+
+* New event-driven alarm evaluator
+
+* Add new alarm type "notification" as well as AlarmNotificationRule
+
+* Add "resource_id" to Alarm model
+
+* Modify existing alarm evaluator to filter out "notification" alarms
+
+* Add new config parameter for alarm request check whether accepting alarms
+ without specifying "resource_id" or not
+
+Future lifecycle
+================
+
+This proposal is key feature to provide information of cloud resources to end
+users in real-time that enables efficient integration with user-side manager
+or Orchestrator, whereas currently those information are considered to be
+consumed by admin side tool or service.
+Based on this change, we will seek orchestrating scenarios including fault
+recovery and add useful event definition as well as additional traits.
+
+Dependencies
+============
+
+None
+
+Testing
+=======
+
+New unit/scenario tests are required for this change.
+
+Documentation Impact
+====================
+
+* Proposed evaluator will be described in the developer document.
+
+* New alarm type and how to use will be explained in user guide.
+
+References
+==========
+
+* OPNFV Doctor project: https://wiki.opnfv.org/doctor
+
+* Blueprint "Alarm type based on notification":
+ https://blueprints.launchpad.net/ceilometer/+spec/alarm-on-notification
Code Review / doctor.git / commitdiff