* *dpdkevents plugin*: A read plugin that retrieves DPDK link status and DPDK
forwarding cores liveliness status (DPDK Keep Alive).
-* `ceilometer plugin`_: A write plugin that pushes the retrieved stats to
- Ceilometer. It's capable of pushing any stats read through collectd to
- Ceilometer, not just the DPDK stats.
+* `gnocchi plugin`_: A write plugin that pushes the retrieved stats to
+ Gnocchi. It's capable of pushing any stats read through collectd to
+ Gnocchi, not just the DPDK stats.
+
+* `aodh plugin`_: A notification plugin that pushes events to Aodh, and
+ creates/updates alarms appropriately.
* *hugepages plugin*: A read plugin that retrieves the number of available
and free hugepages on a platform as well as what is available in terms of
* *Open vSwitch events Plugin*: A read plugin that retrieves events from OVS.
-* *Open vSwitch stats Plugin*: A read plugin that retrieve flow and interface
+* *Open vSwitch stats Plugin*: A read plugin that retrieves flow and interface
stats from OVS.
* *mcelog plugin*: A read plugin that uses mcelog client protocol to check for
memory Machine Check Exceptions and sends the stats for reported exceptions
-* *RDT plugin*: A read plugin that provides the last level cache utilitzation and
+* *RDT plugin*: A read plugin that provides the last level cache utilization and
memory bandwidth utilization
All the plugins above are available on the collectd master, except for the
-ceilometer plugin as it's a python based plugin and only C plugins are accepted
-by the collectd community. The ceilometer plugin lives in the OpenStack
-repositories.
+Gnocchi and Aodh plugins as they are Python-based plugins and only C plugins
+are accepted by the collectd community. The Gnocchi and Aodh plugins live in
+the OpenStack repositories.
Other plugins existing as a pull request into collectd master:
fanspeed, current, flow, power etc. Also, the plugin monitors Intelligent
Platform Management Interface (IPMI) System Event Log (SEL) and sends the
+* *virt*: A read plugin that uses virtualization API *libvirt* to gather
+ statistics about virtualized guests on a system directly from the hypervisor,
+ without a need to install collectd instance on the guest.
+
+Third party application in Barometer repository:
+
+* *Open vSwitch PMD stats*: An aplication that retrieves PMD stats from OVS. It is run
+ through exec plugin.
**Plugins included in the Danube release:**
them as shown in the examples below.
The collectd plugins in OPNFV are configured with reasonable defaults, but can
- be overriden.
+ be overridden.
Building all Barometer upstreamed plugins from scratch
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* sudo permissions are required to install collectd.
* These are instructions for Ubuntu 16.04
-To build and install these dependencies, clone the barometer repo:
+To build all the upstream plugins, clone the barometer repo:
.. code:: c
$ git clone https://gerrit.opnfv.org/gerrit/barometer
-Install the build dependencies
-
-.. code:: bash
-
- $ ./src/install_build_deps.sh
-
To install collectd as a service and install all it's dependencies:
.. code:: bash
- $ cd barometer/src && sudo make && sudo make install
+ $ cd barometer/systems && ./build_base_machine.sh
This will install collectd as a service and the base install directory
will be /opt/collectd.
Sample configuration files can be found in '/opt/collectd/etc/collectd.conf.d'
.. note::
- - If you plan on using the Exec plugin, the plugin requires non-root
- user to execute scripts. By default, `collectd_exec` user is used. Barometer
- scripts do *not* create this user. It needs to be manually added or exec plugin
- configuration has to be changed to use other, existing user before starting
- collectd service.
+ - If you plan on using the Exec plugin (for OVS_PMD_STATS or for executing scripts
+ on notification generation), the plugin requires a non-root
+ user to execute scripts. By default, `collectd_exec` user is used in the exec.conf
+ provided in the sample configurations directory under src/collectd in the Barometer
+ repo. The scripts *DO NOT* create this user. You need to create this user before you
+ run build_base_machine.sh. Or modify configuration in the sample configurations
+ directory under src/collectd to use another existing non root user before running
+ run build_base_machine.sh.
- If you don't want to use one of the Barometer plugins, simply remove the
sample config file from '/opt/collectd/etc/collectd.conf.d'
Below is the per plugin installation and configuration guide, if you only want
to install some/particular plugins.
-DPDK statistics plugin
-^^^^^^^^^^^^^^^^^^^^^^
+DPDK plugins
+^^^^^^^^^^^^^
Repo: https://github.com/collectd/collectd
Branch: master
-Dependencies: DPDK (http://dpdk.org/) Min_Version: 16.04
+Dependencies: DPDK (http://dpdk.org/)
+
+.. note:: DPDK statistics plugin requires DPDK version 16.04 or later
To build and install DPDK to /usr please see:
https://github.com/collectd/collectd/blob/master/docs/BUILD.dpdkstat.md
$ make
$ sudo make install
+.. note:: If DPDK was installed in a non standard location you will need to
+ specify paths to the header files and libraries using *LIBDPDK_CPPFLAGS* and
+ *LIBDPDK_LDFLAGS*. You will also need to add the DPDK library symbols to the
+ shared library path using *ldconfig*. Note that this update to the shared
+ library path is not persistant (i.e. it will not survive a reboot).
+
+Example of specifying custom paths to DPDK headers and libraries:
+
+.. code:: bash
+
+ $ ./configure LIBDPDK_CPPFLAGS="path to DPDK header files" LIBDPDK_LDFLAGS="path to DPDK libraries"
This will install collectd to /opt/collectd
The collectd configuration file can be found at /opt/collectd/etc
+
To configure the dpdkstats plugin you need to modify the configuration file to
include:
PortName "interface2"
</Plugin>
-For more information on the plugin parameters, please see:
-https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
-
-.. note:: If you are not building and installing DPDK system-wide
- you will need to specify the specific paths to the header files and libraries
- using LIBDPDK_CPPFLAGS and LIBDPDK_LDFLAGS. You will also need to add the DPDK
- library symbols to the shared library path using ldconfig. Note that this
- update to the shared library path is not persistant (i.e. it will not survive a
- reboot).
-.. note:: Plugin initialization time depends on read interval. It requires
- 5 read cycles to set up internal buffers and states. During that time
- no statistics are submitted. Also if plugin is running and the number of DPDK
- ports is increased, internal buffers are resized. That requires 3 read cycles
- and no port statistics are submitted in that time.
-
-DPDK events plugin
-^^^^^^^^^^^^^^^^^^^^^^
-Repo: https://github.com/collectd/collectd
-
-Branch: master
-
-Dependencies: DPDK (http://dpdk.org/)
-
-To build and install DPDK to /usr please see:
-https://github.com/collectd/collectd/blob/master/docs/BUILD.dpdkstat.md
-
-Building and installing collectd:
-
-.. code:: bash
-
- $ git clone https://github.com/maryamtahhan/collectd.git
- $ cd collectd
- $ ./build.sh
- $ ./configure --enable-syslog --enable-logfile --enable-debug
- $ make
- $ sudo make install
-
-This will install collectd to /opt/collectd
-The collectd configuration file can be found at /opt/collectd/etc
To configure the dpdkevents plugin you need to modify the configuration file to
include:
</Event>
</Plugin>
+.. note:: Currently, the DPDK library doesn’t support API to de-initialize
+ the DPDK resources allocated on the initialization. It means, the collectd
+ plugin will not be able to release the allocated DPDK resources
+ (locks/memory/pci bindings etc.) correctly on collectd shutdown or reinitialize
+ the DPDK library if primary DPDK process is restarted. The only way to release
+ those resources is to terminate the process itself. For this reason, the plugin
+ forks off a separate collectd process. This child process becomes a secondary
+ DPDK process which can be run on specific CPU cores configured by user through
+ collectd configuration file (“Coremask” EAL configuration option, the
+ hexadecimal bitmask of the cores to run on).
+
For more information on the plugin parameters, please see:
https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
-.. note:: If you are not building and installing DPDK system-wide
- you will need to specify the specific paths to the header files and libraries
- using LIBDPDK_CPPFLAGS and LIBDPDK_LDFLAGS. You will also need to add the DPDK
- library symbols to the shared library path using ldconfig. Note that this
- update to the shared library path is not persistant (i.e. it will not survive a
- reboot).
+.. note:: dpdkstat plugin initialization time depends on read interval. It
+ requires 5 read cycles to set up internal buffers and states. During that time
+ no statistics are submitted. Also if plugin is running and the number of DPDK
+ ports is increased, internal buffers are resized. That requires 3 read cycles
+ and no port statistics are submitted in that time.
+
+The Address-Space Layout Randomization (ASLR) security feature in Linux should be
+disabled, in order for the same hugepage memory mappings to be present in all
+DPDK multi-process applications.
+
+To disable ASLR:
.. code:: bash
- $ ./configure LIBDPDK_CPPFLAGS="path to DPDK header files" LIBDPDK_LDFLAGS="path to DPDK libraries"
+ $ sudo echo 0 > /proc/sys/kernel/randomize_va_space
+
+To fully enable ASLR:
+
+.. code:: bash
+
+ $ sudo echo 2 > /proc/sys/kernel/randomize_va_space
+
+.. warning:: Disabling Address-Space Layout Randomization (ASLR) may have security
+ implications. It is recommended to be disabled only when absolutely necessary,
+ and only when all implications of this change have been understood.
+
+For more information on multi-process support, please see:
+http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html
+
+**DPDK stats plugin limitations:**
+
+1. The DPDK primary process application should use the same version of DPDK
+ that collectd DPDK plugin is using;
+
+2. L2 statistics are only supported;
+
+3. The plugin has been tested on Intel NIC’s only.
+
+**DPDK stats known issues:**
+
+* DPDK port visibility
+
+ When network port controlled by Linux is bound to DPDK driver, the port
+ will not be available in the OS. It affects the SNMP write plugin as those
+ ports will not be present in standard IF-MIB. Thus addition work is
+ required to be done to support DPDK ports and statistics.
Hugepages Plugin
^^^^^^^^^^^^^^^^^
Branch: feat_ipmi_events, feat_ipmi_analog
-Dependencies: OpenIPMI library
+Dependencies: OpenIPMI library (http://openipmi.sourceforge.net/)
The IPMI plugin is already implemented in the latest collectd and sensors
like temperature, voltage, fanspeed, current are already supported there.
* The feat_ipmi_analog branch includes the support of extended IPMI sensors in
collectd IPMI plugin.
-On Ubuntu, install the dependencies:
+**Install dependencies**
+
+On Ubuntu, the OpenIPMI library can be installed via apt package manager:
.. code:: bash
$ sudo apt-get install libopenipmi-dev
+Anyway, it's recommended to use the latest version of the OpenIPMI library as
+it includes fixes of known issues which aren't included in standard OpenIPMI
+library package. The latest version of the library can be found at
+https://sourceforge.net/p/openipmi/code/ci/master/tree/. Steps to install the
+library from sources are described below.
+
+Remove old version of OpenIPMI library:
+
+.. code:: bash
+
+ $ sudo apt-get remove libopenipmi-dev
+
+Download OpenIPMI library sources:
+
+.. code:: bash
+
+ $ git clone https://git.code.sf.net/p/openipmi/code openipmi-code
+ $ cd openipmi-code
+
+Patch the OpenIPMI pkg-config file to provide correct compilation flags
+for collectd IPMI plugin:
+
+.. code:: diff
+
+ diff --git a/OpenIPMIpthread.pc.in b/OpenIPMIpthread.pc.in
+ index 59b52e5..fffa0d0 100644
+ --- a/OpenIPMIpthread.pc.in
+ +++ b/OpenIPMIpthread.pc.in
+ @@ -6,6 +6,6 @@ includedir=@includedir@
+ Name: OpenIPMIpthread
+ Description: Pthread OS handler for OpenIPMI
+ Version: @VERSION@
+ -Requires: OpenIPMI pthread
+ +Requires: OpenIPMI
+ Libs: -L${libdir} -lOpenIPMIutils -lOpenIPMIpthread
+ -Cflags: -I${includedir}
+ +Cflags: -I${includedir} -pthread
+
+Build and install OpenIPMI library:
+
+.. code:: bash
+
+ $ autoreconf --install
+ $ ./configure --prefix=/usr
+ $ make
+ $ sudo make install
+
Enable IPMI support in the kernel:
.. code:: bash
Open vSwitch Plugins
^^^^^^^^^^^^^^^^^^^^^
-OvS Events Repo: https://github.com/collectd/collectd
-
-OvS Stats Repo: https://github.com/maryamtahhan/collectd
+OvS Plugins Repo: https://github.com/collectd/collectd
-OvS Events Branch: master
-
-OvS Stats Branch:feat_ovs_stats
+OvS Plugins Branch: master
OvS Events MIBs: The SNMP OVS interface link status is provided by standard
IF-MIB (http://www.net-snmp.org/docs/mibs/IF-MIB.txt)
$ git clone $REPO
$ cd collectd
- $ git checkout $BRANCH
+ $ git checkout master
$ ./build.sh
$ ./configure --enable-syslog --enable-logfile --enable-debug
$ make
$ sudo make install
-where $REPO is one of the repos listed at the top of this section.
-
-Where $BRANCH is master or feat_ovs_stats.
-
This will install collectd to /opt/collectd. The collectd configuration file
can be found at /opt/collectd/etc. To configure the OVS events plugin you
need to modify the configuration file to include:
For more information on the plugin parameters, please see:
https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
-and
-https://github.com/maryamtahhan/collectd/blob/feat_ovs_stats/src/collectd.conf.pod
+
+OVS PMD stats
+^^^^^^^^^^^^^^
+Repo: https://gerrit.opnfv.org/gerrit/barometer
+
+Prequistes:
+1. Open vSwitch dependencies are installed.
+2. Open vSwitch service is running.
+3. Ovsdb-server manager is configured.
+You can refer `Open vSwitch Plugins`_ section above for each one of them.
+
+OVS PMD stats application is run through the exec plugin.
+
+To configure the OVS PMD stats application you need to modify the exec plugin configuration
+to include:
+
+.. code:: bash
+
+ <LoadPlugin exec>
+ Interval 1
+ </LoadPlugin
+ <Plugin exec>
+ Exec "user:group" "<path to ovs_pmd_stat.sh>"
+ #NotificationExec "nobody" "/usr/lib/collectd/notify.sh"
+ </Plugin>
+
+.. note:: Exec plugin configuration has to be changed to use appropriate user before starting collectd service.
+
+ovs_pmd_stat.sh calls the script for OVS PMD stats application with its argument:
+
+.. code:: bash
+
+ sudo python /usr/local/src/ovs_pmd_stats.py" "--socket-pid-file"
+ "/var/run/openvswitch/ovs-vswitchd.pid"
SNMP Agent Plugin
^^^^^^^^^^^^^^^^^
</Data>
</Plugin>
+**Limitations**
+
+* Object instance with Counter64 type is not supported in SNMPv1. When GetNext
+ request is received, Counter64 type objects will be skipped. When Get
+ request is received for Counter64 type object, the error will be returned.
+* Interfaces that are not visible to Linux like DPDK interfaces cannot be
+ retreived using standard IF-MIB tables.
+
For more information on the plugin parameters, please see:
https://github.com/maryamtahhan/collectd/blob/feat_snmp/src/collectd.conf.pod
For more details on AgentX subagent, please see:
http://www.net-snmp.org/tutorial/tutorial-5/toolkit/demon/
+virt plugin
+^^^^^^^^^^^^
+Repo: https://github.com/maryamtahhan/collectd
+
+Branch: feat_libvirt_upstream
+
+Dependencies: libvirt (https://libvirt.org/), libxml2
+
+On Ubuntu, install the dependencies:
+
+.. code:: bash
+
+ $ sudo apt-get install libxml2-dev
+
+Install libvirt:
+
+libvirt version in package manager might be quite old and offer only limited
+functionality. Hence, building and installing libvirt from sources is recommended.
+Detailed instructions can bet found at:
+https://libvirt.org/compiling.html
+
+Certain metrics provided by the plugin have a requirement on a minimal version of
+the libvirt API. *File system information* statistics require a *Guest Agent (GA)*
+to be installed and configured in a VM. User must make sure that installed GA
+version supports retrieving file system information. Number of *Performance monitoring events*
+metrics depends on running libvirt daemon version.
+
+.. note:: Please keep in mind that RDT metrics (part of *Performance monitoring
+ events*) have to be supported by hardware. For more details on hardware support,
+ please see:
+ https://github.com/01org/intel-cmt-cat
+
+ Additionally perf metrics **cannot** be collected if *Intel RDT* plugin is enabled.
+
+libvirt version can be checked with following commands:
+
+.. code:: bash
+
+ $ virsh --version
+ $ libvirtd --version
+
+.. table:: Extended statistics requirements
+
+ +-------------------------------+--------------------------+-------------+
+ | Statistic | Min. libvirt API version | Requires GA |
+ +===============================+==========================+=============+
+ | Domain reason | 0.9.2 | No |
+ +-------------------------------+--------------------------+-------------+
+ | Disk errors | 0.9.10 | No |
+ +-------------------------------+--------------------------+-------------+
+ | Job statistics | 1.2.9 | No |
+ +-------------------------------+--------------------------+-------------+
+ | File system information | 1.2.11 | Yes |
+ +-------------------------------+--------------------------+-------------+
+ | Performance monitoring events | 1.3.3 | No |
+ +-------------------------------+--------------------------+-------------+
+
+Start libvirt daemon:
+
+.. code:: bash
+
+ $ systemctl start libvirtd
+
+Create domain (VM) XML configuration file. For more information on domain XML
+format and examples, please see:
+https://libvirt.org/formatdomain.html
+
+.. note:: Installing additional hypervisor dependencies might be required before
+ deploying virtual machine.
+
+Create domain, based on created XML file:
+
+.. code:: bash
+
+ $ virsh define DOMAIN_CFG_FILE.xml
+
+Start domain:
+
+.. code:: bash
+
+ $ virsh start DOMAIN_NAME
+
+Check if domain is running:
+
+.. code:: bash
+
+ $ virsh list
+
+Check list of available *Performance monitoring events* and their settings:
+
+.. code:: bash
+
+ $ virsh perf DOMAIN_NAME
+
+Enable or disable *Performance monitoring events* for domain:
+
+.. code:: bash
+
+ $ virsh perf DOMAIN_NAME [--enable | --disable] EVENT_NAME --live
+
+Clone and install the collectd virt plugin:
+
+.. code:: bash
+
+ $ git clone $REPO
+ $ cd collectd
+ $ git checkout $BRANCH
+ $ ./build.sh
+ $ ./configure --enable-syslog --enable-logfile --enable-debug
+ $ make
+ $ sudo make install
+
+Where ``$REPO`` and ``$BRANCH`` are equal to information provided above.
+
+This will install collectd to ``/opt/collectd``. The collectd configuration file
+``collectd.conf`` can be found at ``/opt/collectd/etc``. To load the virt plugin
+user needs to modify the configuration file to include:
+
+.. code:: bash
+
+ LoadPlugin virt
+
+Additionally, user can specify plugin configuration parameters in this file,
+such as connection URI, domain name and much more. By default extended virt plugin
+statistics are disabled. They can be enabled with ``ExtraStats`` option.
+
+.. code:: bash
+
+ <Plugin virt>
+ RefreshInterval 60
+ ExtraStats "cpu_util disk disk_err domain_state fs_info job_stats_background pcpu perf vcpupin"
+ </Plugin>
+
+For more information on the plugin parameters, please see:
+https://github.com/maryamtahhan/collectd/blob/feat_libvirt_upstream/src/collectd.conf.pod
+
Installing collectd as a service
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**NOTE**: In an OPNFV installation, collectd is installed and configured as a
The figure above shows the DPDK L2 forwarding application running on a compute
node, sending and receiving traffic. collectd is also running on this compute
node retrieving the stats periodically from DPDK through the dpdkstat plugin
-and publishing the retrieved stats to Ceilometer through the ceilometer plugin.
+and publishing the retrieved stats to OpenStack through the
+collectd-ceilometer-plugin.
To see this demo in action please checkout: `Barometer OPNFV Summit demo`_
+For more information on configuring and installing OpenStack plugins for
+collectd, check out the `collectd-ceilometer-plugin GSG`_.
+
References
^^^^^^^^^^^
.. [1] https://collectd.org/wiki/index.php/Naming_schema
.. [7] https://collectd.org/wiki/index.php/Meta_Data_Interface
.. _Barometer OPNFV Summit demo: https://prezi.com/kjv6o8ixs6se/software-fastpath-service-quality-metrics-demo/
-.. _ceilometer plugin: https://github.com/openstack/collectd-ceilometer-plugin/tree/stable/mitaka
-
+.. _gnocchi plugin: https://github.com/openstack/collectd-ceilometer-plugin/tree/stable/ocata/
+.. _aodh plugin: https://github.com/openstack/collectd-ceilometer-plugin/tree/stable/ocata/
+.. _collectd-ceilometer-plugin GSG: https://github.com/openstack/collectd-ceilometer-plugin/blob/master/doc/source/GSG.rst