1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
3 .. (c) <optionally add copywriters name>
5 ===================================
6 OPNFV Barometer User Guide
7 ===================================
13 Barometer collectd plugins description
14 ---------------------------------------
15 .. Describe the specific features and how it is realised in the scenario in a brief manner
16 .. to ensure the user understand the context for the user guide instructions to follow.
18 collectd is a daemon which collects system performance statistics periodically
19 and provides a variety of mechanisms to publish the collected metrics. It
20 supports more than 90 different input and output plugins. Input plugins
21 retrieve metrics and publish them to the collectd deamon, while output plugins
22 publish the data they receive to an end point. collectd also has infrastructure
23 to support thresholding and notification.
25 Barometer has enabled the following collectd plugins:
27 * *dpdkstat plugin*: A read plugin that retrieve stats from the DPDK extended
30 * *dpdkevents plugin*: A read plugin that retrieves DPDK link status and DPDK
31 forwarding cores liveliness status (DPDK Keep Alive).
33 * `gnocchi plugin`_: A write plugin that pushes the retrieved stats to
34 Gnocchi. It's capable of pushing any stats read through collectd to
35 Gnocchi, not just the DPDK stats.
37 * `aodh plugin`_: A notification plugin that pushes events to Aodh, and
38 creates/updates alarms appropriately.
40 * *hugepages plugin*: A read plugin that retrieves the number of available
41 and free hugepages on a platform as well as what is available in terms of
44 * *Open vSwitch events Plugin*: A read plugin that retrieves events from OVS.
46 * *Open vSwitch stats Plugin*: A read plugin that retrieves flow and interface
49 * *mcelog plugin*: A read plugin that uses mcelog client protocol to check for
50 memory Machine Check Exceptions and sends the stats for reported exceptions
52 * *PMU plugin*: A read plugin that provides performance counters data on
53 Intel CPUs using Linux perf interface.
55 * *RDT plugin*: A read plugin that provides the last level cache utilization and
56 memory bandwidth utilization
58 * *virt*: A read plugin that uses virtualization API *libvirt* to gather
59 statistics about virtualized guests on a system directly from the hypervisor,
60 without a need to install collectd instance on the guest.
62 * *SNMP Agent*: A write plugin that will act as a AgentX subagent that receives
63 and handles queries from SNMP master agent and returns the data collected
64 by read plugins. The SNMP Agent plugin handles requests only for OIDs
65 specified in configuration file. To handle SNMP queries the plugin gets data
66 from collectd and translates requested values from collectd's internal format
67 to SNMP format. Supports SNMP: get, getnext and walk requests.
69 All the plugins above are available on the collectd master, except for the
70 Gnocchi and Aodh plugins as they are Python-based plugins and only C plugins
71 are accepted by the collectd community. The Gnocchi and Aodh plugins live in
72 the OpenStack repositories.
74 Other plugins existing as a pull request into collectd master:
76 * *Legacy/IPMI*: A read plugin that reports platform thermals, voltages,
77 fanspeed, current, flow, power etc. Also, the plugin monitors Intelligent
78 Platform Management Interface (IPMI) System Event Log (SEL) and sends the
79 appropriate notifications based on monitored SEL events.
81 * *PCIe AER*: A read plugin that monitors PCIe standard and advanced errors and
82 sends notifications about those errors.
85 Third party application in Barometer repository:
87 * *Open vSwitch PMD stats*: An aplication that retrieves PMD stats from OVS. It is run
90 **Plugins included in the Danube release:**
97 collectd capabilities and usage
98 ------------------------------------
99 .. Describe the specific capabilities and usage for <XYZ> feature.
100 .. Provide enough information that a user will be able to operate the feature on a deployed scenario.
102 .. note:: Plugins included in the OPNFV D release will be built-in to the fuel
103 plugin and available in the /opt/opnfv directory on the fuel master. You don't
104 need to clone the barometer/collectd repos to use these, but you can configure
105 them as shown in the examples below.
107 The collectd plugins in OPNFV are configured with reasonable defaults, but can
110 Building all Barometer upstreamed plugins from scratch
111 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
112 The plugins that have been merged to the collectd master branch can all be
113 built and configured through the barometer repository.
116 * sudo permissions are required to install collectd.
117 * These are instructions for Ubuntu 16.04
119 To build all the upstream plugins, clone the barometer repo:
123 $ git clone https://gerrit.opnfv.org/gerrit/barometer
125 To install collectd as a service and install all it's dependencies:
129 $ cd barometer/systems && ./build_base_machine.sh
131 This will install collectd as a service and the base install directory
132 will be /opt/collectd.
134 Sample configuration files can be found in '/opt/collectd/etc/collectd.conf.d'
137 If you don't want to use one of the Barometer plugins, simply remove the
138 sample config file from '/opt/collectd/etc/collectd.conf.d'
140 If you plan on using the Exec plugin (for OVS_PMD_STATS or for executing scripts
141 on notification generation), the plugin requires a non-root
142 user to execute scripts. By default, `collectd_exec` user is used in the exec.conf
143 provided in the sample configurations directory under src/collectd in the Barometer
144 repo. The scripts *DO NOT* create this user. You need to create this user before you
145 run build_base_machine.sh. Or modify configuration in the sample configurations
146 directory under src/collectd to use another existing non root user before running
147 run build_base_machine.sh.
150 If you are using any Open vSwitch plugins you need to run:
154 $ sudo ovs-vsctl set-manager ptcp:6640
156 After this, you should be able to start collectd as a service
160 $ sudo systemctl status collectd
162 If you want to use granfana to display the metrics you collect, please see:
165 For more information on configuring and installing OpenStack plugins for
166 collectd, check out the `collectd-ceilometer-plugin GSG`_.
168 Below is the per plugin installation and configuration guide, if you only want
169 to install some/particular plugins.
173 Repo: https://github.com/collectd/collectd
177 Dependencies: DPDK (http://dpdk.org/)
179 .. note:: DPDK statistics plugin requires DPDK version 16.04 or later
181 To build and install DPDK to /usr please see:
182 https://github.com/collectd/collectd/blob/master/docs/BUILD.dpdkstat.md
184 Building and installing collectd:
188 $ git clone https://github.com/collectd/collectd.git
191 $ ./configure --enable-syslog --enable-logfile --enable-debug
195 .. note:: If DPDK was installed in a non standard location you will need to
196 specify paths to the header files and libraries using *LIBDPDK_CPPFLAGS* and
197 *LIBDPDK_LDFLAGS*. You will also need to add the DPDK library symbols to the
198 shared library path using *ldconfig*. Note that this update to the shared
199 library path is not persistant (i.e. it will not survive a reboot).
201 Example of specifying custom paths to DPDK headers and libraries:
205 $ ./configure LIBDPDK_CPPFLAGS="path to DPDK header files" LIBDPDK_LDFLAGS="path to DPDK libraries"
207 This will install collectd to /opt/collectd
208 The collectd configuration file can be found at /opt/collectd/etc
210 To configure the dpdkstats plugin you need to modify the configuration file to
220 ProcessType "secondary"
223 EnabledPortMask 0xffff
224 PortName "interface1"
225 PortName "interface2"
229 To configure the dpdkevents plugin you need to modify the configuration file to
234 LoadPlugin dpdkevents
235 <Plugin "dpdkevents">
240 ProcessType "secondary"
243 <Event "link_status">
244 SendEventsOnUpdate true
245 EnabledPortMask 0xffff
246 PortName "interface1"
247 PortName "interface2"
248 SendNotification false
251 SendEventsOnUpdate true
253 KeepAliveShmName "/dpdk_keepalive_shm_name"
254 SendNotification false
258 .. note:: Currently, the DPDK library doesn’t support API to de-initialize
259 the DPDK resources allocated on the initialization. It means, the collectd
260 plugin will not be able to release the allocated DPDK resources
261 (locks/memory/pci bindings etc.) correctly on collectd shutdown or reinitialize
262 the DPDK library if primary DPDK process is restarted. The only way to release
263 those resources is to terminate the process itself. For this reason, the plugin
264 forks off a separate collectd process. This child process becomes a secondary
265 DPDK process which can be run on specific CPU cores configured by user through
266 collectd configuration file (“Coremask” EAL configuration option, the
267 hexadecimal bitmask of the cores to run on).
269 For more information on the plugin parameters, please see:
270 https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
272 .. note:: dpdkstat plugin initialization time depends on read interval. It
273 requires 5 read cycles to set up internal buffers and states. During that time
274 no statistics are submitted. Also if plugin is running and the number of DPDK
275 ports is increased, internal buffers are resized. That requires 3 read cycles
276 and no port statistics are submitted in that time.
278 The Address-Space Layout Randomization (ASLR) security feature in Linux should be
279 disabled, in order for the same hugepage memory mappings to be present in all
280 DPDK multi-process applications.
286 $ sudo echo 0 > /proc/sys/kernel/randomize_va_space
288 To fully enable ASLR:
292 $ sudo echo 2 > /proc/sys/kernel/randomize_va_space
294 .. warning:: Disabling Address-Space Layout Randomization (ASLR) may have security
295 implications. It is recommended to be disabled only when absolutely necessary,
296 and only when all implications of this change have been understood.
298 For more information on multi-process support, please see:
299 http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html
301 **DPDK stats plugin limitations:**
303 1. The DPDK primary process application should use the same version of DPDK
304 that collectd DPDK plugin is using;
306 2. L2 statistics are only supported;
308 3. The plugin has been tested on Intel NIC’s only.
310 **DPDK stats known issues:**
312 * DPDK port visibility
314 When network port controlled by Linux is bound to DPDK driver, the port
315 will not be available in the OS. It affects the SNMP write plugin as those
316 ports will not be present in standard IF-MIB. Thus addition work is
317 required to be done to support DPDK ports and statistics.
321 Repo: https://github.com/collectd/collectd
325 Dependencies: None, but assumes hugepages are configured.
327 To configure some hugepages:
331 sudo mkdir -p /mnt/huge
332 sudo mount -t hugetlbfs nodev /mnt/huge
333 sudo echo 14336 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
335 Building and installing collectd:
339 $ git clone https://github.com/collectd/collectd.git
342 $ ./configure --enable-syslog --enable-logfile --enable-hugepages --enable-debug
346 This will install collectd to /opt/collectd
347 The collectd configuration file can be found at /opt/collectd/etc
348 To configure the hugepages plugin you need to modify the configuration file to
359 ValuesPercentage false
362 For more information on the plugin parameters, please see:
363 https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
367 Repo: https://github.com/collectd/collectd
373 * PMU tools (jevents library) https://github.com/andikleen/pmu-tools
375 To be suitable for use in collectd plugin shared library *libjevents* should be
376 compiled as position-independent code. To do this add the following line to
377 *pmu-tools/jevents/Makefile*:
383 Building and installing *jevents* library:
387 $ git clone https://github.com/andikleen/pmu-tools.git
388 $ cd pmu-tools/jevents/
392 Building and installing collectd:
396 $ git clone https://github.com/collectd/collectd.git
399 $ ./configure --enable-syslog --enable-logfile --with-libjevents=/usr/local --enable-debug
403 This will install collectd to /opt/collectd
404 The collectd configuration file can be found at /opt/collectd/etc
405 To configure the PMU plugin you need to modify the configuration file to
410 <LoadPlugin intel_pmu>
414 ReportHardwareCacheEvents true
415 ReportKernelPMUEvents true
416 ReportSoftwareEvents true
419 For more information on the plugin parameters, please see:
420 https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
424 The plugin opens file descriptors whose quantity depends on number of
425 monitored CPUs and number of monitored counters. Depending on configuration,
426 it might be required to increase the limit on the number of open file
427 descriptors allowed. This can be done using 'ulimit -n' command. If collectd
428 is executed as a service 'LimitNOFILE=' directive should be defined in
429 [Service] section of *collectd.service* file.
433 Repo: https://github.com/collectd/collectd
439 * PQoS/Intel RDT library https://github.com/01org/intel-cmt-cat.git
442 Building and installing PQoS/Intel RDT library:
446 $ git clone https://github.com/01org/intel-cmt-cat.git
449 $ make install PREFIX=/usr
451 You will need to insert the msr kernel module:
457 Building and installing collectd:
461 $ git clone https://github.com/collectd/collectd.git
464 $ ./configure --enable-syslog --enable-logfile --with-libpqos=/usr/ --enable-debug
468 This will install collectd to /opt/collectd
469 The collectd configuration file can be found at /opt/collectd/etc
470 To configure the RDT plugin you need to modify the configuration file to
475 <LoadPlugin intel_rdt>
482 For more information on the plugin parameters, please see:
483 https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
487 Repo: https://github.com/maryamtahhan/collectd
489 Branch: feat_ipmi_events, feat_ipmi_analog
491 Dependencies: OpenIPMI library (http://openipmi.sourceforge.net/)
493 The IPMI plugin is already implemented in the latest collectd and sensors
494 like temperature, voltage, fanspeed, current are already supported there.
495 The list of supported IPMI sensors has been extended and sensors like flow,
496 power are supported now. Also, a System Event Log (SEL) notification feature
499 * The feat_ipmi_events branch includes new SEL feature support in collectd
500 IPMI plugin. If this feature is enabled, the collectd IPMI plugin will
501 dispatch notifications about new events in System Event Log.
503 * The feat_ipmi_analog branch includes the support of extended IPMI sensors in
504 collectd IPMI plugin.
506 **Install dependencies**
508 On Ubuntu, the OpenIPMI library can be installed via apt package manager:
512 $ sudo apt-get install libopenipmi-dev
514 Anyway, it's recommended to use the latest version of the OpenIPMI library as
515 it includes fixes of known issues which aren't included in standard OpenIPMI
516 library package. The latest version of the library can be found at
517 https://sourceforge.net/p/openipmi/code/ci/master/tree/. Steps to install the
518 library from sources are described below.
520 Remove old version of OpenIPMI library:
524 $ sudo apt-get remove libopenipmi-dev
526 Download OpenIPMI library sources:
530 $ git clone https://git.code.sf.net/p/openipmi/code openipmi-code
533 Patch the OpenIPMI pkg-config file to provide correct compilation flags
534 for collectd IPMI plugin:
538 diff --git a/OpenIPMIpthread.pc.in b/OpenIPMIpthread.pc.in
539 index 59b52e5..fffa0d0 100644
540 --- a/OpenIPMIpthread.pc.in
541 +++ b/OpenIPMIpthread.pc.in
542 @@ -6,6 +6,6 @@ includedir=@includedir@
543 Name: OpenIPMIpthread
544 Description: Pthread OS handler for OpenIPMI
546 -Requires: OpenIPMI pthread
548 Libs: -L${libdir} -lOpenIPMIutils -lOpenIPMIpthread
549 -Cflags: -I${includedir}
550 +Cflags: -I${includedir} -pthread
552 Build and install OpenIPMI library:
556 $ autoreconf --install
557 $ ./configure --prefix=/usr
561 Enable IPMI support in the kernel:
565 $ sudo modprobe ipmi_devintf
566 $ sudo modprobe ipmi_si
569 If HW supports IPMI, the ``/dev/ipmi0`` character device will be
572 Clone and install the collectd IPMI plugin:
576 $ git clone https://github.com/maryamtahhan/collectd
578 $ git checkout $BRANCH
580 $ ./configure --enable-syslog --enable-logfile --enable-debug
584 Where $BRANCH is feat_ipmi_events or feat_ipmi_analog.
586 This will install collectd to default folder ``/opt/collectd``. The collectd
587 configuration file (``collectd.conf``) can be found at ``/opt/collectd/etc``. To
588 configure the IPMI plugin you need to modify the file to include:
594 SELEnabled true # only feat_ipmi_events branch supports this
598 By default, IPMI plugin will read all available analog sensor values,
599 dispatch the values to collectd and send SEL notifications.
601 For more information on the IPMI plugin parameters and SEL feature configuration,
603 https://github.com/maryamtahhan/collectd/blob/feat_ipmi_events/src/collectd.conf.pod
605 Extended analog sensors support doesn't require additional configuration. The usual
606 collectd IPMI documentation can be used:
608 - https://collectd.org/wiki/index.php/Plugin:IPMI
609 - https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_ipmi
613 - https://www.kernel.org/doc/Documentation/IPMI.txt
614 - http://www.intel.com/content/www/us/en/servers/ipmi/ipmi-second-gen-interface-spec-v2-rev1-1.html
618 Repo: https://github.com/collectd/collectd
624 Start by installing mcelog.
627 The kernel has to have CONFIG_X86_MCE enabled. For 32bit kernels you
628 need at least a 2.6,30 kernel.
634 $ apt-get update && apt-get install mcelog
640 $ git clone git://git.kernel.org/pub/scm/utils/cpu/mce/mcelog.git
645 $ cp mcelog.service /etc/systemd/system/
646 $ systemctl enable mcelog.service
647 $ systemctl start mcelog.service
650 Verify you got a /dev/mcelog. You can verify the daemon is running completely
657 This should query the information in the running daemon. If it prints nothing
658 that is fine (no errors logged yet). More info @
659 http://www.mcelog.org/installation.html
661 Modify the mcelog configuration file "/etc/mcelog/mcelog.conf" to include or
666 socket-path = /var/run/mcelog-client
668 Clone and install the collectd mcelog plugin:
672 $ git clone https://github.com/maryamtahhan/collectd
674 $ git checkout feat_ras
676 $ ./configure --enable-syslog --enable-logfile --enable-debug
680 This will install collectd to /opt/collectd
681 The collectd configuration file can be found at /opt/collectd/etc
682 To configure the mcelog plugin you need to modify the configuration file to
691 McelogClientSocket "/var/run/mcelog-client"
694 For more information on the plugin parameters, please see:
695 https://github.com/maryamtahhan/collectd/blob/feat_ras/src/collectd.conf.pod
697 Simulating a Machine Check Exception can be done in one of 3 ways:
699 * Running $make test in the mcelog cloned directory - mcelog test suite
703 **mcelog test suite:**
705 It is always a good idea to test an error handling mechanism before it is
706 really needed. mcelog includes a test suite. The test suite relies on
707 mce-inject which needs to be installed and in $PATH.
709 You also need the mce-inject kernel module configured (with
710 CONFIG_X86_MCE_INJECT=y), compiled, installed and loaded:
714 $ modprobe mce-inject
716 Then you can run the mcelog test suite with
722 This will inject different classes of errors and check that the mcelog triggers
723 runs. There will be some kernel messages about page offlining attempts. The
724 test will also lose a few pages of memory in your system (not significant)
726 This test will kill any running mcelog, which needs to be restarted
731 A utility to inject corrected, uncorrected and fatal machine check exceptions
735 $ git clone https://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git
738 $ modprobe mce-inject
740 Modify the test/corrected script to include the following:
745 STATUS 0xcc00008000010090
751 $ ./mce-inject < test/corrected
754 The uncorrected and fatal scripts under test will cause a platform reset.
755 Only the fatal script generates the memory errors**. In order to quickly
756 emulate uncorrected memory errors and avoid host reboot following test errors
757 from mce-test suite can be injected:
761 $ mce-inject mce-test/cases/coverage/soft-inj/recoverable_ucr/data/srao_mem_scrub
765 In addition an more in-depth test of the Linux kernel machine check facilities
766 can be done with the mce-test test suite. mce-test supports testing uncorrected
767 error handling, real error injection, handling of different soft offlining
768 cases, and other tests.
770 **Corrected memory error injection:**
772 To inject corrected memory errors:
774 * Remove sb_edac and edac_core kernel modules: rmmod sb_edac rmmod edac_core
775 * Insert einj module: modprobe einj param_extension=1
776 * Inject an error by specifying details (last command should be repeated at least two times):
780 $ APEI_IF=/sys/kernel/debug/apei/einj
781 $ echo 0x8 > $APEI_IF/error_type
782 $ echo 0x01f5591000 > $APEI_IF/param1
783 $ echo 0xfffffffffffff000 > $APEI_IF/param2
784 $ echo 1 > $APEI_IF/notrigger
785 $ echo 1 > $APEI_IF/error_inject
787 * Check the MCE statistic: mcelog --client. Check the mcelog log for injected error details: less /var/log/mcelog.
790 ^^^^^^^^^^^^^^^^^^^^^
791 OvS Plugins Repo: https://github.com/collectd/collectd
793 OvS Plugins Branch: master
795 OvS Events MIBs: The SNMP OVS interface link status is provided by standard
796 IF-MIB (http://www.net-snmp.org/docs/mibs/IF-MIB.txt)
798 Dependencies: Open vSwitch, Yet Another JSON Library (https://github.com/lloyd/yajl)
800 On Ubuntu, install the dependencies:
804 $ sudo apt-get install libyajl-dev openvswitch-switch
806 Start the Open vSwitch service:
810 $ sudo service openvswitch-switch start
812 configure the ovsdb-server manager:
816 $ sudo ovs-vsctl set-manager ptcp:6640
818 Clone and install the collectd ovs plugin:
824 $ git checkout master
826 $ ./configure --enable-syslog --enable-logfile --enable-debug
830 This will install collectd to /opt/collectd. The collectd configuration file
831 can be found at /opt/collectd/etc. To configure the OVS events plugin you
832 need to modify the configuration file to include:
836 <LoadPlugin ovs_events>
839 <Plugin "ovs_events">
841 Socket "/var/run/openvswitch/db.sock"
842 Interfaces "br0" "veth0"
843 SendNotification false
847 To configure the OVS stats plugin you need to modify the configuration file
852 <LoadPlugin ovs_stats>
858 Socket "/var/run/openvswitch/db.sock"
859 Bridges "br0" "br_ext"
862 For more information on the plugin parameters, please see:
863 https://github.com/collectd/collectd/blob/master/src/collectd.conf.pod
867 Repo: https://gerrit.opnfv.org/gerrit/barometer
870 1. Open vSwitch dependencies are installed.
871 2. Open vSwitch service is running.
872 3. Ovsdb-server manager is configured.
873 You can refer `Open vSwitch Plugins`_ section above for each one of them.
875 OVS PMD stats application is run through the exec plugin.
877 To configure the OVS PMD stats application you need to modify the exec plugin configuration
886 Exec "user:group" "<path to ovs_pmd_stat.sh>"
887 #NotificationExec "nobody" "/usr/lib/collectd/notify.sh"
890 .. note:: Exec plugin configuration has to be changed to use appropriate user before starting collectd service.
892 ovs_pmd_stat.sh calls the script for OVS PMD stats application with its argument:
896 sudo python /usr/local/src/ovs_pmd_stats.py" "--socket-pid-file"
897 "/var/run/openvswitch/ovs-vswitchd.pid"
901 Repo: https://github.com/maryamtahhan/collectd/
905 Dependencies: NET-SNMP library
907 Start by installing net-snmp and dependencies.
913 $ apt-get install snmp snmp-mibs-downloader snmpd libsnmp-dev
914 $ systemctl start snmpd.service
918 Become root to install net-snmp dependencies
922 $ apt-get install libperl-dev
924 Clone and build net-snmp
928 $ git clone https://github.com/haad/net-snmp.git
930 $ ./configure --with-persistent-directory="/var/net-snmp" --with-systemd --enable-shared --prefix=/usr
939 Copy default configuration to persistent folder
943 $ cp EXAMPLE.conf /usr/share/snmp/snmpd.conf
945 Set library path and default MIB configuration
950 $ echo export LD_LIBRARY_PATH=/usr/lib >> .bashrc
951 $ net-snmp-config --default-mibdirs
952 $ net-snmp-config --snmpconfpath
954 Configure snmpd as a service
959 $ cp ./dist/snmpd.service /etc/systemd/system/
960 $ systemctl enable snmpd.service
961 $ systemctl start snmpd.service
963 Add the following line to snmpd.conf configuration file
964 "/usr/share/snmp/snmpd.conf" to make all OID tree visible for SNMP clients:
968 view systemonly included .1
970 To verify that SNMP is working you can get IF-MIB table using SNMP client
971 to view the list of Linux interfaces:
975 $ snmpwalk -v 2c -c public localhost IF-MIB::interfaces
977 Clone and install the collectd snmp_agent plugin:
981 $ git clone https://github.com/maryamtahhan/collectd
983 $ git checkout feat_snmp
985 $ ./configure --enable-syslog --enable-logfile --enable-debug --enable-snmp --with-libnetsnmp
989 This will install collectd to /opt/collectd
990 The collectd configuration file can be found at /opt/collectd/etc
991 **SNMP Agent plugin is a generic plugin and cannot work without configuration**.
992 To configure the snmp_agent plugin you need to modify the configuration file to
993 include OIDs mapped to collectd types. The following example maps scalar
994 memAvailReal OID to value represented as free memory type of memory plugin:
998 LoadPlugin snmp_agent
999 <Plugin "snmp_agent">
1000 <Data "memAvailReal">
1004 OIDs "1.3.6.1.4.1.2021.4.6.0"
1010 * Object instance with Counter64 type is not supported in SNMPv1. When GetNext
1011 request is received, Counter64 type objects will be skipped. When Get
1012 request is received for Counter64 type object, the error will be returned.
1013 * Interfaces that are not visible to Linux like DPDK interfaces cannot be
1014 retreived using standard IF-MIB tables.
1016 For more information on the plugin parameters, please see:
1017 https://github.com/maryamtahhan/collectd/blob/feat_snmp/src/collectd.conf.pod
1019 For more details on AgentX subagent, please see:
1020 http://www.net-snmp.org/tutorial/tutorial-5/toolkit/demon/
1024 Repo: https://github.com/maryamtahhan/collectd
1026 Branch: feat_libvirt_upstream
1028 Dependencies: libvirt (https://libvirt.org/), libxml2
1030 On Ubuntu, install the dependencies:
1034 $ sudo apt-get install libxml2-dev
1038 libvirt version in package manager might be quite old and offer only limited
1039 functionality. Hence, building and installing libvirt from sources is recommended.
1040 Detailed instructions can bet found at:
1041 https://libvirt.org/compiling.html
1043 Certain metrics provided by the plugin have a requirement on a minimal version of
1044 the libvirt API. *File system information* statistics require a *Guest Agent (GA)*
1045 to be installed and configured in a VM. User must make sure that installed GA
1046 version supports retrieving file system information. Number of *Performance monitoring events*
1047 metrics depends on running libvirt daemon version.
1049 .. note:: Please keep in mind that RDT metrics (part of *Performance monitoring
1050 events*) have to be supported by hardware. For more details on hardware support,
1052 https://github.com/01org/intel-cmt-cat
1054 Additionally perf metrics **cannot** be collected if *Intel RDT* plugin is enabled.
1056 libvirt version can be checked with following commands:
1061 $ libvirtd --version
1063 .. table:: Extended statistics requirements
1065 +-------------------------------+--------------------------+-------------+
1066 | Statistic | Min. libvirt API version | Requires GA |
1067 +===============================+==========================+=============+
1068 | Domain reason | 0.9.2 | No |
1069 +-------------------------------+--------------------------+-------------+
1070 | Disk errors | 0.9.10 | No |
1071 +-------------------------------+--------------------------+-------------+
1072 | Job statistics | 1.2.9 | No |
1073 +-------------------------------+--------------------------+-------------+
1074 | File system information | 1.2.11 | Yes |
1075 +-------------------------------+--------------------------+-------------+
1076 | Performance monitoring events | 1.3.3 | No |
1077 +-------------------------------+--------------------------+-------------+
1079 Start libvirt daemon:
1083 $ systemctl start libvirtd
1085 Create domain (VM) XML configuration file. For more information on domain XML
1086 format and examples, please see:
1087 https://libvirt.org/formatdomain.html
1089 .. note:: Installing additional hypervisor dependencies might be required before
1090 deploying virtual machine.
1092 Create domain, based on created XML file:
1096 $ virsh define DOMAIN_CFG_FILE.xml
1102 $ virsh start DOMAIN_NAME
1104 Check if domain is running:
1110 Check list of available *Performance monitoring events* and their settings:
1114 $ virsh perf DOMAIN_NAME
1116 Enable or disable *Performance monitoring events* for domain:
1120 $ virsh perf DOMAIN_NAME [--enable | --disable] EVENT_NAME --live
1122 Clone and install the collectd virt plugin:
1128 $ git checkout $BRANCH
1130 $ ./configure --enable-syslog --enable-logfile --enable-debug
1134 Where ``$REPO`` and ``$BRANCH`` are equal to information provided above.
1136 This will install collectd to ``/opt/collectd``. The collectd configuration file
1137 ``collectd.conf`` can be found at ``/opt/collectd/etc``. To load the virt plugin
1138 user needs to modify the configuration file to include:
1144 Additionally, user can specify plugin configuration parameters in this file,
1145 such as connection URI, domain name and much more. By default extended virt plugin
1146 statistics are disabled. They can be enabled with ``ExtraStats`` option.
1152 ExtraStats "cpu_util disk disk_err domain_state fs_info job_stats_background pcpu perf vcpupin"
1155 For more information on the plugin parameters, please see:
1156 https://github.com/maryamtahhan/collectd/blob/feat_libvirt_upstream/src/collectd.conf.pod
1158 Installing collectd as a service
1159 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1160 **NOTE**: In an OPNFV installation, collectd is installed and configured as a
1163 Collectd service scripts are available in the collectd/contrib directory.
1164 To install collectd as a service:
1168 $ sudo cp contrib/systemd.collectd.service /etc/systemd/system/
1169 $ cd /etc/systemd/system/
1170 $ sudo mv systemd.collectd.service collectd.service
1171 $ sudo chmod +x collectd.service
1173 Modify collectd.service
1178 ExecStart=/opt/collectd/sbin/collectd
1179 EnvironmentFile=-/opt/collectd/etc/
1180 EnvironmentFile=-/opt/collectd/etc/
1181 CapabilityBoundingSet=CAP_SETUID CAP_SETGID
1187 $ sudo systemctl daemon-reload
1188 $ sudo systemctl start collectd.service
1189 $ sudo systemctl status collectd.service should show success
1191 Additional useful plugins
1192 ^^^^^^^^^^^^^^^^^^^^^^^^^^
1194 * **Exec Plugin** : Can be used to show you when notifications are being
1195 generated by calling a bash script that dumps notifications to file. (handy
1196 for debug). Modify /opt/collectd/etc/collectd.conf:
1202 # Exec "user:group" "/path/to/exec"
1203 NotificationExec "user" "<path to barometer>/barometer/src/collectd/collectd_sample_configs/write_notification.sh"
1206 write_notification.sh (just writes the notification passed from exec through
1207 STDIN to a file (/tmp/notifications)):
1212 rm -f /tmp/notifications
1215 echo $x$y >> /tmp/notifications
1218 output to /tmp/notifications should look like:
1226 PluginInstance:br-ex
1228 TypeInstance:link_status
1229 uuid:f2aafeec-fa98-4e76-aec5-18ae9fc74589
1231 linkstate of "br-ex" interface has been changed to "DOWN"
1233 * **logfile plugin**: Can be used to log collectd activity. Modify
1234 /opt/collectd/etc/collectd.conf to include:
1241 File "/var/log/collectd.log"
1247 Monitoring Interfaces and Openstack Support
1248 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1249 .. Figure:: monitoring_interfaces.png
1251 Monitoring Interfaces and Openstack Support
1253 The figure above shows the DPDK L2 forwarding application running on a compute
1254 node, sending and receiving traffic. collectd is also running on this compute
1255 node retrieving the stats periodically from DPDK through the dpdkstat plugin
1256 and publishing the retrieved stats to OpenStack through the
1257 collectd-ceilometer-plugin.
1259 To see this demo in action please checkout: `Barometer OPNFV Summit demo`_
1261 For more information on configuring and installing OpenStack plugins for
1262 collectd, check out the `collectd-ceilometer-plugin GSG`_.
1266 .. [1] https://collectd.org/wiki/index.php/Naming_schema
1267 .. [2] https://github.com/collectd/collectd/blob/master/src/daemon/plugin.h
1268 .. [3] https://collectd.org/wiki/index.php/Value_list_t
1269 .. [4] https://collectd.org/wiki/index.php/Data_set
1270 .. [5] https://collectd.org/documentation/manpages/types.db.5.shtml
1271 .. [6] https://collectd.org/wiki/index.php/Data_source
1272 .. [7] https://collectd.org/wiki/index.php/Meta_Data_Interface
1274 .. _Barometer OPNFV Summit demo: https://prezi.com/kjv6o8ixs6se/software-fastpath-service-quality-metrics-demo/
1275 .. _gnocchi plugin: https://github.com/openstack/collectd-ceilometer-plugin/tree/stable/ocata/
1276 .. _aodh plugin: https://github.com/openstack/collectd-ceilometer-plugin/tree/stable/ocata/
1277 .. _collectd-ceilometer-plugin GSG: https://github.com/openstack/collectd-ceilometer-plugin/blob/master/doc/source/GSG.rst
1278 .. _grafana guide: https://wiki.opnfv.org/display/fastpath/Installing+and+configuring+InfluxDB+and+Grafana+to+display+metrics+with+collectd