1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
3 .. (c) <optionally add copywriters name>
5 ===================================
6 OPNFV Barometer Docker User Guide
7 ===================================
13 The intention of this user guide is to outline how to install and test the Barometer project's
14 docker images. The `OPNFV docker hub <https://hub.docker.com/u/opnfv/?page=1>`_ contains 5 docker
15 images from the Barometer project:
17 1. `Collectd docker image <https://hub.docker.com/r/opnfv/barometer-collectd/>`_
18 2. `Influxdb docker image <https://hub.docker.com/r/opnfv/barometer-influxdb/>`_
19 3. `Grafana docker image <https://hub.docker.com/r/opnfv/barometer-grafana/>`_
20 4. `Kafka docker image <https://hub.docker.com/r/opnfv/barometer-kafka/>`_
21 5. `VES application docker image <https://hub.docker.com/r/opnfv/barometer-ves/>`_
23 For description of images please see section `Barometer Docker Images Description`_
25 For steps to build and run Collectd image please see section `Build and Run Collectd Docker Image`_
27 For steps to build and run InfluxDB and Grafana images please see section `Build and Run InfluxDB and Grafana Docker Images`_
29 For steps to build and run VES and Kafka images please see section `Build and Run VES and Kafka Docker Images`_
31 For overview of running VES application with Kafka please see the `VES Application User Guide
32 <http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/collectd.ves.userguide.html>`_
34 Barometer Docker Images Description
35 -----------------------------------
37 .. Describe the specific features and how it is realised in the scenario in a brief manner
38 .. to ensure the user understand the context for the user guide instructions to follow.
40 Barometer Collectd Image
41 ^^^^^^^^^^^^^^^^^^^^^^^^
42 The barometer collectd docker image gives you a collectd installation that includes all
43 the barometer plugins.
46 The Dockerfile is available in the docker/barometer-collectd directory in the barometer repo.
47 The Dockerfile builds a CentOS 7 docker image.
48 The container MUST be run as a privileged container.
50 Collectd is a daemon which collects system performance statistics periodically
51 and provides a variety of mechanisms to publish the collected metrics. It
52 supports more than 90 different input and output plugins. Input plugins
53 retrieve metrics and publish them to the collectd deamon, while output plugins
54 publish the data they receive to an end point. Collectd also has infrastructure
55 to support thresholding and notification.
57 Collectd docker image has enabled the following collectd plugins (in addition
58 to the standard collectd plugins):
61 * Open vSwitch events Plugin
62 * Open vSwitch stats Plugin
70 Plugins and third party applications in Barometer repository that will be available in the
73 * Open vSwitch PMD stats
74 * ONAP VES application
79 InfluxDB + Grafana Docker Images
80 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
82 The Barometer project's InfluxDB and Grafana docker images are 2 docker images that database and graph
83 statistics reported by the Barometer collectd docker. InfluxDB is an open-source time series database
84 tool which stores the data from collectd for future analysis via Grafana, which is a open-source
85 metrics anlytics and visualisation suite which can be accessed through any browser.
87 VES + Kafka Docker Images
88 ^^^^^^^^^^^^^^^^^^^^^^^^^
90 The Barometer project's VES application and Kafka docker images are based on a CentOS 7 image. Kafka
91 docker image has a dependancy on `Zookeeper <https://zookeeper.apache.org/>`_. Kafka must be able to
92 connect and register with an instance of Zookeeper that is either running on local or remote host.
93 Kafka recieves and stores metrics recieved from Collectd. VES application pulls latest metrics from Kafka
94 which it normalizes into VES format for sending to a VES collector. Please see details in `VES Application User Guide
95 <http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/collectd.ves.userguide.html>`_
99 .. Describe the specific capabilities and usage for <XYZ> feature.
100 .. Provide enough information that a user will be able to operate the feature on a deployed scenario.
105 * sudo permissions are required to install docker.
106 * These instructions are for Ubuntu 16.10
112 $ sudo apt-get install curl
113 $ sudo curl -fsSL https://get.docker.com/ | sh
114 $ sudo usermod -aG docker <username>
115 $ sudo systemctl status docker
117 Replace <username> above with an appropriate user name.
122 * sudo permissions are required to install docker.
123 * These instructions are for CentOS 7
129 $ sudo yum remove docker docker-common docker-selinux docker-engine
130 $ sudo yum install -y yum-utils device-mapper-persistent-data lvm2
131 $ sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
132 $ sudo yum-config-manager --enable docker-ce-edge
133 $ sudo yum-config-manager --enable docker-ce-test
134 $ sudo yum install docker-ce
135 $ sudo usermod -aG docker <username>
136 $ sudo systemctl status docker
138 Replace <username> above with an appropriate user name.
141 If this is the first time you are installing a package from a recently added
142 repository, you will be prompted to accept the GPG key, and the key’s
143 fingerprint will be shown. Verify that the fingerprint is correct, and if so,
144 accept the key. The fingerprint should match060A 61C5 1B55 8A7F 742B 77AA C52F
147 Retrieving key from https://download.docker.com/linux/centos/gpg
148 Importing GPG key 0x621E9F35:
149 Userid : "Docker Release (CE rpm) <docker@docker.com>"
150 Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
151 From : https://download.docker.com/linux/centos/gpg
157 This applies for both CentOS and Ubuntu.
159 If you are behind an HTTP or HTTPS proxy server, you will need to add this
160 configuration in the Docker systemd service file.
162 1. Create a systemd drop-in directory for the docker service:
166 $ sudo mkdir -p /etc/systemd/system/docker.service.d
169 called /etc/systemd/system/docker.service.d/http-proxy.conf that adds
170 the HTTP_PROXY environment variable:
175 Environment="HTTP_PROXY=http://proxy.example.com:80/"
177 Or, if you are behind an HTTPS proxy server, create a file
178 called /etc/systemd/system/docker.service.d/https-proxy.conf that adds
179 the HTTPS_PROXY environment variable:
184 Environment="HTTPS_PROXY=https://proxy.example.com:443/"
186 Or create a single file with all the proxy configurations:
187 /etc/systemd/system/docker.service.d/proxy.conf
192 Environment="HTTP_PROXY=http://proxy.example.com:80/"
193 Environment="HTTPS_PROXY=https://proxy.example.com:443/"
194 Environment="FTP_PROXY=ftp://proxy.example.com:443/"
195 Environment="NO_PROXY=localhost"
201 $ sudo systemctl daemon-reload
207 $ sudo systemctl restart docker
209 5. Check docker environment variables:
213 sudo systemctl show --property=Environment docker
215 Test docker installation
216 ^^^^^^^^^^^^^^^^^^^^^^^^
218 This applies for both CentOS and Ubuntu.
222 $ sudo docker run hello-world
224 The output should be something like:
228 Unable to find image 'hello-world:latest' locally
229 latest: Pulling from library/hello-world
230 5b0f327be733: Pull complete
231 Digest: sha256:07d5f7800dfe37b8c2196c7b1c524c33808ce2e0f74e7aa00e603295ca9a0972
232 Status: Downloaded newer image for hello-world:latest
235 This message shows that your installation appears to be working correctly.
237 To generate this message, Docker took the following steps:
238 1. The Docker client contacted the Docker daemon.
239 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
240 3. The Docker daemon created a new container from that image which runs the
241 executable that produces the output you are currently reading.
242 4. The Docker daemon streamed that output to the Docker client, which sent it
245 To try something more ambitious, you can run an Ubuntu container with:
249 $ docker run -it ubuntu bash
251 Build and Run Collectd Docker Image
252 -----------------------------------
254 Download the collectd docker image
255 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
256 If you wish to use a pre-built barometer image, you can pull the barometer
257 image from https://hub.docker.com/r/opnfv/barometer-collectd/
261 $ docker pull opnfv/barometer-collectd
263 Build the collectd docker image
264 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
268 $ git clone https://gerrit.opnfv.org/gerrit/barometer
269 $ cd barometer/docker/barometer-collectd
270 $ sudo docker build -t opnfv/barometer-collectd --build-arg http_proxy=`echo $http_proxy` \
271 --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
274 In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to be
275 passed only if system is behind an HTTP or HTTPS proxy server.
277 Check the docker images:
283 Output should contain a barometer-collectd image:
287 REPOSITORY TAG IMAGE ID CREATED SIZE
288 opnfv/barometer-collectd latest 05f2a3edd96b 3 hours ago 1.2GB
289 centos 7 196e0ce0c9fb 4 weeks ago 197MB
290 centos latest 196e0ce0c9fb 4 weeks ago 197MB
291 hello-world latest 05a3bd381fc2 4 weeks ago 1.84kB
293 Run the collectd docker image
294 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
297 $ sudo docker run -tid --net=host -v `pwd`/../src/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
298 -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd /run_collectd.sh
301 The docker collectd image contains configuration for all the collectd plugins. In the command
302 above we are overriding /opt/collectd/etc/collectd.conf.d by mounting a host directory
303 `pwd`/../src/collectd_sample_configs that contains only the sample configurations we are interested
304 in running. *It's important to do this if you don't have DPDK, or RDT installed on the host*.
305 Sample configurations can be found at:
306 https://github.com/opnfv/barometer/tree/master/src/collectd/collectd_sample_configs
308 To make some changes when the container is running run:
312 sudo docker exec -ti opnfv/barometer-collectd /bin/bash
314 Check your docker image is running
320 Build and Run InfluxDB and Grafana docker images
321 ------------------------------------------------
325 The barometer-influxdb image is based on the influxdb:1.3.7 image from the influxdb dockerhub. To
326 view detils on the base image please visit
327 `https://hub.docker.com/_/influxdb/ <https://hub.docker.com/_/influxdb/>`_ Page includes details of
328 exposed ports and configurable enviromental variables of the base image.
330 The barometer-grafana image is based on grafana:4.6.3 image from the grafana dockerhub. To view
331 details on the base image please visit
332 `https://hub.docker.com/r/grafana/grafana/ <https://hub.docker.com/r/grafana/grafana/>`_ Page
333 includes details on exposed ports and configurable enviromental variables of the base image.
335 The barometer-grafana image includes pre-configured source and dashboards to display statistics exposed
336 by the barometer-collectd image. The default datasource is an influxdb database running on localhost
337 but the address of the influxdb server can be modified when launching the image by setting the
338 environmental variables influxdb_host to IP or hostname of host on which influxdb server is running.
340 Additional dashboards can be added to barometer-grafana by mapping a volume to /opt/grafana/dashboards.
341 Incase where a folder is mounted to this volume only files included in this folder will be visible
342 inside barometer-grafana. To ensure all default files are also loaded please ensure they are included in
343 volume folder been mounted. Appropriate example are given in section `Run the Grafana docker image`_
345 Download the InfluxDB and Grafana docker images
346 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
347 If you wish to use pre-built barometer project's influxdb and grafana images, you can pull the
348 images from https://hub.docker.com/r/opnfv/barometer-influxdb/ and https://hub.docker.com/r/opnfv/barometer-grafana/
351 If your preference is to build images locally please see sections `Build InfluxDB Docker Image`_ and
352 `Build Grafana Docker Image`_
356 $ docker pull opnfv/barometer-influxdb
357 $ docker pull opnfv/barometer-grafana
360 If you have pulled the pre-built barometer-influxdb and barometer-grafana images there is no
361 requirement to complete steps outlined in sections `Build InfluxDB Docker Image`_ and
362 `Build Grafana Docker Image`_ and you can proceed directly to section
363 `Run the Influxdb and Grafana Images`_ If you wish to run the barometer-influxdb and
364 barometer-grafana images via Docker Compose proceed directly to section
367 Build InfluxDB docker image
368 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
370 Build influxdb image from Dockerfile
374 $ cd barometer/docker/barometer-influxdb
375 $ sudo docker build -t opnfv/barometer-influxdb --build-arg http_proxy=`echo $http_proxy` \
376 --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
379 In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to
380 be passed only if system is behind an HTTP or HTTPS proxy server.
382 Check the docker images:
388 Output should contain an influxdb image:
392 REPOSITORY TAG IMAGE ID CREATED SIZE
393 opnfv/barometer-influxdb latest 1e4623a59fe5 3 days ago 191MB
395 Build Grafana docker image
396 ^^^^^^^^^^^^^^^^^^^^^^^^^^
398 Build Grafana image from Dockerfile
402 $ cd barometer/docker/barometer-grafana
403 $ sudo docker build -t opnfv/barometer-grafana --build-arg http_proxy=`echo $http_proxy` \
404 --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
407 In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to
408 be passed only if system is behind an HTTP or HTTPS proxy server.
410 Check the docker images:
416 Output should contain an influxdb image:
420 REPOSITORY TAG IMAGE ID CREATED SIZE
421 opnfv/barometer-grafana latest 05f2a3edd96b 3 hours ago 1.2GB
423 Run the Influxdb and Grafana Images
424 -----------------------------------
426 Run the InfluxDB docker image
427 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
430 $ sudo docker run -tid --net=host -v /var/lib/influxdb:/var/lib/influxdb -p 8086:8086 -p 25826:25826 opnfv/barometer-influxdb
432 To make some changes when the container is running run:
436 sudo docker exec -ti opnfv/barometer-influxdb /bin/bash
438 Check your docker image is running
444 Run the Grafana docker image
445 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
447 Connecting to an influxdb instance running on local system and adding own custom dashboards
451 $ sudo docker run -tid --net=host -v /var/lib/grafana:/var/lib/grafana -v ${PWD}/dashboards:/opt/grafana/dashboards \
452 -p 3000:3000 opnfv/barometer-grafana
454 Connecting to an influxdb instance running on remote system with hostname of someserver and IP address
459 $ sudo docker run -tid --net=host -v /var/lib/grafana:/var/lib/grafana -p 3000:3000 -e \
460 influxdb_host=someserver --add-host someserver:192.168.121.111 opnfv/barometer-grafana
462 To make some changes when the container is running run:
466 sudo docker exec -ti opnfv/barometer-grafana /bin/bash
468 Check your docker image is running
474 Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
477 Build and Run VES and Kafka Docker Images
478 ------------------------------------------
480 Download VES and Kafka docker images
481 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
483 If you wish to use pre-built barometer project's VES and kafka images, you can pull the
484 images from https://hub.docker.com/r/opnfv/barometer-ves/ and https://hub.docker.com/r/opnfv/barometer-kafka/
487 If your preference is to build images locally please see sections `Build the Kafka Image`_ and
492 $ docker pull opnfv/barometer-kafka
493 $ docker pull opnfv/barometer-ves
496 If you have pulled the pre-built images there is no requirement to complete steps outlined
497 in sections `Build Kafka Docker Image`_ and `Build VES Docker Image`_ and you can proceed directly to section
498 `Run Kafka Docker Image`_ If you wish to run the docker images via Docker Compose proceed directly to section `Docker Compose`_.
500 Build Kafka docker image
501 ^^^^^^^^^^^^^^^^^^^^^^^^
503 Build Kafka docker image:
507 $ cd barometer/docker/barometer-kafka
508 $ sudo docker build -t opnfv/barometer-kafka --build-arg http_proxy=`echo $http_proxy` \
509 --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
512 In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs
513 to be passed only if system is behind an HTTP or HTTPS proxy server.
515 Check the docker images:
521 Output should contain a barometer image:
525 REPOSITORY TAG IMAGE ID CREATED SIZE
526 opnfv/barometer-kafka latest 05f2a3edd96b 3 hours ago 1.2GB
528 Build VES docker image
529 ^^^^^^^^^^^^^^^^^^^^^^
531 Build VES application docker image:
535 $ cd barometer/docker/barometer-ves
536 $ sudo docker build -t opnfv/barometer-ves --build-arg http_proxy=`echo $http_proxy` \
537 --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
540 In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs
541 to be passed only if system is behind an HTTP or HTTPS proxy server.
543 Check the docker images:
549 Output should contain a barometer image:
553 REPOSITORY TAG IMAGE ID CREATED SIZE
554 opnfv/barometer-ves latest 05f2a3edd96b 3 hours ago 1.2GB
556 Run Kafka docker image
557 ^^^^^^^^^^^^^^^^^^^^^^
560 Before running Kafka an instance of Zookeeper must be running for the Kafka broker to register
561 with. Zookeeper can be running locally or on a remote platform. Kafka's broker_id and address of
562 its zookeeper instance can be configured by setting values for environmental variables 'broker_id'
563 and 'zookeeper_node'. In instance where 'broker_id' and/or 'zookeeper_node' is not set the default
564 setting of broker_id=0 and zookeeper_node=localhost is used. In intance where Zookeeper is running
565 on same node as Kafka and there is a one to one relationship between Zookeeper and Kafka, default
566 setting can be used. The docker argument `add-host` adds hostname and IP address to
567 /etc/hosts file in container
569 Run zookeeper docker image:
573 $ sudo docker run -tid --net=host -p 2181:2181 31z4/zookeeper
575 Run kafka docker image which connects with a zookeeper instance running on same node with a 1:1 relationship
579 $ sudo docker run -tid --net=host opnfv/barometer_image
582 Run kafka docker image which connects with a zookeeper instance running on a node with IP address of
583 192.168.121.111 using broker ID of 1
587 $ sudo docker run -tid --net=host --env broker_id=1 --env zookeeper_node=zookeeper --add-host \
588 zookeeper:192.168.121.111 opnfv/barometer_image
590 Run VES Application docker image
591 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
593 VES application uses configuration file ves_app_config.conf from directory
594 barometer/3rd_party/collectd-ves-app/ves_app/config/ and host.yaml file from
595 barometer/3rd_party/collectd-ves-app/ves_app/yaml/ by default. If you wish to use a custom config
596 file it should be mounted to mount point /opt/ves/config/ves_app_config.conf. To use an alternative yaml
597 file from folder barometer/3rd_party/collectd-ves-app/ves_app/yaml the name of the yaml file to use
598 should be passed as an additional command. If you wish to use a custom file the file should be
599 mounted to mount point /opt/ves/yaml/ Please see examples below
601 Run VES docker image with default configuration
605 $ sudo docker run -tid --net=host opnfv/barometer-ves
607 Run VES docker image with guest.yaml files from barometer/3rd_party/collectd-ves-app/ves_app/yaml/
611 $ sudo docker run -tid --net=host opnfv/barometer-ves guest.yaml
614 Run VES docker image with using custom config and yaml files. In example below yaml/ folder cotains
615 file named custom.yaml
619 $ sudo docker run -tid --net=host -v ${PWD}/custom.config:/opt/ves/config/ves_app_config.conf \
620 -v ${PWD}/yaml/:/opt/ves/yaml/ opnfv/barometer-ves custom.yaml
625 Install docker-compose
626 ^^^^^^^^^^^^^^^^^^^^^^
628 On the node where you want to run influxdb + grafana or the node where you want to run the VES app
629 zookeeper and Kafka containers together:
632 The default configuration for all these containers is to run on the localhost. If this is not
633 the model you want to use then please make the appropriate configuration changes before launching
634 the docker containers.
636 1. Start by installing docker compose
640 $ sudo curl -L https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
643 Use the latest Compose release number in the download command. The above command is an example,
644 and it may become out-of-date. To ensure you have the latest version, check the Compose repository
645 release page on GitHub.
647 2. Apply executable permissions to the binary:
651 $ sudo chmod +x /usr/bin/docker-compose
653 3. Test the installation.
657 $ sudo docker-compose --version
659 Run the InfluxDB and Grafana containers using docker compose
660 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
666 $ cd barometer/docker/compose/influxdb-grafana/
667 $ sudo docker-compose up -d
669 Check your docker images are running
675 Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
677 Run the Kafka, zookeeper and VES containers using docker compose
678 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
684 $ cd barometer/docker/compose/ves/
685 $ sudo docker-compose up -d
687 Check your docker images are running
693 Testing the docker image
694 ^^^^^^^^^^^^^^^^^^^^^^^^
699 .. [1] https://docs.docker.com/engine/admin/systemd/#httphttps-proxy
700 .. [2] https://docs.docker.com/engine/installation/linux/docker-ce/centos/#install-using-the-repository
701 .. [3] https://docs.docker.com/engine/userguide/