.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) <optionally add copywriters name>
+.. _barometer-docker-userguide:
===================================
OPNFV Barometer Docker User Guide
For steps to build and run VES and Kafka images please see section `Build and Run VES and Kafka Docker Images`_
-For overview of running VES application with Kafka please see the `VES Application User Guide
-<http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/collectd.ves.userguide.html>`_
+For overview of running VES application with Kafka please see the :ref:`VES Application User Guide <barometer-ves-userguide>`
Barometer Docker Images Description
-----------------------------------
docker image has a dependancy on `Zookeeper <https://zookeeper.apache.org/>`_. Kafka must be able to
connect and register with an instance of Zookeeper that is either running on local or remote host.
Kafka recieves and stores metrics recieved from Collectd. VES application pulls latest metrics from Kafka
-which it normalizes into VES format for sending to a VES collector. Please see details in `VES Application User Guide
-<http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/collectd.ves.userguide.html>`_
+which it normalizes into VES format for sending to a VES collector. Please see details in
+:ref:`VES Application User Guide <barometer-ves-userguide>`
-Download and Run Docker Images with Ansible-Playbook
-----------------------------------------------------
+One Click Install with Ansible
+------------------------------
+
+Proxy for package manager on host
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+ This step has to be performed only if host is behind HTTP/HTTPS proxy
+
+Proxy URL have to be set in dedicated config file
+
+1. CentOS - /etc/yum.conf
+
+.. code:: bash
+
+ proxy=http://your.proxy.domain:1234
+
+2. Ubuntu - /etc/apt/apt.conf
+
+.. code:: bash
+
+ Acquire::http::Proxy "http://your.proxy.domain:1234"
+
+After update of config file, apt mirrors have to be updated via 'apt-get update'
+
+.. code:: bash
+
+ $ sudo apt-get update
+
+Proxy environment variables(for docker and pip)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+ This step has to be performed only if host is behind HTTP/HTTPS proxy
+
+Configuring proxy for packaging system is not enough, also some proxy
+environment variables have to be set in the system before ansible scripts
+can be started.
+Barometer configures docker proxy automatically via ansible task as a part
+of 'one click install' process - user only has to provide proxy URL using common
+shell environment variables and ansible will automatically configure proxies
+for docker(to be able to fetch barometer images). Another component used by
+ansible (e.g. pip is used for downloading python dependencies) will also benefit
+from setting proxy variables properly in the system.
+
+Proxy variables used by ansible One Click Install:
+ * http_proxy
+ * https_proxy
+ * ftp_proxy
+ * no_proxy
+
+Variables mentioned above have to be visible for superuser (because most
+actions involving ansible-barometer installation require root privileges).
+Proxy variables are commonly defined in '/etc/environment' file (but any other
+place is good as long as variables can be seen by commands using 'su').
+
+Sample proxy configuration in /etc/environment:
+
+.. code:: bash
+
+ http_proxy=http://your.proxy.domain:1234
+ https_proxy=http://your.proxy.domain:1234
+ ftp_proxy=http://your.proxy.domain:1234
+ no_proxy=localhost
Install Ansible
^^^^^^^^^^^^^^^
* sudo permissions or root access are required to install ansible.
* ansible version needs to be 2.4+, because usage of import/include statements
+The following steps have been verified with Ansible 2.6.3 on Ubuntu 16.04 and 18.04.
To install Ansible 2.6.3 on Ubuntu:
.. code:: bash
$ sudo apt-get install python-pip
$ sudo pip install 'ansible==2.6.3'
+The following steps have been verified with Ansible 2.6.3 on Centos 7.5.
To install Ansible 2.6.3 on Centos:
.. code:: bash
Edit inventory file
^^^^^^^^^^^^^^^^^^^
-Edit inventory file and add hosts: ~/default.inv
+Edit inventory file and add hosts: $barometer_dir/docker/ansible/default.inv
.. code:: bash
[grafana_hosts]
localhost
+ [prometheus_hosts]
+ #localhost
+
[kafka_hosts]
#localhost
install_mcelog=false
insert_ipmi_modules=false
+.. note::
+ On Ubuntu 18.04 to use mcelog plugin the user has to install mcelog daemon
+ manually ahead of installing from ansible scripts as the deb package is not
+ available in official Ubuntu 18.04 repo. It means that setting install_mcelog
+ to true is ignored.
+
Configure ssh keys
^^^^^^^^^^^^^^^^^^
$ sudo ssh-keygen
-Coppy ssh key to all target hosts. It requires to provide root password. The example is for localhost.
+Copy ssh key to all target hosts. It requires to provide root password.
+The example is for localhost.
.. code:: bash
$ sudo ssh-copy-id root@localhost
-Download collectd+influxdb+grafana containers
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Verify that key is added and password is not required to connect.
+
+.. code:: bash
+
+ $ sudo ssh root@localhost
+
+.. note::
+ Keys should be added to every target host and [localhost] is only used as an
+ example. For multinode installation keys need to be copied for each node:
+ [collectd_hostname], [influxdb_hostname] etc.
+
+Download and run Collectd+Influxdb+Grafana containers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The One Click installation features easy and scalable deployment of Collectd,
+Influxdb and Grafana containers using Ansible playbook. The following steps goes
+through more details.
.. code:: bash
- $ sudo ansible-playbook -i ~/default.inv collectd_service.yml
+ $ sudo ansible-playbook -i default.inv collectd_service.yml
Check the three containers are running, the output of docker ps should be similar to:
$ sudo docker exec -ti <CONTAINER ID> /bin/bash
-Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
+Connect to <host_ip>:3000 with a browser and log into Grafana: admin/admin.
+For short introduction please see the:
+`Grafana guide <http://docs.grafana.org/guides/getting_started/>`_.
The collectd configuration files can be accessed directly on target system in '/opt/collectd/etc/collectd.conf.d'.
It can be used for manual changes or enable/disable plugins. If configuration has been modified it is required to
.. code:: bash
- $ sudo ansible-playbook -i ~/default.inv collectd_ves.yml
+ $ sudo ansible-playbook -i default.inv collectd_ves.yml
Check the three containers are running, the output of docker ps should be similar to:
.. code:: bash
- $ sudo ansible-playbook -i ~/default.inv collectd_service.yml --tags "syslog,cpu,uuid"
+ $ sudo ansible-playbook -i default.inv collectd_service.yml --tags "syslog,cpu,uuid"
To disable some parts or plugins:
.. code:: bash
- $ sudo ansible-playbook -i ~/default.inv collectd_service.yml --skip-tags "en_default_all,syslog,cpu,uuid"
+ $ sudo ansible-playbook -i default.inv collectd_service.yml --skip-tags "en_default_all,syslog,cpu,uuid"
List of available tags:
.. Describe the specific capabilities and usage for <XYZ> feature.
.. Provide enough information that a user will be able to operate the feature on a deployed scenario.
+.. note::
+ The below sections provide steps for manual installation and configuration
+ of docker images. They are not neccessary if docker images were installed with
+ use of Ansible-Playbook.
+
On Ubuntu
^^^^^^^^^^
.. note::
From : https://download.docker.com/linux/centos/gpg
Is this ok [y/N]: y
-Proxy Configuration:
-^^^^^^^^^^^^^^^^^^^^
+Manual proxy configuration for docker
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
.. note::
This applies for both CentOS and Ubuntu.
$ sudo docker build -t opnfv/barometer-collectd --build-arg http_proxy=`echo $http_proxy` \
--build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
+.. note::
+ Main directory of barometer source code (directory that contains 'docker',
+ 'docs', 'src' and systems sub-directories) will be referred as
+ ``<BAROMETER_REPO_DIR>``
+
+
.. note::
In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to be
passed only if system is behind an HTTP or HTTPS proxy server.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo docker run -tid --net=host -v `pwd`/../src/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
- -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd /run_collectd.sh
+ $ cd <BAROMETER_REPO_DIR>
+ $ sudo docker run -ti --net=host -v \
+ `pwd`/src/collectd/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
+ -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd
.. note::
- The docker collectd image contains configuration for all the collectd plugins. In the command
- above we are overriding /opt/collectd/etc/collectd.conf.d by mounting a host directory
- `pwd`/../src/collectd_sample_configs that contains only the sample configurations we are interested
- in running. *It's important to do this if you don't have DPDK, or RDT installed on the host*.
+ The docker collectd image contains configuration for all the collectd
+ plugins. In the command above we are overriding
+ /opt/collectd/etc/collectd.conf.d by mounting a host directory
+ src/collectd/collectd_sample_configs that contains only the sample
+ configurations we are interested in running.
+
+ *If some dependencies for plugins listed in configuration directory
+ aren't met, then collectd startup may fail(collectd tries to
+ initialize plugins configurations for all given config files that can
+ be found in shared configs directory and may fail if some dependency
+ is missing).*
+
+ If `DPDK` or `RDT` can't be installed on host, then corresponding config
+ files should be removed from shared configuration directory
+ (`<BAROMETER_REPO_DIR>/src/collectd/collectd_sample_configs/`) prior
+ to starting barometer-collectd container. By example: in case of missing
+ `DPDK` functionality on the host, `dpdkstat.conf` and `dpdkevents.conf`
+ should be removed.
+
Sample configurations can be found at:
https://github.com/opnfv/barometer/tree/master/src/collectd/collectd_sample_configs
+ List of barometer-collectd dependencies on host for various plugins
+ can be found at:
+ https://wiki.opnfv.org/display/fastpath/Barometer-collectd+host+dependencies
+
Check your docker image is running
.. code:: bash
If you have pulled the pre-built barometer-influxdb and barometer-grafana images there is no
requirement to complete steps outlined in sections `Build InfluxDB Docker Image`_ and
`Build Grafana Docker Image`_ and you can proceed directly to section
- `Run the Influxdb and Grafana Images`_ If you wish to run the barometer-influxdb and
- barometer-grafana images via Docker Compose proceed directly to section
- `Docker Compose`_.
+ `Run the Influxdb and Grafana Images`_
Build InfluxDB docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo docker run -tid -v /var/lib/influxdb:/var/lib/influxdb -p 8086:8086 -p 25826:25826 opnfv/barometer-influxdb
+ $ sudo docker run -tid -v /var/lib/influxdb:/var/lib/influxdb --net=host\
+ --name bar-influxdb opnfv/barometer-influxdb
Check your docker image is running
.. code:: bash
- $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana -v ${PWD}/dashboards:/opt/grafana/dashboards \
- -p 3000:3000 opnfv/barometer-grafana
+ $ cd <BAROMETER_REPO_DIR>
+ $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana \
+ -v ${PWD}/docker/barometer-grafana/dashboards:/opt/grafana/dashboards \
+ --name bar-grafana --net=host opnfv/barometer-grafana
Connecting to an influxdb instance running on remote system with hostname of someserver and IP address
of 192.168.121.111
.. code:: bash
- $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana -p 3000:3000 -e \
- influxdb_host=someserver --add-host someserver:192.168.121.111 opnfv/barometer-grafana
+ $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana --net=host -e \
+ influxdb_host=someserver --add-host someserver:192.168.121.111 --name \
+ bar-grafana opnfv/barometer-grafana
Check your docker image is running
.. note::
If you have pulled the pre-built images there is no requirement to complete steps outlined
in sections `Build Kafka Docker Image`_ and `Build VES Docker Image`_ and you can proceed directly to section
- `Run Kafka Docker Image`_ If you wish to run the docker images via Docker Compose proceed directly to section `Docker Compose`_.
+ `Run Kafka Docker Image`_
Build Kafka docker image
^^^^^^^^^^^^^^^^^^^^^^^^
$ sudo docker run -tid --net=host -v ${PWD}/custom.config:/opt/ves/config/ves_app_config.conf \
-v ${PWD}/yaml/:/opt/ves/yaml/ opnfv/barometer-ves custom.yaml
-Build and Run LocalAgent and Redis Docker Images
+Run VES Test Collector application
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VES Test Collector application can be used for displaying platform
+wide metrics that are collected by barometer-ves container.
+Setup instructions are located in: :ref:`Setup VES Test Collector`
+
+Build and Run DMA and Redis Docker Images
-----------------------------------------------------
-Download LocalAgent docker images
+Download DMA docker images
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-If you wish to use pre-built barometer project's LocalAgent images, you can pull the
-images from https://hub.docker.com/r/opnfv/barometer-localagent/
+If you wish to use pre-built barometer project's DMA images, you can pull the
+images from https://hub.docker.com/r/opnfv/barometer-dma/
.. note::
- If your preference is to build images locally please see sections `Build LocalAgent Docker Image`_
+ If your preference is to build images locally please see sections `Build DMA Docker Image`_
.. code:: bash
- $ docker pull opnfv/barometer-localagent
+ $ docker pull opnfv/barometer-dma
.. note::
If you have pulled the pre-built images there is no requirement to complete steps outlined
- in sections `Build LocalAgent Docker Image`_ and you can proceed directly to section
- `Run LocalAgent Docker Image`_ If you wish to run the docker images via Docker Compose proceed directly to section `Docker Compose`_.
+ in sections `Build DMA Docker Image`_ and you can proceed directly to section
+ `Run DMA Docker Image`_
-Build LocalAgent docker image
+Build DMA docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Build LocalAgent docker image:
+Build DMA docker image:
.. code:: bash
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
- Before running LocalAgent, Redis must be running.
+ Before running DMA, Redis must be running.
Run Redis docker image:
sudo docker ps
-Run LocalAgent docker image
+Run DMA docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
-Run LocalAgent docker image with default configuration
+Run DMA docker image with default configuration
.. code:: bash
$ sudo docker cp infofetch:/threshold ./
$ sudo ln -s ${PWD}/threshold /usr/local/bin/
-Docker Compose
---------------
-
-Install docker-compose
-^^^^^^^^^^^^^^^^^^^^^^
-
-On the node where you want to run influxdb + grafana or the node where you want to run the VES app
-zookeeper and Kafka containers together:
-
-.. note::
- The default configuration for all these containers is to run on the localhost. If this is not
- the model you want to use then please make the appropriate configuration changes before launching
- the docker containers.
-
-1. Start by installing docker compose
-
-.. code:: bash
-
- $ sudo curl -L https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
-
-.. note::
- Use the latest Compose release number in the download command. The above command is an example,
- and it may become out-of-date. To ensure you have the latest version, check the Compose repository
- release page on GitHub.
-
-2. Apply executable permissions to the binary:
-
-.. code:: bash
-
- $ sudo chmod +x /usr/bin/docker-compose
-
-3. Test the installation.
-
-.. code:: bash
-
- $ sudo docker-compose --version
-
-Run the InfluxDB and Grafana containers using docker compose
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Launch containers:
-
-.. code:: bash
-
- $ cd barometer/docker/compose/influxdb-grafana/
- $ sudo docker-compose up -d
-
-Check your docker images are running
-
-.. code:: bash
-
- $ sudo docker ps
-
-Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
-
-Run the Kafka, zookeeper and VES containers using docker compose
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Launch containers:
-
-.. code:: bash
-
- $ cd barometer/docker/compose/ves/
- $ sudo docker-compose up -d
-
-Check your docker images are running
-
-.. code:: bash
-
- $ sudo docker ps
-
-Testing the docker image
-^^^^^^^^^^^^^^^^^^^^^^^^
-TODO
-
References
^^^^^^^^^^^
.. [1] https://docs.docker.com/engine/admin/systemd/#httphttps-proxy
Code Review / barometer.git / blobdiff