One Click Install with Ansible
------------------------------
+Proxy for package manager on host
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+ This step has to be performed only if host is behind HTTP/HTTPS proxy
+
+Proxy URL have to be set in dedicated config file
+
+1. CentOS - /etc/yum.conf
+
+.. code:: bash
+
+ proxy=http://your.proxy.domain:1234
+
+2. Ubuntu - /etc/apt/apt.conf
+
+.. code:: bash
+
+ Acquire::http::Proxy "http://your.proxy.domain:1234"
+
+After update of config file, apt mirrors have to be updated via 'apt-get update'
+
+.. code:: bash
+
+ $ sudo apt-get update
+
+Proxy environment variables(for docker and pip)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+ This step has to be performed only if host is behind HTTP/HTTPS proxy
+
+Configuring proxy for packaging system is not enough, also some proxy
+environment variables have to be set in the system before ansible scripts
+can be started.
+Barometer configures docker proxy automatically via ansible task as a part
+of 'one click install' process - user only has to provide proxy URL using common
+shell environment variables and ansible will automatically configure proxies
+for docker(to be able to fetch barometer images). Another component used by
+ansible (e.g. pip is used for downloading python dependencies) will also benefit
+from setting proxy variables properly in the system.
+
+Proxy variables used by ansible One Click Install:
+ * http_proxy
+ * https_proxy
+ * ftp_proxy
+ * no_proxy
+
+Variables mentioned above have to be visible for superuser (because most
+actions involving ansible-barometer installation require root privileges).
+Proxy variables are commonly defined in '/etc/environment' file (but any other
+place is good as long as variables can be seen by commands using 'su').
+
+Sample proxy configuration in /etc/environment:
+
+.. code:: bash
+
+ http_proxy=http://your.proxy.domain:1234
+ https_proxy=http://your.proxy.domain:1234
+ ftp_proxy=http://your.proxy.domain:1234
+ no_proxy=localhost
+
Install Ansible
^^^^^^^^^^^^^^^
.. note::
$ sudo apt-get install python
$ sudo apt-get install python-pip
- $ sudo pip install 'ansible==2.6.3'
+ $ sudo -H pip install 'ansible==2.6.3'
The following steps have been verified with Ansible 2.6.3 on Centos 7.5.
To install Ansible 2.6.3 on Centos:
$ sudo yum install python
$ sudo yum install epel-release
$ sudo yum install python-pip
- $ sudo pip install 'ansible==2.6.3'
+ $ sudo -H pip install 'ansible==2.6.3'
Clone barometer repo
^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo ssh-copy-id root@localhost
+ $ sudo -i
+ $ ssh-copy-id root@localhost
Verify that key is added and password is not required to connect.
.. code:: bash
- $ sudo ansible-playbook -i default.inv collectd_service.yml
+ $ sudo -H ansible-playbook -i default.inv collectd_service.yml
Check the three containers are running, the output of docker ps should be similar to:
From : https://download.docker.com/linux/centos/gpg
Is this ok [y/N]: y
-Proxy Configuration:
-^^^^^^^^^^^^^^^^^^^^
+Manual proxy configuration for docker
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
.. note::
This applies for both CentOS and Ubuntu.
$ git clone https://gerrit.opnfv.org/gerrit/barometer
$ cd barometer/docker/barometer-collectd
$ sudo docker build -t opnfv/barometer-collectd --build-arg http_proxy=`echo $http_proxy` \
- --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
+ --build-arg https_proxy=`echo $https_proxy` --network=host -f Dockerfile .
+
+.. note::
+ Main directory of barometer source code (directory that contains 'docker',
+ 'docs', 'src' and systems sub-directories) will be referred as
+ ``<BAROMETER_REPO_DIR>``
+
.. note::
In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to be
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo docker run -tid --net=host -v `pwd`/../src/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
- -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd /run_collectd.sh
+ $ cd <BAROMETER_REPO_DIR>
+ $ sudo docker run -ti --net=host -v \
+ `pwd`/src/collectd/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
+ -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd
.. note::
- The docker collectd image contains configuration for all the collectd plugins. In the command
- above we are overriding /opt/collectd/etc/collectd.conf.d by mounting a host directory
- `pwd`/../src/collectd_sample_configs that contains only the sample configurations we are interested
- in running. *It's important to do this if you don't have DPDK, or RDT installed on the host*.
+ The docker collectd image contains configuration for all the collectd
+ plugins. In the command above we are overriding
+ /opt/collectd/etc/collectd.conf.d by mounting a host directory
+ src/collectd/collectd_sample_configs that contains only the sample
+ configurations we are interested in running.
+
+ *If some dependencies for plugins listed in configuration directory
+ aren't met, then collectd startup may fail(collectd tries to
+ initialize plugins configurations for all given config files that can
+ be found in shared configs directory and may fail if some dependency
+ is missing).*
+
+ If `DPDK` or `RDT` can't be installed on host, then corresponding config
+ files should be removed from shared configuration directory
+ (`<BAROMETER_REPO_DIR>/src/collectd/collectd_sample_configs/`) prior
+ to starting barometer-collectd container. By example: in case of missing
+ `DPDK` functionality on the host, `dpdkstat.conf` and `dpdkevents.conf`
+ should be removed.
+
Sample configurations can be found at:
https://github.com/opnfv/barometer/tree/master/src/collectd/collectd_sample_configs
+ List of barometer-collectd dependencies on host for various plugins
+ can be found at:
+ https://wiki.opnfv.org/display/fastpath/Barometer-collectd+host+dependencies
+
Check your docker image is running
.. code:: bash
If you have pulled the pre-built barometer-influxdb and barometer-grafana images there is no
requirement to complete steps outlined in sections `Build InfluxDB Docker Image`_ and
`Build Grafana Docker Image`_ and you can proceed directly to section
- `Run the Influxdb and Grafana Images`_ If you wish to run the barometer-influxdb and
- barometer-grafana images via Docker Compose proceed directly to section
- `Docker Compose`_.
+ `Run the Influxdb and Grafana Images`_
Build InfluxDB docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^
$ cd barometer/docker/barometer-influxdb
$ sudo docker build -t opnfv/barometer-influxdb --build-arg http_proxy=`echo $http_proxy` \
- --build-arg https_proxy=`echo $https_proxy` -f Dockerfile .
+ --build-arg https_proxy=`echo $https_proxy` --network=host -f Dockerfile .
.. note::
In the above mentioned ``docker build`` command, http_proxy & https_proxy arguments needs to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo docker run -tid -v /var/lib/influxdb:/var/lib/influxdb -p 8086:8086 -p 25826:25826 opnfv/barometer-influxdb
+ $ sudo docker run -tid -v /var/lib/influxdb:/var/lib/influxdb --net=host\
+ --name bar-influxdb opnfv/barometer-influxdb
Check your docker image is running
sudo docker exec -ti <CONTAINER ID> /bin/bash
+When both collectd and InfluxDB containers are located
+on the same host, then no additional configuration have to be added and you
+can proceed directly to `Run the Grafana docker image`_ section.
+
+Modify collectd to support InfluxDB on another host
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+If InfluxDB and collectd containers are located on separate hosts, then
+additional configuration have to be done in ``collectd`` container - it
+normally sends data using network plugin to 'localhost/127.0.0.1' therefore
+changing output location is required:
+
+1. Stop and remove running bar-collectd container (if it is running)
+
+ .. code:: bash
+
+ $ sudo docker ps #to get collectd container name
+ $ sudo docker rm -f <COLLECTD_CONTAINER_NAME>
+
+2. Go to location where shared collectd config files are stored
+
+ .. code:: bash
+
+ $ cd <BAROMETER_REPO_DIR>
+ $ cd src/collectd/collectd_sample_configs
+
+3. Edit content of ``network.conf`` file.
+ By default this file looks like that:
+
+ .. code::
+
+ LoadPlugin network
+ <Plugin network>
+ Server "127.0.0.1" "25826"
+ </Plugin>
+
+ ``127.0.0.1`` string has to be replaced with the IP address of host where
+ InfluxDB container is running (e.g. ``192.168.121.111``). Edit this using your
+ favorite text editor.
+
+4. Start again collectd container like it is described in
+ `Run the collectd docker image`_ chapter
+
+ .. code:: bash
+
+ $ cd <BAROMETER_REPO_DIR>
+ $ sudo docker run -ti --name bar-collectd --net=host -v \
+ `pwd`/src/collectd/collectd_sample_configs:/opt/collectd/etc/collectd.conf.d \
+ -v /var/run:/var/run -v /tmp:/tmp --privileged opnfv/barometer-collectd
+
+Now collectd container will be sending data to InfluxDB container located on
+remote Host pointed by IP configured in step 3.
+
Run the Grafana docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
- $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana -v ${PWD}/dashboards:/opt/grafana/dashboards \
- -p 3000:3000 opnfv/barometer-grafana
+ $ cd <BAROMETER_REPO_DIR>
+ $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana \
+ -v ${PWD}/docker/barometer-grafana/dashboards:/opt/grafana/dashboards \
+ --name bar-grafana --net=host opnfv/barometer-grafana
Connecting to an influxdb instance running on remote system with hostname of someserver and IP address
of 192.168.121.111
.. code:: bash
- $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana -p 3000:3000 -e \
- influxdb_host=someserver --add-host someserver:192.168.121.111 opnfv/barometer-grafana
+ $ sudo docker run -tid -v /var/lib/grafana:/var/lib/grafana --net=host -e \
+ influxdb_host=someserver --add-host someserver:192.168.121.111 --name \
+ bar-grafana opnfv/barometer-grafana
Check your docker image is running
Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
+Cleanup of influxdb/grafana configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When user wants to remove current grafana and influxdb configuration,
+folowing actions have to be performed
+
+1. Stop and remove running influxdb and grafana containers
+
+.. code:: bash
+
+ sudo docker rm -f bar-grafana bar-influxdb
+
+2. Remove shared influxdb and grafana folders from the Host
+
+.. code:: bash
+
+ sudo rm -rf /var/lib/grafana
+ sudo rm -rf /var/lib/influxdb
+
+.. note::
+ Shared folders are storing configuration of grafana and influxdb
+ containers. In case of changing influxdb or grafana configuration
+ (e.g. moving influxdb to another host) it is good to perform cleanup
+ on shared folders to not affect new setup with an old configuration.
Build and Run VES and Kafka Docker Images
------------------------------------------
.. note::
If you have pulled the pre-built images there is no requirement to complete steps outlined
in sections `Build Kafka Docker Image`_ and `Build VES Docker Image`_ and you can proceed directly to section
- `Run Kafka Docker Image`_ If you wish to run the docker images via Docker Compose proceed directly to section `Docker Compose`_.
+ `Run Kafka Docker Image`_
Build Kafka docker image
^^^^^^^^^^^^^^^^^^^^^^^^
$ sudo docker run -tid --net=host -v ${PWD}/custom.config:/opt/ves/config/ves_app_config.conf \
-v ${PWD}/yaml/:/opt/ves/yaml/ opnfv/barometer-ves custom.yaml
-Build and Run LocalAgent and Redis Docker Images
+Run VES Test Collector application
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VES Test Collector application can be used for displaying platform
+wide metrics that are collected by barometer-ves container.
+Setup instructions are located in: :ref:`Setup VES Test Collector`
+
+Build and Run DMA and Redis Docker Images
-----------------------------------------------------
-Download LocalAgent docker images
+Download DMA docker images
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-If you wish to use pre-built barometer project's LocalAgent images, you can pull the
-images from https://hub.docker.com/r/opnfv/barometer-localagent/
+If you wish to use pre-built barometer project's DMA images, you can pull the
+images from https://hub.docker.com/r/opnfv/barometer-dma/
.. note::
- If your preference is to build images locally please see sections `Build LocalAgent Docker Image`_
+ If your preference is to build images locally please see sections `Build DMA Docker Image`_
.. code:: bash
- $ docker pull opnfv/barometer-localagent
+ $ docker pull opnfv/barometer-dma
.. note::
If you have pulled the pre-built images there is no requirement to complete steps outlined
- in sections `Build LocalAgent Docker Image`_ and you can proceed directly to section
- `Run LocalAgent Docker Image`_ If you wish to run the docker images via Docker Compose proceed directly to section `Docker Compose`_.
+ in sections `Build DMA Docker Image`_ and you can proceed directly to section
+ `Run DMA Docker Image`_
-Build LocalAgent docker image
+Build DMA docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Build LocalAgent docker image:
+Build DMA docker image:
.. code:: bash
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
- Before running LocalAgent, Redis must be running.
+ Before running DMA, Redis must be running.
Run Redis docker image:
sudo docker ps
-Run LocalAgent docker image
+Run DMA docker image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
-Run LocalAgent docker image with default configuration
+Run DMA docker image with default configuration
.. code:: bash
$ sudo docker cp infofetch:/threshold ./
$ sudo ln -s ${PWD}/threshold /usr/local/bin/
-Docker Compose
---------------
-
-Install docker-compose
-^^^^^^^^^^^^^^^^^^^^^^
-
-On the node where you want to run influxdb + grafana or the node where you want to run the VES app
-zookeeper and Kafka containers together:
-
-.. note::
- The default configuration for all these containers is to run on the localhost. If this is not
- the model you want to use then please make the appropriate configuration changes before launching
- the docker containers.
-
-1. Start by installing docker compose
-
-.. code:: bash
-
- $ sudo curl -L https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
-
-.. note::
- Use the latest Compose release number in the download command. The above command is an example,
- and it may become out-of-date. To ensure you have the latest version, check the Compose repository
- release page on GitHub.
-
-2. Apply executable permissions to the binary:
-
-.. code:: bash
-
- $ sudo chmod +x /usr/bin/docker-compose
-
-3. Test the installation.
-
-.. code:: bash
-
- $ sudo docker-compose --version
-
-Run the InfluxDB and Grafana containers using docker compose
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Launch containers:
-
-.. code:: bash
-
- $ cd barometer/docker/compose/influxdb-grafana/
- $ sudo docker-compose up -d
-
-Check your docker images are running
-
-.. code:: bash
-
- $ sudo docker ps
-
-Connect to <host_ip>:3000 with a browser and log into grafana: admin/admin
-
-Run the Kafka, zookeeper and VES containers using docker compose
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Launch containers:
-
-.. code:: bash
-
- $ cd barometer/docker/compose/ves/
- $ sudo docker-compose up -d
-
-Check your docker images are running
-
-.. code:: bash
-
- $ sudo docker ps
-
-Testing the docker image
-^^^^^^^^^^^^^^^^^^^^^^^^
-TODO
-
References
^^^^^^^^^^^
.. [1] https://docs.docker.com/engine/admin/systemd/#httphttps-proxy