* opnfv/functest-core
* opnfv/functest-healthcheck
* opnfv/functest-smoke
- * opnfv/functest-features
- * opnfv/functest-components
+ * opnfv/functest-benchmarking
* opnfv/functest-vnf
- * opnfv/functest-parser
-
-The tag "opnfv-6.0.0" is the official release image in Fraser, but you can
-also pull "fraser" tag as it is being maintained by Functest team and might
-include bugfixes.
-
-The Functest docker container environment can -in principle- be also
-used with non-OPNFV official installers (e.g. 'devstack'), with the
-**disclaimer** that support for such environments is outside of the
-scope and responsibility of the OPNFV project.
Preparing your environment
cat env::
- EXTERNAL_NETWORK=XXX
DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
NAMESERVER=XXX # if not 8.8.8.8
+ EXTERNAL_NETWORK=XXX # if not first network with router:external=True
+ NEW_USER_ROLE=XXX # if not member
+ SDN_CONTROLLER_IP=XXX # if odl scenario
+ VOLUME_DEVICE_NAME=XXX # if not vdb
+ FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
See section on environment variables for details.
export OS_PROJECT_NAME=XXX
export OS_PASSWORD=XXX
export OS_IDENTITY_API_VERSION=3
+ export OS_REGION_NAME=XXX
See section on OpenStack credentials for details.
Create a directory for the different images (attached as a Docker volume)::
- mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/fraser | bash -s -- images && ls -1 images/*
+ mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
- images/CentOS-7-aarch64-GenericCloud.qcow2
- images/CentOS-7-aarch64-GenericCloud.qcow2.xz
- images/CentOS-7-x86_64-GenericCloud.qcow2
+ images/cirros-0.4.0-aarch64-disk.img
images/cirros-0.4.0-x86_64-disk.img
- images/cirros-0.4.0-x86_64-lxc.tar.gz
- images/cirros-d161201-aarch64-disk.img
- images/cirros-d161201-aarch64-initramfs
- images/cirros-d161201-aarch64-kernel
- images/cloudify-manager-premium-4.0.1.qcow2
+ images/cloudify-docker-manager-community-19.01.24.tar
+ images/shaker-image-1.3.0+stretch.qcow2
images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
- images/vyos-1.1.7.img
+ images/vyos-1.1.8-amd64.qcow2
Testing healthcheck suite
^^^^^^^^^^^^^^^^^^^^^^^^^
Results shall be displayed as follows::
- +----------------------------+------------------+---------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +----------------------------+------------------+---------------------+------------------+----------------+
- | connection_check | functest | healthcheck | 00:07 | PASS |
- | api_check | functest | healthcheck | 07:46 | PASS |
- | snaps_health_check | functest | healthcheck | 00:36 | PASS |
- +----------------------------+------------------+---------------------+------------------+----------------+
- NOTE: the duration is a reference and it might vary depending on your SUT.
+ +--------------------------+------------------+---------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +--------------------------+------------------+---------------------+------------------+----------------+
+ | connection_check | functest | healthcheck | 00:02 | PASS |
+ | tenantnetwork1 | functest | healthcheck | 00:06 | PASS |
+ | tenantnetwork2 | functest | healthcheck | 00:06 | PASS |
+ | vmready1 | functest | healthcheck | 00:06 | PASS |
+ | vmready2 | functest | healthcheck | 00:08 | PASS |
+ | singlevm1 | functest | healthcheck | 00:28 | PASS |
+ | singlevm2 | functest | healthcheck | 00:25 | PASS |
+ | vping_ssh | functest | healthcheck | 00:36 | PASS |
+ | vping_userdata | functest | healthcheck | 00:34 | PASS |
+ | cinder_test | functest | healthcheck | 01:03 | PASS |
+ | tempest_smoke | functest | healthcheck | 05:13 | PASS |
+ | odl | functest | healthcheck | 00:00 | SKIP |
+ +--------------------------+------------------+---------------------+------------------+----------------+
+
+NOTE: the duration is a reference and it might vary depending on your SUT.
Testing smoke suite
^^^^^^^^^^^^^^^^^^^
Results shall be displayed as follows::
- +------------------------------+------------------+---------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +------------------------------+------------------+---------------+------------------+----------------+
- | vping_ssh | functest | smoke | 00:57 | PASS |
- | vping_userdata | functest | smoke | 00:33 | PASS |
- | tempest_smoke_serial | functest | smoke | 13:22 | PASS |
- | rally_sanity | functest | smoke | 24:07 | PASS |
- | refstack_defcore | functest | smoke | 05:21 | PASS |
- | patrole | functest | smoke | 04:29 | PASS |
- | snaps_smoke | functest | smoke | 46:54 | PASS |
- | odl | functest | smoke | 00:00 | SKIP |
- | neutron_trunk | functest | smoke | 00:00 | SKIP |
- +------------------------------+------------------+---------------+------------------+----------------+
- Note: if the scenario does not support some tests, they are indicated as SKIP.
- See User guide for details.
-
-Testing features suite
-^^^^^^^^^^^^^^^^^^^^^^
-
-Run features suite::
-
- sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
- -v $(pwd)/images:/home/opnfv/functest/images \
- opnfv/functest-features
-
-Results shall be displayed as follows::
-
- +-----------------------------+------------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-----------------------------+------------------------+------------------+------------------+----------------+
- | doctor-notification | doctor | features | 00:00 | SKIP |
- | bgpvpn | sdnvpn | features | 00:00 | SKIP |
- | functest-odl-sfc | sfc | features | 00:00 | SKIP |
- | barometercollectd | barometer | features | 00:00 | SKIP |
- | fds | fastdatastacks | features | 00:00 | SKIP |
- +-----------------------------+------------------------+------------------+------------------+----------------+
- Note: if the scenario does not support some tests, they are indicated as SKIP.
- See User guide for details.
-
-Testing components suite
-^^^^^^^^^^^^^^^^^^^^^^^^
+ +------------------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +------------------------------------+------------------+---------------+------------------+----------------+
+ | neutron-tempest-plugin-api | functest | smoke | 09:12 | PASS |
+ | rally_sanity | functest | smoke | 16:29 | PASS |
+ | refstack_compute | functest | smoke | 06:25 | PASS |
+ | refstack_object | functest | smoke | 01:54 | PASS |
+ | refstack_platform | functest | smoke | 06:52 | PASS |
+ | tempest_full | functest | smoke | 30:26 | PASS |
+ | tempest_scenario | functest | smoke | 09:23 | PASS |
+ | tempest_slow | functest | smoke | 24:42 | PASS |
+ | patrole | functest | smoke | 02:36 | PASS |
+ | barbican | functest | smoke | 02:13 | PASS |
+ | neutron_trunk | functest | smoke | 00:00 | SKIP |
+ | networking-bgpvpn | functest | smoke | 00:00 | SKIP |
+ | networking-sfc | functest | smoke | 00:00 | SKIP |
+ | octavia | functest | smoke | 00:00 | SKIP |
+ +------------------------------------+------------------+---------------+------------------+----------------+
+
+Note: if the scenario does not support some tests, they are indicated as SKIP.
+See User guide for details.
+
+Testing benchmarking suite
+^^^^^^^^^^^^^^^^^^^^^^^^^^
-Run components suite::
+Run benchmarking suite::
sudo docker run --env-file env \
-v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
-v $(pwd)/images:/home/opnfv/functest/images \
- opnfv/functest-components
+ opnfv/functest-benchmarking
Results shall be displayed as follows::
- +-------------------------------+------------------+--------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-------------------------------+------------------+--------------------+------------------+----------------+
- | tempest_full_parallel | functest | components | 48:28 | PASS |
- | rally_full | functest | components | 126:02 | PASS |
- +-------------------------------+------------------+--------------------+------------------+----------------+
+ +--------------------+------------------+----------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +--------------------+------------------+----------------------+------------------+----------------+
+ | rally_full | functest | benchmarking | 92:16 | PASS |
+ | rally_jobs | functest | benchmarking | 18:49 | PASS |
+ | vmtp | functest | benchmarking | 15:28 | PASS |
+ | shaker | functest | benchmarking | 24:04 | PASS |
+ +--------------------+------------------+----------------------+------------------+----------------+
+
+Note: if the scenario does not support some tests, they are indicated as SKIP.
+See User guide for details.
Testing vnf suite
^^^^^^^^^^^^^^^^^
+----------------------+------------------+--------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------+------------------+--------------+------------------+----------------+
- | cloudify_ims | functest | vnf | 28:15 | PASS |
- | vyos_vrouter | functest | vnf | 17:59 | PASS |
- | juju_epc | functest | vnf | 46:44 | PASS |
+ | cloudify | functest | vnf | 03:49 | PASS |
+ | cloudify_ims | functest | vnf | 24:20 | PASS |
+ | heat_ims | functest | vnf | 32:13 | PASS |
+ | vyos_vrouter | functest | vnf | 14:55 | PASS |
+ | juju_epc | functest | vnf | 41:24 | PASS |
+----------------------+------------------+--------------+------------------+----------------+
-
Functest Dockers for Kubernetes deployment
------------------------------------------
Docker images are available on the dockerhub:
* opnfv/functest-kubernetes-core
* opnfv/functest-kubernetest-healthcheck
* opnfv/functest-kubernetest-smoke
- * opnfv/functest-kubernetest-features
Preparing your environment
^^^^^^^^^^^^^^^^^^^^^^^^^^
cat env::
- INSTALLER_TYPE=XXX
- DEPLOY_SCENARIO=XXX
-
-cat k8s.creds::
-
- export KUBERNETES_PROVIDER=local
- export KUBE_MASTER_URL=XXX
- export KUBE_MASTER_IP=XXX
+ DEPLOY_SCENARIO=k8s-XXX
Testing healthcheck suite
^^^^^^^^^^^^^^^^^^^^^^^^^
Run healthcheck suite::
sudo docker run -it --env-file env \
- -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \
-v $(pwd)/config:/root/.kube/config \
opnfv/functest-kubernetes-healthcheck
Results shall be displayed as follows::
- +-------------------------+------------------+-----------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-------------------------+------------------+-----------------+------------------+----------------+
- | k8s_smoke | functest | healthcheck | 01:54 | PASS |
- +-------------------------+------------------+-----------------+------------------+----------------+
+ +-------------------+------------------+---------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------+------------------+---------------------+------------------+----------------+
+ | k8s_smoke | functest | healthcheck | 01:09 | PASS |
+ +-------------------+------------------+---------------------+------------------+----------------+
Testing smoke suite
^^^^^^^^^^^^^^^^^^^
Run smoke suite::
sudo docker run -it --env-file env \
- -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \
-v $(pwd)/config:/root/.kube/config \
opnfv/functest-kubernetes-smoke
Results shall be displayed as follows::
- +-------------------------+------------------+---------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-------------------------+------------------+---------------+------------------+----------------+
- | k8s_conformance | functest | smoke | 57:47 | PASS |
- +-------------------------+------------------+---------------+------------------+----------------+
-
-Testing features suite
-^^^^^^^^^^^^^^^^^^^^^^
-
-Run features suite::
-
- sudo docker run -it --env-file env \
- -v $(pwd)/k8s.creds:/home/opnfv/functest/conf/env_file \
- -v $(pwd)/config:/root/.kube/config \
- opnfv/functest-kubernetes-features
-
-Results shall be displayed as follows::
-
- +----------------------+------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +----------------------+------------------+------------------+------------------+----------------+
- | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
- | clover_k8s | clover | clover | 00:00 | SKIP |
- +----------------------+------------------+------------------+------------------+----------------+
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | xrally_kubernetes | functest | smoke | 22:04 | PASS |
+ | k8s_conformance | functest | smoke | 173:48 | PASS |
+ +---------------------------+------------------+---------------+------------------+----------------+
Environment variables
=====================
Several environement variables may be specified:
- * INSTALLER_TYPE=(apex|compass|daisy|fuel|joid)
+
* INSTALLER_IP=<Specific IP Address>
* DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
-
+ * NAMESERVER=XXX # if not 8.8.8.8
+ * VOLUME_DEVICE_NAME=XXX # if not vdb
+ * EXTERNAL_NETWORK=XXX # if not first network with router:external=True
+ * NEW_USER_ROLE=XXX # if not member
INSTALLER_IP is required by Barometer in order to access the installer node and
the deployment.
only relevant to Jenkins CI invoked testing and **should not be used**
when performing manual test scenarios:
- * NODE_NAME = <Test POD Name>
- * BUILD_TAG = <Jenkins Build Tag>
+ * INSTALLER_TYPE=(apex|compass|daisy|fuel)
+ * NODE_NAME=<Test POD Name>
+ * BUILD_TAG=<Jenkins Build Tag>
where:
If you want to have more logs in console, you may edit the logging.ini file
manually.
Connect on the docker then edit the file located in
-/usr/lib/python2.7/site-packages/xtesting/ci/logging.ini
+/usr/lib/python3.6/site-packages/xtesting/ci/logging.ini
Change wconsole to console in the desired module to get more traces.
You may also directly modify the python code or the configuration file (e.g.
testcases.yaml used to declare test constraints) under
-/usr/lib/python2.7/site-packages/xtesting and
-/usr/lib/python2.7/site-packages/functest
+/usr/lib/python3.6/site-packages/xtesting and
+/usr/lib/python3.6/site-packages/functest
Tips
Note: In a Jumphost node based on the CentOS family OS, the **nc**
commands might not work. You can use the **curl** command instead.
- curl http://www.opnfv.org:80
+ curl https://www.opnfv.org/
<HTML><HEAD><meta http-equiv="content-type"
.
work with the above pre-requisite actions.
-.. _`[4]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/configguide/index.html
.. _`dockerdocs`: https://docs.docker.com/
.. _`Proxy`: https://docs.docker.com/engine/admin/systemd/#http-proxy
.. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
Code Review / functest.git / blobdiff