-----------------------------------------
Docker images are available on the dockerhub:
- * opnfv/functest-core
* opnfv/functest-healthcheck
* opnfv/functest-smoke
+ * opnfv/functest-smoke-cntt
* opnfv/functest-benchmarking
- * opnfv/functest-features
+ * opnfv/functest-benchmarking-cntt
* opnfv/functest-vnf
DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
NAMESERVER=XXX # if not 8.8.8.8
EXTERNAL_NETWORK=XXX # if not first network with router:external=True
+ DASHBOARD_URL=XXX # else tempest_horizon will be skipped
NEW_USER_ROLE=XXX # if not member
+ SDN_CONTROLLER_IP=XXX # if odl scenario
+ VOLUME_DEVICE_NAME=XXX # if not vdb
+ FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
See section on environment variables for details.
export OS_PROJECT_NAME=XXX
export OS_PASSWORD=XXX
export OS_IDENTITY_API_VERSION=3
+ export OS_REGION_NAME=XXX
See section on OpenStack credentials for details.
mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
- images/cirros-0.4.0-aarch64-disk.img
- images/cirros-0.4.0-x86_64-disk.img
+ images/cirros-0.5.1-aarch64-disk.img
+ images/cirros-0.5.1-x86_64-disk.img
images/cloudify-docker-manager-community-19.01.24.tar
- images/shaker-image-arm64.qcow2
- images/shaker-image.qcow2
+ images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2
+ images/shaker-image-1.3.0+stretch.qcow2
images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
+--------------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+--------------------------+------------------+---------------------+------------------+----------------+
- | connection_check | functest | healthcheck | 00:01 | PASS |
- | tenantnetwork1 | functest | healthcheck | 00:06 | PASS |
+ | connection_check | functest | healthcheck | 00:03 | PASS |
+ | tenantnetwork1 | functest | healthcheck | 00:05 | PASS |
| tenantnetwork2 | functest | healthcheck | 00:06 | PASS |
- | vmready1 | functest | healthcheck | 00:08 | PASS |
+ | vmready1 | functest | healthcheck | 00:06 | PASS |
| vmready2 | functest | healthcheck | 00:08 | PASS |
- | singlevm1 | functest | healthcheck | 00:35 | PASS |
- | singlevm2 | functest | healthcheck | 00:38 | PASS |
- | vping_ssh | functest | healthcheck | 00:47 | PASS |
- | vping_userdata | functest | healthcheck | 00:33 | PASS |
- | cinder_test | functest | healthcheck | 01:16 | PASS |
- | tempest_smoke | functest | healthcheck | 07:05 | PASS |
+ | singlevm1 | functest | healthcheck | 00:32 | PASS |
+ | singlevm2 | functest | healthcheck | 00:37 | PASS |
+ | vping_ssh | functest | healthcheck | 00:46 | PASS |
+ | vping_userdata | functest | healthcheck | 00:39 | PASS |
+ | cinder_test | functest | healthcheck | 01:05 | PASS |
+ | tempest_smoke | functest | healthcheck | 05:39 | PASS |
+ | tempest_horizon | functest | healthcheck | 01:05 | PASS |
| odl | functest | healthcheck | 00:00 | SKIP |
+--------------------------+------------------+---------------------+------------------+----------------+
Results shall be displayed as follows::
- +------------------------------------+------------------+---------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +------------------------------------+------------------+---------------+------------------+----------------+
- | neutron-tempest-plugin-api | functest | smoke | 13:50 | PASS |
- | rally_sanity | functest | smoke | 14:13 | PASS |
- | refstack_compute | functest | smoke | 07:24 | PASS |
- | refstack_object | functest | smoke | 02:08 | PASS |
- | refstack_platform | functest | smoke | 08:32 | PASS |
- | tempest_full | functest | smoke | 58:40 | PASS |
- | tempest_scenario | functest | smoke | 09:47 | PASS |
- | patrole | functest | smoke | 05:34 | PASS |
- | neutron_trunk | functest | smoke | 00:00 | SKIP |
- | networking-bgpvpn | functest | smoke | 01:43 | PASS |
- | networking-sfc | functest | smoke | 04:49 | PASS |
- | barbican | functest | smoke | 05:09 | PASS |
- +------------------------------------+------------------+---------------+------------------+----------------+
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | tempest_neutron | functest | smoke | 15:30 | PASS |
+ | tempest_cinder | functest | smoke | 02:01 | PASS |
+ | tempest_keystone | functest | smoke | 01:17 | PASS |
+ | tempest_heat | functest | smoke | 22:14 | PASS |
+ | tempest_telemetry | functest | smoke | 00:00 | SKIP |
+ | rally_sanity | functest | smoke | 17:24 | PASS |
+ | refstack_compute | functest | smoke | 07:03 | PASS |
+ | refstack_object | functest | smoke | 02:09 | PASS |
+ | refstack_platform | functest | smoke | 07:31 | PASS |
+ | tempest_full | functest | smoke | 41:52 | PASS |
+ | tempest_scenario | functest | smoke | 08:42 | PASS |
+ | tempest_slow | functest | smoke | 43:42 | PASS |
+ | patrole | functest | smoke | 02:42 | PASS |
+ | tempest_barbican | functest | smoke | 02:30 | PASS |
+ | tempest_octavia | functest | smoke | 00:00 | SKIP |
+ +---------------------------+------------------+---------------+------------------+----------------+
Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.
+Testing smoke CNTT suite
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Run smoke-cntt suite::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ opnfv/functest-smoke-cntt
+
+Results shall be displayed as follows::
+
+ +-------------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------------------+------------------+---------------+------------------+----------------+
+ | tempest_neutron_cntt | functest | smoke | 11:35 | PASS |
+ | tempest_cinder_cntt | functest | smoke | 01:58 | PASS |
+ | tempest_keystone_cntt | functest | smoke | 01:13 | PASS |
+ | tempest_heat_cntt | functest | smoke | 22:32 | PASS |
+ | rally_sanity_cntt | functest | smoke | 17:16 | PASS |
+ | tempest_full_cntt | functest | smoke | 41:13 | PASS |
+ | tempest_scenario_cntt | functest | smoke | 08:57 | PASS |
+ | tempest_slow_cntt | functest | smoke | 35:58 | PASS |
+ +-------------------------------+------------------+---------------+------------------+----------------+
+
Testing benchmarking suite
^^^^^^^^^^^^^^^^^^^^^^^^^^
+--------------------+------------------+----------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+--------------------+------------------+----------------------+------------------+----------------+
- | rally_full | functest | benchmarking | 121:55 | PASS |
- | rally_jobs | functest | benchmarking | 24:21 | PASS |
- | vmtp | functest | benchmarking | 00:00 | SKIP |
- | shaker | functest | benchmarking | 00:00 | SKIP |
+ | rally_full | functest | benchmarking | 93:03 | PASS |
+ | rally_jobs | functest | benchmarking | 27:05 | PASS |
+ | vmtp | functest | benchmarking | 17:56 | PASS |
+ | shaker | functest | benchmarking | 24:02 | PASS |
+--------------------+------------------+----------------------+------------------+----------------+
Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.
-Testing features suite
-^^^^^^^^^^^^^^^^^^^^^^
+Testing benchmarking CNTT suite
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Run features suite::
+Run benchmarking-cntt suite::
sudo docker run --env-file env \
-v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
-v $(pwd)/images:/home/opnfv/functest/images \
- opnfv/functest-features
+ opnfv/functest-benchmarking-cntt
Results shall be displayed as follows::
- +-----------------------------+-------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-----------------------------+-------------------+------------------+------------------+----------------+
- | doctor-notification | doctor | features | 00:00 | SKIP |
- | functest-odl-sfc | sfc | features | 00:00 | SKIP |
- | barometercollectd | barometer | features | 00:00 | SKIP |
- | vgpu | functest | features | 00:00 | SKIP |
- | stor4nfv_os | stor4nfv | features | 00:00 | SKIP |
- +-----------------------------+-------------------+------------------+------------------+----------------+
-
-Note: if the scenario does not support some tests, they are indicated as SKIP.
-See User guide for details.
+ +-------------------------+------------------+----------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------------+------------------+----------------------+------------------+----------------+
+ | rally_full_cntt | functest | benchmarking | 89:52 | PASS |
+ | rally_jobs_cntt | functest | benchmarking | 19:39 | PASS |
+ | vmtp | functest | benchmarking | 16:59 | PASS |
+ | shaker | functest | benchmarking | 23:43 | PASS |
+ +-------------------------+------------------+----------------------+------------------+----------------+
Testing vnf suite
^^^^^^^^^^^^^^^^^
+----------------------+------------------+--------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------+------------------+--------------+------------------+----------------+
- | cloudify | functest | vnf | 06:42 | PASS |
- | cloudify_ims | functest | vnf | 39:21 | PASS |
- | heat_ims | functest | vnf | 34:33 | PASS |
- | vyos_vrouter | functest | vnf | 20:20 | PASS |
- | juju_epc | functest | vnf | 56:55 | PASS |
+ | cloudify | functest | vnf | 05:08 | PASS |
+ | cloudify_ims | functest | vnf | 24:46 | PASS |
+ | heat_ims | functest | vnf | 33:12 | PASS |
+ | vyos_vrouter | functest | vnf | 15:53 | PASS |
+ | juju_epc | functest | vnf | 27:52 | PASS |
+----------------------+------------------+--------------+------------------+----------------+
Functest Dockers for Kubernetes deployment
------------------------------------------
Docker images are available on the dockerhub:
- * opnfv/functest-kubernetes-core
- * opnfv/functest-kubernetest-healthcheck
- * opnfv/functest-kubernetest-smoke
- * opnfv/functest-kubernetest-features
+ * opnfv/functest-kubernetes-healthcheck
+ * opnfv/functest-kubernetes-smoke
+ * opnfv/functest-kubernetes-security
+ * opnfv/functest-kubernetes-benchmarking
+ * opnfv/functest-kubernetes-cnf
Preparing your environment
^^^^^^^^^^^^^^^^^^^^^^^^^^
+-------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+-------------------+------------------+---------------------+------------------+----------------+
- | k8s_smoke | functest | healthcheck | 02:27 | PASS |
+ | k8s_quick | functest | healthcheck | 00:18 | PASS |
+ | k8s_smoke | functest | healthcheck | 01:14 | PASS |
+-------------------+------------------+---------------------+------------------+----------------+
Testing smoke suite
Results shall be displayed as follows::
- +-------------------------+------------------+---------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-------------------------+------------------+---------------+------------------+----------------+
- | k8s_conformance | functest | smoke | 57:14 | PASS |
- +-------------------------+------------------+---------------+------------------+----------------+
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------+------------------+---------------+------------------+----------------+
+ | k8s_conformance | functest | smoke | 94:26 | PASS |
+ | xrally_kubernetes | functest | smoke | 13:05 | PASS |
+ +---------------------------+------------------+---------------+------------------+----------------+
-Testing features suite
+Testing security suite
^^^^^^^^^^^^^^^^^^^^^^
-Run features suite::
+Run smoke suite::
+
+ sudo docker run -it --env-file env \
+ -v $(pwd)/config:/root/.kube/config \
+ opnfv/functest-kubernetes-security
+
+Results shall be displayed as follows::
+
+ +---------------------------+------------------+------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +---------------------------+------------------+------------------+------------------+----------------+
+ | kube_hunter | functest | security | 00:19 | PASS |
+ | kube_bench_master | functest | security | 00:02 | PASS |
+ | kube_bench_node | functest | security | 00:01 | PASS |
+ +---------------------------+------------------+------------------+------------------+----------------+
+
+Testing benchmarking suite
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Run benchmarking suite::
+
+ sudo docker run -it --env-file env \
+ -v $(pwd)/config:/root/.kube/config \
+ opnfv/functest-kubernetes-benchmarking
+
+Results shall be displayed as follows::
+
+ +--------------------------------+------------------+----------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +--------------------------------+------------------+----------------------+------------------+----------------+
+ | xrally_kubernetes_full | functest | benchmarking | 34:16 | PASS |
+ +--------------------------------+------------------+----------------------+------------------+----------------+
+
+Testing cnf suite
+^^^^^^^^^^^^^^^^^
+
+Run cnf suite::
sudo docker run -it --env-file env \
-v $(pwd)/config:/root/.kube/config \
- opnfv/functest-kubernetes-features
+ opnfv/functest-kubernetes-cnf
Results shall be displayed as follows::
- +----------------------+------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +----------------------+------------------+------------------+------------------+----------------+
- | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
- | clover_k8s | clover | clover | 00:00 | SKIP |
- +----------------------+------------------+------------------+------------------+----------------+
+ +-------------------------+------------------+--------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------------+------------------+--------------+------------------+----------------+
+ | cnf_conformance | functest | cnf | 05:30 | PASS |
+ | k8s_vims | functest | cnf | 20:28 | PASS |
+ +-------------------------+------------------+--------------+------------------+----------------+
Environment variables
=====================
-Several environement variables may be specified:
+Several environment variables may be specified:
* INSTALLER_IP=<Specific IP Address>
* DEPLOY_SCENARIO=<vim>-<controller>-<nfv_feature>-<ha_mode>
If you want to have more logs in console, you may edit the logging.ini file
manually.
Connect on the docker then edit the file located in
-/usr/lib/python3.6/site-packages/xtesting/ci/logging.ini
+/usr/lib/python3.8/site-packages/xtesting/ci/logging.ini
Change wconsole to console in the desired module to get more traces.
You may also directly modify the python code or the configuration file (e.g.
testcases.yaml used to declare test constraints) under
-/usr/lib/python3.6/site-packages/xtesting and
-/usr/lib/python3.6/site-packages/functest
+/usr/lib/python3.8/site-packages/xtesting and
+/usr/lib/python3.8/site-packages/functest
Tips
.. _`Install Docker on CentOS`: https://docs.docker.com/engine/installation/linux/centos/
.. _`Functest User Guide`: http://docs.opnfv.org/en/stable-danube/submodules/functest/docs/testing/user/userguide/index.html
.. _`images/CentOS-7-x86_64-GenericCloud.qcow2`: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
-.. _`images/cirros-0.4.0-x86_64-disk.img`: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
+.. _`images/cirros-0.5.1-x86_64-disk.img`: http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img
.. _`images/ubuntu-14.04-server-cloudimg-amd64-disk1.img`: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
Code Review / functest.git / blobdiff