3 Network virtualization has dramatically modified our architectures which asks
4 for more automation and powerful testing tools like Functest, a collection of
5 state-of-the-art virtual infrastructure test suites, including automatic VNF
7 [[1]](https://www.linuxfoundation.org/press-release/2019/05/opnfv-hunter-delivers-test-tools-ci-cd-framework-to-enable-common-nfvi-for-verifying-vnfs/)).
9 In context of OPNFV, Functest verifies any kind of OpenStack and Kubernetes
10 deployments including production environments. It conforms to upstream rules
11 and integrates smoothly lots of the test cases available in the opensource
12 market. It includes about 3000+ functional tests and 3 hours upstream API and
13 dataplane benchmarks. It’s completed by Virtual Network Function deployments
14 and testing (vIMS, vRouter and vEPC) to ensure that the platforms meet Network
15 Functions Virtualization requirements. Raspberry PI is also supported to verify
16 datacenters as the lowest cost (50 euros hardware and software included).
18 | Functest releases | OpenStack releases |
19 |-------------------|--------------------|
20 | **Jerma** | **Train** |
24 | Master | next Xena |
26 ## Prepare your environment
30 DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
31 NAMESERVER=XXX # if not 8.8.8.8
32 EXTERNAL_NETWORK=XXX # if not first network with router:external=True
33 DASHBOARD_URL=XXX # else tempest_horizon will be skipped
34 NEW_USER_ROLE=XXX # if not member
35 SDN_CONTROLLER_IP=XXX # if odl scenario
36 VOLUME_DEVICE_NAME=XXX # if not vdb
37 FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
42 export OS_AUTH_URL=XXX
43 export OS_USER_DOMAIN_NAME=XXX
44 export OS_PROJECT_DOMAIN_NAME=XXX
45 export OS_USERNAME=XXX
46 export OS_PROJECT_NAME=XXX
47 export OS_PASSWORD=XXX
48 export OS_IDENTITY_API_VERSION=3
49 export OS_REGION_NAME=XXX
52 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/jerma | bash -s -- images && ls -1 images/*
54 images/cirros-0.4.0-aarch64-disk.img
55 images/cirros-0.4.0-x86_64-disk.img
56 images/cloudify-docker-manager-community-19.01.24.tar
57 images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2
58 images/shaker-image-1.3.0+stretch.qcow2
59 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
60 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
61 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
62 images/vyos-1.1.8-amd64.qcow2
65 ## Run healthcheck suite
68 sudo docker run --env-file env \
69 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
70 -v $(pwd)/images:/home/opnfv/functest/images \
71 opnfv/functest-healthcheck:jerma
75 +--------------------------+------------------+---------------------+------------------+----------------+
76 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
77 +--------------------------+------------------+---------------------+------------------+----------------+
78 | connection_check | functest | healthcheck | 00:02 | PASS |
79 | tenantnetwork1 | functest | healthcheck | 00:04 | PASS |
80 | tenantnetwork2 | functest | healthcheck | 00:05 | PASS |
81 | vmready1 | functest | healthcheck | 00:05 | PASS |
82 | vmready2 | functest | healthcheck | 00:08 | PASS |
83 | singlevm1 | functest | healthcheck | 00:35 | PASS |
84 | singlevm2 | functest | healthcheck | 00:34 | PASS |
85 | vping_ssh | functest | healthcheck | 00:44 | PASS |
86 | vping_userdata | functest | healthcheck | 00:30 | PASS |
87 | cinder_test | functest | healthcheck | 01:10 | PASS |
88 | tempest_smoke | functest | healthcheck | 11:18 | PASS |
89 | tempest_horizon | functest | healthcheck | 01:15 | PASS |
90 | odl | functest | healthcheck | 00:00 | SKIP |
91 +--------------------------+------------------+---------------------+------------------+----------------+
97 sudo docker run --env-file env \
98 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
99 -v $(pwd)/images:/home/opnfv/functest/images \
100 opnfv/functest-smoke:jerma
104 +---------------------------+------------------+---------------+------------------+----------------+
105 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
106 +---------------------------+------------------+---------------+------------------+----------------+
107 | tempest_neutron | functest | smoke | 14:49 | PASS |
108 | tempest_cinder | functest | smoke | 01:40 | PASS |
109 | tempest_keystone | functest | smoke | 01:12 | PASS |
110 | tempest_heat | functest | smoke | 21:53 | PASS |
111 | rally_sanity | functest | smoke | 17:10 | PASS |
112 | refstack_compute | functest | smoke | 07:53 | PASS |
113 | refstack_object | functest | smoke | 01:59 | PASS |
114 | refstack_platform | functest | smoke | 07:52 | PASS |
115 | tempest_full | functest | smoke | 34:39 | PASS |
116 | tempest_scenario | functest | smoke | 08:30 | PASS |
117 | tempest_slow | functest | smoke | 44:40 | PASS |
118 | patrole | functest | smoke | 02:43 | PASS |
119 | tempest_barbican | functest | smoke | 02:26 | PASS |
120 | tempest_octavia | functest | smoke | 00:00 | SKIP |
121 +---------------------------+------------------+---------------+------------------+----------------+
124 ## Run smoke CNTT suite
127 sudo docker run --env-file env \
128 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
129 -v $(pwd)/images:/home/opnfv/functest/images \
130 opnfv/functest-smoke-cntt:jerma
134 +-------------------------------+------------------+---------------+------------------+----------------+
135 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
136 +-------------------------------+------------------+---------------+------------------+----------------+
137 | tempest_neutron_cntt | functest | smoke | 12:21 | PASS |
138 | tempest_cinder_cntt | functest | smoke | 01:31 | PASS |
139 | tempest_keystone_cntt | functest | smoke | 01:08 | PASS |
140 | tempest_heat_cntt | functest | smoke | 21:47 | PASS |
141 | rally_sanity_cntt | functest | smoke | 16:09 | PASS |
142 | tempest_full_cntt | functest | smoke | 39:11 | PASS |
143 | tempest_scenario_cntt | functest | smoke | 08:32 | PASS |
144 | tempest_slow_cntt | functest | smoke | 32:22 | PASS |
145 +-------------------------------+------------------+---------------+------------------+----------------+
148 ## Run benchmarking suite
151 sudo docker run --env-file env \
152 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
153 -v $(pwd)/images:/home/opnfv/functest/images \
154 opnfv/functest-benchmarking:jerma
158 +--------------------+------------------+----------------------+------------------+----------------+
159 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
160 +--------------------+------------------+----------------------+------------------+----------------+
161 | rally_full | functest | benchmarking | 91:42 | PASS |
162 | rally_jobs | functest | benchmarking | 25:24 | PASS |
163 | vmtp | functest | benchmarking | 16:47 | PASS |
164 | shaker | functest | benchmarking | 23:43 | PASS |
165 +--------------------+------------------+----------------------+------------------+----------------+
168 ## Run benchmarking CNTT suite
171 sudo docker run --env-file env \
172 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
173 -v $(pwd)/images:/home/opnfv/functest/images \
174 opnfv/functest-benchmarking-cntt:jerma
178 +-------------------------+------------------+----------------------+------------------+----------------+
179 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
180 +-------------------------+------------------+----------------------+------------------+----------------+
181 | rally_full_cntt | functest | benchmarking | 86:41 | PASS |
182 | rally_jobs_cntt | functest | benchmarking | 19:16 | PASS |
183 | vmtp | functest | benchmarking | 17:05 | PASS |
184 | shaker | functest | benchmarking | 23:33 | PASS |
185 +-------------------------+------------------+----------------------+------------------+----------------+
191 sudo docker run --env-file env \
192 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
193 -v $(pwd)/images:/home/opnfv/functest/images \
194 opnfv/functest-vnf:jerma
198 +----------------------+------------------+--------------+------------------+----------------+
199 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
200 +----------------------+------------------+--------------+------------------+----------------+
201 | cloudify | functest | vnf | 04:41 | PASS |
202 | cloudify_ims | functest | vnf | 21:03 | PASS |
203 | heat_ims | functest | vnf | 30:04 | PASS |
204 | vyos_vrouter | functest | vnf | 16:21 | PASS |
205 | juju_epc | functest | vnf | 28:33 | PASS |
206 +----------------------+------------------+--------------+------------------+----------------+