3 Network virtualization has dramatically modified our architectures which asks
4 for more automation and powerful testing tools like Functest, a collection of
5 state-of-the-art virtual infrastructure test suites, including automatic VNF
7 [[1]](https://www.linuxfoundation.org/press-release/2019/05/opnfv-hunter-delivers-test-tools-ci-cd-framework-to-enable-common-nfvi-for-verifying-vnfs/)).
9 In context of OPNFV, Functest verifies any kind of OpenStack and Kubernetes
10 deployments including production environments. It conforms to upstream rules
11 and integrates smoothly lots of the test cases available in the opensource
12 market. It includes about 3000+ functional tests and 3 hours upstream API and
13 dataplane benchmarks. It’s completed by Virtual Network Function deployments
14 and testing (vIMS, vRouter and vEPC) to ensure that the platforms meet Network
15 Functions Virtualization requirements. Raspberry PI is also supported to verify
16 datacenters as the lowest cost (50 euros hardware and software included).
18 | Functest releases | OpenStack releases |
19 |-------------------|--------------------|
20 | **Hunter** | **Rocky** |
25 | Master | next Wallaby |
27 ## Prepare your environment
31 DEPLOY_SCENARIO=XXX # if not os-nosdn-nofeature-noha scenario
32 NAMESERVER=XXX # if not 8.8.8.8
33 EXTERNAL_NETWORK=XXX # if not first network with router:external=True
34 DASHBOARD_URL=XXX # else tempest_horizon will be skipped
35 NEW_USER_ROLE=XXX # if not member
36 SDN_CONTROLLER_IP=XXX # if odl scenario
37 VOLUME_DEVICE_NAME=XXX # if not vdb
38 FLAVOR_EXTRA_SPECS=hw:mem_page_size:large # if fdio scenarios
43 export OS_AUTH_URL=XXX
44 export OS_USER_DOMAIN_NAME=XXX
45 export OS_PROJECT_DOMAIN_NAME=XXX
46 export OS_USERNAME=XXX
47 export OS_PROJECT_NAME=XXX
48 export OS_PASSWORD=XXX
49 export OS_IDENTITY_API_VERSION=3
50 export OS_REGION_NAME=XXX
53 mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh?h=stable/hunter | bash -s -- images && ls -1 images/*
55 images/cirros-0.4.0-aarch64-disk.img
56 images/cirros-0.4.0-x86_64-disk.img
57 images/cloudify-docker-manager-community-19.01.24.tar
58 images/Fedora-Cloud-Base-30-1.2.x86_64.qcow2
59 images/shaker-image-1.3.0+stretch.qcow2
60 images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
61 images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
62 images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
63 images/vyos-1.1.8-amd64.qcow2
66 ## Run healthcheck suite
69 sudo docker run --env-file env \
70 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
71 -v $(pwd)/images:/home/opnfv/functest/images \
72 opnfv/functest-healthcheck:hunter
76 +--------------------------+------------------+---------------------+------------------+----------------+
77 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
78 +--------------------------+------------------+---------------------+------------------+----------------+
79 | connection_check | functest | healthcheck | 00:03 | PASS |
80 | tenantnetwork1 | functest | healthcheck | 00:08 | PASS |
81 | tenantnetwork2 | functest | healthcheck | 00:16 | PASS |
82 | vmready1 | functest | healthcheck | 00:09 | PASS |
83 | vmready2 | functest | healthcheck | 00:10 | PASS |
84 | singlevm1 | functest | healthcheck | 00:51 | PASS |
85 | singlevm2 | functest | healthcheck | 00:41 | PASS |
86 | vping_ssh | functest | healthcheck | 00:56 | PASS |
87 | vping_userdata | functest | healthcheck | 00:42 | PASS |
88 | cinder_test | functest | healthcheck | 02:19 | PASS |
89 | tempest_smoke | functest | healthcheck | 07:02 | PASS |
90 | tempest_horizon | functest | healthcheck | 00:52 | PASS |
91 | odl | functest | healthcheck | 00:00 | SKIP |
92 +--------------------------+------------------+---------------------+------------------+----------------+
98 sudo docker run --env-file env \
99 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
100 -v $(pwd)/images:/home/opnfv/functest/images \
101 opnfv/functest-smoke:hunter
105 +---------------------------+------------------+---------------+------------------+----------------+
106 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
107 +---------------------------+------------------+---------------+------------------+----------------+
108 | tempest_neutron | functest | smoke | 16:49 | PASS |
109 | tempest_cinder | functest | smoke | 01:39 | PASS |
110 | tempest_keystone | functest | smoke | 00:57 | PASS |
111 | tempest_heat | functest | smoke | 24:33 | PASS |
112 | rally_sanity | functest | smoke | 18:41 | PASS |
113 | refstack_defcore | functest | smoke | 10:38 | PASS |
114 | tempest_full | functest | smoke | 55:19 | PASS |
115 | tempest_scenario | functest | smoke | 11:06 | PASS |
116 | tempest_slow | functest | smoke | 61:39 | PASS |
117 | patrole | functest | smoke | 02:46 | PASS |
118 | networking-bgpvpn | functest | smoke | 00:00 | SKIP |
119 | networking-sfc | functest | smoke | 00:00 | SKIP |
120 | tempest_barbican | functest | smoke | 02:30 | PASS |
121 +---------------------------+------------------+---------------+------------------+----------------+
124 ## Run smoke CNTT suite
127 sudo docker run --env-file env \
128 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
129 -v $(pwd)/images:/home/opnfv/functest/images \
130 opnfv/functest-smoke-cntt:hunter
134 +-------------------------------+------------------+---------------+------------------+----------------+
135 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
136 +-------------------------------+------------------+---------------+------------------+----------------+
137 | tempest_neutron_cntt | functest | smoke | 13:54 | PASS |
138 | tempest_cinder_cntt | functest | smoke | 01:46 | PASS |
139 | tempest_keystone_cntt | functest | smoke | 00:58 | PASS |
140 | tempest_heat_cntt | functest | smoke | 25:31 | PASS |
141 | rally_sanity_cntt | functest | smoke | 18:50 | PASS |
142 | tempest_full_cntt | functest | smoke | 44:32 | PASS |
143 | tempest_scenario_cntt | functest | smoke | 11:14 | PASS |
144 | tempest_slow_cntt | functest | smoke | 43:55 | PASS |
145 +-------------------------------+------------------+---------------+------------------+----------------+
148 ## Run benchmarking suite
151 sudo docker run --env-file env \
152 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
153 -v $(pwd)/images:/home/opnfv/functest/images \
154 opnfv/functest-benchmarking:hunter
158 +--------------------+------------------+----------------------+------------------+----------------+
159 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
160 +--------------------+------------------+----------------------+------------------+----------------+
161 | rally_full | functest | benchmarking | 108:34 | PASS |
162 | rally_jobs | functest | benchmarking | 22:07 | PASS |
163 | vmtp | functest | benchmarking | 15:38 | PASS |
164 | shaker | functest | benchmarking | 25:12 | PASS |
165 +--------------------+------------------+----------------------+------------------+----------------+
168 ## Run benchmarking CNTT suite
171 sudo docker run --env-file env \
172 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
173 -v $(pwd)/images:/home/opnfv/functest/images \
174 opnfv/functest-benchmarking-cntt:hunter
178 +-------------------------+------------------+----------------------+------------------+----------------+
179 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
180 +-------------------------+------------------+----------------------+------------------+----------------+
181 | rally_full_cntt | functest | benchmarking | 106:60 | PASS |
182 | rally_jobs_cntt | functest | benchmarking | 21:16 | PASS |
183 | vmtp | functest | benchmarking | 16:15 | PASS |
184 | shaker | functest | benchmarking | 25:09 | PASS |
185 +-------------------------+------------------+----------------------+------------------+----------------+
191 sudo docker run --env-file env \
192 -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
193 -v $(pwd)/images:/home/opnfv/functest/images \
194 opnfv/functest-vnf:hunter
198 +----------------------+------------------+--------------+------------------+----------------+
199 | TEST CASE | PROJECT | TIER | DURATION | RESULT |
200 +----------------------+------------------+--------------+------------------+----------------+
201 | cloudify | functest | vnf | 04:35 | PASS |
202 | cloudify_ims | functest | vnf | 24:16 | PASS |
203 | heat_ims | functest | vnf | 30:36 | PASS |
204 | vyos_vrouter | functest | vnf | 15:37 | PASS |
205 | juju_epc | functest | vnf | 34:39 | PASS |
206 +----------------------+------------------+--------------+------------------+----------------+