vFW: Adding Virtual Firewall VNF 97/33497/7
authorAnand B Jyoti <anand.b.jyoti@intel.com>
Tue, 18 Apr 2017 08:06:02 +0000 (13:36 +0530)
committerDeepak S <deepak.s@linux.intel.com>
Wed, 19 Apr 2017 10:15:39 +0000 (03:15 -0700)
JIRA: SAMPLEVNF-4

vFW supports following features:
 - Basic packet filtering (malformed packets, IP fragments)
 - Connection tracking for TCP and UDP
 - Access Control List for rule based policy enforcement
 - SYN-flood protection via Synproxy* for TCP
 - UDP, TCP and ICMP protocol pass-through
 - CLI based enable/disable connection tracking, synproxy,
   basic packet filtering
 - Hardware and Software Load Balancing
 - L2L3 stack support for ARP/ICMP handling
 - Multithread support
 - Multiple physical port support

Change-Id: I96d28858488ed8764370d161975bc1e0557c8b20
Signed-off-by: Anand B Jyoti <anand.b.jyoti@intel.com>
[Push patch to gerrit]
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
35 files changed:
Makefile
VNFs/vFW/Makefile [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_script.tc [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_1Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_4Thread.cfg [new file with mode: 0644]
VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_script.tc [new file with mode: 0644]
VNFs/vFW/init.c [new file with mode: 0644]
VNFs/vFW/main.c [new file with mode: 0644]
VNFs/vFW/pipeline/pipeline_vfw.c [new file with mode: 0644]
VNFs/vFW/pipeline/pipeline_vfw.h [new file with mode: 0644]
VNFs/vFW/pipeline/pipeline_vfw_be.c [new file with mode: 0644]
VNFs/vFW/pipeline/pipeline_vfw_be.h [new file with mode: 0644]
docs/vFW/INSTALL.rst [new file with mode: 0644]
docs/vFW/README.rst [new file with mode: 0644]
docs/vFW/RELEASE_NOTES.rst [new file with mode: 0644]

index 9cb9fb7..35aa5f3 100644 (file)
--- a/Makefile
+++ b/Makefile
@@ -22,9 +22,10 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc
 TARGETS      := all clean
 VNF_DIR      := VNFs
 ACL          := $(VNF_DIR)/vACL
+FW           := $(VNF_DIR)/vFW
 CGNAPT       := $(VNF_DIR)/vCGNAPT
 
-subdirs      := $(ACL) $(CGNAPT)
+subdirs      := $(ACL) $(CGNAPT) $(FW)
 
 .PHONY: $(TARGETS) $(subdirs)
 
diff --git a/VNFs/vFW/Makefile b/VNFs/vFW/Makefile
new file mode 100644 (file)
index 0000000..c96246b
--- /dev/null
@@ -0,0 +1,117 @@
+# Copyright (c) 2017 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http:#www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+ifeq ($(VNF_CORE),)
+$(error "Please define VNF_CORE environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+DIRS-(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = vFW
+
+VPATH += $(VNF_CORE)/common/vnf_common
+VPATH += $(VNF_CORE)/common/VIL/pipeline_arpicmp
+VPATH += $(VNF_CORE)/common/VIL/conntrack
+VPATH += $(VNF_CORE)/common/VIL/pipeline_common
+VPATH += $(VNF_CORE)/common/VIL/pipeline_loadb
+VPATH += $(VNF_CORE)/common/VIL/pipeline_master
+VPATH += $(VNF_CORE)/common/VIL/pipeline_passthrough
+VPATH += $(SRCDIR)/pipeline
+VPATH += $(VNF_CORE)/common/VIL/pipeline_txrx
+VPATH += $(VNF_CORE)/common/VIL/acl
+VPATH += $(VNF_CORE)/common/VIL/l2l3_stack
+
+INC += $(wildcard *.h)
+INC += $(wildcard pipeline/*.h)
+INC += $(wildcard $(VNF_CORE)/common/vnf_common/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/l2l3_stack/*.h)
+#INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_arpicmp/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/conntrack/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_loadb/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_common/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_master/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_passthrough/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/pipeline_txrx/*.h)
+INC += $(wildcard $(VNF_CORE)/common/VIL/acl/*.h)
+
+CFLAGS += -I$(SRCDIR) -mrtm -mhle -I$(SRCDIR)/pipeline -I$(VNF_CORE)/common/vnf_common
+CFLAGS += -I$(VNF_CORE)/common/VIL/conntrack -I$(VNF_CORE)/common/VIL/l2l3_stack
+CFLAGS += -I$(VNF_CORE)/common/VIL/pipeline_common -I$(VNF_CORE)/common/VIL/pipeline_loadb
+CFLAGS += -I$(VNF_CORE)/common/VIL/pipeline_master -I$(VNF_CORE)/common/VIL/pipeline_passthrough
+CFLAGS += -I$(VNF_CORE)/common/VIL/pipeline_txrx
+CFLAGS += -I$(VNF_CORE)/common/VIL/acl
+CFLAGS += -I$(VNF_CORE)/common/VIL/pipeline_arpicmp
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := main.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += config_parse.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += config_parse_tm.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += config_check.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += init.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += thread.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += thread_fe.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += cpu_core_map.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib_arp.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib_icmpv6.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += interface.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += hle.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += tsx.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += l2_proto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += l3fwd_main.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += l3fwd_lpm4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += l3fwd_lpm6.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += bond.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_common_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_common_fe.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_master_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_master.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_passthrough_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_passthrough.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_ct_tcp.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_cnxn_tracking.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_ct_udp.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_ct_synproxy.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_vfw_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_vfw.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_loadb.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_loadb_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += vnf_common.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_arpicmp.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_arpicmp_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_txrx.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline_txrx_be.c
+SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += lib_acl.c
+
+
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS) -Wno-error=unused-function -Wno-error=unused-variable
+CFLAGS += -DACL_ENABLE
+CFLAGS += -DIPV6
+CFLAGS += -DHW_CHECKSUM_REQ
+#CFLAGS += -DLSC_GRARP
+#CFLAGS += -Wno-error=unused-function -Wno-error=unused-variable
+
+include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_1Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..a274506
--- /dev/null
@@ -0,0 +1,60 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ad102814,ff000000,2,ad102814) (cb106414,ff000000,0,cb106414)
+arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,2)(1,3)
+prv_que_handler = (0,1)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 TXQ2.1 TXQ3.1 SWQ0
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+
+
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_4Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..7d543c8
--- /dev/null
@@ -0,0 +1,119 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0 SWQ1 SWQ2 SWQ3
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ad102814,ff000000,2,ad102814) (cb106414,ff000000,0,cb106414)
+arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,2)(1,3)
+prv_que_handler = (0,1)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 TXQ2.1 TXQ3.1 SWQ0
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE3]
+type = VFW
+core = 3
+pktq_in  = RXQ0.1 RXQ1.1 RXQ2.1 RXQ3.1
+pktq_out = TXQ0.2 TXQ1.2 TXQ2.2 TXQ3.2 SWQ1
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = RXQ0.2 RXQ1.2 RXQ2.2 RXQ3.2
+pktq_out = TXQ0.3 TXQ1.3 TXQ2.3 TXQ3.3 SWQ2
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = RXQ0.3 RXQ1.3 RXQ2.3 RXQ3.3
+pktq_out = TXQ0.4 TXQ1.4 TXQ2.4 TXQ3.4 SWQ3
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_script.tc b/VNFs/vFW/config/VFW_HWLB_IPV4_MultiPortPair_script.tc
new file mode 100644 (file)
index 0000000..0670207
--- /dev/null
@@ -0,0 +1,86 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config 203.16.100.10 8
+link 0 up
+link 1 down
+link 1 config 202.16.100.10 8
+link 1 up
+link 2 down
+link 2 config 173.16.40.10 8
+link 2 up
+link 3 down
+link 3 config 172.16.40.10 8
+link 3 up
+;p 1 arpadd 0 203.16.100.20 00:00:00:00:00:01
+;p 1 arpadd 2 173.16.40.20 00:00:00:00:00:04
+;p 1 arpadd 1 202.16.100.20 00:00:00:00:00:02
+;p 1 arpadd 3 172.16.40.20 00:00:00:00:00:03
+p action add 0 accept
+p action add 0 fwd 2
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 accept
+p action add 2 fwd 3
+p action add 2 count
+
+p action add 3 accept
+p action add 3 fwd 1
+p action add 3 count
+
+p action add 4 drop
+p action add 4 count
+
+p action add 5 drop
+p action add 5 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+
+p action add 4 conntrack
+p action add 5 conntrack
+
+p vfw add 1 203.16.100.20 8 173.16.40.20 8 0 65535 67 69 0 0 4
+p vfw add 1 202.16.100.20 8 172.16.40.20 8 0 65535 67 69 0 0 5
+
+p vfw add 2 203.16.100.20 8 173.16.40.20 8 0 65535 0 65535 0 0 0
+p vfw add 2 173.16.40.20 8 203.16.100.20 8 0 65535 0 65535 0 0 1
+p vfw add 2 202.16.100.20 8 172.16.40.20 8 0 65535 0 65535 0 0 2
+p vfw add 2 172.16.40.20 8 202.16.100.20 8 0 65535 0 65535 0 0 3
+
+p vfw applyruleset
+set fwd rxonly
+set_sym_hash_ena_per_port 0 enable
+set_hash_global_config 0 simple_xor ipv4-udp enable
+set_sym_hash_ena_per_port 1 enable
+set_hash_global_config 1 simple_xor ipv4-udp enable
+
+set_hash_input_set 0 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+set_hash_input_set 1 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+
+set_sym_hash_ena_per_port 2 enable
+set_hash_global_config 2 simple_xor ipv4-udp enable
+set_sym_hash_ena_per_port 3 enable
+set_hash_global_config 3 simple_xor ipv4-udp enable
+
+set_hash_input_set 2 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+set_hash_input_set 3 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_1Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..b90fad1
--- /dev/null
@@ -0,0 +1,59 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ac102814,ff000000,1,ac102814) (ca106414,ff000000,0,ca106414)
+;arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0
+pktq_out = TXQ0.1 TXQ1.1 SWQ0
+
+;n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 4000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_4Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..dee9d97
--- /dev/null
@@ -0,0 +1,117 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0 SWQ1 SWQ2 SWQ3
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ac102814,ff000000,1,ac102814) (ca106414,ff000000,0,ca106414)
+;arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0
+pktq_out = TXQ0.1 TXQ1.1 SWQ0
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE3]
+type = VFW
+core = 3
+pktq_in  = RXQ0.1 RXQ1.1
+pktq_out = TXQ0.2 TXQ1.2 SWQ1
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = RXQ0.2 RXQ1.2
+pktq_out = TXQ0.3 TXQ1.3 SWQ2
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = RXQ0.3 RXQ1.3
+pktq_out = TXQ0.4 TXQ1.4 SWQ3
+;
+;
+n_rules = 4096
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_script.tc b/VNFs/vFW/config/VFW_HWLB_IPV4_SinglePortPair_script.tc
new file mode 100644 (file)
index 0000000..ef585d7
--- /dev/null
@@ -0,0 +1,60 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config 202.16.100.10 8
+link 0 up
+link 1 down
+link 1 config 172.16.40.10 8
+link 1 up
+
+;p 1 arpadd 1 172.16.40.20 00:00:00:00:00:04
+;p 1 arpadd 0 202.16.100.20 00:00:00:00:00:01
+
+p action add 0 accept
+p action add 0 fwd 0
+p action add 0 count
+p action add 1 accept
+p action add 1 fwd 1
+p action add 1 count
+
+p action add 2 drop
+p action add 2 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+p action add 4 conntrack
+p action add 5 conntrack
+p action add 6 conntrack
+p action add 7 conntrack
+
+p vfw add 1 202.16.100.20 8 172.16.40.20 8 0 65535 67 69 0 0 2
+p vfw add 2 202.16.100.20 8 172.16.40.20 8 0 65535 0 65535 0 0 1
+p vfw add 2 172.16.40.20 8 202.16.100.20 8 0 65535 0 65535 0 0 0
+
+p vfw applyruleset
+
+set fwd rxonly
+set_sym_hash_ena_per_port 0 enable
+set_hash_global_config 0 simple_xor ipv4-udp enable
+set_sym_hash_ena_per_port 1 enable
+set_hash_global_config 1 simple_xor ipv4-udp enable
+
+set_hash_input_set 0 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+set_hash_input_set 1 ipv4-udp src-ipv4 dst-ipv4 udp-src-port udp-dst-port  add
+
+p 1 arpls
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_1Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..ddf746e
--- /dev/null
@@ -0,0 +1,58 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,2,2012::6a05:caff:fe30:2081)
+nd_route_tbl =  (fec1::6a05:caff:fe30:21b0,64,1,fec1::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2016::6a05:caff:fe30:2081,64,3,2016::6a05:caff:fe30:2081)
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,2)(1,3)
+prv_que_handler = (0,1)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 TXQ2.1 TXQ3.1 SWQ0
+
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_4Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..5b1a1cd
--- /dev/null
@@ -0,0 +1,119 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0 SWQ1 SWQ2 SWQ3
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,2,2012::6a05:caff:fe30:2081)
+nd_route_tbl =  (fec1::6a05:caff:fe30:21b0,64,1,fec1::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2016::6a05:caff:fe30:2081,64,3,2016::6a05:caff:fe30:2081)
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,2)(1,3)
+prv_que_handler = (0,1)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 TXQ2.1 TXQ3.1 SWQ0
+
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE3]
+type = VFW
+core = 3
+pktq_in  = RXQ0.1 RXQ1.1 RXQ2.1 RXQ3.1
+pktq_out = TXQ0.2 TXQ1.2 TXQ2.2 TXQ3.2 SWQ1
+
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = RXQ0.2 RXQ1.2 RXQ2.2 RXQ3.2
+pktq_out = TXQ0.3 TXQ1.3 TXQ2.3 TXQ3.3 SWQ2
+
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = RXQ0.3 RXQ1.3 RXQ2.3 RXQ3.3
+pktq_out = TXQ0.4 TXQ1.4 TXQ2.4 TXQ3.4 SWQ3
+
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_script.tc b/VNFs/vFW/config/VFW_HWLB_IPV6_MultiPortPair_script.tc
new file mode 100644 (file)
index 0000000..00bf424
--- /dev/null
@@ -0,0 +1,82 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config fec0:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 0 up
+link 1 down
+link 1 config fec1:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 1 up
+link 2 down
+link 2 config 2012:0000:0000:0000:6a05:caff:fe30:2071 64
+link 2 up
+link 3 down
+link 3 config 2016:0000:0000:0000:6a05:caff:fe30:2071 64
+link 3 up
+;p 1 arpadd 0 fec0::6a05:caff:fe30:21b0 00:00:00:00:00:01
+;p 1 arpadd 2 2012::6a05:caff:fe30:2081 00:00:00:00:00:04
+;p 1 arpadd 1 fec1::6a05:caff:fe30:21b0 00:00:00:00:00:02
+;p 1 arpadd 3 2016::6a05:caff:fe30:2081 00:00:00:00:00:03
+p action add 0 accept
+p action add 0 fwd 2
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 accept
+p action add 2 fwd 3
+p action add 2 count
+
+p action add 3 accept
+p action add 3 fwd 1
+p action add 3 count
+
+p action add 4 drop
+p action add 4 count
+
+p action add 5 drop
+p action add 5 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+p action add 4 conntrack
+p action add 5 conntrack
+
+p vfw add 1 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 4
+p vfw add 1 fec1::6a05:caff:fe30:21b0 64 2016::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 5
+p vfw add 2 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 0
+p vfw add 2 2012::6a05:caff:fe30:2081 64 fec0::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 1
+p vfw add 2 fec1::6a05:caff:fe30:21b0 64 2016::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 2
+p vfw add 2 2016::6a05:caff:fe30:2081 64 fec1::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 3
+p vfw applyruleset
+set fwd rxonly
+set_sym_hash_ena_per_port 0 enable
+set_hash_global_config 0 simple_xor ipv6-udp enable
+set_sym_hash_ena_per_port 1 enable
+set_hash_global_config 1 simple_xor ipv6-udp enable
+
+set_hash_input_set 0 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
+set_hash_input_set 1 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
+
+set_sym_hash_ena_per_port 2 enable
+set_hash_global_config 2 simple_xor ipv6-udp enable
+set_sym_hash_ena_per_port 3 enable
+set_hash_global_config 3 simple_xor ipv6-udp enable
+
+set_hash_input_set 2 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
+set_hash_input_set 3 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_1Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..ed2e842
--- /dev/null
@@ -0,0 +1,59 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0
+pktq_out = TXQ0.0 TXQ1.0 ;TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+;arp_route_tbl = (98102814,ff000000,1,98102814) (c0106414,ff000000,0,c0106414)
+;arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00 ;00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 ;RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1);(1,3)
+prv_que_handler = (0)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 ;RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 SWQ0;TXQ2.1 TXQ3.1 SWQ0
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_4Thread.cfg b/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..fd9da8f
--- /dev/null
@@ -0,0 +1,118 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ0 SWQ1 SWQ2 SWQ3
+pktq_out = TXQ0.0 TXQ1.0 ;TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+;arp_route_tbl = (98102814,ff000000,1,98102814) (c0106414,ff000000,0,c0106414)
+;arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,1,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00 ;00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 ;RXQ1.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1);(1,3)
+prv_que_handler = (0)
+[PIPELINE2]
+type = VFW
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 ;RXQ2.0 RXQ3.0
+pktq_out = TXQ0.1 TXQ1.1 SWQ0;TXQ2.1 TXQ3.1 SWQ0
+
+;n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE3]
+type = VFW
+core = 3
+pktq_in  = RXQ0.1 RXQ1.1 ;RXQ2.1 RXQ3.1
+pktq_out = TXQ0.2 TXQ1.2 SWQ1;TXQ2.2 TXQ3.2 SWQ1
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = RXQ0.2 RXQ1.2 ;RXQ2.2 RXQ3.2
+pktq_out = TXQ0.3 TXQ1.3 SWQ2;TXQ2.3 TXQ3.3 SWQ2
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = RXQ0.3 RXQ1.3 ;RXQ2.2 RXQ3.2
+pktq_out = TXQ0.4 TXQ1.4 SWQ3;TXQ2.3 TXQ3.3 SWQ2
+;
+;
+;n_rules = 10000
+;n_flows gets round up to power of 2
+
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+
+tcp_be_liberal = 0
+
diff --git a/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_script.tc b/VNFs/vFW/config/VFW_HWLB_IPV6_SinglePortPair_script.tc
new file mode 100644 (file)
index 0000000..c157944
--- /dev/null
@@ -0,0 +1,54 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config fec0:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 0 up
+
+link 1 down
+link 1 config 2012:0000:0000:0000:6a05:caff:fe30:2071 64
+link 1 up
+
+;p 1 arpadd 0 fec0::6a05:caff:fe30:21b0 00:00:00:00:00:01
+;p 1 arpadd 1 2012::6a05:caff:fe30:2081 00:00:00:00:00:04
+
+p action add 0 accept
+p action add 0 fwd 1
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 drop
+p action add 2 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+
+p vfw add 1 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 2
+p vfw add 2 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 0
+p vfw add 2 2012::6a05:caff:fe30:2081 64 fec0::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 1
+p vfw applyruleset
+
+set fwd rxonly
+set_sym_hash_ena_per_port 0 enable
+set_hash_global_config 0 simple_xor ipv6-udp enable
+set_sym_hash_ena_per_port 1 enable
+set_hash_global_config 1 simple_xor ipv6-udp enable
+
+
+set_hash_input_set 0 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
+set_hash_input_set 1 ipv6-udp src-ipv6 dst-ipv6 udp-src-port udp-dst-port  add
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_1Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..8e26c28
--- /dev/null
@@ -0,0 +1,121 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2 SWQ9
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ad102814,ff000000,1,ad102814) (cb106414,ff000000,0,cb106414)
+arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,2,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+#ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ2.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)(2,3)
+prv_que_handler = (0,2)
+;lib_arp_debug = 1
+[PIPELINE2]
+type = TXRX
+core = s0c2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 1
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = s0c4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ5 SWQ6;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE5]
+type = TXRX
+core = s0c2h
+pktq_in  = SWQ5 SWQ6 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ0.1 TXQ1.1 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
+[PIPELINE6]
+type = TXRX
+core = s0c5
+pktq_in  = RXQ2.0 RXQ3.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ7 SWQ8 SWQ9;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE7]
+type = LOADB
+core = 6
+pktq_in  = SWQ7 SWQ8
+pktq_out = SWQ9 SWQ10 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 1
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE8]
+type = VFW
+core = s0c7
+pktq_in  = SWQ9 SWQ10
+pktq_out = SWQ11 SWQ12;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE9]
+type = TXRX
+core = s0c5h
+pktq_in  = SWQ11 SWQ12 ;SWQ20 SWQ21 ;RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ2.1 TXQ3.1 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_4Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..adcce34
--- /dev/null
@@ -0,0 +1,155 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2 SWQ13
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ad102814,ff000000,1,ad102814) (cb106414,ff000000,0,cb106414)
+arp_route_tbl = (ac102814,ff000000,3,ac102814) (ca106414,ff000000,2,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+#ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ2.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)(2,3)
+prv_que_handler = (0,2)
+
+;lib_arp_debug = 1
+[PIPELINE2]
+type = TXRX
+core = s0c2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 SWQ5 SWQ6 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 2
+prv_que_handler = (0)
+
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = s0c4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ7 SWQ8;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = s0c4h
+pktq_in  = SWQ5 SWQ6
+pktq_out = SWQ9 SWQ10;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE6]
+type = TXRX
+core = s0c2h
+pktq_in  = SWQ7 SWQ8 SWQ9 SWQ10 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
+[PIPELINE7]
+type = TXRX
+core = s0c5
+pktq_in  = RXQ2.0 RXQ3.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ11 SWQ12 SWQ13;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE8]
+type = LOADB
+core = 6
+pktq_in  = SWQ11 SWQ12
+pktq_out = SWQ14 SWQ15 SWQ16 SWQ17 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 2
+n_lb_tuples = 5
+prv_que_handler = (0)
+;loadb_debug = 0
+[PIPELINE9]
+type = VFW
+core = s0c7
+pktq_in  = SWQ14 SWQ15
+pktq_out = SWQ18 SWQ19;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE10]
+type = VFW
+core = s0c7h
+pktq_in  = SWQ16 SWQ17
+pktq_out = SWQ20 SWQ21;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE11]
+type = TXRX
+core = s0c5h
+pktq_in  = SWQ18 SWQ19 SWQ20 SWQ21 ;RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ2.1 TXQ3.1 TXQ2.2 TXQ3.2 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_script.tc b/VNFs/vFW/config/VFW_SWLB_IPV4_MultiPortPair_script.tc
new file mode 100644 (file)
index 0000000..2e328a7
--- /dev/null
@@ -0,0 +1,67 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config 203.16.100.10 8
+link 0 up
+link 1 down
+link 1 config 202.16.100.10 8
+link 1 up
+link 2 down
+link 2 config 173.16.40.10 8
+link 2 up
+link 3 down
+link 3 config 172.16.40.10 8
+link 3 up
+;p 1 arpadd 0 203.16.100.20 00:cb:10:64:14:00
+;p 1 arpadd 1 173.16.40.20 00:ca:10:64:14:00
+;p 1 arpadd 2 202.16.100.20 00:ad:10:28:14:00
+;p 1 arpadd 3 172.16.40.20 00:ac:10:28:14:00
+p action add 0 accept
+p action add 0 fwd 1
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 accept
+p action add 2 fwd 3
+p action add 2 count
+
+p action add 3 accept
+p action add 3 fwd 2
+p action add 3 count
+
+p action add 4 drop
+p action add 4 count
+
+p action add 5 drop
+p action add 5 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+p action add 4 conntrack
+p action add 5 conntrack
+
+p vfw add 1 203.16.100.20 8 173.16.40.20 8 0 65535 67 69 0 0 4
+p vfw add 1 202.16.100.20 8 172.16.40.20 8 0 65535 67 69 0 0 5
+
+p vfw add 2 203.16.100.20 8 173.16.40.20 8 0 65535 0 65535 0 0 0
+p vfw add 2 173.16.40.20 8 203.16.100.20 8 0 65535 0 65535 0 0 1
+p vfw add 2 202.16.100.20 8 172.16.40.20 8 0 65535 0 65535 0 0 2
+p vfw add 2 172.16.40.20 8 202.16.100.20 8 0 65535 0 65535 0 0 3
+p vfw applyruleset
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_1Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..5499ea8
--- /dev/null
@@ -0,0 +1,81 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ac102814,ff000000,1,ac102814) (ca106414,ff000000,0,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+
+[PIPELINE2]
+type = TXRX
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0
+pktq_out = SWQ0 SWQ1 SWQ2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4
+outport_offset = 136
+n_vnf_threads = 1
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ5 SWQ6
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 4096000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+;udp_replied = 20
+
+[PIPELINE5]
+type = TXRX
+core = 5
+pktq_in  = SWQ5 SWQ6
+pktq_out = TXQ0.1 TXQ1.1
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..abb4735
--- /dev/null
@@ -0,0 +1,130 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+arp_route_tbl = (ac102814,ff000000,1,ac102814) (ca106414,ff000000,0,ca106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+
+[PIPELINE2]
+type = TXRX
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0
+pktq_out = SWQ0 SWQ1 SWQ2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 SWQ5 SWQ6 SWQ7 SWQ8 SWQ9 SWQ10
+outport_offset = 136
+n_vnf_threads = 4
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ11 SWQ12;TXQ0.0 TXQ1.0
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = SWQ5 SWQ6
+pktq_out = SWQ13 SWQ14;TXQ0.0 TXQ1.0
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE6]
+type = VFW
+core = 6
+pktq_in  = SWQ7 SWQ8
+pktq_out = SWQ15 SWQ16;TXQ0.0 TXQ1.0
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE7]
+type = VFW
+core = 7
+pktq_in  = SWQ9 SWQ10
+pktq_out = SWQ17 SWQ18;TXQ0.0 TXQ1.0
+
+n_rules = 4096
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv4
+traffic_type = 4
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE8]
+type = TXRX
+core = 8
+pktq_in  = SWQ11 SWQ12 SWQ13 SWQ14 SWQ15 SWQ16 SWQ17 SWQ18
+pktq_out = TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2 TXQ0.3 TXQ1.3 TXQ0.4 TXQ1.4
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_script.tc b/VNFs/vFW/config/VFW_SWLB_IPV4_SinglePortPair_script.tc
new file mode 100644 (file)
index 0000000..213f58f
--- /dev/null
@@ -0,0 +1,49 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config 202.16.100.10 8
+link 0 up
+link 1 down
+link 1 config 172.16.40.10 8
+link 1 up
+
+;p 1 arpadd 1 172.16.40.20 00:00:00:00:00:04
+;p 1 arpadd 0 202.16.100.20 00:00:00:00:00:01
+
+p action add 0 accept
+p action add 0 fwd 0
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 1
+p action add 1 count
+
+p action add 2 drop
+p action add 2 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+p action add 4 conntrack
+p action add 5 conntrack
+p action add 6 conntrack
+p action add 7 conntrack
+
+p vfw add 1 202.16.100.20 8 172.16.40.20 8 0 65535 67 69 0 0 2
+p vfw add 2 202.16.100.20 8 172.16.40.20 8 0 65535 0 65535 0 0 1
+p vfw add 2 172.16.40.20 8 202.16.100.20 8 0 65535 0 65535 0 0 0
+
+p vfw applyruleset
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_1Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..aefb37a
--- /dev/null
@@ -0,0 +1,122 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2 SWQ9
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+nd_route_tbl =  (fec1::6a05:caff:fe30:21b0,64,2,fec1::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2016::6a05:caff:fe30:2081,64,3,2016::6a05:caff:fe30:2081)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ2.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)(2,3)
+prv_que_handler = (0,2)
+;lib_arp_debug = 1
+[PIPELINE2]
+type = TXRX
+core = s0c2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 1
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = s0c4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ5 SWQ6;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+
+[PIPELINE5]
+type = TXRX
+core = s0c2h
+pktq_in  = SWQ5 SWQ6 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ0.1 TXQ1.1 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
+[PIPELINE6]
+type = TXRX
+core = s0c5
+pktq_in  = RXQ2.0 RXQ3.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ7 SWQ8 SWQ9;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE7]
+type = LOADB
+core = 6
+pktq_in  = SWQ7 SWQ8
+pktq_out = SWQ9 SWQ10 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 1
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE8]
+type = VFW
+core = s0c7
+pktq_in  = SWQ9 SWQ10
+pktq_out = SWQ11 SWQ12;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE9]
+type = TXRX
+core = s0c5h
+pktq_in  = SWQ11 SWQ12 ;SWQ20 SWQ21 ;RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ2.1 TXQ3.1 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_4Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..bc30db8
--- /dev/null
@@ -0,0 +1,152 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2 SWQ13
+pktq_out = TXQ0.0 TXQ1.0 TXQ2.0 TXQ3.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+nd_route_tbl =  (fec1::6a05:caff:fe30:21b0,64,2,fec1::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2016::6a05:caff:fe30:2081,64,3,2016::6a05:caff:fe30:2081)
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ca:10:64:14:00 00:ad:10:28:14:00 00:ac:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0 RXQ2.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)(2,3)
+prv_que_handler = (0,2)
+;lib_arp_debug = 1
+[PIPELINE2]
+type = TXRX
+core = s0c2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 SWQ5 SWQ6 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 2
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = s0c4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ7 SWQ8;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = s0c4h
+pktq_in  = SWQ5 SWQ6
+pktq_out = SWQ9 SWQ10;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE6]
+type = TXRX
+core = s0c2h
+pktq_in  = SWQ7 SWQ8 SWQ9 SWQ10 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2 ;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
+[PIPELINE7]
+type = TXRX
+core = s0c5
+pktq_in  = RXQ2.0 RXQ3.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ11 SWQ12 SWQ13;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE8]
+type = LOADB
+core = 6
+pktq_in  = SWQ11 SWQ12
+pktq_out = SWQ14 SWQ15 SWQ16 SWQ17 ;SWQ10 SWQ11
+outport_offset = 136; 8
+n_vnf_threads = 2
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE9]
+type = VFW
+core = s0c7
+pktq_in  = SWQ14 SWQ15
+pktq_out = SWQ18 SWQ19;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE10]
+type = VFW
+core = s0c7h
+pktq_in  = SWQ16 SWQ17
+pktq_out = SWQ20 SWQ21;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE11]
+type = TXRX
+core = s0c5h
+pktq_in  = SWQ18 SWQ19 SWQ20 SWQ21 ;RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = TXQ2.1 TXQ3.1 TXQ2.2 TXQ3.2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_script.tc b/VNFs/vFW/config/VFW_SWLB_IPV6_MultiPortPair_script.tc
new file mode 100644 (file)
index 0000000..048ff73
--- /dev/null
@@ -0,0 +1,68 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config fec0:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 0 up
+link 1 down
+link 1 config fec1:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 1 up
+link 2 down
+link 2 config 2012:0000:0000:0000:6a05:caff:fe30:2071 64
+link 2 up
+link 3 down
+link 3 config 2016:0000:0000:0000:6a05:caff:fe30:2071 64
+link 3 up
+;p 1 arpadd 0 fec0::6a05:caff:fe30:21b0 00:cb:10:64:14:00
+;p 1 arpadd 1 2012::6a05:caff:fe30:2081 00:ca:10:64:14:00
+;p 1 arpadd 2 fec1::6a05:caff:fe30:21b0 00:ad:10:28:14:00
+;p 1 arpadd 3 2016::6a05:caff:fe30:2081 00:ac:10:28:14:00
+p action add 0 accept
+p action add 0 fwd 1
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 accept
+p action add 2 fwd 3
+p action add 2 count
+
+p action add 3 accept
+p action add 3 fwd 2
+p action add 3 count
+
+p action add 4 drop
+p action add 4 count
+
+p action add 5 drop
+p action add 5 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+p action add 3 conntrack
+p action add 4 conntrack
+p action add 5 conntrack
+
+p vfw add 1 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 4
+p vfw add 1 fec1::6a05:caff:fe30:21b0 64 2016::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 5
+p vfw add 2 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 0
+p vfw add 2 2012::6a05:caff:fe30:2081 64 fec0::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 1
+p vfw add 2 fec1::6a05:caff:fe30:21b0 64 2016::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 2
+p vfw add 2 2016::6a05:caff:fe30:2081 64 fec1::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 3
+p vfw applyruleset
+
+
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_1Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_1Thread.cfg
new file mode 100644 (file)
index 0000000..4fb0fad
--- /dev/null
@@ -0,0 +1,79 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+;arp_route_tbl = (98102814,ff000000,1,98102814) (c0106414,ff000000,0,c0106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+[PIPELINE2]
+type = TXRX
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4
+outport_offset = 136; 8
+n_vnf_threads = 1
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ5 SWQ6;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = TXRX
+core = 5
+pktq_in  = SWQ5 SWQ6
+pktq_out = TXQ0.1 TXQ1.1
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_4Thread.cfg b/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_4Thread.cfg
new file mode 100644 (file)
index 0000000..4c372f7
--- /dev/null
@@ -0,0 +1,131 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+[PIPELINE0]
+type = MASTER
+core = 0
+[PIPELINE1]
+type =  ARPICMP
+core = 1
+
+pktq_in  = SWQ2
+pktq_out = TXQ0.0 TXQ1.0
+; ARP route table entries (ip, mask, if_port, nh) hex values with no 0x
+nd_route_tbl =  (fec0::6a05:caff:fe30:21b0,64,0,fec0::6a05:caff:fe30:21b0)
+nd_route_tbl =  (2012::6a05:caff:fe30:2081,64,1,2012::6a05:caff:fe30:2081)
+;arp_route_tbl = (98102814,ff000000,1,98102814) (c0106414,ff000000,0,c0106414)
+; Link MAC addresses in order aa:bb:cc:dd:ee:ff separated by space
+;
+; Hex values with no leading 0x, MACs in ascending port order starting @P0
+ports_mac_list = 00:cb:10:64:14:00 00:ad:10:28:14:00
+;
+; egress (private interface) info
+pktq_in_prv =  RXQ0.0
+;
+;for pub port <-> prv port mapping (prv, pub)
+prv_to_pub_map = (0,1)
+prv_que_handler = (0)
+[PIPELINE2]
+type = TXRX
+core = 2
+pktq_in  = RXQ0.0 RXQ1.0 ;SWQ4 SWQ5 ;SWQ8 SWQ9 SWQ12 SWQ13
+pktq_out = SWQ0 SWQ1 SWQ2;TXQ0.0 TXQ1.0 ;TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2
+pipeline_txrx_type = RXRX
+;
+[PIPELINE3]
+type = LOADB
+core = 3
+pktq_in  = SWQ0 SWQ1
+pktq_out = SWQ3 SWQ4 SWQ5 SWQ6 SWQ7 SWQ8 SWQ9 SWQ10
+outport_offset = 136; 8
+n_vnf_threads = 4
+prv_que_handler = (0)
+n_lb_tuples = 5
+;loadb_debug = 0
+[PIPELINE4]
+type = VFW
+core = 4
+pktq_in  = SWQ3 SWQ4
+pktq_out = SWQ11 SWQ12;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE5]
+type = VFW
+core = 5
+pktq_in  = SWQ5 SWQ6
+pktq_out = SWQ13 SWQ14;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE6]
+type = VFW
+core = 6
+pktq_in  = SWQ7 SWQ8
+pktq_out = SWQ15 SWQ16;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+;udp_unreplied and udp_replied controls udp "connection" timeouts, normally 30/180
+;udp_unreplied = 20
+;udp_replied = 20
+[PIPELINE7]
+type = VFW
+core = 7
+pktq_in  = SWQ9 SWQ10
+pktq_out = SWQ17 SWQ18;TXQ0.0 TXQ1.0
+
+n_rules = 10000
+
+;n_flows gets round up to power of 2
+n_flows = 1000000
+pkt_type = ipv6
+traffic_type = 6
+; tcp_time_wait controls timeout for closed connection, normally 120
+tcp_time_wait = 10
+tcp_be_liberal = 0
+
+[PIPELINE8]
+type = TXRX
+core = 8
+pktq_in  = SWQ11 SWQ12 SWQ13 SWQ14 SWQ15 SWQ16 SWQ17 SWQ18
+pktq_out = TXQ0.1 TXQ1.1 TXQ0.2 TXQ1.2 TXQ0.3 TXQ1.3 TXQ0.4 TXQ1.4
+pipeline_txrx_type = TXTX
diff --git a/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_script.tc b/VNFs/vFW/config/VFW_SWLB_IPV6_SinglePortPair_script.tc
new file mode 100644 (file)
index 0000000..b333d5c
--- /dev/null
@@ -0,0 +1,44 @@
+; Copyright (c) 2017 Intel Corporation
+;
+; Licensed under the Apache License, Version 2.0 (the "License");
+; you may not use this file except in compliance with the License.
+; You may obtain a copy of the License at
+;
+;      http:#www.apache.org/licenses/LICENSE-2.0
+;
+; Unless required by applicable law or agreed to in writing, software
+; distributed under the License is distributed on an "AS IS" BASIS,
+; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+; See the License for the specific language governing permissions and
+; limitations under the License.
+
+link 0 down
+link 0 config fec0:0000:0000:0000:6a05:caff:fe30:21a0 64
+link 0 up
+
+link 1 down
+link 1 config 2012:0000:0000:0000:6a05:caff:fe30:2071 64
+link 1 up
+
+;p 1 arpadd 0 fec0::6a05:caff:fe30:21b0 00:00:00:00:00:01
+;p 1 arpadd 1 2012::6a05:caff:fe30:2081 00:00:00:00:00:04
+
+p action add 0 accept
+p action add 0 fwd 1
+p action add 0 count
+
+p action add 1 accept
+p action add 1 fwd 0
+p action add 1 count
+
+p action add 2 drop
+p action add 2 count
+
+p action add 0 conntrack
+p action add 1 conntrack
+p action add 2 conntrack
+
+p vfw add 1 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 67 69 0 0 2
+p vfw add 2 fec0::6a05:caff:fe30:21b0 64 2012::6a05:caff:fe30:2081 64 0 65535 0 65535 0 0 0
+p vfw add 2 2012::6a05:caff:fe30:2081 64 fec0::6a05:caff:fe30:21b0 64 0 65535 0 65535 0 0 1
+p vfw applyruleset
diff --git a/VNFs/vFW/init.c b/VNFs/vFW/init.c
new file mode 100644 (file)
index 0000000..676cdb3
--- /dev/null
@@ -0,0 +1,1452 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+#include <inttypes.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <rte_cycles.h>
+#include <rte_ethdev.h>
+#include <rte_ether.h>
+#include <rte_ip.h>
+#include <rte_eal.h>
+#include <rte_malloc.h>
+
+#include "app.h"
+#include "pipeline.h"
+#include "pipeline_common_fe.h"
+#include "pipeline_master.h"
+#include "pipeline_passthrough.h"
+#include "thread_fe.h"
+#include "pipeline_vfw.h"
+#include "pipeline_loadb.h"
+#include "pipeline_txrx.h"
+#include "pipeline_arpicmp.h"
+#include "interface.h"
+#include "l3fwd_common.h"
+#include "l3fwd_lpm4.h"
+#include "l3fwd_lpm6.h"
+#include "lib_arp.h"
+#include "vnf_define.h"
+#define APP_NAME_SIZE  32
+port_config_t *port_config;
+
+static void
+app_init_core_map(struct app_params *app)
+{
+       APP_LOG(app, HIGH, "Initializing CPU core map ...");
+       app->core_map = cpu_core_map_init(4, 32, 4, 0);
+
+       if (app->core_map == NULL)
+               rte_panic("Cannot create CPU core map\n");
+
+       if (app->log_level >= APP_LOG_LEVEL_LOW)
+               cpu_core_map_print(app->core_map);
+}
+
+/* Core Mask String in Hex Representation */
+#define APP_CORE_MASK_STRING_SIZE ((64 * APP_CORE_MASK_SIZE) / 8 * 2 + 1)
+
+static void
+app_init_core_mask(struct app_params *app)
+{
+       char core_mask_str[APP_CORE_MASK_STRING_SIZE];
+       uint32_t i;
+
+       for (i = 0; i < app->n_pipelines; i++) {
+               struct app_pipeline_params *p = &app->pipeline_params[i];
+               int lcore_id;
+
+               lcore_id = cpu_core_map_get_lcore_id(app->core_map,
+                       p->socket_id,
+                       p->core_id,
+                       p->hyper_th_id);
+
+               if (lcore_id < 0)
+                       rte_panic("Cannot create CPU core mask\n");
+
+               app_core_enable_in_core_mask(app, lcore_id);
+       }
+
+       app_core_build_core_mask_string(app, core_mask_str);
+       APP_LOG(app, HIGH, "CPU core mask = 0x%s", core_mask_str);
+
+}
+
+static void
+app_init_eal(struct app_params *app)
+{
+       char buffer[256];
+       char core_mask_str[APP_CORE_MASK_STRING_SIZE];
+       struct app_eal_params *p = &app->eal_params;
+       uint8_t n_args = 0;
+       uint32_t i;
+       int status;
+
+       app->eal_argv[n_args++] = strdup(app->app_name);
+
+       app_core_build_core_mask_string(app, core_mask_str);
+       snprintf(buffer, sizeof(buffer), "-c%s", core_mask_str);
+       app->eal_argv[n_args++] = strdup(buffer);
+
+       if (p->coremap) {
+               snprintf(buffer, sizeof(buffer), "--lcores=%s", p->coremap);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->master_lcore_present) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--master-lcore=%" PRIu32,
+                       p->master_lcore);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       snprintf(buffer, sizeof(buffer), "-n%" PRIu32, p->channels);
+       app->eal_argv[n_args++] = strdup(buffer);
+
+       if (p->memory_present) {
+               snprintf(buffer, sizeof(buffer), "-m%" PRIu32, p->memory);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->ranks_present) {
+               snprintf(buffer, sizeof(buffer), "-r%" PRIu32, p->ranks);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       for (i = 0; i < APP_MAX_LINKS; i++) {
+               if (p->pci_blacklist[i] == NULL)
+                       break;
+
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--pci-blacklist=%s",
+                       p->pci_blacklist[i]);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (app->port_mask != 0)
+               for (i = 0; i < APP_MAX_LINKS; i++) {
+                       if (p->pci_whitelist[i] == NULL)
+                               break;
+
+                       snprintf(buffer,
+                               sizeof(buffer),
+                               "--pci-whitelist=%s",
+                               p->pci_whitelist[i]);
+                       app->eal_argv[n_args++] = strdup(buffer);
+               }
+       else
+               for (i = 0; i < app->n_links; i++) {
+                       char *pci_bdf = app->link_params[i].pci_bdf;
+
+                       snprintf(buffer,
+                               sizeof(buffer),
+                               "--pci-whitelist=%s",
+                               pci_bdf);
+                       app->eal_argv[n_args++] = strdup(buffer);
+               }
+
+       for (i = 0; i < APP_MAX_LINKS; i++) {
+               if (p->vdev[i] == NULL)
+                       break;
+
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--vdev=%s",
+                       p->vdev[i]);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->vmware_tsc_map_present) && p->vmware_tsc_map) {
+               snprintf(buffer, sizeof(buffer), "--vmware-tsc-map");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->proc_type) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--proc-type=%s",
+                       p->proc_type);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->syslog) {
+               snprintf(buffer, sizeof(buffer), "--syslog=%s", p->syslog);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->log_level_present) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--log-level=%" PRIu32,
+                       p->log_level);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->version_present) && p->version) {
+               snprintf(buffer, sizeof(buffer), "-v");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->help_present) && p->help) {
+               snprintf(buffer, sizeof(buffer), "--help");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->no_huge_present) && p->no_huge) {
+               snprintf(buffer, sizeof(buffer), "--no-huge");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->no_pci_present) && p->no_pci) {
+               snprintf(buffer, sizeof(buffer), "--no-pci");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->no_hpet_present) && p->no_hpet) {
+               snprintf(buffer, sizeof(buffer), "--no-hpet");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->no_shconf_present) && p->no_shconf) {
+               snprintf(buffer, sizeof(buffer), "--no-shconf");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->add_driver) {
+               snprintf(buffer, sizeof(buffer), "-d=%s", p->add_driver);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->socket_mem) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--socket-mem=%s",
+                       p->socket_mem);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->huge_dir) {
+               snprintf(buffer, sizeof(buffer), "--huge-dir=%s", p->huge_dir);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->file_prefix) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--file-prefix=%s",
+                       p->file_prefix);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->base_virtaddr) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--base-virtaddr=%s",
+                       p->base_virtaddr);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->create_uio_dev_present) && p->create_uio_dev) {
+               snprintf(buffer, sizeof(buffer), "--create-uio-dev");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if (p->vfio_intr) {
+               snprintf(buffer,
+                       sizeof(buffer),
+                       "--vfio-intr=%s",
+                       p->vfio_intr);
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       if ((p->xen_dom0_present) && (p->xen_dom0)) {
+               snprintf(buffer, sizeof(buffer), "--xen-dom0");
+               app->eal_argv[n_args++] = strdup(buffer);
+       }
+
+       snprintf(buffer, sizeof(buffer), "--");
+       app->eal_argv[n_args++] = strdup(buffer);
+
+       app->eal_argc = n_args;
+
+       APP_LOG(app, HIGH, "Initializing EAL ...");
+       if (app->log_level >= APP_LOG_LEVEL_LOW) {
+               int i;
+
+               fprintf(stdout, "[APP] EAL arguments: \"");
+               for (i = 1; i < app->eal_argc; i++)
+                       fprintf(stdout, "%s ", app->eal_argv[i]);
+               fprintf(stdout, "\"\n");
+       }
+
+       status = rte_eal_init(app->eal_argc, app->eal_argv);
+       if (status < 0)
+               rte_panic("EAL init error\n");
+}
+static inline int
+app_link_filter_arp_add(struct app_link_params *link)
+{
+       struct rte_eth_ethertype_filter filter = {
+               .ether_type = ETHER_TYPE_ARP,
+               .flags = 0,
+               .queue = link->arp_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(link->pmd_id,
+               RTE_ETH_FILTER_ETHERTYPE,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_tcp_syn_add(struct app_link_params *link)
+{
+       struct rte_eth_syn_filter filter = {
+               .hig_pri = 1,
+               .queue = link->tcp_syn_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(link->pmd_id,
+               RTE_ETH_FILTER_SYN,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_ip_add(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = 0,
+               .proto_mask = 0, /* Disable */
+               .tcp_flags = 0,
+               .priority = 1, /* Lowest */
+               .queue = l1->ip_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_ip_del(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = 0,
+               .proto_mask = 0, /* Disable */
+               .tcp_flags = 0,
+               .priority = 1, /* Lowest */
+               .queue = l1->ip_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_DELETE,
+               &filter);
+}
+
+static inline int
+app_link_filter_tcp_add(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_TCP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->tcp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_tcp_del(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_TCP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->tcp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_DELETE,
+               &filter);
+}
+
+static inline int
+app_link_filter_udp_add(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_UDP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->udp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_udp_del(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_UDP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->udp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_DELETE,
+               &filter);
+}
+
+static inline int
+app_link_filter_sctp_add(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_SCTP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->sctp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_ADD,
+               &filter);
+}
+
+static inline int
+app_link_filter_sctp_del(struct app_link_params *l1, struct app_link_params *l2)
+{
+       struct rte_eth_ntuple_filter filter = {
+               .flags = RTE_5TUPLE_FLAGS,
+               .dst_ip = rte_bswap32(l2->ip),
+               .dst_ip_mask = UINT32_MAX, /* Enable */
+               .src_ip = 0,
+               .src_ip_mask = 0, /* Disable */
+               .dst_port = 0,
+               .dst_port_mask = 0, /* Disable */
+               .src_port = 0,
+               .src_port_mask = 0, /* Disable */
+               .proto = IPPROTO_SCTP,
+               .proto_mask = UINT8_MAX, /* Enable */
+               .tcp_flags = 0,
+               .priority = 2, /* Higher priority than IP */
+               .queue = l1->sctp_local_q,
+       };
+
+       return rte_eth_dev_filter_ctrl(l1->pmd_id,
+               RTE_ETH_FILTER_NTUPLE,
+               RTE_ETH_FILTER_DELETE,
+               &filter);
+}
+
+static int
+app_link_is_virtual(struct app_link_params *p)
+{
+        uint32_t pmd_id = p->pmd_id;
+        struct rte_eth_dev *dev = &rte_eth_devices[pmd_id];
+
+        if (dev->dev_type == RTE_ETH_DEV_VIRTUAL)
+                return 1;
+
+        return 0;
+}
+
+
+void
+app_link_up_internal(__rte_unused struct app_params *app,
+               struct app_link_params *cp)
+{
+       if(app == NULL || cp == NULL)
+               printf("NULL Pointers");
+
+        if (app_link_is_virtual(cp)) {
+                cp->state = 1;
+                return;
+        }
+
+       ifm_update_linkstatus(cp->pmd_id, IFM_ETH_LINK_UP);
+
+       /* Mark link as UP */
+       cp->state = 1;
+}
+
+void
+app_link_down_internal(__rte_unused struct app_params *app,
+               struct app_link_params *cp)
+{
+       if(app == NULL || cp == NULL)
+               printf("NULL Pointers");
+
+        if (app_link_is_virtual(cp)) {
+                cp->state = 0;
+                return;
+        }
+
+       ifm_update_linkstatus(cp->pmd_id, IFM_ETH_LINK_DOWN);
+       /* Mark link as DOWN */
+       cp->state = 0;
+
+}
+
+static void
+app_check_link(struct app_params *app)
+{
+       uint32_t all_links_up, i;
+
+       all_links_up = 1;
+
+       for (i = 0; i < app->n_links; i++) {
+               struct app_link_params *p = &app->link_params[i];
+               struct rte_eth_link link_params;
+
+               memset(&link_params, 0, sizeof(link_params));
+               rte_eth_link_get(p->pmd_id, &link_params);
+
+               APP_LOG(app, HIGH, "%s (%" PRIu32 ") (%" PRIu32 " Gbps) %s",
+                       p->name,
+                       p->pmd_id,
+                       link_params.link_speed / 1000,
+                       link_params.link_status ? "UP" : "DOWN");
+
+               if (link_params.link_status == ETH_LINK_DOWN)
+                       all_links_up = 0;
+       }
+
+       if (all_links_up == 0)
+               rte_panic("Some links are DOWN\n");
+}
+
+static uint32_t
+is_any_swq_frag_or_ras(struct app_params *app)
+{
+       uint32_t i;
+
+       for (i = 0; i < app->n_pktq_swq; i++) {
+               struct app_pktq_swq_params *p = &app->swq_params[i];
+
+               if ((p->ipv4_frag == 1) || (p->ipv6_frag == 1) ||
+                       (p->ipv4_ras == 1) || (p->ipv6_ras == 1))
+                       return 1;
+       }
+
+       return 0;
+}
+
+static void
+app_init_link_frag_ras(struct app_params *app)
+{
+       uint32_t i;
+
+       if (is_any_swq_frag_or_ras(app)) {
+               for (i = 0; i < app->n_pktq_hwq_out; i++) {
+                       struct app_pktq_hwq_out_params *p_txq =
+                               &app->hwq_out_params[i];
+
+                       p_txq->conf.txq_flags &= ~ETH_TXQ_FLAGS_NOMULTSEGS;
+               }
+       }
+}
+
+static inline int
+app_get_cpu_socket_id(uint32_t pmd_id)
+{
+       int status = rte_eth_dev_socket_id(pmd_id);
+
+       return (status != SOCKET_ID_ANY) ? status : 0;
+}
+
+struct rte_eth_rxmode rx_mode = {
+       .max_rx_pkt_len = ETHER_MAX_LEN, /**< Default maximum frame length. */
+       .split_hdr_size = 0,
+       .header_split   = 0, /**< Header Split disabled. */
+       .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+       .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+       .hw_vlan_strip  = 1, /**< VLAN strip enabled. */
+       .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+       .jumbo_frame    = 0, /**< Jumbo Frame Support disabled. */
+       .hw_strip_crc   = 0, /**< CRC stripping by hardware disabled. */
+};
+struct rte_fdir_conf fdir_conf = {
+       .mode = RTE_FDIR_MODE_NONE,
+       .pballoc = RTE_FDIR_PBALLOC_64K,
+       .status = RTE_FDIR_REPORT_STATUS,
+       .mask = {
+               .vlan_tci_mask = 0x0,
+               .ipv4_mask     = {
+                       .src_ip = 0xFFFFFFFF,
+                       .dst_ip = 0xFFFFFFFF,
+               },
+               .ipv6_mask     = {
+               .src_ip = {0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF},
+               .dst_ip = {0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF},
+               },
+               .src_port_mask = 0xFFFF,
+               .dst_port_mask = 0xFFFF,
+               .mac_addr_byte_mask = 0xFF,
+               .tunnel_type_mask = 1,
+               .tunnel_id_mask = 0xFFFFFFFF,
+       },
+       .drop_queue = 127,
+};
+
+       static void
+app_init_link(struct app_params *app)
+{
+       uint32_t i, size;
+
+       app_init_link_frag_ras(app);
+
+       /* Configuring port_config_t structure for interface
+        * manager initialization
+        */
+       size = RTE_CACHE_LINE_ROUNDUP(sizeof(port_config_t));
+       port_config = rte_zmalloc(NULL, (app->n_links * size),
+                       RTE_CACHE_LINE_SIZE);
+       if (port_config == NULL)
+               rte_panic("port_config is NULL: Memory Allocation failure\n");
+
+       for (i = 0; i < app->n_links; i++) {
+               struct app_link_params *p_link = &app->link_params[i];
+               uint32_t link_id, n_hwq_in, n_hwq_out;
+               int status;
+
+               status = sscanf(p_link->name, "LINK%" PRIu32, &link_id);
+               if (status < 0)
+                       rte_panic("%s (%" PRId32 "): "
+                                       "init error (%" PRId32 ")\n",
+                                       p_link->name, link_id, status);
+
+               n_hwq_in = app_link_get_n_rxq(app, p_link);
+               n_hwq_out = app_link_get_n_txq(app, p_link);
+
+               printf("\n\nn_hwq_in %d\n", n_hwq_in);
+               struct rte_eth_conf *My_local_conf = &p_link->conf;
+               if (enable_hwlb) {
+                       My_local_conf->rxmode = rx_mode;
+                       My_local_conf->fdir_conf = fdir_conf;
+                       My_local_conf->rxmode.mq_mode = ETH_MQ_RX_RSS;
+                       My_local_conf->rx_adv_conf.rss_conf.rss_key = NULL;
+                       My_local_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP
+                               | ETH_RSS_UDP | ETH_RSS_TCP;
+               } else {/* disable-rss */
+                       My_local_conf->rx_adv_conf.rss_conf.rss_hf = 0;
+                       /* pkt-filter-mode is perfect */
+                       My_local_conf->fdir_conf.mode = RTE_FDIR_MODE_PERFECT;
+               }
+
+               /* Set the hardware CRC stripping to avoid double stripping
+                * of FCS in VM */
+               p_link->conf.rxmode.hw_strip_crc = 1;
+
+               APP_LOG(app, HIGH, "Initializing %s (%" PRIu32") "
+                               "(%" PRIu32 " RXQ, %" PRIu32 " TXQ) ...",
+                               p_link->name,
+                               p_link->pmd_id,
+                               n_hwq_in,
+                               n_hwq_out);
+
+               port_config[i].port_id = p_link->pmd_id;
+               port_config[i].nrx_queue = n_hwq_in;
+               port_config[i].ntx_queue = n_hwq_out;
+               port_config[i].state = 1;
+               port_config[i].promisc = p_link->promisc;
+               port_config[i].mempool.pool_size =
+                       app->mempool_params[0].pool_size;
+               port_config[i].mempool.buffer_size =
+                       app->mempool_params[0].buffer_size;
+               port_config[i].mempool.cache_size =
+                       app->mempool_params[0].cache_size;
+               port_config[i].mempool.cpu_socket_id =
+                       app->mempool_params[0].cpu_socket_id;
+               memcpy(&port_config[i].port_conf, &p_link->conf,
+                               sizeof(struct rte_eth_conf));
+               memcpy(&port_config[i].rx_conf, &app->hwq_in_params[0].conf,
+                               sizeof(struct rte_eth_rxconf));
+               memcpy(&port_config[i].tx_conf, &app->hwq_out_params[0].conf,
+                               sizeof(struct rte_eth_txconf));
+
+               if (app->header_csum_req) {
+                       /* Enable TCP and UDP HW Checksum */
+                       port_config[i].tx_conf.txq_flags &=
+                               ~(ETH_TXQ_FLAGS_NOXSUMTCP |
+                                               ETH_TXQ_FLAGS_NOXSUMUDP);
+               }
+
+               if (ifm_port_setup(p_link->pmd_id, &port_config[i]))
+                       rte_panic("Port Setup Failed: %s - %" PRIu32
+                                       "\n", p_link->name, p_link->pmd_id);
+
+               app_link_up_internal(app, p_link);
+       }
+
+       app_check_link(app);
+}
+
+static void
+app_init_swq(struct app_params *app)
+{
+       uint32_t i;
+
+       for (i = 0; i < app->n_pktq_swq; i++) {
+               struct app_pktq_swq_params *p = &app->swq_params[i];
+               unsigned int flags = 0;
+
+               if (app_swq_get_readers(app, p) == 1)
+                       flags |= RING_F_SC_DEQ;
+               if (app_swq_get_writers(app, p) == 1)
+                       flags |= RING_F_SP_ENQ;
+
+               APP_LOG(app, HIGH, "Initializing %s...", p->name);
+               app->swq[i] = rte_ring_create(
+                               p->name,
+                               p->size,
+                               p->cpu_socket_id,
+                               flags);
+
+               if (app->swq[i] == NULL)
+                       rte_panic("%s init error\n", p->name);
+       }
+}
+
+static void
+app_init_tm(struct app_params *app)
+{
+       uint32_t i;
+
+       for (i = 0; i < app->n_pktq_tm; i++) {
+               struct app_pktq_tm_params *p_tm = &app->tm_params[i];
+               struct app_link_params *p_link;
+               struct rte_eth_link link_eth_params;
+               struct rte_sched_port *sched;
+               uint32_t n_subports, subport_id;
+               int status;
+
+               p_link = app_get_link_for_tm(app, p_tm);
+               /* LINK */
+               rte_eth_link_get(p_link->pmd_id, &link_eth_params);
+
+               /* TM */
+               p_tm->sched_port_params.name = p_tm->name;
+               p_tm->sched_port_params.socket =
+                       app_get_cpu_socket_id(p_link->pmd_id);
+               p_tm->sched_port_params.rate =
+                       (uint64_t) link_eth_params.link_speed * 1000 * 1000 / 8;
+
+               APP_LOG(app, HIGH, "Initializing %s ...", p_tm->name);
+               sched = rte_sched_port_config(&p_tm->sched_port_params);
+               if (sched == NULL)
+                       rte_panic("%s init error\n", p_tm->name);
+               app->tm[i] = sched;
+
+               /* Subport */
+               n_subports = p_tm->sched_port_params.n_subports_per_port;
+               for (subport_id = 0; subport_id < n_subports; subport_id++) {
+                       uint32_t n_pipes_per_subport, pipe_id;
+
+                       status = rte_sched_subport_config(sched,
+                               subport_id,
+                               &p_tm->sched_subport_params[subport_id]);
+                       if (status)
+                               rte_panic("%s subport %" PRIu32
+                                       " init error (%" PRId32 ")\n",
+                                       p_tm->name, subport_id, status);
+
+                       /* Pipe */
+                       n_pipes_per_subport =
+                               p_tm->sched_port_params.n_pipes_per_subport;
+                       for (pipe_id = 0;
+                               pipe_id < n_pipes_per_subport;
+                               pipe_id++) {
+                               int profile_id = p_tm->sched_pipe_to_profile[
+                                       subport_id * APP_MAX_SCHED_PIPES +
+                                       pipe_id];
+
+                               if (profile_id == -1)
+                                       continue;
+
+                               status = rte_sched_pipe_config(sched,
+                                       subport_id,
+                                       pipe_id,
+                                       profile_id);
+                               if (status)
+                                       rte_panic("%s subport %" PRIu32
+                                               " pipe %" PRIu32
+                                               " (profile %" PRId32 ") "
+                                               "init error (% " PRId32 ")\n",
+                                               p_tm->name, subport_id, pipe_id,
+                                               profile_id, status);
+                       }
+               }
+       }
+}
+
+static void
+app_init_msgq(struct app_params *app)
+{
+       uint32_t i;
+
+       for (i = 0; i < app->n_msgq; i++) {
+               struct app_msgq_params *p = &app->msgq_params[i];
+
+               APP_LOG(app, HIGH, "Initializing %s ...", p->name);
+               app->msgq[i] = rte_ring_create(
+                               p->name,
+                               p->size,
+                               p->cpu_socket_id,
+                               RING_F_SP_ENQ | RING_F_SC_DEQ);
+
+               if (app->msgq[i] == NULL)
+                       rte_panic("%s init error\n", p->name);
+       }
+}
+
+static void app_pipeline_params_get(struct app_params *app,
+       struct app_pipeline_params *p_in,
+       struct pipeline_params *p_out)
+{
+       uint32_t i;
+       uint32_t mempool_id;
+
+       snprintf(p_out->name, PIPELINE_NAME_SIZE, "%s", p_in->name);
+
+       p_out->socket_id = (int) p_in->socket_id;
+
+       p_out->log_level = app->log_level;
+
+       /* pktq_in */
+       p_out->n_ports_in = p_in->n_pktq_in;
+       for (i = 0; i < p_in->n_pktq_in; i++) {
+               struct app_pktq_in_params *in = &p_in->pktq_in[i];
+               struct pipeline_port_in_params *out = &p_out->port_in[i];
+
+               switch (in->type) {
+               case APP_PKTQ_IN_HWQ:
+               {
+                       struct app_pktq_hwq_in_params *p_hwq_in =
+                               &app->hwq_in_params[in->id];
+                       struct app_link_params *p_link =
+                               app_get_link_for_rxq(app, p_hwq_in);
+                       uint32_t rxq_link_id, rxq_queue_id;
+
+                       int status =
+                       sscanf(p_hwq_in->name, "RXQ%" SCNu32 ".%" SCNu32,
+                               &rxq_link_id,
+                               &rxq_queue_id);
+                       if (status < 0)
+                               rte_panic("%s (%" PRId32 "): "
+                               "init error (%" PRId32 ")\n",
+                               p_hwq_in->name, rxq_link_id, status);
+
+                       out->type = PIPELINE_PORT_IN_ETHDEV_READER;
+                       out->params.ethdev.port_id = p_link->pmd_id;
+                       out->params.ethdev.queue_id = rxq_queue_id;
+                       out->burst_size = p_hwq_in->burst;
+                       break;
+               }
+               case APP_PKTQ_IN_SWQ:
+               {
+                       struct app_pktq_swq_params *swq_params =
+                               &app->swq_params[in->id];
+
+                       if ((swq_params->ipv4_frag == 0) &&
+                               (swq_params->ipv6_frag == 0)) {
+                               if (app_swq_get_readers(app,
+                                       swq_params) == 1) {
+                                       out->type =
+                                               PIPELINE_PORT_IN_RING_READER;
+                                       out->params.ring.ring =
+                                               app->swq[in->id];
+                                       out->burst_size =
+                                               app->swq_params[in->id].
+                                                       burst_read;
+                               } else {
+                               out->type = PIPELINE_PORT_IN_RING_MULTI_READER;
+                               out->params.ring_multi.ring = app->swq[in->id];
+                               out->burst_size = swq_params->burst_read;
+                               }
+                       } else {
+                               if (swq_params->ipv4_frag == 1) {
+                               struct rte_port_ring_reader_ipv4_frag_params
+                                       *params =
+                                               &out->params.ring_ipv4_frag;
+
+                               out->type =
+                                       PIPELINE_PORT_IN_RING_READER_IPV4_FRAG;
+                               params->ring = app->swq[in->id];
+                               params->mtu = swq_params->mtu;
+                               params->metadata_size =
+                                       swq_params->metadata_size;
+                               params->pool_direct =
+                                       app->mempool
+                                       [swq_params->mempool_direct_id];
+                               params->pool_indirect =
+                                       app->mempool
+                                       [swq_params->mempool_indirect_id];
+                               out->burst_size = swq_params->burst_read;
+                               } else {
+                               struct rte_port_ring_reader_ipv6_frag_params
+                                       *params =
+                                               &out->params.ring_ipv6_frag;
+
+                               out->type =
+                                       PIPELINE_PORT_IN_RING_READER_IPV6_FRAG;
+                               params->ring = app->swq[in->id];
+                               params->mtu = swq_params->mtu;
+                               params->metadata_size =
+                                       swq_params->metadata_size;
+                               params->pool_direct =
+                                       app->mempool
+                                       [swq_params->mempool_direct_id];
+                               params->pool_indirect =
+                                       app->mempool
+                                       [swq_params->mempool_indirect_id];
+                               out->burst_size = swq_params->burst_read;
+                               }
+                       }
+                       break;
+               }
+               case APP_PKTQ_IN_TM:
+                       out->type = PIPELINE_PORT_IN_SCHED_READER;
+                       out->params.sched.sched = app->tm[in->id];
+                       out->burst_size = app->tm_params[in->id].burst_read;
+                       break;
+               case APP_PKTQ_IN_SOURCE:
+                       mempool_id = app->source_params[in->id].mempool_id;
+                       out->type = PIPELINE_PORT_IN_SOURCE;
+                       out->params.source.mempool = app->mempool[mempool_id];
+                       out->burst_size = app->source_params[in->id].burst;
+
+#ifdef RTE_NEXT_ABI
+                       if (app->source_params[in->id].file_name
+                               != NULL) {
+                               out->params.source.file_name = strdup(
+                                       app->source_params[in->id].
+                                       file_name);
+                               if (out->params.source.file_name == NULL) {
+                                       out->params.source.
+                                               n_bytes_per_pkt = 0;
+                                       break;
+                               }
+                               out->params.source.n_bytes_per_pkt =
+                                       app->source_params[in->id].
+                                       n_bytes_per_pkt;
+                       }
+#endif
+
+                       break;
+               default:
+                       break;
+               }
+       }
+
+       /* pktq_out */
+       p_out->n_ports_out = p_in->n_pktq_out;
+       for (i = 0; i < p_in->n_pktq_out; i++) {
+               struct app_pktq_out_params *in = &p_in->pktq_out[i];
+               struct pipeline_port_out_params *out = &p_out->port_out[i];
+
+               switch (in->type) {
+               case APP_PKTQ_OUT_HWQ:
+               {
+                       struct app_pktq_hwq_out_params *p_hwq_out =
+                               &app->hwq_out_params[in->id];
+                       struct app_link_params *p_link =
+                               app_get_link_for_txq(app, p_hwq_out);
+                       uint32_t txq_link_id, txq_queue_id;
+
+                       int status =
+                       sscanf(p_hwq_out->name,
+                               "TXQ%" SCNu32 ".%" SCNu32,
+                               &txq_link_id,
+                               &txq_queue_id);
+                       if (status < 0)
+                               rte_panic("%s (%" PRId32 "): "
+                               "init error (%" PRId32 ")\n",
+                               p_hwq_out->name, txq_link_id, status);
+
+                       if (p_hwq_out->dropless == 0) {
+                               struct rte_port_ethdev_writer_params *params =
+                                       &out->params.ethdev;
+
+                               out->type = PIPELINE_PORT_OUT_ETHDEV_WRITER;
+                               params->port_id = p_link->pmd_id;
+                               params->queue_id = txq_queue_id;
+                               params->tx_burst_sz =
+                                       app->hwq_out_params[in->id].burst;
+                       } else {
+                               struct rte_port_ethdev_writer_nodrop_params
+                                       *params = &out->params.ethdev_nodrop;
+
+                               out->type =
+                                       PIPELINE_PORT_OUT_ETHDEV_WRITER_NODROP;
+                               params->port_id = p_link->pmd_id;
+                               params->queue_id = txq_queue_id;
+                               params->tx_burst_sz = p_hwq_out->burst;
+                               params->n_retries = p_hwq_out->n_retries;
+                       }
+                       break;
+               }
+               case APP_PKTQ_OUT_SWQ:
+               {
+               struct app_pktq_swq_params *swq_params =
+                       &app->swq_params[in->id];
+
+               if ((swq_params->ipv4_ras == 0) &&
+                       (swq_params->ipv6_ras == 0)) {
+                       if (app_swq_get_writers(app, swq_params) == 1) {
+                               if (app->swq_params[in->id].dropless == 0) {
+                                       struct rte_port_ring_writer_params
+                                               *params = &out->params.ring;
+
+                               out->type = PIPELINE_PORT_OUT_RING_WRITER;
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz =
+                                       app->swq_params[in->id].burst_write;
+                               } else {
+                               struct rte_port_ring_writer_nodrop_params
+                                       *params = &out->params.ring_nodrop;
+
+                               out->type =
+                                       PIPELINE_PORT_OUT_RING_WRITER_NODROP;
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz =
+                                       app->swq_params[in->id].burst_write;
+                               params->n_retries =
+                               app->swq_params[in->id].n_retries;
+                               }
+                       } else {
+                               if (swq_params->dropless == 0) {
+                                       struct rte_port_ring_multi_writer_params
+                                               *params =
+                                               &out->params.ring_multi;
+
+                               out->type =
+                                       PIPELINE_PORT_OUT_RING_MULTI_WRITER;
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz = swq_params->burst_write;
+                               } else {
+                               struct rte_port_ring_multi_writer_nodrop_params
+                                       *params =
+                                               &out->params.ring_multi_nodrop;
+
+                               out->type =
+                               PIPELINE_PORT_OUT_RING_MULTI_WRITER_NODROP;
+
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz = swq_params->burst_write;
+                               params->n_retries = swq_params->n_retries;
+                               }
+                               }
+                       } else {
+                       if (swq_params->ipv4_ras == 1) {
+                               struct rte_port_ring_writer_ipv4_ras_params
+                                       *params =
+                                               &out->params.ring_ipv4_ras;
+
+                               out->type =
+                                       PIPELINE_PORT_OUT_RING_WRITER_IPV4_RAS;
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz = swq_params->burst_write;
+                       } else {
+                               struct rte_port_ring_writer_ipv6_ras_params
+                                       *params =
+                                               &out->params.ring_ipv6_ras;
+
+                               out->type =
+                                       PIPELINE_PORT_OUT_RING_WRITER_IPV6_RAS;
+                               params->ring = app->swq[in->id];
+                               params->tx_burst_sz = swq_params->burst_write;
+                       }
+                       }
+                       break;
+               }
+               case APP_PKTQ_OUT_TM: {
+                       struct rte_port_sched_writer_params *params =
+                               &out->params.sched;
+
+                       out->type = PIPELINE_PORT_OUT_SCHED_WRITER;
+                       params->sched = app->tm[in->id];
+                       params->tx_burst_sz =
+                               app->tm_params[in->id].burst_write;
+                       break;
+               }
+               case APP_PKTQ_OUT_SINK:
+                       out->type = PIPELINE_PORT_OUT_SINK;
+                       if (app->sink_params[in->id].file_name != NULL) {
+                               out->params.sink.file_name = strdup(
+                                       app->sink_params[in->id].
+                                       file_name);
+                               if (out->params.sink.file_name == NULL) {
+                                       out->params.sink.max_n_pkts = 0;
+                                       break;
+                               }
+                               out->params.sink.max_n_pkts =
+                                       app->sink_params[in->id].
+                                       n_pkts_to_dump;
+                       } else {
+                               out->params.sink.file_name = NULL;
+                               out->params.sink.max_n_pkts = 0;
+                       }
+                       break;
+               default:
+                       break;
+               }
+       }
+
+       /* msgq */
+       p_out->n_msgq = p_in->n_msgq_in;
+
+       for (i = 0; i < p_in->n_msgq_in; i++)
+               p_out->msgq_in[i] = app->msgq[p_in->msgq_in[i]];
+
+       for (i = 0; i < p_in->n_msgq_out; i++)
+               p_out->msgq_out[i] = app->msgq[p_in->msgq_out[i]];
+
+       /* args */
+       p_out->n_args = p_in->n_args;
+       for (i = 0; i < p_in->n_args; i++) {
+               p_out->args_name[i] = p_in->args_name[i];
+               p_out->args_value[i] = p_in->args_value[i];
+       }
+}
+
+static void
+app_init_pipelines(struct app_params *app)
+{
+       uint32_t p_id;
+
+       for (p_id = 0; p_id < app->n_pipelines; p_id++) {
+               struct app_pipeline_params *params =
+                       &app->pipeline_params[p_id];
+               struct app_pipeline_data *data = &app->pipeline_data[p_id];
+               struct pipeline_type *ptype;
+               struct pipeline_params pp;
+
+               APP_LOG(app, HIGH, "Initializing %s ...", params->name);
+
+               ptype = app_pipeline_type_find(app, params->type);
+               if (ptype == NULL)
+                       rte_panic("Init error: Unknown pipeline type \"%s\"\n",
+                               params->type);
+
+               app_pipeline_params_get(app, params, &pp);
+
+               /* Back-end */
+               data->be = NULL;
+               if (ptype->be_ops->f_init) {
+                       data->be = ptype->be_ops->f_init(&pp, (void *) app);
+
+                       if (data->be == NULL)
+                               rte_panic("Pipeline instance \"%s\" back-end "
+                                       "init error\n", params->name);
+               }
+
+               /* Front-end */
+               data->fe = NULL;
+               if (ptype->fe_ops->f_init) {
+                       data->fe = ptype->fe_ops->f_init(&pp, (void *) app);
+
+                       if (data->fe == NULL)
+                               rte_panic("Pipeline instance \"%s\" front-end "
+                               "init error\n", params->name);
+               }
+
+               data->ptype = ptype;
+
+               data->timer_period = (rte_get_tsc_hz() *
+                       params->timer_period) / 100;
+       }
+}
+
+static void
+app_init_threads(struct app_params *app)
+{
+       uint64_t time = rte_get_tsc_cycles();
+       uint32_t p_id;
+
+       for (p_id = 0; p_id < app->n_pipelines; p_id++) {
+               struct app_pipeline_params *params =
+                       &app->pipeline_params[p_id];
+               struct app_pipeline_data *data = &app->pipeline_data[p_id];
+               struct pipeline_type *ptype;
+               struct app_thread_data *t;
+               struct app_thread_pipeline_data *p;
+               int lcore_id;
+
+               lcore_id = cpu_core_map_get_lcore_id(app->core_map,
+                       params->socket_id,
+                       params->core_id,
+                       params->hyper_th_id);
+
+               if (lcore_id < 0)
+                       rte_panic("Invalid core s%" PRIu32 "c%" PRIu32 "%s\n",
+                               params->socket_id,
+                               params->core_id,
+                               (params->hyper_th_id) ? "h" : "");
+
+               t = &app->thread_data[lcore_id];
+
+               t->timer_period = (rte_get_tsc_hz() *
+                       APP_THREAD_TIMER_PERIOD) / DIV_CONV_HZ_SEC;
+               t->thread_req_deadline = time + t->timer_period;
+
+               t->headroom_cycles = 0;
+               t->headroom_time = rte_get_tsc_cycles();
+               t->headroom_ratio = 0.0;
+
+               t->msgq_in = app_thread_msgq_in_get(app,
+                               params->socket_id,
+                               params->core_id,
+                               params->hyper_th_id);
+               if (t->msgq_in == NULL)
+                       rte_panic("Init error: Cannot find MSGQ_IN "
+                               "for thread %" PRId32, lcore_id);
+
+               t->msgq_out = app_thread_msgq_out_get(app,
+                               params->socket_id,
+                               params->core_id,
+                               params->hyper_th_id);
+               if (t->msgq_out == NULL)
+                       rte_panic("Init error: Cannot find MSGQ_OUT "
+                               "for thread %" PRId32, lcore_id);
+
+               ptype = app_pipeline_type_find(app, params->type);
+               if (ptype == NULL)
+                       rte_panic("Init error: Unknown pipeline "
+                               "type \"%s\"\n", params->type);
+
+               p = (ptype->be_ops->f_run == NULL) ?
+                       &t->regular[t->n_regular] :
+                       &t->custom[t->n_custom];
+
+               p->pipeline_id = p_id;
+               p->be = data->be;
+               p->f_run = ptype->be_ops->f_run;
+               p->f_timer = ptype->be_ops->f_timer;
+               p->timer_period = data->timer_period;
+               p->deadline = time + data->timer_period;
+
+               data->enabled = 1;
+
+               if (ptype->be_ops->f_run == NULL)
+                       t->n_regular++;
+               else
+                       t->n_custom++;
+       }
+}
+
+int app_init(struct app_params *app)
+{
+       app_init_core_map(app);
+       app_init_core_mask(app);
+
+       app_init_eal(app);
+       ifm_init();
+       /*app_init_mempool(app);*/
+       app_init_link(app);
+       app_init_swq(app);
+       app_init_tm(app);
+       app_init_msgq(app);
+
+       app_pipeline_common_cmd_push(app);
+       app_pipeline_thread_cmd_push(app);
+       app_pipeline_type_register(app, &pipeline_master);
+       app_pipeline_type_register(app, &pipeline_passthrough);
+       app_pipeline_type_register(app, &pipeline_vfw);
+       app_pipeline_type_register(app, &pipeline_loadb);
+       app_pipeline_type_register(app, &pipeline_txrx);
+       app_pipeline_type_register(app, &pipeline_arpicmp);
+
+       app_init_pipelines(app);
+       app_init_threads(app);
+
+       l3fwd_init();
+       create_arp_table();
+       create_nd_table();
+       populate_lpm_routes();
+       print_interface_details();
+
+       return 0;
+}
+
+static int
+app_pipeline_type_cmd_push(struct app_params *app,
+       struct pipeline_type *ptype)
+{
+       cmdline_parse_ctx_t *cmds;
+       uint32_t n_cmds, i;
+
+       /* Check input arguments */
+       if ((app == NULL) ||
+               (ptype == NULL))
+               return -EINVAL;
+
+       n_cmds = pipeline_type_cmds_count(ptype);
+       if (n_cmds == 0)
+               return 0;
+
+       cmds = ptype->fe_ops->cmds;
+
+       /* Check for available slots in the application commands array */
+       if (n_cmds > APP_MAX_CMDS - app->n_cmds)
+               return -ENOMEM;
+
+       /* Push pipeline commands into the application */
+       memcpy(&app->cmds[app->n_cmds],
+               cmds,
+               n_cmds * sizeof(cmdline_parse_ctx_t));
+
+       for (i = 0; i < n_cmds; i++)
+               app->cmds[app->n_cmds + i]->data = app;
+
+       app->n_cmds += n_cmds;
+       app->cmds[app->n_cmds] = NULL;
+
+       return 0;
+}
+
+int
+app_pipeline_type_register(struct app_params *app, struct pipeline_type *ptype)
+{
+       uint32_t n_cmds, i;
+
+       /* Check input arguments */
+       if ((app == NULL) ||
+               (ptype == NULL) ||
+               (ptype->name == NULL) ||
+               (strlen(ptype->name) == 0) ||
+               (ptype->be_ops->f_init == NULL) ||
+               (ptype->be_ops->f_timer == NULL))
+               return -EINVAL;
+
+       /* Check for duplicate entry */
+       for (i = 0; i < app->n_pipeline_types; i++)
+               if (strcmp(app->pipeline_type[i].name, ptype->name) == 0)
+                       return -EEXIST;
+
+       /* Check for resource availability */
+       n_cmds = pipeline_type_cmds_count(ptype);
+       if ((app->n_pipeline_types == APP_MAX_PIPELINE_TYPES) ||
+               (n_cmds > APP_MAX_CMDS - app->n_cmds))
+               return -ENOMEM;
+
+       /* Copy pipeline type */
+       memcpy(&app->pipeline_type[app->n_pipeline_types++],
+               ptype,
+               sizeof(struct pipeline_type));
+
+       /* Copy CLI commands */
+       if (n_cmds)
+               app_pipeline_type_cmd_push(app, ptype);
+
+       return 0;
+}
+
+struct
+pipeline_type *app_pipeline_type_find(struct app_params *app, char *name)
+{
+       uint32_t i;
+
+       for (i = 0; i < app->n_pipeline_types; i++)
+               if (strcmp(app->pipeline_type[i].name, name) == 0)
+                       return &app->pipeline_type[i];
+
+       return NULL;
+}
diff --git a/VNFs/vFW/main.c b/VNFs/vFW/main.c
new file mode 100644 (file)
index 0000000..9ebf6fc
--- /dev/null
@@ -0,0 +1,50 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+#include "app.h"
+
+static struct app_params app;
+
+int
+main(int argc, char **argv)
+{
+       rte_openlog_stream(stderr);
+
+       /* Config */
+       app_config_init(&app);
+
+       app_config_args(&app, argc, argv);
+
+       app_config_preproc(&app);
+
+       app_config_parse(&app, app.parser_file);
+
+       app_config_check(&app);
+
+       /* Timer subsystem init*/
+       rte_timer_subsystem_init();
+
+       /* Init */
+       app_init(&app);
+
+       /* Run-time */
+       rte_eal_mp_remote_launch(
+               app_thread,
+               (void *) &app,
+               CALL_MASTER);
+
+       return 0;
+}
diff --git a/VNFs/vFW/pipeline/pipeline_vfw.c b/VNFs/vFW/pipeline/pipeline_vfw.c
new file mode 100644 (file)
index 0000000..934e442
--- /dev/null
@@ -0,0 +1,4684 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+/**
+ * @file
+ * Pipeline VFW FE Implementation.
+ *
+ * Implementation of the Pipeline VFW Front End (FE).
+ * Runs on the Master pipeline, responsible for CLI commands.
+ *
+ */
+
+#include <stdio.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/queue.h>
+#include <netinet/in.h>
+
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_malloc.h>
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_num.h>
+#include <cmdline_parse_string.h>
+#include <cmdline_parse_ipaddr.h>
+#include <cmdline_parse_etheraddr.h>
+#include <cmdline_socket.h>
+#include <cmdline.h>
+#include <rte_table_acl.h>
+
+#include "app.h"
+#include "pipeline_common_fe.h"
+#include "pipeline_vfw.h"
+#include "pipeline_vfw_be.h"
+#include "rte_cnxn_tracking.h"
+
+/**
+ * A structure defining the VFW rule for the TAILQ Tables.
+ */
+struct app_pipeline_vfw_rule {
+       struct pipeline_vfw_key key;
+       int32_t priority;
+       uint32_t port_id;
+       uint32_t action_id;
+       uint32_t command;
+       void *entry_ptr;
+
+        TAILQ_ENTRY(app_pipeline_vfw_rule) node;
+};
+
+/**
+ * A structure defining the VFW pipeline front end data.
+ */
+struct app_pipeline_vfw {
+       /* parameters */
+       uint32_t n_ports_in;
+       uint32_t n_ports_out;
+};
+
+/*
+ * Define a structure to calculate performance measurements for VFW.
+ * VFW continually updates counters for total number of packets
+ * processed, and total number of bytes processed. Each VFW backend thread
+ * i.e.the packet processing instances updates their own copy of these counters.
+ * An optional, 1 second periodic timer fires on the master core, which combines
+ * those numbers to perform byte and packet per second calculations, without
+ * burdening the packet processors.
+ */
+#define RTE_VFW_PERF_MSR_BUFF_SIZE 8       /* must be power of 2 */
+#define RTE_VFW_PERF_MSR_BUFF_SIZE_MASK (RTE_VFW_PERF_MSR_BUFF_SIZE - 1)
+
+/**
+ * A structure defining the VFW performance measurements.
+ */
+struct rte_vfw_performance_measures_t {
+       /* two circular buffers */
+       uint64_t total_packets[RTE_VFW_PERF_MSR_BUFF_SIZE];
+       uint64_t total_bytes[RTE_VFW_PERF_MSR_BUFF_SIZE];
+       uint32_t bytes_last_second;
+       uint32_t ave_bytes_per_second;
+       uint32_t pkts_last_second;
+       uint32_t ave_pkts_per_second;
+       /* times data has been (over-)written into buffers */
+       uint64_t total_entries;
+       uint8_t current_index;       /* for circular buffers */
+};
+
+struct rte_vfw_performance_measures_t rte_vfw_performance_measures;
+
+/*
+ * Active and Standby Tables
+ * Active and standby tables exist to allow modifying VFW rules and
+ * actions and having no impact on the packet processing running on
+ * the multiple VFW threads/pipelines. The packet processing does a
+ * lookup on the active tables. Each VFW thread/pipeline runs on
+ * a separate core (i.e. 2,3,4, etc).
+ *
+ * All CLI actions run on the VFW Front End (FE) code on Core 0.
+ * All changes, adding/delete rules and action occurs on the standby tables.
+ * In activate the changes in the standby table, the CLI command is entered:
+ * p vfw applyruleset
+ *
+ * The standby tables become active. The active table becomes the standby.
+ * The new standby table gets updated with the changes that were done.
+ *
+ * Table descriptions:
+ * VFW Rule Tables TAILQ - 2 global tables active/standby per ipv4,ipv6
+ * The TAILQ tables are required for the LS CLI command and in order
+ * to do a lookup using a rule when adding or deleting a rule.
+ * The VFW TRIE tables in DPDK do not allow this type of listing or lookup.
+ *
+ * VFW Rule Tables TRIE - 2 global tables active/standby per ipv4, ipv6
+ * The TRIE tables are the tables used during packet processing.
+ * A bulk lookup can be performed by passing in a burst of packets.
+ * Unfortunately, the current implementation of the TRIE tables does
+ * not allow lookup using a rule. Hence the need for the TAILQ tables.
+ *
+ * VFW Action Tables ARRAY - 2 global tables active/standby
+ * The action tables stores the VFW actions.
+ * Every rule has an action id which defines what action to take
+ * when a packet matching that rule is received.
+ * Actions: accept, drop, fwd, count, nat, dscp, conntrack
+ *
+ * Command Table TAILQ - 1 table
+ * After the active and standby tables are swithover, the new standby
+ * table needs to be updated with all the changes that were done.
+ * This table stores all the add and delete commands and updates
+ * the new standby table when the applyruleset command executes.
+ *
+ * The active and standby tables can be displayed individually:
+ * p vfw ls 0    <== active VFW rules
+ * p vfw ls 1    <== standby VFW rules
+ * p action ls 0 <== active VFW actions
+ * p action ls 1 <== standby VFW actions
+ */
+
+/* Only create global VFW tables once */
+int vfw_rule_table_created;
+
+/*
+ * VFW Rule Tables TAILQ - see description above
+ * Two tables/counters are required for active and standby.
+ * The A and B tables/counters are the actual instances.
+ * The pointers are set to point to these tables/counters.
+ * The pointers are updated during the switchover for the applyruleset.
+ */
+
+/* Definition of the the TAILQ table */
+TAILQ_HEAD(app_pipeline_vfw_rule_type, app_pipeline_vfw_rule);
+/* Instances of tables and counters */
+struct app_pipeline_vfw_rule_type vfw_tailq_rules_ipv4a;
+struct app_pipeline_vfw_rule_type vfw_tailq_rules_ipv4b;
+struct app_pipeline_vfw_rule_type vfw_tailq_rules_ipv6a;
+struct app_pipeline_vfw_rule_type vfw_tailq_rules_ipv6b;
+uint32_t vfw_n_tailq_rules_ipv4a;
+uint32_t vfw_n_tailq_rules_ipv6a;
+uint32_t vfw_n_tailq_rules_ipv4b;
+uint32_t vfw_n_tailq_rules_ipv6b;
+/* Pointers to tables and counters for switchover in applyruleset */
+struct app_pipeline_vfw_rule_type *vfw_tailq_rules_ipv4_active;
+struct app_pipeline_vfw_rule_type *vfw_tailq_rules_ipv4_standby;
+struct app_pipeline_vfw_rule_type *vfw_tailq_rules_ipv6_active;
+struct app_pipeline_vfw_rule_type *vfw_tailq_rules_ipv6_standby;
+struct app_pipeline_vfw_rule_type *vfw_tailq_rules_temp_ptr;
+uint32_t *vfw_n_tailq_rules_ipv4_active;
+uint32_t *vfw_n_tailq_rules_ipv4_standby;
+uint32_t *vfw_n_tailq_rules_ipv6_active;
+uint32_t *vfw_n_tailq_rules_ipv6_standby;
+
+/* VFW commands to update new standby tables after switchover */
+TAILQ_HEAD(, app_pipeline_vfw_rule) vfw_commands;
+
+/* VFW IPV4 and IPV6 enable flags for debugging (Default both on) */
+int vfw_ipv4_enabled = 1;
+int vfw_ipv6_enabled = 1;
+
+/* Number of VFW Rules, default 4 * 1024 */
+uint32_t vfw_n_rules = 4 * 1024;
+/* VFW Rule Table TRIE - 2 (Active, Standby) Global table per ipv4, ipv6 */
+void *vfw_rule_table_ipv4_active;
+void *vfw_rule_table_ipv4_standby;
+void *vfw_rule_table_ipv6_active;
+void *vfw_rule_table_ipv6_standby;
+
+/**
+ * Reset running averages for performance measurements.
+ *
+ */
+static void rte_vfw_reset_running_averages(void)
+{
+       memset(&rte_vfw_performance_measures, 0,
+              sizeof(rte_vfw_performance_measures));
+};
+
+/**
+ * Compute performance calculations on master to reduce computing on
+ * packet processor.
+ *
+ * @param total_bytes
+ *  Total bytes processed during this interval.
+ * @param total_packets
+ *  Total packets processed during this interval.
+ *
+ */
+static void rte_vfw_update_performance_measures(uint64_t total_bytes,
+                                             uint64_t total_packets)
+{
+       /* make readable */
+       struct rte_vfw_performance_measures_t *pm =
+           &rte_vfw_performance_measures;
+
+       if (unlikely(pm->total_entries == 0 && total_packets == 0))
+              /* the timer is running, but no traffic started yet,
+               * so do nothing */
+              return;
+
+       if (likely(pm->total_entries > 0)) {
+              uint8_t oldest_index;
+              uint8_t divisor;
+
+              pm->bytes_last_second =
+                  total_bytes - pm->total_bytes[pm->current_index];
+              pm->pkts_last_second =
+                  total_packets - pm->total_packets[pm->current_index];
+
+              /* if total_entries zero, current_index must remain as zero */
+              pm->current_index =
+                  (pm->current_index +
+                   1) & RTE_VFW_PERF_MSR_BUFF_SIZE_MASK;
+
+              if (unlikely
+                  (pm->total_entries <= RTE_VFW_PERF_MSR_BUFF_SIZE)) {
+                     /* oldest value is at element 0 */
+                     oldest_index = 0;
+                     divisor = pm->total_entries;
+                     /* note, prior to incrementing total_entries */
+              } else {
+                     /* oldest value is at element about to be overwritten */
+                     oldest_index = pm->current_index;
+                     divisor = RTE_VFW_PERF_MSR_BUFF_SIZE;
+              }
+
+              pm->ave_bytes_per_second =
+                  (total_bytes - pm->total_bytes[oldest_index]) / divisor;
+              pm->ave_pkts_per_second =
+                  (total_packets - pm->total_packets[oldest_index]) / divisor;
+       }
+
+       pm->total_bytes[pm->current_index] = total_bytes;
+       pm->total_packets[pm->current_index] = total_packets;
+       pm->total_entries++;
+}
+
+/**
+ * Combine data from all vfw+connection tracking instances.
+ * Calculate various statistics. Dump to console.
+ *
+ */
+static void rte_vfw_sum_and_print_counters(void)
+{
+       int i;
+       struct rte_VFW_counter_block vfw_counter_sums;
+       struct rte_CT_counter_block ct_counter_sums;
+       /* For ct instance with this fw instance */
+       struct rte_CT_counter_block *ct_counters;
+
+       memset(&vfw_counter_sums, 0, sizeof(vfw_counter_sums));
+       memset(&ct_counter_sums, 0, sizeof(ct_counter_sums));
+
+       for (i = 0; i <= rte_VFW_hi_counter_block_in_use; i++) {
+              struct rte_VFW_counter_block *vfw_ctrs =
+                  &rte_vfw_counter_table[i];
+              ct_counters = rte_vfw_counter_table[i].ct_counters;
+
+              uint64_t average_internal_time =
+                  vfw_ctrs->time_measurements ==
+                  0 ? 0 : vfw_ctrs->internal_time_sum /
+                  vfw_ctrs->time_measurements;
+              uint64_t average_external_time =
+                  vfw_ctrs->time_measurements ==
+                  0 ? 0 : vfw_ctrs->external_time_sum /
+                  vfw_ctrs->time_measurements;
+              uint64_t average_pkts_in_batch =
+                  vfw_ctrs->num_pkts_measurements ==
+                  0 ? 0 : vfw_ctrs->num_batch_pkts_sum /
+                  vfw_ctrs->num_pkts_measurements;
+
+              printf("{\"VFW counters\" : {\"id\" : \"%s\",\"packets_processed\" : %"
+                     PRIu64 ", \"bytes_processed\" : %"
+                     PRIu64 ", \"average_pkts_in_batch\" : %"
+                     PRIu64 ", \"average_internal_time_in_clocks\" : %"
+                     PRIu64 ", \"average_external_time_in_clocks\" : %"
+                     PRIu64 ", \"total_time_measures\" : %"
+                     PRIu32 ", \"ct_packets_forwarded\" : %"
+                     PRIu64 ", \"ct_packets_dropped\" : %"
+                     PRIu64 "}}\n",
+                     vfw_ctrs->name,
+                     vfw_ctrs->pkts_received,
+                     vfw_ctrs->bytes_processed,
+                     average_pkts_in_batch,
+                     average_internal_time,
+                     average_external_time,
+                     vfw_ctrs->time_measurements,
+                     ct_counters->pkts_forwarded, ct_counters->pkts_drop);
+
+              /* sum VFW counters */
+              vfw_counter_sums.bytes_processed +=
+                  vfw_ctrs->bytes_processed;
+              vfw_counter_sums.pkts_drop_without_rule +=
+                  vfw_ctrs->pkts_drop_without_rule;
+              vfw_counter_sums.pkts_received += vfw_ctrs->pkts_received;
+              vfw_counter_sums.pkts_drop_ttl += vfw_ctrs->pkts_drop_ttl;
+              vfw_counter_sums.pkts_drop_bad_size +=
+                  vfw_ctrs->pkts_drop_bad_size;
+              vfw_counter_sums.pkts_drop_fragmented +=
+                  vfw_ctrs->pkts_drop_fragmented;
+              vfw_counter_sums.pkts_drop_without_arp_entry +=
+                  vfw_ctrs->pkts_drop_without_arp_entry;
+              vfw_counter_sums.sum_latencies += vfw_ctrs->sum_latencies;
+              vfw_counter_sums.count_latencies +=
+                  vfw_ctrs->count_latencies;
+
+              vfw_counter_sums.internal_time_sum +=
+                  vfw_ctrs->internal_time_sum;
+              vfw_counter_sums.external_time_sum +=
+                  vfw_ctrs->external_time_sum;
+              vfw_counter_sums.time_measurements +=
+                  vfw_ctrs->time_measurements;
+              vfw_counter_sums.pkts_drop_unsupported_type +=
+                  vfw_ctrs->pkts_drop_unsupported_type;
+
+              /* sum cnxn tracking counters */
+              ct_counter_sums.current_active_sessions +=
+                  ct_counters->current_active_sessions;
+              ct_counter_sums.sessions_activated +=
+                  ct_counters->sessions_activated;
+              ct_counter_sums.sessions_reactivated +=
+                  ct_counters->sessions_reactivated;
+              ct_counter_sums.sessions_established +=
+                  ct_counters->sessions_established;
+              ct_counter_sums.sessions_closed += ct_counters->sessions_closed;
+              ct_counter_sums.sessions_timedout +=
+                  ct_counters->sessions_timedout;
+              ct_counter_sums.pkts_forwarded += ct_counters->pkts_forwarded;
+              ct_counter_sums.pkts_drop += ct_counters->pkts_drop;
+              ct_counter_sums.pkts_drop_invalid_conn +=
+                  ct_counters->pkts_drop_invalid_conn;
+              ct_counter_sums.pkts_drop_invalid_state +=
+                  ct_counters->pkts_drop_invalid_state;
+              ct_counter_sums.pkts_drop_invalid_rst +=
+                  ct_counters->pkts_drop_invalid_rst;
+              ct_counter_sums.pkts_drop_outof_window +=
+                  ct_counters->pkts_drop_outof_window;
+       }
+
+       rte_vfw_update_performance_measures(vfw_counter_sums.
+                                          bytes_processed,
+                                          vfw_counter_sums.
+                                          pkts_received);
+       uint64_t average_latency =
+           vfw_counter_sums.count_latencies ==
+           0 ? 0 : vfw_counter_sums.sum_latencies /
+           vfw_counter_sums.count_latencies;
+
+       printf("{\"VFW sum counters\" : {"
+              "\"packets_last_sec\" : %"
+              PRIu32 ", \"average_packets_per_sec\" : %"
+              PRIu32 ", \"bytes_last_sec\" : %"
+              PRIu32 ", \"average_bytes_per_sec\" : %"
+              PRIu32 ", \"pkts_received\" : %"
+              PRIu64 ", \"bytes_processed\" : %"
+              PRIu64 ", \"average_latency_in_clocks\" : %"
+              PRIu64 ", \"ct_packets_forwarded\" : %"
+              PRIu64 ", \"ct_packets_dropped\" : %"
+              PRIu64 ", \"drops\" : {"
+              "\"TTL_zero\" : %" PRIu64 ", \"bad_size\" : %"
+              PRIu64 ", \"fragmented_packet\" : %"
+              PRIu64 ", \"unsupported_packet_types\" : %"
+              PRIu64 ", \"no_arp_entry\" : %"
+              PRIu64 "}, \"ct_sessions\" : {"
+              "\"active\" : %" PRIu64 ", \"open\" : %"
+              PRIu64 ", \"re-open_attempt\" : %"
+              PRIu64 ", \"established\" : %"
+              PRIu64 ", \"closed\" : %"
+              PRIu64 ", \"timeout\" : %"
+              PRIu64 "}, \"ct_drops\" : {"
+              "\"out_of_window\" : %" PRIu64 ", \"invalid_conn\" : %"
+              PRIu64 ", \"invalid_state_transition\" : %"
+              PRIu64 " \"RST\" : %"
+              PRIu64 "}}}\n",
+              rte_vfw_performance_measures.pkts_last_second,
+              rte_vfw_performance_measures.ave_pkts_per_second,
+              rte_vfw_performance_measures.bytes_last_second,
+              rte_vfw_performance_measures.ave_bytes_per_second,
+              vfw_counter_sums.pkts_received,
+              vfw_counter_sums.bytes_processed,
+              average_latency,
+              ct_counter_sums.pkts_forwarded,
+              ct_counter_sums.pkts_drop,
+              vfw_counter_sums.pkts_drop_ttl,
+              vfw_counter_sums.pkts_drop_bad_size,
+              vfw_counter_sums.pkts_drop_fragmented,
+              vfw_counter_sums.pkts_drop_unsupported_type,
+              vfw_counter_sums.pkts_drop_without_arp_entry,
+              ct_counter_sums.current_active_sessions,
+              ct_counter_sums.sessions_activated,
+              ct_counter_sums.sessions_reactivated,
+              ct_counter_sums.sessions_established,
+              ct_counter_sums.sessions_closed,
+              ct_counter_sums.sessions_timedout,
+              ct_counter_sums.pkts_drop_outof_window,
+              ct_counter_sums.pkts_drop_invalid_conn,
+              ct_counter_sums.pkts_drop_invalid_state,
+              ct_counter_sums.pkts_drop_invalid_rst);
+
+}
+
+/**
+ * Callback routine for 1 second, periodic timer.
+ *
+ * @param rt
+ *  A pointer to the rte_timer.
+ * @param arg
+ *  A pointer to application specific arguments (not used).
+ *
+ * @return
+ *  0 on success and port_id is filled, negative on error.
+ */
+static void rte_dump_vfw_counters_from_master(
+              __rte_unused struct rte_timer *rt, __rte_unused void *arg)
+{
+       rte_vfw_sum_and_print_counters();
+}
+
+int rte_vfw_hertz_computed;       /* only launch timer once */
+uint64_t rte_vfw_ticks_in_one_second;
+/* TODO: is processor hertz computed/stored elsewhere? */
+struct rte_timer rte_vfw_one_second_timer = RTE_TIMER_INITIALIZER;
+
+/**
+ * Print IPv4 Rule.
+ *
+ * @param rule
+ *  A pointer to the rule.
+ *
+ */
+static void print_vfw_ipv4_rule(struct app_pipeline_vfw_rule *rule)
+{
+       printf("Prio = %" PRId32 " (SA = %" PRIu32 ".%" PRIu32
+              ".%" PRIu32 ".%" PRIu32 "/%" PRIu32 ", DA = %"
+              PRIu32 ".%" PRIu32
+              ".%" PRIu32 ".%" PRIu32 "/%" PRIu32 ", SP = %"
+              PRIu32 "-%" PRIu32 ", DP = %"
+              PRIu32 "-%" PRIu32 ", Proto = %"
+              PRIu32 " / 0x%" PRIx32 ") => Action ID = %"
+              PRIu32 " (entry ptr = %p)\n",
+              rule->priority,
+              (rule->key.key.ipv4_5tuple.src_ip >> 24) & 0xFF,
+              (rule->key.key.ipv4_5tuple.src_ip >> 16) & 0xFF,
+              (rule->key.key.ipv4_5tuple.src_ip >> 8) & 0xFF,
+              rule->key.key.ipv4_5tuple.src_ip & 0xFF,
+              rule->key.key.ipv4_5tuple.src_ip_mask,
+              (rule->key.key.ipv4_5tuple.dst_ip >> 24) & 0xFF,
+              (rule->key.key.ipv4_5tuple.dst_ip >> 16) & 0xFF,
+              (rule->key.key.ipv4_5tuple.dst_ip >> 8) & 0xFF,
+              rule->key.key.ipv4_5tuple.dst_ip & 0xFF,
+              rule->key.key.ipv4_5tuple.dst_ip_mask,
+              rule->key.key.ipv4_5tuple.src_port_from,
+              rule->key.key.ipv4_5tuple.src_port_to,
+              rule->key.key.ipv4_5tuple.dst_port_from,
+              rule->key.key.ipv4_5tuple.dst_port_to,
+              rule->key.key.ipv4_5tuple.proto,
+              rule->key.key.ipv4_5tuple.proto_mask,
+              rule->action_id, rule->entry_ptr);
+}
+
+/**
+ * Print IPv6 Rule.
+ *
+ * @param rule
+ *  A pointer to the rule.
+ *
+ */
+static void print_vfw_ipv6_rule(struct app_pipeline_vfw_rule *rule)
+{
+       printf("Prio = %" PRId32 " (SA = %02" PRIx8 "%02" PRIx8
+              ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8
+              ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8
+              ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8
+              ":%02" PRIx8 "%02" PRIx8 "/" "%" PRIu32 ", DA = %02"
+              PRIx8 "%02" PRIx8 ":%02" PRIx8
+              "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8
+              "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8
+              "%02" PRIx8 ":%02" PRIx8 "%02" PRIx8 ":%02" PRIx8
+              "%02" PRIx8 "/" "%" PRIu32", " "SP = %" PRIu32 "-%" PRIu32
+              ", DP = %" PRIu32 "-%" PRIu32 ", Proto = %"
+              PRIu32 " / 0x%" PRIx32 ") => Action ID = %"
+              PRIu32 " (entry ptr = %p)\n", rule->priority,
+              (rule->key.key.ipv6_5tuple.src_ip[0]),
+              (rule->key.key.ipv6_5tuple.src_ip[1]),
+              (rule->key.key.ipv6_5tuple.src_ip[2]),
+              (rule->key.key.ipv6_5tuple.src_ip[3]),
+              (rule->key.key.ipv6_5tuple.src_ip[4]),
+              (rule->key.key.ipv6_5tuple.src_ip[5]),
+              (rule->key.key.ipv6_5tuple.src_ip[6]),
+              (rule->key.key.ipv6_5tuple.src_ip[7]),
+              (rule->key.key.ipv6_5tuple.src_ip[8]),
+              (rule->key.key.ipv6_5tuple.src_ip[9]),
+              (rule->key.key.ipv6_5tuple.src_ip[10]),
+              (rule->key.key.ipv6_5tuple.src_ip[11]),
+              (rule->key.key.ipv6_5tuple.src_ip[12]),
+              (rule->key.key.ipv6_5tuple.src_ip[13]),
+              (rule->key.key.ipv6_5tuple.src_ip[14]),
+              (rule->key.key.ipv6_5tuple.src_ip[15]),
+              rule->key.key.ipv6_5tuple.src_ip_mask,
+              (rule->key.key.ipv6_5tuple.dst_ip[0]),
+              (rule->key.key.ipv6_5tuple.dst_ip[1]),
+              (rule->key.key.ipv6_5tuple.dst_ip[2]),
+              (rule->key.key.ipv6_5tuple.dst_ip[3]),
+              (rule->key.key.ipv6_5tuple.dst_ip[4]),
+              (rule->key.key.ipv6_5tuple.dst_ip[5]),
+              (rule->key.key.ipv6_5tuple.dst_ip[6]),
+              (rule->key.key.ipv6_5tuple.dst_ip[7]),
+              (rule->key.key.ipv6_5tuple.dst_ip[8]),
+              (rule->key.key.ipv6_5tuple.dst_ip[9]),
+              (rule->key.key.ipv6_5tuple.dst_ip[10]),
+              (rule->key.key.ipv6_5tuple.dst_ip[11]),
+              (rule->key.key.ipv6_5tuple.dst_ip[12]),
+              (rule->key.key.ipv6_5tuple.dst_ip[13]),
+              (rule->key.key.ipv6_5tuple.dst_ip[14]),
+              (rule->key.key.ipv6_5tuple.dst_ip[15]),
+              rule->key.key.ipv6_5tuple.dst_ip_mask,
+              rule->key.key.ipv6_5tuple.src_port_from,
+              rule->key.key.ipv6_5tuple.src_port_to,
+              rule->key.key.ipv6_5tuple.dst_port_from,
+              rule->key.key.ipv6_5tuple.dst_port_to,
+              rule->key.key.ipv6_5tuple.proto,
+              rule->key.key.ipv6_5tuple.proto_mask, rule->action_id,
+              rule->entry_ptr);
+}
+
+/**
+ * Find an VFW rule.
+ * This function is used by the add and delete rule functions.
+ * Since all updates are done on the standby tables,
+ * only search the standby tables.
+ * Both IPv4 and IPv6 rules can be searched
+ *
+ * @param key
+ *  A pointer to the rule to be found.
+ *
+ * @return
+ *  - Pointer to the rule found.
+ *  - NULL if no rule found.
+ */
+static struct app_pipeline_vfw_rule *app_pipeline_vfw_rule_find(
+              struct pipeline_vfw_key *key)
+{
+       /*
+        * This function is used by the add and delete rule functions.
+        * Since all updates are done on the standby tables,
+        * only search the standby tables.
+        */
+
+       struct app_pipeline_vfw_rule *r;
+
+       if (key->type == PIPELINE_VFW_IPV4_5TUPLE) {
+              TAILQ_FOREACH(r, vfw_tailq_rules_ipv4_standby, node)
+                  if (memcmp(key,
+                            &r->key,
+                            sizeof(struct pipeline_vfw_key)) == 0)
+                     return r;
+       } else {              /* IPV6 */
+              TAILQ_FOREACH(r, vfw_tailq_rules_ipv6_standby, node)
+                  if (memcmp(key,
+                            &r->key,
+                            sizeof(struct pipeline_vfw_key)) == 0)
+                     return r;
+       }
+
+       return NULL;
+}
+
+
+/**
+ * Synproxy ON/OFF CLI command.
+ *
+ * @param app
+ *  A pointer to the application parameter.
+ * @param pipeline_id
+ *  pipeline ID.
+ * @param synproxy_flag
+ * 0-OFF,1-ON.
+ *
+ * @return
+ *  Response message contains status.
+ */
+
+static int
+app_pipeline_vfw_synproxy_flag(struct app_params *app,
+                              uint32_t pipeline_id, uint8_t synproxy_flag)
+{
+       struct app_pipeline_acl *p;
+       struct pipeline_vfw_synproxy_flag_msg_req *req;
+       struct pipeline_vfw_synproxy_flag_msg_rsp *rsp;
+
+       /* Check input arguments */
+       if (app == NULL)
+              return -1;
+       p = app_pipeline_data_fe(app, pipeline_id, &pipeline_vfw);
+       if (p == NULL)
+              return -1;
+       /* Allocate and write request */
+       req = app_msg_alloc(app);
+       if (req == NULL)
+              return -1;
+       req->type = PIPELINE_MSG_REQ_CUSTOM;
+       req->subtype = PIPELINE_VFW_MSG_REQ_SYNPROXY_FLAGS;
+       req->synproxy_flag = synproxy_flag;
+
+       /* Send request and wait for response */
+       rsp = app_msg_send_recv(app, pipeline_id, req, MSG_TIMEOUT_DEFAULT);
+       if (rsp == NULL) {
+              printf("Failed communication with TCP firewall\n");
+              return -1;
+       }
+       /* Read response and write rule */
+       if (rsp->status) {
+              printf("res status=%d", rsp->status);
+              app_msg_free(app, rsp);
+              return -1;
+       }
+
+       /* Free response */
+       app_msg_free(app, rsp);
+
+       return 0;
+}
+
+/**
+ * Display VFW Rules to the console.
+ * Rules from Active and standby tables can be dispayed.
+ * Both IPv4 and IPv6 will be displayed.
+ *
+ * @param app
+ *  A pointer to application specific data.
+ * @param active_standby_table
+ *  Specifies which table to display:
+ *    - active_rule_table (0)
+ *    - standby_rule_table (1)
+ *
+ */
+static void
+app_pipeline_vfw_ls(__attribute__ ((unused)) struct app_params *app,
+              uint32_t active_standby_table)
+{
+       struct app_pipeline_vfw_rule *rule;
+       uint32_t n_rules;
+       int priority;
+
+       if (active_standby_table == active_rule_table) {
+              n_rules = *vfw_n_tailq_rules_ipv4_active;
+              if (n_rules > 0)
+                     printf("VFW Active Table IPV4 Rules\n");
+              for (priority = 0; n_rules; priority++)
+                     TAILQ_FOREACH(rule, vfw_tailq_rules_ipv4_active,
+                                  node)
+                         if (rule->priority == priority) {
+                            print_vfw_ipv4_rule(rule);
+                            n_rules--;
+                     }
+
+              n_rules = *vfw_n_tailq_rules_ipv6_active;
+              if (n_rules > 0)
+                     printf("VFW Active Table IPV6 Rules\n");
+              for (priority = 0; n_rules; priority++)
+                     TAILQ_FOREACH(rule, vfw_tailq_rules_ipv6_active,
+                                  node)
+                         if (rule->priority == priority) {
+                            print_vfw_ipv6_rule(rule);
+                            n_rules--;
+                     }
+       } else {
+              n_rules = *vfw_n_tailq_rules_ipv4_standby;
+              if (n_rules > 0)
+                     printf("VFW Standby Table IPV4 Rules\n");
+              for (priority = 0; n_rules; priority++)
+                     TAILQ_FOREACH(rule, vfw_tailq_rules_ipv4_standby,
+                                  node)
+                         if (rule->priority == priority) {
+                            print_vfw_ipv4_rule(rule);
+                            n_rules--;
+                     }
+
+              n_rules = *vfw_n_tailq_rules_ipv6_standby;
+              if (n_rules > 0)
+                     printf("VFW Standby Table IPV6a Rules\n");
+              for (priority = 0; n_rules; priority++)
+                     TAILQ_FOREACH(rule, vfw_tailq_rules_ipv6_standby,
+                                  node)
+                         if (rule->priority == priority) {
+                            print_vfw_ipv6_rule(rule);
+                            n_rules--;
+                     }
+       }
+       printf("\n");
+}
+
+/**
+ * Initialize VFW pipeline Front End (FE).
+ *
+ * @param params
+ *  A pointer to pipeline parameters
+ * @param arg
+ *  A pointer to pipeline specific data (not used).
+ *
+ * @return
+ *  - A pointer to the pipeline FE
+ *  - NULL if initialization failed.
+ */
+static void *app_pipeline_vfw_init(struct pipeline_params *params,
+                                  __rte_unused void *arg)
+{
+       struct app_pipeline_vfw *p;
+       uint32_t size;
+
+       /* Check input arguments */
+       if ((params == NULL) ||
+           (params->n_ports_in == 0) || (params->n_ports_out == 0))
+              return NULL;
+
+       /* Memory allocation */
+       size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct app_pipeline_vfw));
+       p = rte_zmalloc(NULL, size, RTE_CACHE_LINE_SIZE);
+       if (p == NULL)
+              return NULL;
+
+       /* Initialization */
+       p->n_ports_in = params->n_ports_in;
+       p->n_ports_out = params->n_ports_out;
+
+       if (!vfw_rule_table_created) {
+              /* Only create and init once when first VFW pipeline/thread
+               * comes up */
+
+              /* Init tailq tables */
+              TAILQ_INIT(&vfw_tailq_rules_ipv4a);
+              vfw_n_tailq_rules_ipv4a = 0;
+              TAILQ_INIT(&vfw_tailq_rules_ipv4b);
+              vfw_n_tailq_rules_ipv4b = 0;
+              TAILQ_INIT(&vfw_tailq_rules_ipv6a);
+              vfw_n_tailq_rules_ipv6a = 0;
+              TAILQ_INIT(&vfw_tailq_rules_ipv6b);
+              vfw_n_tailq_rules_ipv6b = 0;
+              TAILQ_INIT(&vfw_commands);
+              vfw_tailq_rules_ipv4_active = &vfw_tailq_rules_ipv4a;
+              vfw_tailq_rules_ipv4_standby = &vfw_tailq_rules_ipv4b;
+              vfw_tailq_rules_ipv6_active = &vfw_tailq_rules_ipv6a;
+              vfw_tailq_rules_ipv6_standby = &vfw_tailq_rules_ipv6b;
+              vfw_n_tailq_rules_ipv4_active = &vfw_n_tailq_rules_ipv4a;
+              vfw_n_tailq_rules_ipv4_standby = &vfw_n_tailq_rules_ipv4b;
+              vfw_n_tailq_rules_ipv6_active = &vfw_n_tailq_rules_ipv6a;
+              vfw_n_tailq_rules_ipv6_standby = &vfw_n_tailq_rules_ipv6b;
+
+              /* Both IPV4 and IPV6 enabled by default */
+              vfw_ipv4_enabled = 1;
+              vfw_ipv6_enabled = 1;
+
+              printf("VFW FE Init Create Tables vfw_n_rules = %i\n",
+                   vfw_n_rules);
+
+              /* Init Action Array and Counter Table */
+              action_array_size =
+                  RTE_CACHE_LINE_ROUNDUP(sizeof(struct pipeline_action_key) *
+                                      action_array_max);
+              action_array_a =
+                  rte_zmalloc(NULL, action_array_size, RTE_CACHE_LINE_SIZE);
+              if (action_array_a == NULL)
+                     return NULL;
+              action_array_b =
+                  rte_zmalloc(NULL, action_array_size, RTE_CACHE_LINE_SIZE);
+              if (action_array_b == NULL)
+                     return NULL;
+              memset(action_array_a, 0, action_array_size);
+              memset(action_array_b, 0, action_array_size);
+              action_array_active = action_array_a;
+              action_array_standby = action_array_b;
+              memset(&action_counter_table, 0, sizeof(action_counter_table));
+
+              vfw_rule_table_created = 1;
+       }
+
+       if (!rte_vfw_hertz_computed) {
+              /* all initialization serialized on core 0,
+               * so no need for lock */
+              rte_vfw_ticks_in_one_second = rte_get_tsc_hz();
+              rte_vfw_hertz_computed = 1;
+       }
+
+       return (void *)p;
+}
+
+/**
+ * Free VFW pipeline resources.
+ *
+ * @param pipeline
+ *  A pointer to the pipeline to delete.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int app_pipeline_vfw_free(void *pipeline)
+{
+       struct app_pipeline_vfw *p = pipeline;
+
+       /* Check input arguments */
+       if (p == NULL)
+              return -1;
+
+       /* Free resources */
+       /* Ignore Klockwork infinite loop issues for all while loops */
+       while (!TAILQ_EMPTY(&vfw_tailq_rules_ipv4a)) {
+              struct app_pipeline_vfw_rule *rule;
+
+              rule = TAILQ_FIRST(&vfw_tailq_rules_ipv4a);
+              TAILQ_REMOVE(&vfw_tailq_rules_ipv4a, rule, node);
+              rte_free(rule);
+       }
+       while (!TAILQ_EMPTY(&vfw_tailq_rules_ipv4b)) {
+              struct app_pipeline_vfw_rule *rule;
+
+              rule = TAILQ_FIRST(&vfw_tailq_rules_ipv4b);
+              TAILQ_REMOVE(&vfw_tailq_rules_ipv4b, rule, node);
+              rte_free(rule);
+       }
+       while (!TAILQ_EMPTY(&vfw_tailq_rules_ipv6a)) {
+              struct app_pipeline_vfw_rule *rule;
+
+              rule = TAILQ_FIRST(&vfw_tailq_rules_ipv6a);
+              TAILQ_REMOVE(&vfw_tailq_rules_ipv6a, rule, node);
+              rte_free(rule);
+       }
+       while (!TAILQ_EMPTY(&vfw_tailq_rules_ipv6b)) {
+              struct app_pipeline_vfw_rule *rule;
+
+              rule = TAILQ_FIRST(&vfw_tailq_rules_ipv6b);
+              TAILQ_REMOVE(&vfw_tailq_rules_ipv6b, rule, node);
+              rte_free(rule);
+       }
+       while (!TAILQ_EMPTY(&vfw_commands)) {
+              struct app_pipeline_vfw_rule *command;
+
+              command = TAILQ_FIRST(&vfw_commands);
+              TAILQ_REMOVE(&vfw_commands, command, node);
+              rte_free(command);
+       }
+       rte_free(action_array_a);
+       rte_free(action_array_b);
+       rte_free(p);
+       return 0;
+}
+
+/**
+ * Verify that the VFW rule is valid.
+ * Both IPv4 and IPv6 rules
+ *
+ * @param key
+ *  A pointer to the VFW rule to verify.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int
+app_pipeline_vfw_key_check_and_normalize(struct pipeline_vfw_key *key)
+{
+       switch (key->type) {
+       case PIPELINE_VFW_IPV4_5TUPLE:
+              {
+                     uint32_t src_ip_depth =
+                         key->key.ipv4_5tuple.src_ip_mask;
+                     uint32_t dst_ip_depth =
+                         key->key.ipv4_5tuple.dst_ip_mask;
+                     uint16_t src_port_from =
+                         key->key.ipv4_5tuple.src_port_from;
+                     uint16_t src_port_to = key->key.ipv4_5tuple.src_port_to;
+                     uint16_t dst_port_from =
+                         key->key.ipv4_5tuple.dst_port_from;
+                     uint16_t dst_port_to = key->key.ipv4_5tuple.dst_port_to;
+
+                     uint32_t src_ip_netmask = 0;
+                     uint32_t dst_ip_netmask = 0;
+
+                     if ((src_ip_depth > 32) ||
+                         (dst_ip_depth > 32) ||
+                         (src_port_from > src_port_to) ||
+                         (dst_port_from > dst_port_to))
+                            return -1;
+
+                     if (src_ip_depth)
+                            src_ip_netmask = (~0) << (32 - src_ip_depth);
+
+                     if (dst_ip_depth)
+                            dst_ip_netmask = ((~0) << (32 - dst_ip_depth));
+
+                     key->key.ipv4_5tuple.src_ip &= src_ip_netmask;
+                     key->key.ipv4_5tuple.dst_ip &= dst_ip_netmask;
+
+                     return 0;
+              }
+       case PIPELINE_VFW_IPV6_5TUPLE:
+              {
+                     uint32_t src_ip_depth =
+                         key->key.ipv6_5tuple.src_ip_mask;
+                     uint32_t dst_ip_depth =
+                         key->key.ipv6_5tuple.dst_ip_mask;
+                     uint8_t src_ip_netmask[16];
+                     uint8_t dst_ip_netmask[16];
+                     int i;
+
+                     convert_prefixlen_to_netmask_ipv6(src_ip_depth,
+                                                   src_ip_netmask);
+                     convert_prefixlen_to_netmask_ipv6(dst_ip_depth,
+                                                   dst_ip_netmask);
+                     for (i = 0; i < 16; i++) {
+                            key->key.ipv6_5tuple.src_ip[i] &=
+                                src_ip_netmask[i];
+                            key->key.ipv6_5tuple.dst_ip[i] &=
+                                dst_ip_netmask[i];
+                     }
+                     return 0;
+              }
+
+       default:
+              return -1;
+       }
+}
+
+/**
+ * Add VFW rule to the VFW rule table.
+ * Rules are added standby table.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules can be added.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the VFW rule to add.
+ * @param priority
+ *  Priority of the VFW rule.
+ * @param port_id
+ *  Port ID of the VFW rule.
+ * @param action_id
+ *  Action ID of the VFW rule. Defined in Action Table.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_vfw_add_rule(struct app_params *app,
+                          struct pipeline_vfw_key *key,
+                          uint32_t priority,
+                          uint32_t port_id, uint32_t action_id)
+{
+       struct app_pipeline_vfw_rule *rule;
+       struct pipeline_vfw_add_msg_rsp *rsp;
+       int new_rule, src_field_start, dst_field_start, i;
+       uint32_t *ip1, *ip2, *ip3, *ip4, src_mask, dest_mask;
+       uint32_t src_ip[IPV6_32BIT_LENGTH], dst_ip[IPV6_32BIT_LENGTH];
+       const uint32_t nbu32 = sizeof(uint32_t) * CHAR_BIT;
+
+       struct rte_table_acl_rule_add_params params;
+       struct lib_acl_table_entry entry = {
+              .head = {
+                      .action = RTE_PIPELINE_ACTION_PORT,
+                      {.port_id = port_id},
+                      },
+              .action_id = action_id,
+       };
+
+       memset(&params, 0, sizeof(params));
+
+       /* Check input arguments */
+       if ((app == NULL) ||
+           (key == NULL) || !((key->type == PIPELINE_VFW_IPV4_5TUPLE) ||
+                            (key->type == PIPELINE_VFW_IPV6_5TUPLE)))
+              return -1;
+
+       if (action_id > action_array_max) {
+              printf("Action ID greater than max\n");
+              return -1;
+       }
+
+       if (app_pipeline_vfw_key_check_and_normalize(key) != 0)
+              return -1;
+
+       /* Find existing rule or allocate new rule */
+       rule = app_pipeline_vfw_rule_find(key);
+       new_rule = (rule == NULL);
+       if (rule == NULL) {
+              rule = rte_malloc(NULL, sizeof(*rule), RTE_CACHE_LINE_SIZE);
+
+              if (rule == NULL)
+                     return -1;
+       }
+
+       /* Allocate Response */
+       rsp = app_msg_alloc(app);
+       if (rsp == NULL) {
+              if (new_rule)
+                     rte_free(rule);
+              return -1;
+       }
+
+       switch (key->type) {
+       case PIPELINE_VFW_IPV4_5TUPLE:
+              params.priority = priority;
+              params.field_value[0].value.u8 = key->key.ipv4_5tuple.proto;
+              params.field_value[0].mask_range.u8 =
+                  key->key.ipv4_5tuple.proto_mask;
+              params.field_value[1].value.u32 = key->key.ipv4_5tuple.src_ip;
+              params.field_value[1].mask_range.u32 =
+                  key->key.ipv4_5tuple.src_ip_mask;
+              params.field_value[2].value.u32 = key->key.ipv4_5tuple.dst_ip;
+              params.field_value[2].mask_range.u32 =
+                  key->key.ipv4_5tuple.dst_ip_mask;
+              params.field_value[3].value.u16 =
+                  key->key.ipv4_5tuple.src_port_from;
+              params.field_value[3].mask_range.u16 =
+                  key->key.ipv4_5tuple.src_port_to;
+              params.field_value[4].value.u16 =
+                  key->key.ipv4_5tuple.dst_port_from;
+              params.field_value[4].mask_range.u16 =
+                  key->key.ipv4_5tuple.dst_port_to;
+
+              rsp->status =
+                  rte_table_acl_ops.f_add(vfw_rule_table_ipv4_standby,
+                                       &params,
+                                       (struct rte_pipeline_table_entry *)
+                                       &entry, &rsp->key_found,
+                                       (void **)&rsp->entry_ptr);
+
+              if (rsp->status != 0)
+                     printf
+                         ("IPV4 Add Rule Command failed key_found: %i\n",
+                          rsp->key_found);
+              else
+                     printf
+                         ("IPV4 Add Rule Command success key_found: %i\n",
+                          rsp->key_found);
+
+              break;
+
+       case PIPELINE_VFW_IPV6_5TUPLE:
+              ip1 = (uint32_t *) (key->key.ipv6_5tuple.src_ip);
+              ip2 = ip1 + 1;
+              ip3 = ip1 + 2;
+              ip4 = ip1 + 3;
+
+              params.priority = priority;
+              params.field_value[0].value.u8 = key->key.ipv6_5tuple.proto;
+              params.field_value[0].mask_range.u8 =
+                  key->key.ipv6_5tuple.proto_mask;
+
+              src_ip[0] = rte_bswap32(*ip1);
+              src_ip[1] = rte_bswap32(*ip2);
+              src_ip[2] = rte_bswap32(*ip3);
+              src_ip[3] = rte_bswap32(*ip4);
+
+              src_mask = key->key.ipv6_5tuple.src_ip_mask;
+
+              src_field_start = 1;
+              for (i = 0; i != RTE_DIM(src_ip); i++, src_field_start++) {
+                     if (src_mask >= (i + 1) * nbu32)
+                            params.field_value[src_field_start].mask_range.
+                                u32 = nbu32;
+                     else
+                            params.field_value[src_field_start].mask_range.
+                                u32 =
+                                src_mask >
+                                (i * nbu32) ? src_mask - (i * 32) : 0;
+                     params.field_value[src_field_start].value.u32 =
+                         src_ip[i];
+              }
+
+              ip1 = (uint32_t *) (key->key.ipv6_5tuple.dst_ip);
+              ip2 = ip1 + 1;
+              ip3 = ip1 + 2;
+              ip4 = ip1 + 3;
+
+              dst_ip[0] = rte_bswap32(*ip1);
+              dst_ip[1] = rte_bswap32(*ip2);
+              dst_ip[2] = rte_bswap32(*ip3);
+              dst_ip[3] = rte_bswap32(*ip4);
+
+              dest_mask = key->key.ipv6_5tuple.dst_ip_mask;
+
+              dst_field_start = 5;
+              for (i = 0; i != RTE_DIM(dst_ip); i++, dst_field_start++) {
+                     if (dest_mask >= (i + 1) * nbu32)
+                            params.field_value[dst_field_start].mask_range.
+                                u32 = nbu32;
+                     else
+                            params.field_value[dst_field_start].mask_range.
+                                u32 =
+                                dest_mask >
+                                (i * nbu32) ? dest_mask - (i * 32) : 0;
+                     params.field_value[dst_field_start].value.u32 =
+                         dst_ip[i];
+              }
+
+              params.field_value[9].value.u16 =
+                  key->key.ipv6_5tuple.src_port_from;
+              params.field_value[9].mask_range.u16 =
+                  key->key.ipv6_5tuple.src_port_to;
+              params.field_value[10].value.u16 =
+                  key->key.ipv6_5tuple.dst_port_from;
+              params.field_value[10].mask_range.u16 =
+                  key->key.ipv6_5tuple.dst_port_to;
+
+              rsp->status =
+                  rte_table_acl_ops.f_add(vfw_rule_table_ipv6_standby,
+                                       &params,
+                                       (struct rte_pipeline_table_entry *)
+                                       &entry, &rsp->key_found,
+                                       (void **)&rsp->entry_ptr);
+
+              if (rsp->status != 0)
+                     printf
+                         ("IPV6 Add Rule Command failed key_found: %i\n",
+                          rsp->key_found);
+              else
+                     printf
+                         ("IPV6 Add Rule Command success key_found: %i\n",
+                          rsp->key_found);
+
+              break;
+
+       default:
+              /* Error */
+              app_msg_free(app, rsp);
+              if (new_rule)
+                     rte_free(rule);
+              return -1;
+       }
+
+       /* Read response and write rule */
+       if (rsp->status ||
+           (rsp->entry_ptr == NULL) ||
+           ((new_rule == 0) && (rsp->key_found == 0)) ||
+           ((new_rule == 1) && (rsp->key_found == 1))) {
+              app_msg_free(app, rsp);
+              if (new_rule)
+                     rte_free(rule);
+              return -1;
+       }
+
+       memcpy(&rule->key, key, sizeof(*key));
+       rule->priority = priority;
+       rule->port_id = port_id;
+       rule->action_id = action_id;
+       rule->entry_ptr = rsp->entry_ptr;
+
+       /* Commit rule */
+       if (new_rule) {
+              if (key->type == PIPELINE_VFW_IPV4_5TUPLE) {
+                     TAILQ_INSERT_TAIL(vfw_tailq_rules_ipv4_standby, rule,
+                                     node);
+                     (*vfw_n_tailq_rules_ipv4_standby)++;
+              } else {       /* IPV6 */
+                     TAILQ_INSERT_TAIL(vfw_tailq_rules_ipv6_standby, rule,
+                                     node);
+                     (*vfw_n_tailq_rules_ipv6_standby)++;
+              }
+       }
+
+       if (key->type == PIPELINE_VFW_IPV4_5TUPLE)
+              print_vfw_ipv4_rule(rule);
+       else
+              print_vfw_ipv6_rule(rule);
+
+       /* Free response */
+       app_msg_free(app, rsp);
+
+       return 0;
+}
+
+/**
+ * Delete VFW rule from the VFW rule table.
+ * Rules deleted from standby tables.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules can be deleted.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the VFW rule to delete.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_vfw_delete_rule(struct app_params *app,
+                            struct pipeline_vfw_key *key)
+{
+       struct app_pipeline_vfw_rule *rule;
+       int status, key_found;
+       int  src_field_start, dst_field_start, i;
+       uint32_t *ip1, *ip2, *ip3, *ip4, src_mask, dest_mask;
+       uint32_t src_ip[IPV6_32BIT_LENGTH], dst_ip[IPV6_32BIT_LENGTH];
+       const uint32_t nbu32 = sizeof(uint32_t) * CHAR_BIT;
+
+
+       struct rte_table_acl_rule_delete_params params;
+
+       memset(&params, 0, sizeof(params));
+
+       /* Check input arguments */
+       if ((app == NULL) ||
+           (key == NULL) || !((key->type == PIPELINE_VFW_IPV4_5TUPLE) ||
+                            (key->type == PIPELINE_VFW_IPV6_5TUPLE)))
+              return -1;
+
+       if (app_pipeline_vfw_key_check_and_normalize(key) != 0)
+              return -1;
+
+       /* Find rule */
+       rule = app_pipeline_vfw_rule_find(key);
+       if (rule == NULL) {
+              printf("VFW Delete Rule - Rule does not exist\n");
+              return 0;
+       }
+
+       switch (key->type) {
+       case PIPELINE_VFW_IPV4_5TUPLE:
+              params.field_value[0].value.u8 = key->key.ipv4_5tuple.proto;
+              params.field_value[0].mask_range.u8 =
+                  key->key.ipv4_5tuple.proto_mask;
+              params.field_value[1].value.u32 = key->key.ipv4_5tuple.src_ip;
+              params.field_value[1].mask_range.u32 =
+                  key->key.ipv4_5tuple.src_ip_mask;
+              params.field_value[2].value.u32 = key->key.ipv4_5tuple.dst_ip;
+              params.field_value[2].mask_range.u32 =
+                  key->key.ipv4_5tuple.dst_ip_mask;
+              params.field_value[3].value.u16 =
+                  key->key.ipv4_5tuple.src_port_from;
+              params.field_value[3].mask_range.u16 =
+                  key->key.ipv4_5tuple.src_port_to;
+              params.field_value[4].value.u16 =
+                  key->key.ipv4_5tuple.dst_port_from;
+              params.field_value[4].mask_range.u16 =
+                  key->key.ipv4_5tuple.dst_port_to;
+
+              status =
+                  rte_table_acl_ops.f_delete(vfw_rule_table_ipv4_standby,
+                                          &params, &key_found, NULL);
+
+              if (status != 0)
+                     printf
+                         ("IPV4 Del Rule Command failed key_found: %i\n",
+                          key_found);
+              else
+                     printf
+                         ("IPV4 Del Rule Command success key_found: %i\n",
+                          key_found);
+
+              break;
+
+       case PIPELINE_VFW_IPV6_5TUPLE:
+              ip1 = (uint32_t *) (key->key.ipv6_5tuple.src_ip);
+              ip2 = ip1 + 1;
+              ip3 = ip1 + 2;
+              ip4 = ip1 + 3;
+
+              params.field_value[0].value.u8 = key->key.ipv6_5tuple.proto;
+              params.field_value[0].mask_range.u8 =
+                  key->key.ipv6_5tuple.proto_mask;
+
+              src_ip[0] = rte_bswap32(*ip1);
+              src_ip[1] = rte_bswap32(*ip2);
+              src_ip[2] = rte_bswap32(*ip3);
+              src_ip[3] = rte_bswap32(*ip4);
+
+              src_mask = key->key.ipv6_5tuple.src_ip_mask;
+
+              src_field_start = 1;
+              for (i = 0; i != RTE_DIM(src_ip); i++, src_field_start++) {
+                     if (src_mask >= (i + 1) * nbu32)
+                            params.field_value[src_field_start].mask_range.
+                                u32 = nbu32;
+                     else
+                            params.field_value[src_field_start].mask_range.
+                                u32 =
+                                src_mask >
+                                (i * nbu32) ? src_mask - (i * 32) : 0;
+                     params.field_value[src_field_start].value.u32 =
+                         src_ip[i];
+              }
+
+              ip1 = (uint32_t *) (key->key.ipv6_5tuple.dst_ip);
+              ip2 = ip1 + 1;
+              ip3 = ip1 + 2;
+              ip4 = ip1 + 3;
+
+              dst_ip[0] = rte_bswap32(*ip1);
+              dst_ip[1] = rte_bswap32(*ip2);
+              dst_ip[2] = rte_bswap32(*ip3);
+              dst_ip[3] = rte_bswap32(*ip4);
+
+              dest_mask = key->key.ipv6_5tuple.dst_ip_mask;
+
+              dst_field_start = 5;
+              for (i = 0; i != RTE_DIM(dst_ip); i++, dst_field_start++) {
+                     if (dest_mask >= (i + 1) * nbu32)
+                            params.field_value[dst_field_start].mask_range.
+                                u32 = nbu32;
+                     else
+                            params.field_value[dst_field_start].mask_range.
+                                u32 =
+                                dest_mask >
+                                (i * nbu32) ? dest_mask - (i * 32) : 0;
+                     params.field_value[dst_field_start].value.u32 =
+                         dst_ip[i];
+              }
+
+              params.field_value[9].value.u16 =
+                  key->key.ipv6_5tuple.src_port_from;
+              params.field_value[9].mask_range.u16 =
+                  key->key.ipv6_5tuple.src_port_to;
+              params.field_value[10].value.u16 =
+                  key->key.ipv6_5tuple.dst_port_from;
+              params.field_value[10].mask_range.u16 =
+                  key->key.ipv6_5tuple.dst_port_to;
+
+
+              status =
+                  rte_table_acl_ops.f_delete(vfw_rule_table_ipv6_standby,
+                                          &params, &key_found, NULL);
+
+              if (status != 0)
+                     printf("IPV6 Del Rule Command failed key_found: %i\n",
+                          key_found);
+              else
+                     printf("IPV6 Del Rule Command success key_found: %i\n",
+                          key_found);
+
+              break;
+
+       default:
+              /* Error */
+              return -1;
+       }
+
+       /* Read response */
+       if (status || !key_found)
+              return -1;
+
+       /* Remove rule */
+       if (key->type == PIPELINE_VFW_IPV4_5TUPLE) {
+              TAILQ_REMOVE(vfw_tailq_rules_ipv4_standby, rule, node);
+              (*vfw_n_tailq_rules_ipv4_standby)--;
+       } else {              /* IPV6 */
+              TAILQ_REMOVE(vfw_tailq_rules_ipv6_standby, rule, node);
+              (*vfw_n_tailq_rules_ipv6_standby)--;
+       }
+
+       rte_free(rule);
+
+       return 0;
+}
+
+/**
+ * Clear all VFW rules from the VFW rule table.
+ * Rules cleared from standby tables.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules will be cleared.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int app_pipeline_vfw_clearrules(struct app_params *app)
+{
+       struct app_pipeline_vfw_rule *rule;
+       struct app_pipeline_vfw_rule *command;
+       uint32_t n_rules;
+
+       int priority;
+
+       /* Check input arguments */
+       if (app == NULL)
+              return -1;
+
+       n_rules = *vfw_n_tailq_rules_ipv4_standby;
+       for (priority = 0; n_rules; priority++) {
+              TAILQ_FOREACH(rule, vfw_tailq_rules_ipv4_standby, node) {
+                     if (rule->priority == priority) {
+                            struct pipeline_vfw_key key = rule->key;
+
+                            /* Store command to update standby tables
+                             * after switchover */
+                            command =
+                                rte_malloc(NULL, sizeof(*command),
+                                          RTE_CACHE_LINE_SIZE);
+                            if (command == NULL) {
+                                   printf("Cannot allocation command\n");
+                                   return -1;
+                            }
+                            memset(command, 0,
+                                   sizeof(struct app_pipeline_vfw_rule));
+                            memcpy(&command->key, &key, sizeof(key));
+                            command->command = vfw_delete_command;
+                            TAILQ_INSERT_TAIL(&vfw_commands, command,
+                                            node);
+
+                            /* Delete rule */
+                            app_pipeline_vfw_delete_rule(app, &key);
+                            n_rules--;
+                     }
+              }
+       }
+
+       n_rules = *vfw_n_tailq_rules_ipv6_standby;
+       for (priority = 0; n_rules; priority++) {
+              TAILQ_FOREACH(rule, vfw_tailq_rules_ipv6_standby, node) {
+                     if (rule->priority == priority) {
+                            struct pipeline_vfw_key key = rule->key;
+
+                            /* Store command to update standby tables
+                             * after switchover */
+                            command =
+                                rte_malloc(NULL, sizeof(*command),
+                                          RTE_CACHE_LINE_SIZE);
+                            if (command == NULL) {
+                                   printf("Cannot allocation command\n");
+                                   return -1;
+                            }
+                            memset(command, 0,
+                                   sizeof(struct app_pipeline_vfw_rule));
+                            memcpy(&command->key, &key, sizeof(key));
+                            command->command = vfw_delete_command;
+                            TAILQ_INSERT_TAIL(&vfw_commands, command,
+                                            node);
+
+                            /* Delete rule */
+                            app_pipeline_vfw_delete_rule(app, &key);
+                            n_rules--;
+                     }
+              }
+       }
+
+       /* Clear Action Array */
+       memset(action_array_standby, 0, action_array_size);
+
+       return 0;
+}
+
+/*
+ * loadrules
+ */
+
+/**
+ * Open file and process all commands in the file.
+ *
+ * @param ctx
+ *  A pointer to the CLI context
+ * @param file_name
+ *  A pointer to the file to process.
+ *
+ */
+static void app_loadrules_file(cmdline_parse_ctx_t *ctx, const char *file_name)
+{
+       struct cmdline *file_cl;
+       int fd;
+
+       fd = open(file_name, O_RDONLY);
+       if (fd < 0) {
+              printf("Cannot open file \"%s\"\n", file_name);
+              return;
+       }
+
+       file_cl = cmdline_new(ctx, "", fd, 1);
+       cmdline_interact(file_cl);
+       close(fd);
+}
+
+struct cmd_loadrules_file_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t loadrules_string;
+       char file_name[APP_FILE_NAME_SIZE];
+};
+
+/**
+ * Parse load rules command.
+ * Verify that file exists.
+ * Clear existing rules and action.
+ * Process commands in command file.
+ *
+ * @param parsed_result
+ *  A pointer to the CLI command parsed result
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data.
+ *
+ * @return
+ *  0 on success, negative on error.
+ *
+ */
+static void
+cmd_loadrules_parsed(void *parsed_result, struct cmdline *cl, void *data)
+{
+       struct cmd_loadrules_file_result *params = parsed_result;
+       struct app_params *app = data;
+       int status;
+       int fd;
+
+       /* Make sure the file exists before clearing rules and actions */
+       fd = open(params->file_name, O_RDONLY);
+       if (fd < 0) {
+              printf("Cannot open file \"%s\"\n", params->file_name);
+              return;
+       }
+       close(fd);
+
+       /* Clear all rules and actions */
+       status = app_pipeline_vfw_clearrules(app);
+
+       if (status != 0) {
+              printf("Command clearrules failed\n");
+              return;
+       }
+
+       /* Process commands in script file */
+       app_loadrules_file(cl->ctx, params->file_name);
+}
+
+cmdline_parse_token_string_t cmd_loadrules_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_loadrules_file_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_loadrules_vfw_string =
+TOKEN_STRING_INITIALIZER(struct cmd_loadrules_file_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_loadrules_loadrules_string =
+TOKEN_STRING_INITIALIZER(struct cmd_loadrules_file_result, loadrules_string,
+                      "loadrules");
+
+cmdline_parse_token_string_t cmd_loadrules_file_name =
+TOKEN_STRING_INITIALIZER(struct cmd_loadrules_file_result, file_name, NULL);
+
+cmdline_parse_inst_t cmd_loadrules = {
+       .f = cmd_loadrules_parsed,
+       .data = NULL,
+       .help_str = "VFW Load Rules",
+       .tokens = {
+                 (void *)&cmd_loadrules_p_string,
+                 (void *)&cmd_loadrules_vfw_string,
+                 (void *)&cmd_loadrules_loadrules_string,
+                 (void *)&cmd_loadrules_file_name,
+                 NULL,
+                 },
+};
+
+/**
+ * Add Action to the Action table.
+ * Actions are added standby table.
+ * Applyruleset command will activate the change.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the Action to add.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_action_add(__attribute__ ((unused)) struct app_params *app,
+              struct pipeline_action_key *key)
+{
+
+       /*
+        * This function will update the action IDs on the standby table.
+        * Activating the changes is done with the applyruleset command.
+        */
+
+       uint32_t action_bitmap = key->action_bitmap;
+       uint32_t action_id = key->action_id;
+
+       if (action_id >= action_array_max) {
+              if (VFW_DEBUG)
+                     printf("Action id: %u out of range\n", action_id);
+              return -1;
+       }
+
+       action_array_standby[action_id].action_id = action_id;
+
+       if (VFW_DEBUG)
+              printf("Adding action id: %u Type: ", action_id);
+       if (action_bitmap == lib_acl_action_packet_accept) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_packet_accept;
+              if (VFW_DEBUG)
+                     printf("Accept\n");
+       }
+       if (action_bitmap == lib_acl_action_packet_drop) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_packet_drop;
+              if (VFW_DEBUG)
+                     printf("Drop\n");
+       }
+       if (action_bitmap == lib_acl_action_nat) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_nat;
+              action_array_standby[action_id].nat_port = key->nat_port;
+              if (VFW_DEBUG)
+                     printf("NAT  Port ID: %u\n", key->nat_port);
+       }
+       if (action_bitmap == lib_acl_action_fwd) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_fwd;
+              action_array_standby[action_id].fwd_port = key->fwd_port;
+              if (VFW_DEBUG)
+                     printf("FWD  Port ID: %u\n", key->fwd_port);
+       }
+       if (action_bitmap == lib_acl_action_count) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_count;
+              if (VFW_DEBUG)
+                     printf("Count\n");
+       }
+       if (action_bitmap == lib_acl_action_conntrack) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_conntrack;
+              if (VFW_DEBUG)
+                     printf("Conntrack\n");
+       }
+       if (action_bitmap == lib_acl_action_connexist) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_connexist;
+              action_array_standby[action_id].private_public =
+                  key->private_public;
+              if (VFW_DEBUG)
+                     printf("Conntrack   prvpub: %i\n", key->private_public);
+       }
+       if (action_bitmap == lib_acl_action_dscp) {
+              action_array_standby[action_id].action_bitmap |=
+                  lib_acl_action_dscp;
+              action_array_standby[action_id].dscp_priority =
+                  key->dscp_priority;
+              if (VFW_DEBUG)
+                     printf("DSCP  Priority: %u\n", key->dscp_priority);
+       }
+
+       if (VFW_DEBUG)
+              printf("action_bitmap: %" PRIu32 "\n",
+                     action_array_standby[action_id].action_bitmap);
+
+       return 0;
+}
+
+/**
+ * Delete Action from the Action table.
+ * Actions are deleted from the standby table.
+ * Applyruleset command will activate the change.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the Action to delete.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_action_delete(__attribute__ ((unused)) struct app_params *app,
+                        struct pipeline_action_key *key)
+{
+       /*
+        * This function will update the action IDs on the standby table.
+        * Activating the changes is done with the applyruleset command.
+        */
+
+       uint32_t action_bitmap = key->action_bitmap;
+       uint32_t action_id = key->action_id;
+
+       if (action_id >= action_array_max) {
+              if (VFW_DEBUG)
+                     printf("Action id: %u out of range\n", action_id);
+              return -1;
+       }
+
+       if (action_array_standby[action_id].action_bitmap & action_bitmap)
+              action_array_standby[action_id].action_bitmap &= ~action_bitmap;
+       else
+              printf("VFW Action Delete - Action not set\n");
+
+       if (VFW_DEBUG) {
+              printf("Deleting action id: %u Type: ", key->action_id);
+              if (action_bitmap == lib_acl_action_packet_accept)
+                     printf("Accept\n");
+              if (action_bitmap == lib_acl_action_packet_drop)
+                     printf("Drop\n");
+              if (action_bitmap == lib_acl_action_nat)
+                     printf("NAT\n");
+              if (action_bitmap == lib_acl_action_fwd)
+                     printf("FWD\n");
+              if (action_bitmap == lib_acl_action_count)
+                     printf("Count\n");
+              if (action_bitmap == lib_acl_action_conntrack)
+                     printf("Conntrack\n");
+              if (action_bitmap == lib_acl_action_connexist)
+                     printf("Connexist\n");
+              if (action_bitmap == lib_acl_action_dscp)
+                     printf("DSCP\n");
+
+              printf("action_bitmap: %" PRIu32 "\n",
+                     action_array_standby[action_id].action_bitmap);
+       }
+
+       return 0;
+}
+
+/*
+ * p vfw add
+ */
+
+/**
+ * A structure defining the VFW add rule command.
+ */
+struct cmd_vfw_add_ip_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t add_string;
+       int32_t priority;
+       cmdline_ipaddr_t src_ip;
+       uint32_t src_ip_mask;
+       cmdline_ipaddr_t dst_ip;
+       uint32_t dst_ip_mask;
+       uint16_t src_port_from;
+       uint16_t src_port_to;
+       uint16_t dst_port_from;
+       uint16_t dst_port_to;
+       uint8_t proto;
+       uint8_t proto_mask;
+       uint8_t port_id;
+       uint32_t action_id;
+};
+
+/**
+ * Parse VFW add rule CLI command.
+ * Add rule to standby table.
+ * Store command to update standby table
+ * after applyruleset command is invoked.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_vfw_add_ip_parsed(void *parsed_result, __attribute__ ((unused))
+                      struct cmdline *cl, void *data)
+{
+       struct cmd_vfw_add_ip_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_vfw_key key;
+       struct app_pipeline_vfw_rule *command;
+       int status;
+
+       memset(&key, 0, sizeof(struct pipeline_vfw_key));
+
+       if (params->src_ip.family == AF_INET) {
+              key.type = PIPELINE_VFW_IPV4_5TUPLE;
+              key.key.ipv4_5tuple.src_ip = rte_bswap32((uint32_t)
+                                                  params->src_ip.addr.
+                                                  ipv4.s_addr);
+              key.key.ipv4_5tuple.src_ip_mask = params->src_ip_mask;
+              key.key.ipv4_5tuple.dst_ip = rte_bswap32((uint32_t)
+                                                  params->dst_ip.addr.
+                                                  ipv4.s_addr);
+              key.key.ipv4_5tuple.dst_ip_mask = params->dst_ip_mask;
+              key.key.ipv4_5tuple.src_port_from = params->src_port_from;
+              key.key.ipv4_5tuple.src_port_to = params->src_port_to;
+              key.key.ipv4_5tuple.dst_port_from = params->dst_port_from;
+              key.key.ipv4_5tuple.dst_port_to = params->dst_port_to;
+              key.key.ipv4_5tuple.proto = params->proto;
+              key.key.ipv4_5tuple.proto_mask = params->proto_mask;
+       }
+       if (params->src_ip.family == AF_INET6) {
+              if (VFW_DEBUG)
+                     printf("entered IPV6");
+              key.type = PIPELINE_VFW_IPV6_5TUPLE;
+              memcpy(key.key.ipv6_5tuple.src_ip,
+                     params->src_ip.addr.ipv6.s6_addr,
+                     sizeof(params->src_ip.addr.ipv6.s6_addr));
+              key.key.ipv6_5tuple.src_ip_mask = params->src_ip_mask;
+              memcpy(key.key.ipv6_5tuple.dst_ip,
+                     params->dst_ip.addr.ipv6.s6_addr,
+                     sizeof(params->src_ip.addr.ipv6.s6_addr));
+              key.key.ipv6_5tuple.dst_ip_mask = params->dst_ip_mask;
+              key.key.ipv6_5tuple.src_port_from = params->src_port_from;
+              key.key.ipv6_5tuple.src_port_to = params->src_port_to;
+              key.key.ipv6_5tuple.dst_port_from = params->dst_port_from;
+              key.key.ipv6_5tuple.dst_port_to = params->dst_port_to;
+              key.key.ipv6_5tuple.proto = params->proto;
+              key.key.ipv6_5tuple.proto_mask = params->proto_mask;
+       }
+       /* Set to 1 as default, overwritten by Action FWD/NAT Port */
+       status = app_pipeline_vfw_add_rule(app, &key, params->priority, 1,
+                                         params->action_id);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+
+       /* Store command to update standby tables after switchover */
+       command = rte_malloc(NULL, sizeof(*command), RTE_CACHE_LINE_SIZE);
+       if (command == NULL) {
+              printf("Cannot allocation command\n");
+              return;
+       }
+       memset(command, 0, sizeof(struct app_pipeline_vfw_rule));
+       memcpy(&command->key, &key, sizeof(key));
+       command->priority = params->priority;
+       command->port_id = params->port_id;
+       command->action_id = params->action_id;
+       command->command = vfw_add_command;
+       TAILQ_INSERT_TAIL(&vfw_commands, command, node);
+}
+
+cmdline_parse_token_string_t cmd_vfw_add_ip_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_add_ip_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_vfw_add_ip_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_add_ip_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_add_ip_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_add_ip_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_priority =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result, priority,
+                    INT32);
+
+cmdline_parse_token_ipaddr_t cmd_vfw_add_ip_src_ip =
+TOKEN_IPADDR_INITIALIZER(struct cmd_vfw_add_ip_result, src_ip);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_src_ip_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result, src_ip_mask,
+                    UINT32);
+
+cmdline_parse_token_ipaddr_t cmd_vfw_add_ip_dst_ip =
+TOKEN_IPADDR_INITIALIZER(struct cmd_vfw_add_ip_result, dst_ip);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_dst_ip_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result, dst_ip_mask,
+                    UINT32);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_src_port_from =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    src_port_from, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_src_port_to =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    src_port_to, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_dst_port_from =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    dst_port_from, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_dst_port_to =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    dst_port_to, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_proto =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    proto, UINT8);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_proto_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    proto_mask, UINT8);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_port_id =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    port_id, UINT8);
+
+cmdline_parse_token_num_t cmd_vfw_add_ip_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_add_ip_result,
+                    action_id, UINT32);
+
+cmdline_parse_inst_t cmd_vfw_add_ip = {
+       .f = cmd_vfw_add_ip_parsed,
+       .data = NULL,
+       .help_str = "VFW rule add",
+       .tokens = {
+                 (void *)&cmd_vfw_add_ip_p_string,
+                 (void *)&cmd_vfw_add_ip_acl_string,
+                 (void *)&cmd_vfw_add_ip_add_string,
+                 (void *)&cmd_vfw_add_ip_priority,
+                 (void *)&cmd_vfw_add_ip_src_ip,
+                 (void *)&cmd_vfw_add_ip_src_ip_mask,
+                 (void *)&cmd_vfw_add_ip_dst_ip,
+                 (void *)&cmd_vfw_add_ip_dst_ip_mask,
+                 (void *)&cmd_vfw_add_ip_src_port_from,
+                 (void *)&cmd_vfw_add_ip_src_port_to,
+                 (void *)&cmd_vfw_add_ip_dst_port_from,
+                 (void *)&cmd_vfw_add_ip_dst_port_to,
+                 (void *)&cmd_vfw_add_ip_proto,
+                 (void *)&cmd_vfw_add_ip_proto_mask,
+                 (void *)&cmd_vfw_add_ip_action_id,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw del
+ */
+
+/**
+ * A structure defining the VFW delete rule command.
+ */
+struct cmd_vfw_del_ip_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t del_string;
+       cmdline_ipaddr_t src_ip;
+       uint32_t src_ip_mask;
+       cmdline_ipaddr_t dst_ip;
+       uint32_t dst_ip_mask;
+       uint16_t src_port_from;
+       uint16_t src_port_to;
+       uint16_t dst_port_from;
+       uint16_t dst_port_to;
+       uint8_t proto;
+       uint8_t proto_mask;
+};
+
+/**
+ * Parse VFW delete rule CLI command.
+ * Delete rule from standby table.
+ * Store command to update standby table
+ * after applyruleset command is invoked.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_vfw_del_ip_parsed(void *parsed_result, __attribute__ ((unused))
+                      struct cmdline *cl, void *data)
+{
+       struct cmd_vfw_del_ip_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_vfw_key key;
+       struct app_pipeline_vfw_rule *command;
+       int status;
+
+       memset(&key, 0, sizeof(struct pipeline_vfw_key));
+
+       if (params->src_ip.family == AF_INET) {
+              key.type = PIPELINE_VFW_IPV4_5TUPLE;
+              key.key.ipv4_5tuple.src_ip = rte_bswap32((uint32_t)
+                                                  params->src_ip.addr.
+                                                  ipv4.s_addr);
+              key.key.ipv4_5tuple.src_ip_mask = params->src_ip_mask;
+              key.key.ipv4_5tuple.dst_ip = rte_bswap32((uint32_t)
+                                                  params->dst_ip.addr.
+                                                  ipv4.s_addr);
+              key.key.ipv4_5tuple.dst_ip_mask = params->dst_ip_mask;
+              key.key.ipv4_5tuple.src_port_from = params->src_port_from;
+              key.key.ipv4_5tuple.src_port_to = params->src_port_to;
+              key.key.ipv4_5tuple.dst_port_from = params->dst_port_from;
+              key.key.ipv4_5tuple.dst_port_to = params->dst_port_to;
+              key.key.ipv4_5tuple.proto = params->proto;
+              key.key.ipv4_5tuple.proto_mask = params->proto_mask;
+       }
+       if (params->src_ip.family == AF_INET6) {
+              key.type = PIPELINE_VFW_IPV6_5TUPLE;
+              memcpy(key.key.ipv6_5tuple.src_ip,
+                     params->src_ip.addr.ipv6.s6_addr,
+                     sizeof(params->src_ip.addr.ipv6.s6_addr));
+              key.key.ipv6_5tuple.src_ip_mask = params->src_ip_mask;
+              memcpy(key.key.ipv6_5tuple.dst_ip,
+                     params->dst_ip.addr.ipv6.s6_addr,
+                     sizeof(params->dst_ip.addr.ipv6.s6_addr));
+              key.key.ipv6_5tuple.dst_ip_mask = params->dst_ip_mask;
+              key.key.ipv6_5tuple.src_port_from = params->src_port_from;
+              key.key.ipv6_5tuple.src_port_to = params->src_port_to;
+              key.key.ipv6_5tuple.dst_port_from = params->dst_port_from;
+              key.key.ipv6_5tuple.dst_port_to = params->dst_port_to;
+              key.key.ipv6_5tuple.proto = params->proto;
+              key.key.ipv6_5tuple.proto_mask = params->proto_mask;
+       }
+
+       status = app_pipeline_vfw_delete_rule(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+
+       /* Store command to update standby tables after switchover */
+       command = rte_malloc(NULL, sizeof(*command), RTE_CACHE_LINE_SIZE);
+       if (command == NULL) {
+              printf("Cannot allocation command\n");
+              return;
+       }
+       memset(command, 0, sizeof(struct app_pipeline_vfw_rule));
+       memcpy(&command->key, &key, sizeof(key));
+       command->command = vfw_delete_command;
+       TAILQ_INSERT_TAIL(&vfw_commands, command, node);
+}
+
+cmdline_parse_token_string_t cmd_vfw_del_ip_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_del_ip_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_vfw_del_ip_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_del_ip_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_del_ip_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_del_ip_result,
+                      del_string, "del");
+
+cmdline_parse_token_ipaddr_t cmd_vfw_del_ip_src_ip =
+TOKEN_IPADDR_INITIALIZER(struct cmd_vfw_del_ip_result, src_ip);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_src_ip_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result, src_ip_mask,
+                    UINT32);
+
+cmdline_parse_token_ipaddr_t cmd_vfw_del_ip_dst_ip =
+TOKEN_IPADDR_INITIALIZER(struct cmd_vfw_del_ip_result, dst_ip);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_dst_ip_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result, dst_ip_mask,
+                    UINT32);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_src_port_from =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result,
+                    src_port_from, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_src_port_to =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result, src_port_to,
+                    UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_dst_port_from =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result,
+                    dst_port_from, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_dst_port_to =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result,
+                    dst_port_to, UINT16);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_proto =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result,
+                    proto, UINT8);
+
+cmdline_parse_token_num_t cmd_vfw_del_ip_proto_mask =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_del_ip_result, proto_mask,
+                    UINT8);
+
+cmdline_parse_inst_t cmd_vfw_del_ip = {
+       .f = cmd_vfw_del_ip_parsed,
+       .data = NULL,
+       .help_str = "VFW rule delete",
+       .tokens = {
+                 (void *)&cmd_vfw_del_ip_p_string,
+                 (void *)&cmd_vfw_del_ip_acl_string,
+                 (void *)&cmd_vfw_del_ip_del_string,
+                 (void *)&cmd_vfw_del_ip_src_ip,
+                 (void *)&cmd_vfw_del_ip_src_ip_mask,
+                 (void *)&cmd_vfw_del_ip_dst_ip,
+                 (void *)&cmd_vfw_del_ip_dst_ip_mask,
+                 (void *)&cmd_vfw_del_ip_src_port_from,
+                 (void *)&cmd_vfw_del_ip_src_port_to,
+                 (void *)&cmd_vfw_del_ip_dst_port_from,
+                 (void *)&cmd_vfw_del_ip_dst_port_to,
+                 (void *)&cmd_vfw_del_ip_proto,
+                 (void *)&cmd_vfw_del_ip_proto_mask,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw stats
+ */
+
+/**
+ * A structure defining the VFW stats command.
+ */
+struct cmd_vfw_stats_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t stats_string;
+};
+
+/**
+ * Display VFW and Connection Tracker stats to the console.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_vfw_stats_parsed(__attribute__ ((unused)) void *parsed_result,
+              __attribute__ ((unused)) struct cmdline *cl,
+              __attribute__ ((unused)) void *data)
+{
+       int i, j;
+       struct rte_VFW_counter_block vfw_counter_sums;
+       struct rte_CT_counter_block ct_counter_sums;
+       struct rte_CT_counter_block *ct_counters;
+       struct action_counter_block action_counter_sum[action_array_max];
+       uint64_t sum_pkts_drop_fw = 0;
+
+       memset(&vfw_counter_sums, 0, sizeof(vfw_counter_sums));
+       memset(&ct_counter_sums, 0, sizeof(ct_counter_sums));
+       memset(&action_counter_sum, 0, sizeof(action_counter_sum));
+
+       printf("VFW Stats\n");
+       for (i = 0; i <= rte_VFW_hi_counter_block_in_use; i++) {
+              struct rte_VFW_counter_block *vfw_ctrs =
+                  &rte_vfw_counter_table[i];
+              ct_counters = rte_vfw_counter_table[i].ct_counters;
+
+              uint64_t average_internal_time =
+                  vfw_ctrs->time_measurements ==
+                  0 ? 0 : vfw_ctrs->internal_time_sum /
+                  vfw_ctrs->time_measurements;
+              uint64_t average_external_time =
+                  vfw_ctrs->time_measurements ==
+                  0 ? 0 : vfw_ctrs->external_time_sum /
+                  vfw_ctrs->time_measurements;
+              uint64_t average_pkts_in_batch =
+                  vfw_ctrs->num_pkts_measurements ==
+                  0 ? 0 : vfw_ctrs->num_batch_pkts_sum /
+                  vfw_ctrs->num_pkts_measurements;
+              uint64_t pkts_drop_fw = vfw_ctrs->pkts_drop_ttl +
+                                   vfw_ctrs->pkts_drop_bad_size +
+                                   vfw_ctrs->pkts_drop_fragmented +
+                                   vfw_ctrs->pkts_drop_unsupported_type;
+
+              printf("{\"VFW_counters\" : {\"id\" : \"%s\", \" pkts_received\": %"
+                     PRIu64 ", \" pkts_fw_forwarded\": %"
+                     PRIu64 ", \" pkts_drop_fw\": %"
+                     PRIu64 ", \" pkts_acl_forwarded\": %"
+                     PRIu64 ", \"pkts_drop_without_rule\" : %"
+                     PRIu64 ", \"average_pkts_in_batch\" : %"
+                     PRIu64 ", \"average_internal_time_in_clocks\" : %"
+                     PRIu64 ", \"average_external_time_in_clocks\" : %"
+                     PRIu64 ", \"total_time_measures\" : %"
+                     PRIu32 ", \"ct_packets_forwarded\" : %"
+                     PRIu64 ", \"ct_packets_dropped\" : %"
+                     PRIu64 ", \"bytes_processed \": %"
+                     PRIu64 ", \"ct_sessions\" : {"
+                     "\"active\" : %" PRIu64 ", \"open_attempt\" : %"
+                     PRIu64 ", \"re-open_attempt\" : %"
+                     PRIu64 ", \"established\" : %"
+                     PRIu64 ", \"closed\" : %"
+                     PRIu64 ", \"timeout\" : %"
+                     PRIu64 "}, \"ct_drops\" : {"
+                     "\"out_of_window\" : %" PRIu64 ", \"invalid_conn\" : %"
+                     PRIu64 ", \"invalid_state_transition\" : %"
+                     PRIu64 " \"RST\" : %"
+                     PRIu64 "}}\n",
+                     vfw_ctrs->name,
+                     vfw_ctrs->pkts_received,
+                     vfw_ctrs->pkts_fw_forwarded,
+                     pkts_drop_fw,
+                     vfw_ctrs->pkts_acl_forwarded,
+                     vfw_ctrs->pkts_drop_without_rule,
+                     average_pkts_in_batch,
+                     average_internal_time,
+                     average_external_time,
+                     vfw_ctrs->time_measurements,
+                     ct_counters->pkts_forwarded,
+                     ct_counters->pkts_drop,
+                     vfw_ctrs->bytes_processed,
+                     ct_counters->current_active_sessions,
+                     ct_counters->sessions_activated,
+                     ct_counters->sessions_reactivated,
+                     ct_counters->sessions_established,
+                     ct_counters->sessions_closed,
+                     ct_counters->sessions_timedout,
+                     ct_counters->pkts_drop_outof_window,
+                     ct_counters->pkts_drop_invalid_conn,
+                     ct_counters->pkts_drop_invalid_state,
+                     ct_counters->pkts_drop_invalid_rst);
+
+              vfw_counter_sums.bytes_processed +=
+                  vfw_ctrs->bytes_processed;
+
+              vfw_counter_sums.internal_time_sum +=
+                  vfw_ctrs->internal_time_sum;
+              vfw_counter_sums.external_time_sum +=
+                  vfw_ctrs->external_time_sum;
+              vfw_counter_sums.time_measurements +=
+                  vfw_ctrs->time_measurements;
+
+              vfw_counter_sums.pkts_drop_ttl += vfw_ctrs->pkts_drop_ttl;
+              vfw_counter_sums.pkts_drop_bad_size +=
+                  vfw_ctrs->pkts_drop_bad_size;
+              vfw_counter_sums.pkts_drop_fragmented +=
+                  vfw_ctrs->pkts_drop_fragmented;
+              vfw_counter_sums.pkts_drop_unsupported_type +=
+                  vfw_ctrs->pkts_drop_unsupported_type;
+              vfw_counter_sums.pkts_drop_without_arp_entry +=
+                  vfw_ctrs->pkts_drop_without_arp_entry;
+
+              vfw_counter_sums.pkts_drop_without_rule +=
+                  vfw_ctrs->pkts_drop_without_rule;
+              vfw_counter_sums.pkts_received += vfw_ctrs->pkts_received;
+              vfw_counter_sums.pkts_fw_forwarded +=
+                     vfw_ctrs->pkts_fw_forwarded;
+              vfw_counter_sums.pkts_acl_forwarded +=
+                     vfw_ctrs->pkts_acl_forwarded;
+              sum_pkts_drop_fw += pkts_drop_fw;
+              ct_counter_sums.pkts_forwarded += ct_counters->pkts_forwarded;
+              ct_counter_sums.pkts_drop += ct_counters->pkts_drop;
+              ct_counter_sums.current_active_sessions +=
+                  ct_counters->current_active_sessions;
+              ct_counter_sums.sessions_activated +=
+                  ct_counters->sessions_activated;
+              ct_counter_sums.sessions_reactivated +=
+                  ct_counters->sessions_reactivated;
+              ct_counter_sums.sessions_established +=
+                  ct_counters->sessions_established;
+              ct_counter_sums.sessions_closed += ct_counters->sessions_closed;
+              ct_counter_sums.sessions_timedout +=
+                  ct_counters->sessions_timedout;
+              ct_counter_sums.pkts_drop_invalid_conn +=
+                  ct_counters->pkts_drop_invalid_conn;
+              ct_counter_sums.pkts_drop_invalid_state +=
+                  ct_counters->pkts_drop_invalid_state;
+              ct_counter_sums.pkts_drop_invalid_rst +=
+                  ct_counters->pkts_drop_invalid_rst;
+              ct_counter_sums.pkts_drop_outof_window +=
+                  ct_counters->pkts_drop_outof_window;
+
+       }
+
+       printf("VFW TOTAL: pkts_received: %"
+                     PRIu64 ", \"pkts_fw_forwarded\": %"
+                     PRIu64 ", \"pkts_drop_fw\": %"
+                     PRIu64 ", \"fw_drops\" : {"
+                     "\"TTL_zero\" : %" PRIu64 ", \"bad_size\" : %"
+                     PRIu64 ", \"fragmented_packet\" : %"
+                     PRIu64 ", \"unsupported_packet_types\" : %"
+                     PRIu64 ", \"no_arp_entry\" : %"
+                     PRIu64 "}, \"pkts_acl_forwarded\": %"
+                     PRIu64 ", \"pkts_drop_without_rule\": %"
+                     PRIu64 ", \"packets_last_sec\" : %"
+                     PRIu32 ", \"average_packets_per_sec\" : %"
+                     PRIu32 ", \"bytes_last_sec\" : %"
+                     PRIu32 ", \"average_bytes_per_sec\" : %"
+                     PRIu32 ", \"bytes_processed \": %"
+                     PRIu64 "\n",
+                     vfw_counter_sums.pkts_received,
+                     vfw_counter_sums.pkts_fw_forwarded,
+                     sum_pkts_drop_fw,
+                     vfw_counter_sums.pkts_drop_ttl,
+                     vfw_counter_sums.pkts_drop_bad_size,
+                     vfw_counter_sums.pkts_drop_fragmented,
+                     vfw_counter_sums.pkts_drop_unsupported_type,
+                     vfw_counter_sums.pkts_drop_without_arp_entry,
+                     vfw_counter_sums.pkts_acl_forwarded,
+                     vfw_counter_sums.pkts_drop_without_rule,
+                     rte_vfw_performance_measures.pkts_last_second,
+                     rte_vfw_performance_measures.ave_pkts_per_second,
+                     rte_vfw_performance_measures.bytes_last_second,
+                     rte_vfw_performance_measures.ave_bytes_per_second,
+                     vfw_counter_sums.bytes_processed);
+
+       printf("\"CT TOTAL: ct_packets_forwarded\" : %"
+                     PRIu64 ", \" ct_packets_dropped\" : %"
+                     PRIu64 ", \"ct_sessions\" : {"
+                     "\"active\" : %" PRIu64 ", \"open_attempt\" : %"
+                     PRIu64 ", \"re-open_attempt\" : %"
+                     PRIu64 ", \"established\" : %"
+                     PRIu64 ", \"closed\" : %"
+                     PRIu64 ", \"timeout\" : %"
+                     PRIu64 "}, \"ct_drops\" : {"
+                     "\"out_of_window\" : %" PRIu64 ", \"invalid_conn\" : %"
+                     PRIu64 ", \"invalid_state_transition\" : %"
+                     PRIu64 " \"RST\" : %"
+                     PRIu64 "}\n",
+                     ct_counter_sums.pkts_forwarded,
+                     ct_counter_sums.pkts_drop,
+                     ct_counter_sums.current_active_sessions,
+                     ct_counter_sums.sessions_activated,
+                     ct_counter_sums.sessions_reactivated,
+                     ct_counter_sums.sessions_established,
+                     ct_counter_sums.sessions_closed,
+                     ct_counter_sums.sessions_timedout,
+                     ct_counter_sums.pkts_drop_outof_window,
+                     ct_counter_sums.pkts_drop_invalid_conn,
+                     ct_counter_sums.pkts_drop_invalid_state,
+                     ct_counter_sums.pkts_drop_invalid_rst);
+
+       for (i = 0; i <= rte_VFW_hi_counter_block_in_use; i++) {
+              for (j = 0; j < action_array_max; j++) {
+                     if (action_array_active[j].
+                         action_bitmap & lib_acl_action_count) {
+                            action_counter_sum[j].packetCount +=
+                                action_counter_table[i][j].packetCount;
+                            action_counter_sum[j].byteCount +=
+                                action_counter_table[i][j].byteCount;
+                     }
+              }
+       }
+
+       for (j = 0; j < action_array_max; j++) {
+              if (action_array_active[j].action_bitmap & lib_acl_action_count)
+                     printf("Action ID: %02u, packetCount: %" PRIu64
+                            ", byteCount: %" PRIu64 "\n", j,
+                            action_counter_sum[j].packetCount,
+                            action_counter_sum[j].byteCount);
+       }
+}
+
+cmdline_parse_token_string_t cmd_vfw_stats_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_stats_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_stats_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_stats_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_stats_stats_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_stats_result,
+                      stats_string, "stats");
+
+cmdline_parse_inst_t cmd_vfw_stats = {
+       .f = cmd_vfw_stats_parsed,
+       .data = NULL,
+       .help_str = "VFW stats",
+       .tokens = {
+                 (void *)&cmd_vfw_stats_p_string,
+                 (void *)&cmd_vfw_stats_acl_string,
+                 (void *)&cmd_vfw_stats_stats_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw clearstats
+ */
+
+/**
+ * A structure defining the VFW clear stats command.
+ */
+struct cmd_vfw_clearstats_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t clearstats_string;
+};
+
+/**
+ * Clear VFW and Connection Tracker stats.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_clearstats_parsed(__attribute__ ((unused))
+                                    void *parsed_result,
+                                    __attribute__ ((unused))
+                                    struct cmdline *cl,
+                                    __attribute__ ((unused))
+                                    void *data)
+{
+       int i;
+       struct rte_CT_counter_block *ct_counters;
+
+       for (i = 0; i <= rte_VFW_hi_counter_block_in_use; i++) {
+              ct_counters = rte_vfw_counter_table[i].ct_counters;
+              rte_vfw_counter_table[i].bytes_processed = 0;
+              rte_vfw_counter_table[i].pkts_drop_without_rule = 0;
+              rte_vfw_counter_table[i].pkts_received = 0;
+              rte_vfw_counter_table[i].pkts_drop_ttl = 0;
+              rte_vfw_counter_table[i].pkts_drop_bad_size = 0;
+              rte_vfw_counter_table[i].pkts_drop_fragmented = 0;
+              rte_vfw_counter_table[i].pkts_drop_without_arp_entry = 0;
+              rte_vfw_counter_table[i].internal_time_sum = 0;
+              rte_vfw_counter_table[i].external_time_sum = 0;
+              rte_vfw_counter_table[i].time_measurements = 0;
+              rte_vfw_counter_table[i].ct_counters->pkts_forwarded = 0;
+              rte_vfw_counter_table[i].ct_counters->pkts_drop = 0;
+              rte_vfw_counter_table[i].pkts_fw_forwarded = 0;
+              rte_vfw_counter_table[i].pkts_acl_forwarded = 0;
+              ct_counters->current_active_sessions = 0;
+              ct_counters->sessions_activated = 0;
+              ct_counters->sessions_reactivated = 0;
+              ct_counters->sessions_established = 0;
+              ct_counters->sessions_closed = 0;
+              ct_counters->sessions_timedout = 0;
+              ct_counters->pkts_drop_invalid_conn = 0;
+              ct_counters->pkts_drop_invalid_state = 0;
+              ct_counters->pkts_drop_invalid_rst = 0;
+              ct_counters->pkts_drop_outof_window = 0;
+       }
+
+       memset(&action_counter_table, 0, sizeof(action_counter_table));
+       rte_vfw_reset_running_averages();
+}
+
+cmdline_parse_token_string_t cmd_vfw_clearstats_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearstats_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_clearstats_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearstats_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_clearstats_clearstats_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearstats_result,
+                      clearstats_string, "clearstats");
+
+cmdline_parse_inst_t cmd_vfw_clearstats = {
+       .f = cmd_vfw_clearstats_parsed,
+       .data = NULL,
+       .help_str = "VFW clearstats",
+       .tokens = {
+                 (void *)&cmd_vfw_clearstats_p_string,
+                 (void *)&cmd_vfw_clearstats_acl_string,
+                 (void *)&cmd_vfw_clearstats_clearstats_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw dbg
+ */
+
+/**
+ * A structure defining the VFW debug command.
+ */
+struct cmd_vfw_dbg_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t dbg_string;
+       uint8_t dbg;
+};
+
+/**
+ * Parse and handle VFW debug command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_dbg_parsed(void *parsed_result, __attribute__ ((unused))
+                              struct cmdline *cl, __rte_unused void *data)
+{
+
+        struct cmd_vfw_dbg_result *params = parsed_result;
+       if (params->dbg == 0) {
+              printf("DBG turned OFF\n");
+              VFW_DEBUG = 0;
+       } else if (params->dbg == 1) {
+              printf("DBG turned ON\n");
+              VFW_DEBUG = 1;
+       } else if (params->dbg == 2) {
+              printf("VFW IPV4 enabled\n");
+              printf("VFW IPV6 enabled\n");
+              vfw_ipv4_enabled = 1;
+              vfw_ipv6_enabled = 1;
+       } else if (params->dbg == 3) {
+              printf("VFW IPV4 enabled\n");
+              printf("VFW IPV6 disabled\n");
+              vfw_ipv4_enabled = 1;
+              vfw_ipv6_enabled = 0;
+       } else if (params->dbg == 4) {
+              printf("VFW IPV4 disabled\n");
+              printf("VFW IPV6 enabled\n");
+              vfw_ipv4_enabled = 0;
+              vfw_ipv6_enabled = 1;
+       } else if (params->dbg == 5) {
+              printf("VFW Version: 3.03\n");
+       } else
+              printf("Invalid DBG setting\n");
+}
+
+cmdline_parse_token_string_t cmd_vfw_dbg_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_dbg_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_dbg_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_dbg_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_dbg_dbg_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_dbg_result,
+                      dbg_string, "dbg");
+
+cmdline_parse_token_num_t cmd_vfw_dbg_dbg =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_dbg_result, dbg,
+                    UINT8);
+
+cmdline_parse_inst_t cmd_vfw_dbg = {
+       .f = cmd_vfw_dbg_parsed,
+       .data = NULL,
+       .help_str = "VFW dbg",
+       .tokens = {
+                 (void *)&cmd_vfw_dbg_p_string,
+                 (void *)&cmd_vfw_dbg_acl_string,
+                 (void *)&cmd_vfw_dbg_dbg_string,
+                 (void *)&cmd_vfw_dbg_dbg,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw clearrules
+ */
+
+/**
+ * A structure defining the VFW clear rules command.
+ */
+struct cmd_vfw_clearrules_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t clearrules_string;
+};
+
+/**
+ * Parse clear rule command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_clearrules_parsed(__attribute__ ((unused))
+                                    void *parsed_result,
+                                    __attribute__ ((unused))
+                                    struct cmdline *cl, void *data)
+{
+       struct app_params *app = data;
+       int status;
+
+       status = app_pipeline_vfw_clearrules(app);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_vfw_clearrules_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearrules_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_clearrules_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearrules_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_clearrules_clearrules_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_clearrules_result,
+                      clearrules_string, "clearrules");
+
+cmdline_parse_inst_t cmd_vfw_clearrules = {
+       .f = cmd_vfw_clearrules_parsed,
+       .data = NULL,
+       .help_str = "VFW clearrules",
+       .tokens = {
+                 (void *)&cmd_vfw_clearrules_p_string,
+                 (void *)&cmd_vfw_clearrules_acl_string,
+                 (void *)&cmd_vfw_clearrules_clearrules_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw ls
+ */
+
+/**
+ * A structure defining the VFW ls CLI command result.
+ */
+struct cmd_vfw_ls_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t ls_string;
+       uint32_t table_instance;
+};
+
+/**
+ * Parse VFW ls command to display rules to the console.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_ls_parsed(__attribute__ ((unused))
+                             void *parsed_result, __attribute__ ((unused))
+                             struct cmdline *cl, void *data)
+{
+       struct app_params *app = data;
+       struct cmd_vfw_ls_result *params = parsed_result;
+
+       app_pipeline_vfw_ls(app, params->table_instance);
+}
+
+cmdline_parse_token_string_t cmd_vfw_ls_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_ls_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_vfw_ls_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_ls_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_ls_ls_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_ls_result, ls_string,
+                      "ls");
+
+cmdline_parse_token_num_t cmd_vfw_ls_table_instance =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_ls_result, table_instance,
+                    UINT32);
+
+cmdline_parse_inst_t cmd_vfw_ls = {
+       .f = cmd_vfw_ls_parsed,
+       .data = NULL,
+       .help_str = "VFW rule list",
+       .tokens = {
+                 (void *)&cmd_vfw_ls_p_string,
+                 (void *)&cmd_vfw_ls_acl_string,
+                 (void *)&cmd_vfw_ls_ls_string,
+                 (void *)&cmd_vfw_ls_table_instance,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw applyruleset
+ */
+
+/**
+ * A structure defining the VFW apply ruleset command.
+ */
+struct cmd_vfw_applyruleset_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t applyruleset_string;
+};
+
+/**
+ * Parse VFW Apply Ruleset Command.
+ * Switchover active and standby tables.
+ * Sync newly standby tables to match updated data.
+ * Both VFW rule and VFW action tables updated.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_applyruleset_parsed(__attribute__ ((unused))
+                                      void *parsed_result,
+                                      __attribute__ ((unused))
+                                      struct cmdline *cl, void *data)
+{
+       struct app_params *app = data;
+       void *temp_ptr;
+       uint32_t *temp_count_ptr;
+       struct pipeline_action_key *action_array_temp_ptr;
+       int status;
+
+       printf("VFW Apply Ruleset\n");
+
+       /* Switchover Active and Standby TRIE rule tables */
+       temp_ptr = vfw_rule_table_ipv4_active;
+       vfw_rule_table_ipv4_active = vfw_rule_table_ipv4_standby;
+       vfw_rule_table_ipv4_standby = temp_ptr;
+       temp_ptr = vfw_rule_table_ipv6_active;
+       vfw_rule_table_ipv6_active = vfw_rule_table_ipv6_standby;
+       vfw_rule_table_ipv6_standby = temp_ptr;
+
+       /* Switchover tailq tables */
+       vfw_tailq_rules_temp_ptr = vfw_tailq_rules_ipv4_active;
+       vfw_tailq_rules_ipv4_active = vfw_tailq_rules_ipv4_standby;
+       vfw_tailq_rules_ipv4_standby = vfw_tailq_rules_temp_ptr;
+       vfw_tailq_rules_temp_ptr = vfw_tailq_rules_ipv6_active;
+       vfw_tailq_rules_ipv6_active = vfw_tailq_rules_ipv6_standby;
+       vfw_tailq_rules_ipv6_standby = vfw_tailq_rules_temp_ptr;
+       temp_count_ptr = vfw_n_tailq_rules_ipv4_active;
+       vfw_n_tailq_rules_ipv4_active = vfw_n_tailq_rules_ipv4_standby;
+       vfw_n_tailq_rules_ipv4_standby = temp_count_ptr;
+       temp_count_ptr = vfw_n_tailq_rules_ipv6_active;
+       vfw_n_tailq_rules_ipv6_active = vfw_n_tailq_rules_ipv6_standby;
+       vfw_n_tailq_rules_ipv6_standby = temp_count_ptr;
+
+       /* Switchover Active and Standby action table */
+       action_array_temp_ptr = action_array_active;
+       action_array_active = action_array_standby;
+       action_array_standby = action_array_temp_ptr;
+       /* Update Standby action table with all changes */
+       memcpy(action_array_standby, action_array_active, action_array_size);
+
+       /* Update Standby Rule Tables with all changes */
+       while (!TAILQ_EMPTY(&vfw_commands)) {
+              struct app_pipeline_vfw_rule *command;
+
+              command = TAILQ_FIRST(&vfw_commands);
+              TAILQ_REMOVE(&vfw_commands, command, node);
+
+              if (command->command == vfw_add_command) {
+                     status = app_pipeline_vfw_add_rule(app,
+                                                       &command->key,
+                                                       command->priority,
+                                                       command->port_id,
+                                                       command->
+                                                       action_id);
+              } else
+                     status =
+                         app_pipeline_vfw_delete_rule(app, &command->key);
+
+              if (status != 0) {
+                     printf("Command applyruleset add rule failed\n");
+                     rte_free(command);
+                     return;
+              }
+
+              rte_free(command);
+       }
+}
+
+cmdline_parse_token_string_t cmd_vfw_applyruleset_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_applyruleset_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_vfw_applyruleset_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_applyruleset_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_applyruleset_applyruleset_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_applyruleset_result,
+                      applyruleset_string,
+                      "applyruleset");
+
+cmdline_parse_inst_t cmd_vfw_applyruleset = {
+       .f = cmd_vfw_applyruleset_parsed,
+       .data = NULL,
+       .help_str = "VFW applyruleset",
+       .tokens = {
+                 (void *)&cmd_vfw_applyruleset_p_string,
+                 (void *)&cmd_vfw_applyruleset_acl_string,
+                 (void *)&cmd_vfw_applyruleset_applyruleset_string,
+                 NULL,
+                 },
+};
+/*
+ * p action add accept
+ */
+
+/**
+ * A structure defining the add accept action command.
+ */
+struct cmd_action_add_accept_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t accept_string;
+};
+
+/**
+ * Parse Accept Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_accept_parsed(void *parsed_result, __attribute__ ((unused))
+                          struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_accept_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_packet_accept;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_accept_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_accept_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_accept_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_accept_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_accept_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_accept_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_accept_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_accept_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_accept_accept_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_accept_result,
+                      accept_string, "accept");
+
+cmdline_parse_inst_t cmd_action_add_accept = {
+       .f = cmd_action_add_accept_parsed,
+       .data = NULL,
+       .help_str = "VFW action add accept",
+       .tokens = {
+                 (void *)&cmd_action_add_accept_p_string,
+                 (void *)&cmd_action_add_accept_action_string,
+                 (void *)&cmd_action_add_accept_add_string,
+                 (void *)&cmd_action_add_accept_action_id,
+                 (void *)&cmd_action_add_accept_accept_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del accept
+ */
+
+/**
+ * A structure defining the delete accept action command.
+ */
+struct cmd_action_del_accept_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t accept_string;
+};
+
+/**
+ * Parse Accept Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_accept_parsed(void *parsed_result, __attribute__ ((unused))
+                          struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_accept_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_packet_accept;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_accept_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_accept_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_accept_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_accept_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_accept_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_accept_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_accept_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_accept_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_accept_accept_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_accept_result,
+                      accept_string, "accept");
+
+cmdline_parse_inst_t cmd_action_del_accept = {
+       .f = cmd_action_del_accept_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete accept",
+       .tokens = {
+                 (void *)&cmd_action_del_accept_p_string,
+                 (void *)&cmd_action_del_accept_action_string,
+                 (void *)&cmd_action_del_accept_del_string,
+                 (void *)&cmd_action_del_accept_action_id,
+                 (void *)&cmd_action_del_accept_accept_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add drop
+ */
+
+/**
+ * A structure defining the add drop action command.
+ */
+struct cmd_action_add_drop_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t drop_string;
+};
+
+/**
+ * Parse Drop Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_drop_parsed(void *parsed_result, __attribute__ ((unused))
+                        struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_drop_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_packet_drop;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_drop_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_drop_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_drop_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_drop_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_drop_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_drop_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_drop_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_drop_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_drop_drop_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_drop_result,
+                      drop_string, "drop");
+
+cmdline_parse_inst_t cmd_action_add_drop = {
+       .f = cmd_action_add_drop_parsed,
+       .data = NULL,
+       .help_str = "VFW action add drop",
+       .tokens = {
+                 (void *)&cmd_action_add_drop_p_string,
+                 (void *)&cmd_action_add_drop_action_string,
+                 (void *)&cmd_action_add_drop_add_string,
+                 (void *)&cmd_action_add_drop_action_id,
+                 (void *)&cmd_action_add_drop_drop_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del drop
+ */
+
+/**
+ * A structure defining the delete drop action command.
+ */
+struct cmd_action_del_drop_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t drop_string;
+};
+
+/**
+ * Parse Drop Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_drop_parsed(void *parsed_result, __attribute__ ((unused))
+                        struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_drop_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_packet_drop;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_drop_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_drop_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_drop_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_drop_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_drop_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_drop_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_drop_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_drop_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_drop_drop_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_drop_result,
+                      drop_string, "drop");
+
+cmdline_parse_inst_t cmd_action_del_drop = {
+       .f = cmd_action_del_drop_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete drop",
+       .tokens = {
+                 (void *)&cmd_action_del_drop_p_string,
+                 (void *)&cmd_action_del_drop_action_string,
+                 (void *)&cmd_action_del_drop_del_string,
+                 (void *)&cmd_action_del_drop_action_id,
+                 (void *)&cmd_action_del_drop_drop_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add fwd
+ */
+
+/**
+ * A structure defining the add forward action command.
+ */
+struct cmd_action_add_fwd_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t fwd_string;
+       int32_t port_id;
+};
+
+/**
+ * Parse Forward Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_fwd_parsed(void *parsed_result, __attribute__ ((unused))
+                       struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_fwd_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_fwd;
+       key.fwd_port = params->port_id;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_fwd_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_fwd_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_fwd_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_fwd_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_fwd_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_fwd_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_fwd_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_fwd_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_fwd_fwd_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_fwd_result,
+                      fwd_string, "fwd");
+
+cmdline_parse_token_num_t cmd_action_add_fwd_port_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_fwd_result, port_id,
+                    UINT32);
+
+cmdline_parse_inst_t cmd_action_add_fwd = {
+       .f = cmd_action_add_fwd_parsed,
+       .data = NULL,
+       .help_str = "VFW action add fwd",
+       .tokens = {
+                 (void *)&cmd_action_add_fwd_p_string,
+                 (void *)&cmd_action_add_fwd_action_string,
+                 (void *)&cmd_action_add_fwd_add_string,
+                 (void *)&cmd_action_add_fwd_action_id,
+                 (void *)&cmd_action_add_fwd_fwd_string,
+                 (void *)&cmd_action_add_fwd_port_id,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del fwd
+ */
+
+/**
+ * A structure defining the delete forward action command.
+ */
+struct cmd_action_del_fwd_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t fwd_string;
+};
+
+/**
+ * Parse Forward Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_fwd_parsed(void *parsed_result, __attribute__ ((unused))
+                       struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_fwd_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_fwd;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_fwd_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_fwd_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_fwd_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_fwd_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_fwd_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_fwd_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_fwd_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_fwd_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_fwd_fwd_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_fwd_result,
+                      fwd_string, "fwd");
+
+cmdline_parse_inst_t cmd_action_del_fwd = {
+       .f = cmd_action_del_fwd_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete fwd",
+       .tokens = {
+                 (void *)&cmd_action_del_fwd_p_string,
+                 (void *)&cmd_action_del_fwd_action_string,
+                 (void *)&cmd_action_del_fwd_del_string,
+                 (void *)&cmd_action_del_fwd_action_id,
+                 (void *)&cmd_action_del_fwd_fwd_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add nat
+ */
+
+/**
+ * A structure defining the add NAT action command.
+ */
+struct cmd_action_add_nat_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t nat_string;
+       int32_t port_id;
+};
+
+/**
+ * Parse NAT Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_nat_parsed(void *parsed_result, __attribute__ ((unused))
+                       struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_nat_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_nat;
+       key.nat_port = params->port_id;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_nat_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_nat_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_nat_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_nat_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_nat_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_nat_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_nat_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_nat_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_nat_nat_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_nat_result,
+                      nat_string, "nat");
+
+cmdline_parse_token_num_t cmd_action_add_nat_port_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_nat_result, port_id,
+                    UINT32);
+
+cmdline_parse_inst_t cmd_action_add_nat = {
+       .f = cmd_action_add_nat_parsed,
+       .data = NULL,
+       .help_str = "VFW action add nat",
+       .tokens = {
+                 (void *)&cmd_action_add_nat_p_string,
+                 (void *)&cmd_action_add_nat_action_string,
+                 (void *)&cmd_action_add_nat_add_string,
+                 (void *)&cmd_action_add_nat_action_id,
+                 (void *)&cmd_action_add_nat_nat_string,
+                 (void *)&cmd_action_add_nat_port_id,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del nat
+ */
+
+/**
+ * A structure defining the delete NAT action command.
+ */
+struct cmd_action_del_nat_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t nat_string;
+};
+
+/**
+ * Parse NAT Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_nat_parsed(void *parsed_result, __attribute__ ((unused))
+                       struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_nat_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_nat;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_nat_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_nat_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_nat_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_nat_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_nat_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_nat_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_nat_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_nat_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_nat_nat_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_nat_result,
+                      nat_string, "nat");
+
+cmdline_parse_inst_t cmd_action_del_nat = {
+       .f = cmd_action_del_nat_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete nat",
+       .tokens = {
+                 (void *)&cmd_action_del_nat_p_string,
+                 (void *)&cmd_action_del_nat_action_string,
+                 (void *)&cmd_action_del_nat_del_string,
+                 (void *)&cmd_action_del_nat_action_id,
+                 (void *)&cmd_action_del_nat_nat_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add count
+ */
+
+/**
+ * A structure defining the add count action command.
+ */
+struct cmd_action_add_count_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t count_string;
+};
+
+/**
+ * Parse Count Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_count_parsed(void *parsed_result, __attribute__ ((unused))
+                         struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_count_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_count;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_count_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_count_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_count_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_count_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_count_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_count_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_count_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_count_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_count_count_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_count_result,
+                      count_string, "count");
+
+cmdline_parse_inst_t cmd_action_add_count = {
+       .f = cmd_action_add_count_parsed,
+       .data = NULL,
+       .help_str = "VFW action add count",
+       .tokens = {
+                 (void *)&cmd_action_add_count_p_string,
+                 (void *)&cmd_action_add_count_action_string,
+                 (void *)&cmd_action_add_count_add_string,
+                 (void *)&cmd_action_add_count_action_id,
+                 (void *)&cmd_action_add_count_count_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del count
+ */
+
+/**
+ * A structure defining the delete count action command.
+ */
+struct cmd_action_del_count_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t count_string;
+};
+
+/**
+ * Parse Count Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_count_parsed(void *parsed_result, __attribute__ ((unused))
+                         struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_count_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_count;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_count_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_count_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_count_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_count_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_count_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_count_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_count_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_count_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_count_count_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_count_result,
+                      count_string, "count");
+
+cmdline_parse_inst_t cmd_action_del_count = {
+       .f = cmd_action_del_count_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete count",
+       .tokens = {
+                 (void *)&cmd_action_del_count_p_string,
+                 (void *)&cmd_action_del_count_action_string,
+                 (void *)&cmd_action_del_count_del_string,
+                 (void *)&cmd_action_del_count_action_id,
+                 (void *)&cmd_action_del_count_count_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add dscp
+ */
+
+/**
+ * A structure defining the add DSCP action command.
+ */
+struct cmd_action_add_dscp_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t dscp_string;
+       uint8_t dscp_priority;
+};
+
+/**
+ * Parse DSCP Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_dscp_parsed(void *parsed_result, __attribute__ ((unused))
+                        struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_dscp_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_dscp;
+       key.dscp_priority = params->dscp_priority;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_dscp_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_dscp_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_dscp_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_dscp_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_dscp_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_dscp_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_dscp_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_dscp_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_dscp_dscp_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_dscp_result,
+                      dscp_string, "dscp");
+
+cmdline_parse_token_num_t cmd_action_add_dscp_dscp_priority =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_dscp_result, dscp_priority,
+                    UINT8);
+
+cmdline_parse_inst_t cmd_action_add_dscp = {
+       .f = cmd_action_add_dscp_parsed,
+       .data = NULL,
+       .help_str = "VFW action add dscp",
+       .tokens = {
+                 (void *)&cmd_action_add_dscp_p_string,
+                 (void *)&cmd_action_add_dscp_action_string,
+                 (void *)&cmd_action_add_dscp_add_string,
+                 (void *)&cmd_action_add_dscp_action_id,
+                 (void *)&cmd_action_add_dscp_dscp_string,
+                 (void *)&cmd_action_add_dscp_dscp_priority,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del dscp
+ */
+
+/**
+ * A structure defining the delete DSCP action command.
+ */
+struct cmd_action_del_dscp_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t dscp_string;
+};
+
+/**
+ * Parse DSCP Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_dscp_parsed(void *parsed_result, __attribute__ ((unused))
+                        struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_dscp_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_dscp;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_dscp_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_dscp_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_dscp_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_dscp_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_dscp_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_dscp_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_dscp_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_dscp_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_dscp_dscp_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_dscp_result,
+                      dscp_string, "dscp");
+
+cmdline_parse_inst_t cmd_action_del_dscp = {
+       .f = cmd_action_del_dscp_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete dscp",
+       .tokens = {
+                 (void *)&cmd_action_del_dscp_p_string,
+                 (void *)&cmd_action_del_dscp_action_string,
+                 (void *)&cmd_action_del_dscp_del_string,
+                 (void *)&cmd_action_del_dscp_action_id,
+                 (void *)&cmd_action_del_dscp_dscp_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add conntrack
+ */
+
+/**
+ * A structure defining the add Connection Tracking action command.
+ */
+struct cmd_action_add_conntrack_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t conntrack_string;
+};
+
+/**
+ * Parse Connection Tracking Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_conntrack_parsed(void *parsed_result, __attribute__ ((unused))
+                            struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_conntrack_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_conntrack;
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_conntrack_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_conntrack_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_conntrack_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_conntrack_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_conntrack_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_conntrack_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_conntrack_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_conntrack_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_conntrack_conntrack_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_conntrack_result,
+                      conntrack_string, "conntrack");
+
+cmdline_parse_inst_t cmd_action_add_conntrack = {
+       .f = cmd_action_add_conntrack_parsed,
+       .data = NULL,
+       .help_str = "VFW action add conntrack",
+       .tokens = {
+                 (void *)&cmd_action_add_conntrack_p_string,
+                 (void *)&cmd_action_add_conntrack_action_string,
+                 (void *)&cmd_action_add_conntrack_add_string,
+                 (void *)&cmd_action_add_conntrack_action_id,
+                 (void *)&cmd_action_add_conntrack_conntrack_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del conntrack
+ */
+
+/**
+ * A structure defining the delete Connection Tracking action command.
+ */
+struct cmd_action_del_conntrack_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t conntrack_string;
+};
+
+/**
+ * Parse Connection Tracking Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_conntrack_parsed(void *parsed_result, __attribute__ ((unused))
+                            struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_conntrack_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_conntrack;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_conntrack_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_conntrack_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_conntrack_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_conntrack_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_conntrack_del_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_conntrack_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_conntrack_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_conntrack_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_conntrack_conntrack_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_conntrack_result,
+                      conntrack_string, "conntrack");
+
+cmdline_parse_inst_t cmd_action_del_conntrack = {
+       .f = cmd_action_del_conntrack_parsed,
+       .data = NULL,
+       .help_str = "VFW action delete conntrack",
+       .tokens = {
+                 (void *)&cmd_action_del_conntrack_p_string,
+                 (void *)&cmd_action_del_conntrack_action_string,
+                 (void *)&cmd_action_del_conntrack_del_string,
+                 (void *)&cmd_action_del_conntrack_action_id,
+                 (void *)&cmd_action_del_conntrack_conntrack_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action add connexist
+ */
+
+/**
+ * A structure defining the add Connection Exist action command.
+ */
+struct cmd_action_add_connexist_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t add_string;
+       int32_t action_id;
+       cmdline_fixed_string_t connexist_string;
+       cmdline_fixed_string_t private_public_string;
+};
+
+/**
+ * Parse Connection Exist Action Add Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_add_connexist_parsed(void *parsed_result, __attribute__ ((unused))
+                            struct cmdline *cl, void *data)
+{
+       struct cmd_action_add_connexist_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       if (VFW_DEBUG)
+              printf("public_private: %s\n", params->private_public_string);
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_connexist;
+       if (strcmp(params->private_public_string, "prvpub") == 0)
+              key.private_public = lib_acl_private_public;
+       else if (strcmp(params->private_public_string, "pubprv") == 0)
+              key.private_public = lib_acl_public_private;
+       else {
+              printf("Command failed - Invalid string: %s\n",
+                     params->private_public_string);
+              return;
+       }
+
+       status = app_pipeline_action_add(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_add_connexist_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_connexist_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_add_connexist_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_connexist_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_add_connexist_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_connexist_result,
+                      add_string, "add");
+
+cmdline_parse_token_num_t cmd_action_add_connexist_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_add_connexist_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_add_connexist_connexist_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_connexist_result,
+                      connexist_string, "connexist");
+
+cmdline_parse_token_string_t cmd_action_add_connexist_private_public =
+TOKEN_STRING_INITIALIZER(struct cmd_action_add_connexist_result,
+                      private_public_string,
+                      NULL);
+
+cmdline_parse_inst_t cmd_action_add_connexist = {
+       .f = cmd_action_add_connexist_parsed,
+       .data = NULL,
+       .help_str = "VFW action add connexist",
+       .tokens = {
+                 (void *)&cmd_action_add_connexist_p_string,
+                 (void *)&cmd_action_add_connexist_action_string,
+                 (void *)&cmd_action_add_connexist_add_string,
+                 (void *)&cmd_action_add_connexist_action_id,
+                 (void *)&cmd_action_add_connexist_connexist_string,
+                 (void *)&cmd_action_add_connexist_private_public,
+                 NULL,
+                 },
+};
+
+/*
+ * p action del connexist
+ */
+
+/**
+ * A structure defining the delete Connection Exist action command.
+ */
+struct cmd_action_del_connexist_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t del_string;
+       int32_t action_id;
+       cmdline_fixed_string_t connexist_string;
+};
+
+/**
+ * Parse Connection Exist Action Delete Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_action_del_connexist_parsed(void *parsed_result, __attribute__ ((unused))
+                            struct cmdline *cl, void *data)
+{
+       struct cmd_action_del_connexist_result *params = parsed_result;
+       struct app_params *app = data;
+       struct pipeline_action_key key;
+       int status;
+
+       key.action_id = params->action_id;
+       key.action_bitmap = lib_acl_action_connexist;
+
+       status = app_pipeline_action_delete(app, &key);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_del_connexist_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_connexist_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_del_connexist_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_connexist_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_del_connexist_add_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_connexist_result,
+                      del_string, "del");
+
+cmdline_parse_token_num_t cmd_action_del_connexist_action_id =
+TOKEN_NUM_INITIALIZER(struct cmd_action_del_connexist_result, action_id,
+                    UINT32);
+
+cmdline_parse_token_string_t cmd_action_del_connexist_connexist_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_del_connexist_result,
+                      connexist_string, "connexist");
+
+cmdline_parse_inst_t cmd_action_del_connexist = {
+       .f = cmd_action_del_connexist_parsed,
+       .data = NULL,
+       .help_str = "VFW action del connexist",
+       .tokens = {
+                 (void *)&cmd_action_del_connexist_p_string,
+                 (void *)&cmd_action_del_connexist_action_string,
+                 (void *)&cmd_action_del_connexist_add_string,
+                 (void *)&cmd_action_del_connexist_action_id,
+                 (void *)&cmd_action_del_connexist_connexist_string,
+                 NULL,
+                 },
+};
+
+/*
+ * p action ls
+ */
+
+/**
+ * A structure defining the action ls command.
+ */
+struct cmd_action_ls_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t action_string;
+       cmdline_fixed_string_t ls_string;
+       uint32_t table_instance;
+};
+
+/**
+ * Parse Action LS Command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_action_ls_parsed(void *parsed_result, __attribute__ ((unused))
+              struct cmdline *cl, __attribute__ ((unused)) void *data)
+{
+       struct cmd_action_ls_result *params = parsed_result;
+       uint32_t action_bitmap, i, j;
+       uint8_t action_found = 0;
+       struct action_counter_block action_counter_sum;
+
+       if (params->table_instance == active_rule_table) {
+              printf("Active Action Table:\n");
+              printf("Action ID     Action\n");
+              printf("=========     ======\n");
+
+              for (i = 0; i < action_array_max; i++) {
+                     action_bitmap = action_array_active[i].action_bitmap;
+                     if (action_bitmap != 0) {
+                            action_found = 1;
+                            if (action_bitmap &
+                                          lib_acl_action_packet_accept)
+                                   printf("  %04u        Accept\n", i);
+                            if (action_bitmap & lib_acl_action_packet_drop)
+                                   printf("  %04u        Drop\n", i);
+                            if (action_bitmap & lib_acl_action_nat)
+                                   printf("  %04"PRIu32
+                                          "        NAT       NAT Port: %"
+                                          PRIu32"\n",
+                                          i,
+                                 action_array_active[i].nat_port);
+                            if (action_bitmap & lib_acl_action_fwd)
+                                   printf("  %04"PRIu32
+                                          "        FWD       FWD Port: %"
+                                          PRIu32"\n",
+                                          i,
+                                          action_array_active[i].
+                                          fwd_port);
+                            if (action_bitmap & lib_acl_action_count) {
+                                   action_counter_sum.packetCount = 0;
+                                   action_counter_sum.byteCount = 0;
+                                   for (j = 0; j <= (uint32_t)
+                                   rte_VFW_hi_counter_block_in_use;
+                                   j++) {
+                                          action_counter_sum.
+                                              packetCount +=
+                                              action_counter_table[j][i].
+                                              packetCount;
+                                          action_counter_sum.byteCount +=
+                                              action_counter_table[j][i].
+                                              byteCount;
+                                   }
+                                   printf("  %04"PRIu32
+                                          "       Count    Packet Count:%"
+                                          PRIu64 "   Byte Count: %" PRIu64
+                                          "\n", i,
+                                          action_counter_sum.packetCount,
+                                          action_counter_sum.byteCount);
+                            }
+                            if (action_bitmap & lib_acl_action_conntrack)
+                                   printf("  %04u        Conntrack\n", i);
+                            if (action_bitmap & lib_acl_action_connexist) {
+                                   printf("  %04u        Connexist", i);
+                                   if (action_array_active[i].
+                                       private_public ==
+                                       lib_acl_private_public)
+                                          printf(" prvpub\n");
+                                   else
+                                          printf(" pubprv\n");
+                            }
+                            if (action_bitmap & lib_acl_action_dscp)
+                                   printf
+                                       ("  %04"PRIu32
+                                        "        DSCP     DSCP Priority: %"
+                                        PRIu8"\n",
+                                        i,
+                                        action_array_active[i].
+                                        dscp_priority);
+                     }
+              }
+
+              if (!action_found)
+                     printf("None\n");
+
+       } else {
+              action_found = 0;
+              printf("Standby Action Table:\n");
+              printf("Action ID     Action\n");
+              printf("=========     ======\n");
+
+              for (i = 0; i < action_array_max; i++) {
+                     action_bitmap = action_array_standby[i].action_bitmap;
+                     if (action_bitmap != 0) {
+                            action_found = 1;
+                            if (action_bitmap &
+                                          lib_acl_action_packet_accept)
+                                   printf("  %04u        Accept\n", i);
+                            if (action_bitmap & lib_acl_action_packet_drop)
+                                   printf("  %04u        Drop\n", i);
+                            if (action_bitmap & lib_acl_action_nat)
+                                   printf
+                                          ("  %04"PRIu32
+                                           "        NAT       NAT Port: %"
+                                           PRIu32"\n", i,
+                                           action_array_standby[i].
+                                           nat_port);
+                            if (action_bitmap & lib_acl_action_fwd)
+                                   printf
+                                          ("  %04"PRIu32
+                                           "        FWD       FWD Port: %"
+                                           PRIu32"\n", i,
+                                           action_array_standby[i].
+                                           fwd_port);
+                            if (action_bitmap & lib_acl_action_count)
+                                   printf("  %04u        Count\n", i);
+                            if (action_bitmap & lib_acl_action_conntrack)
+                                   printf("  %04u        Conntrack\n", i);
+                            if (action_bitmap & lib_acl_action_connexist) {
+                                   printf("  %04u        Connexist", i);
+                                   if (action_array_standby[i].
+                                       private_public ==
+                                       lib_acl_private_public)
+                                          printf(" prvpub\n");
+                                   else
+                                          printf(" pubprv\n");
+                            }
+                            if (action_bitmap & lib_acl_action_dscp)
+                                   printf("  %04"PRIu32
+                                    "       DSCP     DSCP Priority: %"
+                                    PRIu8"\n", i,
+                                    action_array_standby[i].
+                                    dscp_priority);
+                     }
+              }
+
+              if (!action_found)
+                     printf("None\n");
+       }
+}
+
+cmdline_parse_token_string_t cmd_action_ls_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_ls_result, p_string,
+                      "p");
+
+cmdline_parse_token_string_t cmd_action_ls_action_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_ls_result,
+                      action_string, "action");
+
+cmdline_parse_token_string_t cmd_action_ls_ls_string =
+TOKEN_STRING_INITIALIZER(struct cmd_action_ls_result, ls_string,
+                      "ls");
+
+cmdline_parse_token_num_t cmd_action_ls_table_instance =
+TOKEN_NUM_INITIALIZER(struct cmd_action_ls_result, table_instance,
+                    UINT32);
+
+cmdline_parse_inst_t cmd_action_ls = {
+       .f = cmd_action_ls_parsed,
+       .data = NULL,
+       .help_str = "VFW action list",
+       .tokens = {
+                 (void *)&cmd_action_ls_p_string,
+                 (void *)&cmd_action_ls_action_string,
+                 (void *)&cmd_action_ls_ls_string,
+                 (void *)&cmd_action_ls_table_instance,
+                 NULL,
+                 },
+};
+
+/*
+ * p vfw onesectimer start/stop
+ */
+
+/**
+ * A structure defining the VFW Dump Counter start/stop command.
+ */
+struct cmd_vfw_per_sec_ctr_dump_result {
+       cmdline_fixed_string_t p_string;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t per_sec_ctr_dump_string;
+       cmdline_fixed_string_t stop_string;
+};
+
+/**
+ * Parse Dump Counter Start Command.
+ * Start timer to display stats to console at regular intervals.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+       cmd_vfw_per_sec_ctr_dump_start_parsed(
+                     __attribute__ ((unused)) void *parsed_result,
+                     __attribute__ ((unused))
+                     struct cmdline *cl, __attribute__ ((unused)) void *data)
+{
+       rte_vfw_reset_running_averages();
+       /* execute timeout on current core */
+       uint32_t core_id = rte_lcore_id();
+       int success =
+           rte_timer_reset(&rte_vfw_one_second_timer,
+                         rte_vfw_ticks_in_one_second, PERIODICAL, core_id,
+                         rte_dump_vfw_counters_from_master, NULL);
+        if (success < 0)
+              printf("CNXN_TRACKER: Failed to set connection timer.\n");
+}
+
+/**
+ * Parse Dump Counter Stop Command.
+ * Stop timer that was started to display stats.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+       cmd_vfw_per_sec_ctr_dump_stop_parsed(
+                     __attribute__ ((unused)) void *parsed_result,
+                     __attribute__ ((unused))
+                     struct cmdline *cl, __attribute__ ((unused)) void *data)
+{
+       rte_timer_stop(&rte_vfw_one_second_timer);
+}
+
+cmdline_parse_token_string_t cmd_vfw_per_sec_ctr_dump_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_per_sec_ctr_dump_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_per_sec_ctr_dump_acl_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_per_sec_ctr_dump_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_per_sec_ctr_dump_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_per_sec_ctr_dump_result,
+                      per_sec_ctr_dump_string, "counterdump");
+
+cmdline_parse_token_string_t cmd_vfw_stop_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_per_sec_ctr_dump_result,
+                      stop_string, "stop");
+
+cmdline_parse_token_string_t cmd_vfw_start_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_per_sec_ctr_dump_result,
+                      stop_string, "start");
+
+cmdline_parse_inst_t cmd_vfw_per_sec_ctr_dump_stop = {
+       .f = cmd_vfw_per_sec_ctr_dump_stop_parsed,
+       .data = NULL,
+       .help_str = "VFW counterdump stop",
+       .tokens = {
+                 (void *)&cmd_vfw_per_sec_ctr_dump_p_string,
+                 (void *)&cmd_vfw_per_sec_ctr_dump_acl_string,
+                 (void *)&cmd_vfw_per_sec_ctr_dump_string,
+                 (void *)&cmd_vfw_stop_string,
+                 NULL,
+                 },
+};
+
+cmdline_parse_inst_t cmd_vfw_per_sec_ctr_dump_start = {
+       .f = cmd_vfw_per_sec_ctr_dump_start_parsed,
+       .data = NULL,
+       .help_str = "VFW counterdump start",
+       .tokens = {
+                 (void *)&cmd_vfw_per_sec_ctr_dump_p_string,
+                 (void *)&cmd_vfw_per_sec_ctr_dump_acl_string,
+                 (void *)&cmd_vfw_per_sec_ctr_dump_string,
+                 (void *)&cmd_vfw_start_string,
+                 NULL,
+                 },
+};
+
+/**
+ * A structure defining the VFW firewall ON/OFF command.
+ */
+struct cmd_vfw_firewall_flag_result {
+       cmdline_fixed_string_t p_string;
+       uint32_t pipeline_id;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t firewall_flag_string;
+       uint8_t firewall_flag;
+};
+
+/**
+ * Parse VFW Firewall ON/OFF CLI command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_vfw_firewall_flag_parsed(void *parsed_result, __attribute__ ((unused))
+              struct cmdline *cl, __attribute__ ((unused)) void *data)
+{
+       struct cmd_vfw_firewall_flag_result *params = parsed_result;
+
+       if (params->firewall_flag == 0) {
+              printf("firewall turned OFF\n");
+              firewall_flag = 0;
+       } else if (params->firewall_flag == 1) {
+              printf("firewall turned ON\n");
+              firewall_flag = 1;
+       } else
+              printf("Invalid firewall setting\n");
+}
+
+cmdline_parse_token_string_t cmd_vfw_firewall_flag_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_firewall_flag_result,
+                      p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_firewall_flag_vfw_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_firewall_flag_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_firewall_flag_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_firewall_flag_result,
+                      firewall_flag_string, "firewall");
+
+cmdline_parse_token_num_t cmd_vfw_firewall_flag =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_firewall_flag_result, firewall_flag,
+                    UINT8);
+
+cmdline_parse_inst_t cmd_vfw_firewall = {
+       .f = cmd_vfw_firewall_flag_parsed,
+       .data = NULL,
+       .help_str = "VFW firewall_flag",
+       .tokens = {
+                 (void *)&cmd_vfw_firewall_flag_p_string,
+                 (void *)&cmd_vfw_firewall_flag_vfw_string,
+                 (void *)&cmd_vfw_firewall_flag_string,
+                 (void *)&cmd_vfw_firewall_flag,
+                 NULL,
+                 },
+};
+
+
+/**
+ * A structure defining the TCPFW conntrack ON/OFF command.
+ */
+struct cmd_vfw_fw_conntrack_result {
+       cmdline_fixed_string_t p_string;
+       uint32_t pipeline_id;
+       cmdline_fixed_string_t tcpfw_string;
+       cmdline_fixed_string_t conntrack_string;
+       uint8_t conntrack_flag;
+};
+
+/**
+ * Parse VFW_TCPFW conntrack ON/OFF CLI command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void cmd_vfw_fw_conntrack_parsed(
+              void *parsed_result,
+              __attribute__((unused)) struct cmdline *cl,
+              __rte_unused void *data)
+{
+       struct cmd_vfw_fw_conntrack_result *params = parsed_result;
+
+       if (params->conntrack_flag == 0) {
+              printf("firewall conntrack turned OFF\n");
+              cnxn_tracking_is_active = 0;
+       } else if (params->conntrack_flag == 1) {
+              printf("firewall conntrack turned ON\n");
+              cnxn_tracking_is_active = 1;
+       } else
+              printf("Invalid firewall conntrack setting\n");
+
+}
+cmdline_parse_token_string_t cmd_vfw_fw_conntrack_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_fw_conntrack_result,
+              p_string, "p");
+
+cmdline_parse_token_string_t cmd_vfw_fw_conntrack_fw_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_fw_conntrack_result,
+              tcpfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_fw_conntrack_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_fw_conntrack_result,
+              conntrack_string, "conntrack");
+
+cmdline_parse_token_num_t cmd_vfw_fw_conntrack_flag =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_fw_conntrack_result, conntrack_flag,
+              UINT8);
+cmdline_parse_inst_t cmd_vfw_fw_conntrack = {
+       .f = cmd_vfw_fw_conntrack_parsed,
+       .data = NULL,
+       .help_str = "VFW FW conntrack",
+       .tokens = {
+              (void *) &cmd_vfw_fw_conntrack_p_string,
+              (void *) &cmd_vfw_fw_conntrack_fw_string,
+              (void *) &cmd_vfw_fw_conntrack_string,
+              (void *) &cmd_vfw_fw_conntrack_flag,
+              NULL,
+       },
+};
+
+/**
+ * A structure defining the VFW synproxy ON/OFFcommand.
+ */
+struct cmd_vfw_synproxy_flag_result {
+       cmdline_fixed_string_t p_string;
+       uint32_t pipeline_id;
+       cmdline_fixed_string_t vfw_string;
+       cmdline_fixed_string_t synproxy_flag_string;
+       uint8_t synproxy_flag;
+};
+
+/**
+ * Parse TCPFW synproxy ON/OFF CLI command.
+ *
+ * @param parsed_result
+ *  A pointer to CLI command parsed result.
+ * @param cl
+ *  A pointer to command line context.
+ * @param data
+ *  A pointer to command specific data
+ *
+ */
+static void
+cmd_vfw_synproxy_flag_parsed(void *parsed_result, __attribute__ ((unused))
+                            struct cmdline *cl, void *data)
+{
+       struct cmd_vfw_synproxy_flag_result *params = parsed_result;
+       struct app_params *app = data;
+       int status;
+
+       status = app_pipeline_vfw_synproxy_flag(app,
+                                             params->pipeline_id,
+                                             params->synproxy_flag);
+
+       if (status != 0) {
+              printf("Command failed\n");
+              return;
+       }
+}
+
+cmdline_parse_token_string_t cmd_vfw_synproxy_flag_p_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_synproxy_flag_result,
+                      p_string, "p");
+
+cmdline_parse_token_num_t cmd_vfw_synproxy_flag_pipeline_id =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_synproxy_flag_result,
+                    pipeline_id, UINT32);
+
+cmdline_parse_token_string_t cmd_vfw_synproxy_flag_vfw_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_synproxy_flag_result,
+                      vfw_string, "vfw");
+
+cmdline_parse_token_string_t cmd_vfw_synproxy_flag_string =
+TOKEN_STRING_INITIALIZER(struct cmd_vfw_synproxy_flag_result,
+                      synproxy_flag_string, "synproxy");
+
+cmdline_parse_token_num_t cmd_vfw_synproxy_flag =
+TOKEN_NUM_INITIALIZER(struct cmd_vfw_synproxy_flag_result, synproxy_flag,
+              UINT8);
+
+cmdline_parse_inst_t cmd_vfw_synproxy = {
+       .f = cmd_vfw_synproxy_flag_parsed,
+       .data = NULL,
+       .help_str = "VFW synproxy_flag",
+       .tokens = {
+              (void *)&cmd_vfw_synproxy_flag_p_string,
+              (void *)&cmd_vfw_synproxy_flag_pipeline_id,
+              (void *)&cmd_vfw_synproxy_flag_vfw_string,
+              (void *)&cmd_vfw_synproxy_flag_string,
+              (void *)&cmd_vfw_synproxy_flag,
+              NULL,
+       },
+};
+
+static cmdline_parse_ctx_t pipeline_cmds[] = {
+#ifdef ACL_ENABLE
+       (cmdline_parse_inst_t *) &cmd_vfw_add_ip,
+       (cmdline_parse_inst_t *) &cmd_vfw_del_ip,
+       (cmdline_parse_inst_t *) &cmd_vfw_dbg,
+       (cmdline_parse_inst_t *) &cmd_vfw_clearrules,
+       (cmdline_parse_inst_t *) &cmd_loadrules,
+       (cmdline_parse_inst_t *) &cmd_vfw_ls,
+       (cmdline_parse_inst_t *) &cmd_action_add_accept,
+       (cmdline_parse_inst_t *) &cmd_action_del_accept,
+       (cmdline_parse_inst_t *) &cmd_action_add_drop,
+       (cmdline_parse_inst_t *) &cmd_action_del_drop,
+       (cmdline_parse_inst_t *) &cmd_action_add_fwd,
+       (cmdline_parse_inst_t *) &cmd_action_del_fwd,
+       (cmdline_parse_inst_t *) &cmd_action_add_nat,
+       (cmdline_parse_inst_t *) &cmd_action_del_nat,
+       (cmdline_parse_inst_t *) &cmd_action_add_count,
+       (cmdline_parse_inst_t *) &cmd_action_del_count,
+       (cmdline_parse_inst_t *) &cmd_action_add_dscp,
+       (cmdline_parse_inst_t *) &cmd_action_del_dscp,
+       (cmdline_parse_inst_t *) &cmd_action_add_conntrack,
+       (cmdline_parse_inst_t *) &cmd_action_del_conntrack,
+       (cmdline_parse_inst_t *) &cmd_action_add_connexist,
+       (cmdline_parse_inst_t *) &cmd_action_del_connexist,
+       (cmdline_parse_inst_t *) &cmd_action_ls,
+       (cmdline_parse_inst_t *) &cmd_vfw_applyruleset,
+#endif
+       (cmdline_parse_inst_t *) &cmd_vfw_stats,
+       (cmdline_parse_inst_t *) &cmd_vfw_clearstats,
+       (cmdline_parse_inst_t *) &cmd_vfw_per_sec_ctr_dump_stop,
+       (cmdline_parse_inst_t *) &cmd_vfw_per_sec_ctr_dump_start,
+       (cmdline_parse_inst_t *) &cmd_vfw_synproxy,
+       (cmdline_parse_inst_t *) &cmd_vfw_firewall,
+#ifndef ACL_ENABLE
+       (cmdline_parse_inst_t *) &cmd_vfw_fw_conntrack,
+#endif
+       NULL,
+};
+
+static struct pipeline_fe_ops pipeline_vfw_fe_ops = {
+       .f_init = app_pipeline_vfw_init,
+       .f_free = app_pipeline_vfw_free,
+       .cmds = pipeline_cmds,
+};
+
+struct pipeline_type pipeline_vfw = {
+       .name = "VFW",
+       .be_ops = &pipeline_vfw_be_ops,
+       .fe_ops = &pipeline_vfw_fe_ops,
+};
diff --git a/VNFs/vFW/pipeline/pipeline_vfw.h b/VNFs/vFW/pipeline/pipeline_vfw.h
new file mode 100644 (file)
index 0000000..3b1b25f
--- /dev/null
@@ -0,0 +1,145 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+#ifndef __INCLUDE_PIPELINE_VFW_H__
+#define __INCLUDE_PIPELINE_VFW_H__
+
+/**
+ * @file
+ * Pipeline VFW FE.
+ *
+ * Pipeline VFW Front End (FE).
+ * Runs on the Master pipeline, responsible for CLI commands.
+ *
+ */
+
+#include "pipeline.h"
+#include "app.h"
+#include "pipeline_vfw_be.h"
+
+/* VFW IPV4 and IPV6 enable flags for debugging (Default both on) */
+extern int vfw_ipv4_enabled;
+extern int vfw_ipv6_enabled;
+
+/* Number of VFW Rules, default 4 * 1024 */
+extern uint32_t vfw_n_rules;
+/* VFW Rule Table TRIE - 2 (Active, Standby Global table per ipv4, ipv6 */
+extern void *vfw_rule_table_ipv4_active;
+extern void *vfw_rule_table_ipv4_standby;
+extern void *vfw_rule_table_ipv6_active;
+extern void *vfw_rule_table_ipv6_standby;
+
+#define active_rule_table       0
+#define standby_rule_table       1
+#define vfw_add_command              0
+#define vfw_delete_command       1
+#define IPV6_32BIT_LENGTH       4
+
+/**
+ * Add VFW rule to the VFW rule table.
+ * Rules are added standby table.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules can be added.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the VFW rule to add.
+ * @param priority
+ *  Priority of the VFW rule.
+ * @param port_id
+ *  Port ID of the VFW rule.
+ * @param action_id
+ *  Action ID of the VFW rule. Defined in Action Table.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_vfw_add_rule(struct app_params *app,
+                          struct pipeline_vfw_key *key,
+                          uint32_t priority,
+                          uint32_t port_id, uint32_t action_id);
+
+/**
+ * Delete VFW rule from the VFW rule table.
+ * Rules deleted from standby tables.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules can be deleted.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the VFW rule to delete.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_vfw_delete_rule(struct app_params *app,
+                            struct pipeline_vfw_key *key);
+
+/**
+ * Clear all VFW rules from the VFW rule table.
+ * Rules cleared from standby tables.
+ * Applyruleset command will activate the change.
+ * Both IPv4 and IPv6 rules will be cleared.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int app_pipeline_vfw_clearrules(struct app_params *app);
+
+/**
+ * Add Action to the Action table.
+ * Actions are added standby table.
+ * Applyruleset command will activate the change.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the Action to add.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_action_add(struct app_params *app,
+                     struct pipeline_action_key *key);
+
+/**
+ * Delete Action from the Action table.
+ * Actions are deleted from the standby table.
+ * Applyruleset command will activate the change.
+ *
+ * @param app
+ *  A pointer to the VFW pipeline parameters.
+ * @param key
+ *  A pointer to the Action to delete.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+int
+app_pipeline_action_delete(struct app_params *app,
+                        struct pipeline_action_key *key);
+
+extern struct pipeline_type pipeline_vfw;
+
+#endif
diff --git a/VNFs/vFW/pipeline/pipeline_vfw_be.c b/VNFs/vFW/pipeline/pipeline_vfw_be.c
new file mode 100644 (file)
index 0000000..0d3f527
--- /dev/null
@@ -0,0 +1,3674 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+/**
+ * @file
+ * Pipeline VFW BE Implementation.
+ *
+ * Implementation of Pipeline VFW Back End (BE).
+ * Responsible for packet processing.
+ *
+ */
+
+#define EN_SWP_ACL 1
+#define EN_SWP_ARP 1
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <stddef.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_ip.h>
+#include <rte_udp.h>
+#include <rte_icmp.h>
+#include <rte_byteorder.h>
+
+#include <rte_table_lpm.h>
+#include <rte_table_hash.h>
+#include <rte_table_array.h>
+#include <rte_table_acl.h>
+#include <rte_table_stub.h>
+#include <rte_timer.h>
+#include <rte_cycles.h>
+#include <rte_pipeline.h>
+#include <rte_spinlock.h>
+#include <rte_prefetch.h>
+#include "pipeline_actions_common.h"
+#include "hash_func.h"
+#include "pipeline_vfw.h"
+#include "pipeline_vfw_be.h"
+#include "rte_cnxn_tracking.h"
+#include "pipeline_arpicmp_be.h"
+#include "vnf_common.h"
+#include "vnf_define.h"
+
+#include "lib_arp.h"
+#include "lib_icmpv6.h"
+#include "pipeline_common_fe.h"
+
+uint32_t timer_lcore;
+
+uint8_t firewall_flag = 1;
+uint8_t VFW_DEBUG;
+uint8_t cnxn_tracking_is_active = 1;
+/**
+ * A structure defining the VFW pipeline input port per thread data.
+ */
+struct vfw_ports_in_args {
+       struct pipeline *pipe;
+       struct rte_ct_cnxn_tracker *cnxn_tracker;
+} __rte_cache_aligned;
+/**
+ * A structure defining the VFW pipeline per thread data.
+ */
+struct pipeline_vfw {
+       struct pipeline pipe;
+       pipeline_msg_req_handler custom_handlers[PIPELINE_VFW_MSG_REQS];
+
+       struct rte_ct_cnxn_tracker *cnxn_tracker;
+       struct rte_VFW_counter_block *counters;
+       struct rte_mbuf *pkt_buffer[PKT_BUFFER_SIZE];
+       struct lib_acl *plib_acl;
+       /* timestamp retrieved during in-port computations */
+       uint32_t n_flows;
+       uint8_t pipeline_num;
+       uint8_t traffic_type;
+       uint8_t links_map[PIPELINE_MAX_PORT_IN];
+       uint8_t outport_id[PIPELINE_MAX_PORT_IN];
+       /* Local ARP & ND Tables */
+       struct lib_arp_route_table_entry
+              local_lib_arp_route_table[MAX_ARP_RT_ENTRY];
+       uint8_t local_lib_arp_route_ent_cnt;
+       struct lib_nd_route_table_entry
+              local_lib_nd_route_table[MAX_ND_RT_ENTRY];
+       uint8_t local_lib_nd_route_ent_cnt;
+
+} __rte_cache_aligned;
+/**
+ * A structure defining the mbuf meta data for VFW.
+ */
+struct mbuf_tcp_meta_data {
+/* output port stored for RTE_PIPELINE_ACTION_PORT_META */
+       uint32_t output_port;
+       struct rte_mbuf *next;       /* next pointer for chained buffers */
+} __rte_cache_aligned;
+
+#define DONT_CARE_TCP_PACKET 0
+#define IS_NOT_TCP_PACKET 0
+#define IS_TCP_PACKET 1
+
+#define META_DATA_OFFSET 128
+
+#define RTE_PKTMBUF_HEADROOM 128       /* where is this defined ? */
+#define ETHERNET_START (META_DATA_OFFSET + RTE_PKTMBUF_HEADROOM)
+#define ETH_HDR_SIZE 14
+#define PROTOCOL_START (IP_START + 9)
+
+#define TCP_START (IP_START + 20)
+#define RTE_LB_PORT_OFFSET 204       /* TODO: Need definition in LB header */
+#define TCP_START_IPV6 (IP_START + 40)
+#define PROTOCOL_START_IPV6 (IP_START + 6)
+#define IP_HDR_DSCP_OFST 1
+
+#define TCP_PROTOCOL 6
+#define UDP_PROTOCOL 17
+
+#define DELETE_BUFFERED_PACKETS 0
+#define FORWARD_BUFFERED_PACKETS 1
+#define DO_ARP 1
+#define NO_ARP 0
+
+#define IPv4_HEADER_SIZE 20
+#define IPv6_HEADER_SIZE 40
+
+#define IP_VERSION_4 4
+#define IP_VERSION_6 6
+#define MIX 10
+/* IPv6 */
+#define IP_HDR_SIZE_IPV6  40
+#define IP_HDR_DSCP_OFST_IPV6 0
+#define IP_HDR_LENGTH_OFST_IPV6 4
+#define IP_HDR_PROTOCOL_OFST_IPV6 6
+#define IP_HDR_DST_ADR_OFST_IPV6 24
+#define MAX_NUM_LOCAL_MAC_ADDRESS 16
+/** The counter table for VFW pipeline per thread data.*/
+struct rte_VFW_counter_block rte_vfw_counter_table[MAX_VFW_INSTANCES]
+__rte_cache_aligned;
+int rte_VFW_hi_counter_block_in_use = -1;
+
+/* a spin lock used during vfw initialization only */
+rte_spinlock_t rte_VFW_init_lock = RTE_SPINLOCK_INITIALIZER;
+
+/* Action Array */
+struct pipeline_action_key *action_array_a;
+struct pipeline_action_key *action_array_b;
+struct pipeline_action_key *action_array_active;
+struct pipeline_action_key *action_array_standby;
+uint32_t action_array_size;
+struct action_counter_block
+action_counter_table[MAX_VFW_INSTANCES][action_array_max]
+__rte_cache_aligned;
+/*
+  * Pipeline table strategy for firewall. Unfortunately, there does not seem to
+  * be any use for the built-in table lookup of ip_pipeline for the firewall.
+  * The main table requirement of the firewall is the hash table to maintain
+  * connection info, but that is implemented seperately in the connection
+  * tracking library. So a "dummy" table lookup will be performed.
+  * TODO: look into "stub" table and see if that can be used
+  * to avoid useless table lookup
+  */
+/***** ARP local cache *****/
+uint8_t link_hw_laddr_valid[MAX_NUM_LOCAL_MAC_ADDRESS] = {
+       0, 0, 0, 0, 0, 0, 0, 0,
+       0, 0, 0, 0, 0, 0, 0, 0
+};
+
+static struct ether_addr link_hw_laddr[MAX_NUM_LOCAL_MAC_ADDRESS] = {
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} },
+       {.addr_bytes = {0x00, 0x00, 0x00, 0x00, 0x00, 0x00} }
+};
+
+/* Start TSC measurement */
+/* Prefetch counters and pipe before this function */
+static inline void start_tsc_measure(struct pipeline_vfw *vfw_pipe) {
+       vfw_pipe->counters->entry_timestamp = rte_get_tsc_cycles();
+       if (likely(vfw_pipe->counters->exit_timestamp))
+              vfw_pipe->counters->external_time_sum +=
+                     vfw_pipe->counters->entry_timestamp -
+                     vfw_pipe->counters->exit_timestamp;
+}
+
+/* End TSC measurement */
+static inline void end_tsc_measure(
+       struct pipeline_vfw *vfw_pipe,
+       uint8_t n_pkts)
+{
+       if (likely(n_pkts > 1)) {
+              vfw_pipe->counters->exit_timestamp = rte_get_tsc_cycles();
+              vfw_pipe->counters->internal_time_sum +=
+                     vfw_pipe->counters->exit_timestamp -
+                     vfw_pipe->counters->entry_timestamp;
+              vfw_pipe->counters->time_measurements++;
+       } else {
+              /* small counts skew results, ignore */
+              vfw_pipe->counters->exit_timestamp = 0;
+       }
+}
+
+static struct ether_addr *get_local_link_hw_addr(uint8_t out_port)
+{
+       return &link_hw_laddr[out_port];
+}
+
+static uint8_t local_dest_mac_present(uint8_t out_port)
+{
+       return link_hw_laddr_valid[out_port];
+}
+
+static uint32_t local_get_nh_ipv4(
+       uint32_t ip,
+       uint32_t *port,
+       uint32_t *nhip,
+       struct pipeline_vfw *vfw_pipe)
+{
+       int i;
+
+       for (i = 0; i < vfw_pipe->local_lib_arp_route_ent_cnt; i++) {
+              if (((vfw_pipe->local_lib_arp_route_table[i].ip &
+              vfw_pipe->local_lib_arp_route_table[i].mask) ==
+              (ip & vfw_pipe->local_lib_arp_route_table[i].mask))) {
+                     *port = vfw_pipe->local_lib_arp_route_table[i].port;
+
+                     *nhip = vfw_pipe->local_lib_arp_route_table[i].nh;
+                     return 1;
+              }
+       }
+       return 0;
+}
+
+static void do_local_nh_ipv4_cache(uint32_t dest_if,
+              struct pipeline_vfw *vfw_pipe)
+{
+
+       /* Search for the entry and do local copy */
+       int i;
+
+       for (i = 0; i < MAX_ARP_RT_ENTRY; i++) {
+              if (lib_arp_route_table[i].port == dest_if) {
+
+                     struct lib_arp_route_table_entry *lentry =
+                            &vfw_pipe->
+                            local_lib_arp_route_table[vfw_pipe->
+                            local_lib_arp_route_ent_cnt];
+
+                     lentry->ip   = lib_arp_route_table[i].ip;
+                     lentry->mask = lib_arp_route_table[i].mask;
+                     lentry->port = lib_arp_route_table[i].port;
+                     lentry->nh   = lib_arp_route_table[i].nh;
+
+                     vfw_pipe->local_lib_arp_route_ent_cnt++;
+                     break;
+              }
+       }
+}
+static uint32_t local_get_nh_ipv6(
+       uint8_t *ip,
+       uint32_t *port,
+       uint8_t nhip[],
+        struct pipeline_vfw *vfw_pipe)
+{
+       uint8_t netmask_ipv6[IPV6_ADD_SIZE], netip_nd[IPV6_ADD_SIZE],
+              netip_in[IPV6_ADD_SIZE];
+       uint8_t i = 0, j = 0, k = 0, l = 0, depthflags = 0, depthflags1 = 0;
+       memset(netmask_ipv6, 0, sizeof(netmask_ipv6));
+       memset(netip_nd, 0, sizeof(netip_nd));
+       memset(netip_in, 0, sizeof(netip_in));
+
+       for (i = 0; i < vfw_pipe->local_lib_nd_route_ent_cnt; i++) {
+
+              convert_prefixlen_to_netmask_ipv6(
+                     vfw_pipe->local_lib_nd_route_table[i].depth,
+                     netmask_ipv6);
+
+              for (k = 0; k < IPV6_ADD_SIZE; k++)
+                     if (vfw_pipe->local_lib_nd_route_table[i].ipv6[k] &
+                                   netmask_ipv6[k]) {
+                            depthflags++;
+                            netip_nd[k] = vfw_pipe->
+                                   local_lib_nd_route_table[i].ipv6[k];
+                     }
+
+              for (l = 0; l < IPV6_ADD_SIZE; l++)
+                     if (ip[l] & netmask_ipv6[l]) {
+                            depthflags1++;
+                            netip_in[l] = ip[l];
+                     }
+
+
+              if ((depthflags == depthflags1) && (memcmp(netip_nd, netip_in,
+                                          sizeof(netip_nd)) == 0)) {
+
+                     *port = vfw_pipe->local_lib_nd_route_table[i].port;
+
+                     for (j = 0; j < IPV6_ADD_SIZE; j++)
+                            nhip[j] = vfw_pipe->
+                                   local_lib_nd_route_table[i].nhipv6[j];
+                     return 1;
+              }
+
+              depthflags = 0;
+              depthflags1 = 0;
+                     }
+                     return 0;
+}
+
+static void do_local_nh_ipv6_cache(uint32_t dest_if,
+              struct pipeline_vfw *vfw_pipe)
+{
+              /* Search for the entry and do local copy */
+       int i, l;
+
+       for (i = 0; i < MAX_ND_RT_ENTRY; i++) {
+
+              if (lib_nd_route_table[i].port == dest_if) {
+
+                     struct lib_nd_route_table_entry *lentry = &vfw_pipe->
+                            local_lib_nd_route_table[vfw_pipe->
+                            local_lib_nd_route_ent_cnt];
+
+                     for (l = 0; l < IPV6_ADD_SIZE; l++) {
+                            lentry->ipv6[l]   =
+                                   lib_nd_route_table[i].ipv6[l];
+                            lentry->nhipv6[l] =
+                                   lib_nd_route_table[i].nhipv6[l];
+                     }
+                     lentry->depth = lib_nd_route_table[i].depth;
+                     lentry->port  = lib_nd_route_table[i].port;
+
+                     vfw_pipe->local_lib_nd_route_ent_cnt++;
+                     break;
+                     } /* if */
+              } /* for */
+}
+/**
+ * Print packet for debugging.
+ *
+ * @param pkt
+ *  A pointer to the packet.
+ *
+ */
+static __rte_unused  void print_pkt(struct rte_mbuf *pkt)
+{
+       int i;
+       int size = (int)sizeof(struct mbuf_tcp_meta_data);
+       uint8_t *rd = RTE_MBUF_METADATA_UINT8_PTR(pkt, META_DATA_OFFSET);
+
+       printf("Meta-data:\n");
+       for (i = 0; i < size; i++) {
+              printf("%02x ", rd[i]);
+              if ((i & TWO_BYTE_PRINT) == TWO_BYTE_PRINT)
+                     printf("\n");
+       }
+       printf("\n");
+       printf("IP and TCP/UDP headers:\n");
+       rd = RTE_MBUF_METADATA_UINT8_PTR(pkt, IP_START);
+       for (i = 0; i < IP_HDR_SIZE_IPV6; i++) {
+              printf("%02x ", rd[i]);
+              if ((i & TWO_BYTE_PRINT) == TWO_BYTE_PRINT)
+                     printf("\n");
+       }
+       printf("\n");
+}
+
+/* TODO: are the protocol numbers defined somewhere with meaningful names? */
+#define IP_ICMP_PROTOCOL 1
+#define IP_TCP_PROTOCOL 6
+#define IP_UDP_PROTOCOL 17
+#define IPv6_FRAGMENT_HEADER 44
+
+/**
+ * Return ethernet header structure form packet.
+ *
+ * @param pkt
+ *  A pointer to the packet.
+ *
+ */
+static inline struct ether_hdr *rte_vfw_get_ether_addr(struct rte_mbuf *pkt)
+{
+       return (struct ether_hdr *)RTE_MBUF_METADATA_UINT32_PTR(pkt,
+                                                        ETHERNET_START);
+}
+
+/**
+ * Return IPV4 header structure form packet.
+ *
+ * @param pkt
+ *  A pointer to the packet.
+ *
+ */
+
+static inline struct ipv4_hdr *rte_vfw_get_IPv4_hdr_addr(
+              struct rte_mbuf *pkt)
+{
+       return (struct ipv4_hdr *)RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+}
+
+static inline int rte_vfw_is_IPv4(struct rte_mbuf *pkt)
+{
+       /* NOTE: Only supporting IP headers with no options,
+        * so header is fixed size */
+       uint8_t ip_type = RTE_MBUF_METADATA_UINT8(pkt, IP_START)
+              >> VERSION_NO_BYTE;
+
+       return ip_type == IPv4_HDR_VERSION;
+}
+
+static inline int rte_vfw_is_IPv6(struct rte_mbuf *pkt)
+{
+       /* NOTE: Only supporting IP headers with no options,
+        * so header is fixed size */
+       uint8_t ip_type = RTE_MBUF_METADATA_UINT8(pkt, IP_START)
+              >> VERSION_NO_BYTE;
+
+       return ip_type == IPv6_HDR_VERSION;
+}
+
+static inline void rte_vfw_incr_drop_ctr(uint64_t *counter)
+{
+       if (likely(firewall_flag))
+              (*counter)++;
+}
+
+static uint8_t check_arp_icmp(
+              struct rte_mbuf *pkt,
+              struct pipeline_vfw *vfw_pipe)
+{
+       struct ether_hdr *ehdr;
+       struct app_link_params *link;
+        uint8_t solicited_node_multicast_addr[IPV6_ADD_SIZE] = {
+                0xff, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+                0x00, 0x00, 0x00, 0x01, 0xff, 0x00, 0x00, 0x00};
+
+        /* ARP outport number */
+       uint16_t out_port = vfw_pipe->pipe.n_ports_out - 1;
+       struct ipv4_hdr *ipv4_h;
+       struct ipv6_hdr *ipv6_h;
+       link = &myApp->link_params[pkt->port];
+
+       ehdr = rte_vfw_get_ether_addr(pkt);
+       switch (rte_be_to_cpu_16(ehdr->ether_type)) {
+
+       case ETH_TYPE_ARP:
+              rte_pipeline_port_out_packet_insert(
+                            vfw_pipe->pipe.p,
+                            out_port,
+                            pkt);
+
+              vfw_pipe->counters->arpicmpPktCount++;
+
+              return 0;
+       case ETH_TYPE_IPV4:
+              ipv4_h = (struct ipv4_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+              if ((ipv4_h->next_proto_id == IP_PROTOCOL_ICMP) &&
+                            link->ip ==
+                            rte_be_to_cpu_32(ipv4_h->dst_addr)) {
+                     if (is_phy_port_privte(pkt->port)) {
+                            rte_pipeline_port_out_packet_insert(
+                                          vfw_pipe->pipe.p,
+                                          out_port,
+                                          pkt);
+
+                     vfw_pipe->counters->arpicmpPktCount++;
+                            return 0;
+                     }
+              }
+              break;
+#ifdef IPV6
+        case ETH_TYPE_IPV6:
+                ipv6_h = (struct ipv6_hdr *)
+                        RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+                if (ipv6_h->proto == ICMPV6_PROTOCOL_ID) {
+                        if (!memcmp(ipv6_h->dst_addr, link->ipv6, IPV6_ADD_SIZE)
+                                        || !memcmp(ipv6_h->dst_addr,
+                                                solicited_node_multicast_addr,
+                                                IPV6_ADD_CMP_MULTI)) {
+
+                                rte_pipeline_port_out_packet_insert(
+                                                vfw_pipe->pipe.p,
+                                                out_port,
+                                                pkt);
+
+                                vfw_pipe->counters->arpicmpPktCount++;
+
+                        } else
+                                vfw_pipe->counters->
+                                        pkts_drop_unsupported_type++;
+
+                        return 0;
+                }
+                break;
+#endif
+       default:
+              break;
+}
+       return 1;
+}
+
+
+/**
+ * Performs basic VFW packet filtering.
+ * @param pkts
+ *  A pointer to the packets.
+ * @param pkts_mask
+ *  packet mask.
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+static uint64_t
+rte_vfw_packet_filter_and_process(struct rte_mbuf **pkts,
+                                 uint64_t pkts_mask,
+                                 struct pipeline_vfw *vfw_pipe)
+{
+
+       /*
+        * Make use of cache prefetch. At beginning of loop, want to prefetch
+        * mbuf data for next iteration (not current one).
+        * Note that ethernet header (14 bytes) is cache aligned. IPv4 header
+        * is 20 bytes (extensions not supported), while the IPv6 header is 40
+        * bytes. TCP header is 20 bytes, UDP is 8. One cache line prefetch
+        * will cover IPv4 and TCP or UDP, but to get IPv6 and TCP,
+        * need two pre-fetches.
+        */
+
+       uint8_t pos, next_pos = 0;
+       uint64_t pkt_mask;       /* bitmask representing a single packet */
+       struct rte_mbuf *pkt;
+       struct rte_mbuf *next_pkt = NULL;
+       void *iphdr;
+       void *next_iphdr = NULL;
+
+       if (unlikely(pkts_mask == 0))
+              return pkts_mask;
+       pos = (uint8_t) __builtin_ctzll(pkts_mask);
+       pkt_mask = 1LLU << pos;       /* bitmask representing only this packet */
+       pkt = pkts[pos];
+       iphdr = RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+       rte_prefetch0(iphdr);
+
+       uint64_t bytes_processed = 0;
+       /* bitmap of packets left to process */
+       uint64_t pkts_to_process = pkts_mask;
+       /* bitmap of valid packets to return */
+       uint64_t valid_packets = pkts_mask;
+
+       /* prefetch counters, updated below. Most likely counters to update
+        * at beginnning */
+       rte_prefetch0(&vfw_pipe->counters);
+
+       do {                     /* always execute at least once */
+
+              /* remove this packet from remaining list */
+              uint64_t next_pkts_to_process = pkts_to_process &= ~pkt_mask;
+
+              if (likely(next_pkts_to_process)) {
+                     /* another packet to process after this, prefetch it */
+
+                     next_pos =
+                         (uint8_t) __builtin_ctzll(next_pkts_to_process);
+                     next_pkt = pkts[next_pos];
+                     next_iphdr =
+                         RTE_MBUF_METADATA_UINT32_PTR(next_pkt, IP_START);
+                     rte_prefetch0(next_iphdr);
+              }
+
+              int discard = 0;
+              /* remove this packet from remaining list */
+              pkts_to_process &= ~pkt_mask;
+              if (enable_hwlb)
+                     if (!check_arp_icmp(pkt, vfw_pipe))
+                            discard = 1;
+              uint32_t packet_length = rte_pktmbuf_pkt_len(pkt);
+
+              bytes_processed += packet_length;
+
+              if (rte_vfw_is_IPv4(pkt)) {
+                     struct ipv4_hdr *ihdr4 = (struct ipv4_hdr *)iphdr;
+
+                     /* verify that packet size according to mbuf is at least
+                      * as large as the size according to the IP header.
+                      */
+
+                     uint32_t ip_length = rte_bswap16(ihdr4->total_length);
+
+                     if (unlikely
+                         (ip_length > (packet_length - ETH_HDR_SIZE))) {
+                            discard = 1;
+                            vfw_pipe->counters->pkts_drop_bad_size++;
+                     }
+
+                     /*
+                      * IPv4 fragmented if: MF (more fragments) or Fragment
+                      * Offset are non-zero. Header in Intel order, so flip
+                      * constant to compensate. Note that IPv6 uses a header
+                      * extension for identifying fragments.
+                      */
+
+                     int fragmented = (ihdr4->fragment_offset & 0xff3f) != 0;
+                     uint8_t ttl = ihdr4->time_to_live;
+
+                     if (unlikely(fragmented)) {
+                            discard = 1;
+                            vfw_pipe->counters->pkts_drop_fragmented++;
+                     }
+
+                     /*
+                      * Behave like a router, and decrement the TTL of an
+                      * IP packet. If this causes the TTL to become zero,
+                      * the packet will be discarded. Unlike a router,
+                      * no ICMP code 11 (Time * Exceeded) message will be
+                      * sent back to the packet originator.
+                      */
+
+                     if (unlikely(ttl <= 1)) {
+                            /*
+                             * about to decrement to zero (or is somehow
+                             * already zero), so discard
+                             */
+                            discard = 1;
+                            vfw_pipe->counters->pkts_drop_ttl++;
+                     }
+
+                     /*
+                      * Dropping the packets other than TCP AND UDP.
+                      */
+
+                     uint8_t proto = ihdr4->next_proto_id;
+
+                     if (unlikely(!(proto == IP_TCP_PROTOCOL ||
+                                   proto == IP_UDP_PROTOCOL ||
+                                   proto == IP_ICMP_PROTOCOL))) {
+                            discard = 1;
+                            vfw_pipe->counters->
+                                   pkts_drop_unsupported_type++;
+                     }
+
+                     if (unlikely(discard)) {
+                            valid_packets &= ~pkt_mask;
+                     }
+
+              } else if (likely(rte_vfw_is_IPv6(pkt))) {
+                     struct ipv6_hdr *ihdr6 = (struct ipv6_hdr *)iphdr;
+
+                     /*
+                      * verify that packet size according to mbuf is at least
+                      * as large as the size according to the IP header.
+                      * For IPv6, note that size includes header extensions
+                      * but not the base header size
+                      */
+
+                     uint32_t ip_length =
+                         rte_bswap16(ihdr6->payload_len) + IPv6_HEADER_SIZE;
+
+                     if (unlikely
+                         (ip_length > (packet_length - ETH_HDR_SIZE))) {
+                            discard = 1;
+                            vfw_pipe->counters->pkts_drop_bad_size++;
+                     }
+
+                     /*
+                      * Dropping the packets other than TCP AND UDP.
+                      */
+
+                     uint8_t proto = ihdr6->proto;
+
+                     if (unlikely(!(proto == IP_TCP_PROTOCOL ||
+                                   proto == IP_UDP_PROTOCOL ||
+                                   proto == IP_ICMP_PROTOCOL))) {
+                            discard = 1;
+                            if (proto == IPv6_FRAGMENT_HEADER)
+                                   vfw_pipe->counters->
+                                       pkts_drop_fragmented++;
+                            else
+                                   vfw_pipe->counters->
+                                       pkts_drop_unsupported_type++;
+                     }
+
+                     /*
+                      * Behave like a router, and decrement the TTL of an
+                      * IP packet. If this causes the TTL to become zero,
+                      * the packet will be discarded. Unlike a router,
+                      * no ICMP code 11 (Time * Exceeded) message will be
+                      * sent back to the packet originator.
+                      */
+
+                     if (unlikely(ihdr6->hop_limits <= 1)) {
+                            /*
+                             * about to decrement to zero (or is somehow
+                             * already zero), so discard
+                             */
+                            discard = 1;
+                            vfw_pipe->counters->pkts_drop_ttl++;
+                     }
+
+                     if (unlikely(discard))
+                            valid_packets &= ~pkt_mask;
+                     else
+                            ihdr6->hop_limits--;
+              } else
+                     /* discard non-ip */
+                     valid_packets &= ~pkt_mask;
+
+              /* make next packet data the current */
+              pkts_to_process = next_pkts_to_process;
+              pos = next_pos;
+              pkt = next_pkt;
+              iphdr = next_iphdr;
+              pkt_mask = 1LLU << pos;
+
+       } while (pkts_to_process);
+
+       /* finalize counters, etc. */
+       vfw_pipe->counters->bytes_processed += bytes_processed;
+
+       if (likely(firewall_flag))
+              return valid_packets;
+       else
+              return pkts_mask;
+}
+/**
+ * Performs basic VFW ipv4 packet filtering.
+ * @param pkts
+ *  A pointer to the packets.
+ * @param pkts_mask
+ *  packet mask.
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+static uint64_t
+rte_vfw_ipv4_packet_filter_and_process(struct rte_mbuf **pkts,
+                                 uint64_t pkts_mask,
+                                 struct pipeline_vfw *vfw_pipe)
+{
+
+       /*
+        * Make use of cache prefetch. At beginning of loop, want to prefetch
+        * mbuf data for next iteration (not current one).
+        * Note that ethernet header (14 bytes) is cache aligned. IPv4 header
+        * is 20 bytes (extensions not supported), while the IPv6 header is 40
+        * bytes. TCP header is 20 bytes, UDP is 8. One cache line prefetch
+        * will cover IPv4 and TCP or UDP, but to get IPv6 and TCP,
+        * need two pre-fetches.
+        */
+
+       uint8_t pos, next_pos = 0;
+       uint64_t pkt_mask;       /* bitmask representing a single packet */
+       struct rte_mbuf *pkt;
+       struct rte_mbuf *next_pkt = NULL;
+       struct ipv4_hdr *ihdr4;
+       void *next_iphdr = NULL;
+
+       if (unlikely(pkts_mask == 0))
+              return pkts_mask;
+       pos = (uint8_t) __builtin_ctzll(pkts_mask);
+       pkt_mask = 1LLU << pos;       /* bitmask representing only this packet */
+       pkt = pkts[pos];
+
+       uint64_t bytes_processed = 0;
+       /* bitmap of packets left to process */
+       uint64_t pkts_to_process = pkts_mask;
+       /* bitmap of valid packets to return */
+       uint64_t valid_packets = pkts_mask;
+
+       rte_prefetch0(pkt);
+       /* prefetch counters, updated below. Most likely counters to update
+        * at beginnning */
+       rte_prefetch0(&vfw_pipe->counters);
+
+       do {                     /* always execute at least once */
+
+              /* remove this packet from remaining list */
+              uint64_t next_pkts_to_process = pkts_to_process &= ~pkt_mask;
+
+              if (likely(next_pkts_to_process)) {
+                     /* another packet to process after this, prefetch it */
+
+                     next_pos =
+                            (uint8_t) __builtin_ctzll(next_pkts_to_process);
+                     next_pkt = pkts[next_pos];
+                     next_iphdr = RTE_MBUF_METADATA_UINT32_PTR(next_pkt,
+                                   IP_START);
+                     rte_prefetch0(next_iphdr);
+              }
+
+              int discard = 0;
+              /* remove this packet from remaining list */
+              pkts_to_process &= ~pkt_mask;
+
+              if (enable_hwlb)
+                     if (!check_arp_icmp(pkt, vfw_pipe))
+                            discard = 1;
+
+              uint32_t packet_length = rte_pktmbuf_pkt_len(pkt);
+
+              bytes_processed += packet_length;
+
+              ihdr4 = (struct ipv4_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              /* verify that packet size according to mbuf is at least
+               * as large as the size according to the IP header.
+               */
+
+              uint32_t ip_length = rte_bswap16(ihdr4->total_length);
+
+              if (unlikely
+                            (ip_length > (packet_length - ETH_HDR_SIZE))) {
+                     discard = 1;
+                     vfw_pipe->counters->pkts_drop_bad_size++;
+              }
+
+              /*
+               * IPv4 fragmented if: MF (more fragments) or Fragment
+               * Offset are non-zero. Header in Intel order, so flip
+               * constant to compensate. Note that IPv6 uses a header
+               * extension for identifying fragments.
+               */
+
+              int fragmented = (ihdr4->fragment_offset & 0xff3f) != 0;
+              uint8_t ttl = ihdr4->time_to_live;
+
+              if (unlikely(fragmented)) {
+                     discard = 1;
+                     vfw_pipe->counters->pkts_drop_fragmented++;
+              }
+
+              /*
+               * Behave like a router, and decrement the TTL of an
+               * IP packet. If this causes the TTL to become zero,
+               * the packet will be discarded. Unlike a router,
+               * no ICMP code 11 (Time * Exceeded) message will be
+               * sent back to the packet originator.
+               */
+
+              if (unlikely(ttl <= 1)) {
+                     /*
+                      * about to decrement to zero (or is somehow
+                      * already zero), so discard
+                      */
+                     discard = 1;
+                     vfw_pipe->counters->pkts_drop_ttl++;
+              }
+
+              /*
+               * Dropping the packets other than TCP AND UDP.
+               */
+
+              uint8_t proto = ihdr4->next_proto_id;
+
+              if (unlikely(!(proto == IP_TCP_PROTOCOL ||
+                                          proto == IP_UDP_PROTOCOL ||
+                                          proto == IP_ICMP_PROTOCOL))) {
+                     discard = 1;
+                     vfw_pipe->counters->
+                            pkts_drop_unsupported_type++;
+              }
+
+              if (unlikely(discard)) {
+                     valid_packets &= ~pkt_mask;
+              } else {
+                     ihdr4->time_to_live = ttl - 1;
+
+                     /* update header checksum, from rfc 1141 */
+                     uint32_t sum;
+                     uint16_t checksum = rte_bswap16(
+                                   ihdr4->hdr_checksum);
+                     /* increment checksum high byte */
+                     sum = checksum + 0x100;
+                     /* add carry */
+                     checksum = (sum + (sum >> BIT_CARRY));
+                     ihdr4->hdr_checksum = rte_bswap16(checksum);
+              }
+
+              /* make next packet data the current */
+              pkts_to_process = next_pkts_to_process;
+              pos = next_pos;
+              pkt = next_pkt;
+              ihdr4 = next_iphdr;
+              pkt_mask = 1LLU << pos;
+
+       } while (pkts_to_process);
+
+       /* finalize counters, etc. */
+       vfw_pipe->counters->bytes_processed += bytes_processed;
+
+       if (likely(firewall_flag))
+              return valid_packets;
+       else
+              return pkts_mask;
+}
+/**
+ * Performs basic VFW IPV6 packet filtering.
+ * @param pkts
+ *  A pointer to the packets.
+ * @param pkts_mask
+ *  packet mask.
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+       static uint64_t
+rte_vfw_ipv6_packet_filter_and_process(struct rte_mbuf **pkts,
+              uint64_t pkts_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+
+       /*
+        * Make use of cache prefetch. At beginning of loop, want to prefetch
+        * mbuf data for next iteration (not current one).
+        * Note that ethernet header (14 bytes) is cache aligned. IPv4 header
+        * is 20 bytes (extensions not supported), while the IPv6 header is 40
+        * bytes. TCP header is 20 bytes, UDP is 8. One cache line prefetch
+        * will cover IPv4 and TCP or UDP, but to get IPv6 and TCP,
+        * need two pre-fetches.
+        */
+
+       uint8_t pos, next_pos = 0;
+       uint64_t pkt_mask;       /* bitmask representing a single packet */
+       struct rte_mbuf *pkt;
+       struct rte_mbuf *next_pkt = NULL;
+       struct ipv6_hdr *ihdr6;
+       void *next_iphdr = NULL;
+
+       if (unlikely(pkts_mask == 0))
+              return pkts_mask;
+       pos = (uint8_t) __builtin_ctzll(pkts_mask);
+       pkt_mask = 1LLU << pos;       /* bitmask representing only this packet */
+       pkt = pkts[pos];
+
+       uint64_t bytes_processed = 0;
+       /* bitmap of packets left to process */
+       uint64_t pkts_to_process = pkts_mask;
+       /* bitmap of valid packets to return */
+       uint64_t valid_packets = pkts_mask;
+
+       /* prefetch counters, updated below. Most likely counters to update
+        * at beginnning */
+       rte_prefetch0(&vfw_pipe->counters);
+
+       do {                     /* always execute at least once */
+
+              /* remove this packet from remaining list */
+              uint64_t next_pkts_to_process = pkts_to_process &= ~pkt_mask;
+
+              if (likely(next_pkts_to_process)) {
+                     /* another packet to process after this, prefetch it */
+
+                     next_pos =
+                         (uint8_t) __builtin_ctzll(next_pkts_to_process);
+                     next_pkt = pkts[next_pos];
+                     next_iphdr =
+                         RTE_MBUF_METADATA_UINT32_PTR(next_pkt, IP_START);
+                     rte_prefetch0(next_iphdr);
+              }
+
+              int discard = 0;
+              /* remove this packet from remaining list */
+              pkts_to_process &= ~pkt_mask;
+
+              if (enable_hwlb)
+                     if (!check_arp_icmp(pkt, vfw_pipe))
+                            discard = 1;
+
+              uint32_t packet_length = rte_pktmbuf_pkt_len(pkt);
+
+              bytes_processed += packet_length;
+
+              ihdr6 = (struct ipv6_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              /*
+               * verify that packet size according to mbuf is at least
+               * as large as the size according to the IP header.
+               * For IPv6, note that size includes header extensions
+               * but not the base header size
+               */
+
+              uint32_t ip_length =
+                     rte_bswap16(ihdr6->payload_len) + IPv6_HEADER_SIZE;
+
+              if (unlikely
+                            (ip_length > (packet_length - ETH_HDR_SIZE))) {
+                     discard = 1;
+                     vfw_pipe->counters->pkts_drop_bad_size++;
+              }
+
+              /*
+               * Dropping the packets other than TCP AND UDP.
+               */
+
+              uint8_t proto = ihdr6->proto;
+
+              if (unlikely(!(proto == IP_TCP_PROTOCOL ||
+                                          proto == IP_UDP_PROTOCOL ||
+                                          proto == IP_ICMP_PROTOCOL))) {
+                     discard = 1;
+                     if (proto == IPv6_FRAGMENT_HEADER)
+                            vfw_pipe->counters->
+                                   pkts_drop_fragmented++;
+                     else
+                            vfw_pipe->counters->
+                                   pkts_drop_unsupported_type++;
+              }
+
+              /*
+               * Behave like a router, and decrement the TTL of an
+               * IP packet. If this causes the TTL to become zero,
+               * the packet will be discarded. Unlike a router,
+               * no ICMP code 11 (Time * Exceeded) message will be
+               * sent back to the packet originator.
+               */
+
+              if (unlikely(ihdr6->hop_limits <= 1)) {
+                     /*
+                      * about to decrement to zero (or is somehow
+                      * already zero), so discard
+                      */
+                     discard = 1;
+                     vfw_pipe->counters->pkts_drop_ttl++;
+              }
+
+              if (unlikely(discard))
+                     valid_packets &= ~pkt_mask;
+              else
+                     ihdr6->hop_limits--;
+
+              /* make next packet data the current */
+              pkts_to_process = next_pkts_to_process;
+              pos = next_pos;
+              pkt = next_pkt;
+              ihdr6 = next_iphdr;
+              pkt_mask = 1LLU << pos;
+
+       } while (pkts_to_process);
+
+       /* finalize counters, etc. */
+       vfw_pipe->counters->bytes_processed += bytes_processed;
+
+       if (likely(firewall_flag))
+              return valid_packets;
+       else
+              return pkts_mask;
+}
+
+/**
+ * exchange the mac address so source becomes destination and vice versa.
+ *
+ * @param ehdr
+ *  A pointer to the ethernet header.
+ *
+ */
+static inline void rte_sp_exchange_mac_addresses(struct ether_hdr *ehdr)
+{
+       struct ether_addr saved_copy;
+
+       ether_addr_copy(&ehdr->d_addr, &saved_copy);
+       ether_addr_copy(&ehdr->s_addr, &ehdr->d_addr);
+       ether_addr_copy(&saved_copy, &ehdr->s_addr);
+}
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packet.
+ * @param pkts_mask
+ *  Packet mask
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+static uint64_t
+rte_vfw_arp_packets(struct rte_mbuf **pkts,
+              uint64_t pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+       uint64_t pkts_to_arp = pkts_mask;
+       uint32_t ret;
+       uint32_t dest_if = INVALID_DESTIF;
+       int ret_mac;
+
+       for (; pkts_to_arp;) {
+              struct ether_addr hw_addr;
+              struct mbuf_tcp_meta_data *meta_data_addr;
+              struct ether_hdr *ehdr;
+              struct rte_mbuf *pkt;
+              uint16_t phy_port;
+              uint8_t pos = (uint8_t) __builtin_ctzll(pkts_to_arp);
+              /* bitmask representing only this packet */
+              uint64_t pkt_mask = 1LLU << pos;
+              /* remove this packet from remaining list */
+              pkts_to_arp &= ~pkt_mask;
+              pkt = pkts[pos];
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+              void *iphdr = RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              if (rte_vfw_is_IPv4(pkt)) {
+                     struct ipv4_hdr *ihdr = (struct ipv4_hdr *)iphdr;
+                     uint32_t nhip = 0;
+
+                     uint32_t dest_address = rte_bswap32(ihdr->dst_addr);
+
+                     ret = local_get_nh_ipv4(dest_address, &dest_if,
+                                   &nhip, vfw_pipe);
+                     if (must_reverse) {
+                            rte_sp_exchange_mac_addresses(ehdr);
+                            if (is_phy_port_privte(phy_port)) {
+                                   if (!ret) {
+                                          dest_if = get_pub_to_prv_port(
+                                                        &dest_address,
+                                                        IP_VERSION_4);
+                                          if (dest_if == INVALID_DESTIF) {
+                                                 pkts_mask &= ~pkt_mask;
+                                                 vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                          }
+                                          do_local_nh_ipv4_cache(dest_if,
+                                                        vfw_pipe);
+                                   }
+
+                            } else {
+                                   if (!ret) {
+                                          dest_if = get_prv_to_pub_port(
+                                                        &dest_address,
+                                                        IP_VERSION_4);
+                                          if (dest_if == INVALID_DESTIF) {
+                                                 pkts_mask &= ~pkt_mask;
+                                                 vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                          }
+                                          do_local_nh_ipv4_cache(dest_if,
+                                                        vfw_pipe);
+                                   }
+                            }
+
+                     } else if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+
+                     meta_data_addr->output_port =
+                            vfw_pipe->outport_id[dest_if];
+                     if (local_dest_mac_present(dest_if)) {
+                            ether_addr_copy(get_local_link_hw_addr(dest_if),
+                                          &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+                     } else {
+                            ret_mac = get_dest_mac_addr_port(dest_address,
+                                          &dest_if, &hw_addr);
+                            if (ret_mac == ARP_FOUND) {
+
+                                   link_hw_laddr_valid[dest_if] = 1;
+                                   memcpy(&link_hw_laddr[dest_if], &hw_addr,
+                                                 sizeof(struct ether_addr));
+
+                                   ether_addr_copy(&hw_addr,
+                                          &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC found for ip 0x%"PRIx32
+                                                 ",dest_if %d: %s, ",
+                                                 dest_address,
+                                                 dest_if, buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->s_addr);
+                                   printf("new eth hdr src: %s, ", buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->d_addr);
+                                   printf("new eth hdr dst: %s\n", buf);
+                            }
+
+                     } else {
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC NOT FOUND for ip 0x%"
+                                                 PRIx32", dest_if %"
+                                                 PRId16": %s, ",
+                                                 dest_address,
+                                                 dest_if, buf);
+                            }
+                            /* ICMP req sent, drop packet by
+                             * changing the mask */
+                            pkts_mask &= ~pkt_mask;
+                            vfw_pipe->
+                                   counters->pkts_drop_without_arp_entry++;
+                     }
+
+                     }
+              } else if (likely(rte_vfw_is_IPv6(pkt))) {
+                     struct ipv6_hdr *ihdr = (struct ipv6_hdr *)iphdr;
+                     uint8_t dest_addr_ipv6[IPV6_ADD_SIZE];
+
+                     rte_mov16(dest_addr_ipv6, ihdr->dst_addr);
+                     uint8_t nh_ipv6[IPV6_ADD_SIZE];
+
+                     memset(nh_ipv6, 0, IPV6_ADD_SIZE);
+                     ret = local_get_nh_ipv6(&dest_addr_ipv6[0], &dest_if,
+                                   &nh_ipv6[0], vfw_pipe);
+                     if (must_reverse) {
+                            rte_sp_exchange_mac_addresses(ehdr);
+                            if (is_phy_port_privte(phy_port)) {
+                                   if (!ret) {
+                                          dest_if = get_pub_to_prv_port(
+                                                 (uint32_t *)
+                                                 &dest_addr_ipv6[0],
+                                                 IP_VERSION_6);
+                                          if (dest_if == INVALID_DESTIF) {
+                                                 pkts_mask &= ~pkt_mask;
+                                                 vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                          }
+                                          do_local_nh_ipv6_cache(dest_if,
+                                                        vfw_pipe);
+                                   }
+
+                            } else {
+                                   if (!ret) {
+                                          dest_if = get_prv_to_pub_port(
+                                                 (uint32_t *)
+                                                 &dest_addr_ipv6[0],
+                                                 IP_VERSION_6);
+                                          if (dest_if == INVALID_DESTIF) {
+                                                 pkts_mask &= ~pkt_mask;
+                                                 vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                          }
+                                          do_local_nh_ipv6_cache(dest_if,
+                                                        vfw_pipe);
+                                   }
+
+
+                            }
+
+                     } else if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 (uint32_t *)
+                                                 &dest_addr_ipv6[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 (uint32_t *)
+                                                 &dest_addr_ipv6[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     }
+                     meta_data_addr->output_port = vfw_pipe->
+                                          outport_id[dest_if];
+
+                     memset(nh_ipv6, 0, IPV6_ADD_SIZE);
+                     if (get_dest_mac_address_ipv6_port(
+                                          &dest_addr_ipv6[0],
+                                          &dest_if,
+                                          &hw_addr,
+                                          &nh_ipv6[0])) {
+                            ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC found for  dest_if %d: %s,",
+                                                 dest_if, buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->s_addr);
+                                   printf("new eth hdr src: %s, ", buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->d_addr);
+                                   printf("new eth hdr dst: %s\n", buf);
+                            }
+
+                     } else {
+                            printf("deleting ipv6\n");
+                            pkts_mask &= ~pkt_mask;
+                            /*Next Neighbor is not yet implemented
+                             * for ipv6.*/
+                            vfw_pipe->counters->
+                                   pkts_drop_without_arp_entry++;
+                     }
+
+              } else
+                     /* neither IPv4 or IPv6, drop quietly */
+                     pkts_mask &= ~pkt_mask;
+       }
+       return pkts_mask;
+}
+
+#ifdef EN_SWP_ARP
+
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packet array.
+ * @param pkt_num
+ *  Packet num to start processing
+ * @param pkts_mask
+ *  Packet mask
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+static void
+pkt4_work_vfw_arp_ipv4_packets(struct rte_mbuf **pkts,
+              uint16_t pkt_num,
+              uint64_t *pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+
+       uint32_t ret;
+       int ret_mac;
+       uint8_t i;
+
+       struct ether_addr hw_addr;
+       struct mbuf_tcp_meta_data *meta_data_addr;
+       struct ether_hdr *ehdr;
+       struct rte_mbuf *pkt;
+       uint16_t phy_port;
+
+       for (i = 0; i < 4; i++) {
+              uint32_t dest_if = INVALID_DESTIF;
+              /* bitmask representing only this packet */
+              uint64_t pkt_mask = 1LLU << (pkt_num + i);
+
+              pkt = pkts[i];
+
+              if(!(*pkts_mask & pkt_mask))
+                     continue;
+
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+
+              struct ipv4_hdr *ihdr = (struct ipv4_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+              uint32_t nhip = 0;
+
+              uint32_t dest_address = rte_bswap32(ihdr->dst_addr);
+
+              ret = local_get_nh_ipv4(dest_address, &dest_if,
+                            &nhip, vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(
+                                                 dest_if, vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+              meta_data_addr->output_port =  vfw_pipe->outport_id[dest_if];
+              if (local_dest_mac_present(dest_if)) {
+                     ether_addr_copy(get_local_link_hw_addr(dest_if),
+                                   &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+              } else {
+                     ret_mac = get_dest_mac_addr_port(dest_address,
+                                   &dest_if, &hw_addr);
+                     if (ret_mac == ARP_FOUND) {
+
+                            link_hw_laddr_valid[dest_if] = 1;
+                            memcpy(&link_hw_laddr[dest_if], &hw_addr,
+                                          sizeof(struct ether_addr));
+
+                            ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC found for ip 0x%"
+                                          PRIx32", dest_if %d: %s, ",
+                                          dest_address,
+                                          dest_if, buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->s_addr);
+                                   printf("new eth hdr src: %s, ", buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->d_addr);
+                                   printf("new eth hdr dst: %s\n", buf);
+                            }
+
+                     } else {
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                          &hw_addr);
+                                   printf("MAC NOT FOUND for ip 0x%"
+                                          PRIx32", dest_if %"
+                                          PRId16": %s, ",
+                                          dest_address,
+                                          dest_if, buf);
+                            }
+                            /* ICMP req sent, drop packet by
+                             * changing the mask */
+                            *pkts_mask &= ~pkt_mask;
+                            vfw_pipe->
+                                   counters->pkts_drop_without_arp_entry++;
+                     }
+              }
+       }
+}
+
+
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packet.
+ * @param packet_num
+ *  Packet number to process
+ * @param pkts_mask
+ *  Packet mask pointer
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+static void
+pkt_work_vfw_arp_ipv4_packets(struct rte_mbuf *pkts,
+              uint16_t pkt_num,
+              uint64_t *pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+
+       uint32_t ret;
+       uint32_t dest_if = INVALID_DESTIF;
+       int ret_mac;
+
+       struct ether_addr hw_addr;
+       struct mbuf_tcp_meta_data *meta_data_addr;
+       struct ether_hdr *ehdr;
+       struct rte_mbuf *pkt;
+       uint16_t phy_port;
+       uint64_t pkt_mask = 1LLU << pkt_num;
+
+       pkt = pkts;
+
+       if(*pkts_mask & pkt_mask) {
+
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+
+              struct ipv4_hdr *ihdr = (struct ipv4_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+              uint32_t nhip = 0;
+
+              uint32_t dest_address = rte_bswap32(ihdr->dst_addr);
+
+              ret = local_get_nh_ipv4(dest_address, &dest_if,
+                            &nhip, vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(
+                                                 dest_if, vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+              meta_data_addr->output_port =  vfw_pipe->outport_id[dest_if];
+              if (local_dest_mac_present(dest_if)) {
+                     ether_addr_copy(get_local_link_hw_addr(dest_if),
+                                   &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+              } else {
+                     ret_mac = get_dest_mac_addr_port(dest_address,
+                                   &dest_if, &hw_addr);
+                     if (ret_mac) {
+                            link_hw_laddr_valid[dest_if] = 1;
+                            memcpy(&link_hw_laddr[dest_if], &hw_addr,
+                                          sizeof(struct ether_addr));
+
+                            ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC found for ip 0x%"
+                                          PRIx32", dest_if %d: %s, ",
+                                          dest_address,
+                                          dest_if, buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->s_addr);
+                                   printf("new eth hdr src: %s, ", buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->d_addr);
+                                   printf("new eth hdr dst: %s\n", buf);
+                            }
+
+                     } else {
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC NOT FOUND for ip 0x%"
+                                                 PRIx32", dest_if %"
+                                                 PRId16": %s, ",
+                                                 dest_address,
+                                                 dest_if, buf);
+                            }
+                            /* ICMP req sent, drop packet by
+                             * changing the mask */
+                            *pkts_mask &= ~pkt_mask;
+                            vfw_pipe->
+                                   counters->pkts_drop_without_arp_entry++;
+                     }
+              }
+
+       }
+}
+
+
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packets array.
+ * @param pkt_num
+ *  Packet number to start processing.
+ * @param pkts_mask
+ *  Packet mask pointer
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+static void
+pkt4_work_vfw_arp_ipv6_packets(struct rte_mbuf **pkts,
+              uint16_t pkt_num,
+              uint64_t *pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+       uint8_t nh_ipv6[IPV6_ADD_SIZE];
+       uint32_t ret;
+       struct ether_addr hw_addr;
+       struct mbuf_tcp_meta_data *meta_data_addr;
+       struct ether_hdr *ehdr;
+       struct rte_mbuf *pkt;
+       uint16_t phy_port;
+       uint8_t i;
+
+       for (i = 0; i < 4; i++) {
+              uint32_t dest_if = INVALID_DESTIF;
+              /* bitmask representing only this packet */
+              uint64_t pkt_mask = 1LLU << (pkt_num + i);
+
+              pkt = pkts[i];
+
+              if(!(*pkts_mask & pkt_mask))
+                     continue;
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+              struct ipv6_hdr *ihdr = (struct ipv6_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              uint8_t nhip[IPV6_ADD_SIZE];
+              uint8_t dest_address[IPV6_ADD_SIZE];
+
+              memset(nhip, 0, IPV6_ADD_SIZE);
+
+              rte_mov16(dest_address, ihdr->dst_addr);
+              ret = local_get_nh_ipv6(&dest_address[0], &dest_if,
+                            &nhip[0], vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+
+              meta_data_addr->output_port = vfw_pipe->outport_id[dest_if];
+
+              memset(nh_ipv6, 0, IPV6_ADD_SIZE);
+              if (get_dest_mac_address_ipv6_port(
+                                   &dest_address[0],
+                                   &dest_if,
+                                   &hw_addr,
+                                   &nh_ipv6[0])) {
+                     ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+
+                     if (vfw_debug >= DEBUG_LEVEL_4) {
+                            char buf[HW_ADDR_SIZE];
+
+                            ether_format_addr(buf, sizeof(buf),
+                                          &hw_addr);
+                            printf("MAC found for  dest_if %d: %s, ",
+                                          dest_if, buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->s_addr);
+                            printf("new eth hdr src: %s, ", buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->d_addr);
+                            printf("new eth hdr dst: %s\n", buf);
+                     }
+
+              } else {
+                     printf("deleting ipv6\n");
+                     *pkts_mask &= ~pkt_mask;
+                     /*Next Neighbor is not yet implemented
+                      * for ipv6.*/
+                     vfw_pipe->counters->
+                            pkts_drop_without_arp_entry++;
+              }
+
+       }
+}
+
+
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packets.
+ * @param pkt_num
+ *  Packet number to process.
+ * @param pkts_mask
+ *  Packet mask pointer
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+static void
+pkt_work_vfw_arp_ipv6_packets(struct rte_mbuf *pkts,
+              uint16_t pkt_num,
+              uint64_t *pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+       uint8_t nh_ipv6[IPV6_ADD_SIZE];
+       uint32_t ret;
+       struct ether_addr hw_addr;
+       struct mbuf_tcp_meta_data *meta_data_addr;
+       struct ether_hdr *ehdr;
+       struct rte_mbuf *pkt;
+       uint16_t phy_port;
+
+       uint32_t dest_if = INVALID_DESTIF;
+       /* bitmask representing only this packet */
+       uint64_t pkt_mask = 1LLU << pkt_num;
+
+       pkt = pkts;
+
+       if(*pkts_mask & pkt_mask) {
+
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+              struct ipv6_hdr *ihdr = (struct ipv6_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              uint8_t nhip[IPV6_ADD_SIZE];
+              uint8_t dest_address[IPV6_ADD_SIZE];
+
+              memset(nhip, 0, IPV6_ADD_SIZE);
+
+              rte_mov16(dest_address, ihdr->dst_addr);
+              ret = local_get_nh_ipv6(&dest_address[0], &dest_if,
+                            &nhip[0], vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          *pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   *pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+
+              meta_data_addr->output_port = vfw_pipe->outport_id[dest_if];
+
+              memset(nh_ipv6, 0, IPV6_ADD_SIZE);
+              if (get_dest_mac_address_ipv6_port(
+                                   &dest_address[0],
+                                   &dest_if,
+                                   &hw_addr,
+                                   &nh_ipv6[0])) {
+                     ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+
+                     if (vfw_debug >= DEBUG_LEVEL_4) {
+                            char buf[HW_ADDR_SIZE];
+
+                            ether_format_addr(buf, sizeof(buf),
+                                          &hw_addr);
+                            printf("MAC found for  dest_if %d: %s, ",
+                                          dest_if, buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->s_addr);
+                            printf("new eth hdr src: %s, ", buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->d_addr);
+                            printf("new eth hdr dst: %s\n", buf);
+                     }
+
+              } else {
+                     printf("deleting ipv6\n");
+                     *pkts_mask &= ~pkt_mask;
+                     /*Next Neighbor is not yet implemented
+                      * for ipv6.*/
+                     vfw_pipe->counters->
+                            pkts_drop_without_arp_entry++;
+              }
+
+       }
+
+}
+
+#else
+
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packet.
+ * @param pkts_mask
+ *  Packet mask
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+static uint64_t
+rte_vfw_arp_ipv4_packets(struct rte_mbuf **pkts,
+              uint64_t pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+       uint64_t pkts_to_arp = pkts_mask;
+
+       uint32_t ret;
+       uint32_t dest_if = INVALID_DESTIF;
+       int ret_mac;
+       for (; pkts_to_arp;) {
+              struct ether_addr hw_addr;
+              struct mbuf_tcp_meta_data *meta_data_addr;
+              struct ether_hdr *ehdr;
+              struct rte_mbuf *pkt;
+              uint16_t phy_port;
+
+              uint8_t pos = (uint8_t) __builtin_ctzll(pkts_to_arp);
+              /* bitmask representing only this packet */
+              uint64_t pkt_mask = 1LLU << pos;
+              /* remove this packet from remaining list */
+              pkts_to_arp &= ~pkt_mask;
+              pkt = pkts[pos];
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+
+              struct ipv4_hdr *ihdr = (struct ipv4_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+              uint32_t nhip = 0;
+
+              uint32_t dest_address = rte_bswap32(ihdr->dst_addr);
+
+              ret = local_get_nh_ipv4(dest_address, &dest_if,
+                            &nhip, vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(
+                                                 dest_if, vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 &dest_address,
+                                                 IP_VERSION_4);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv4_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port(&dest_address,
+                                          IP_VERSION_4);
+                            if (dest_if == INVALID_DESTIF) {
+                                   pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv4_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+              meta_data_addr->output_port =  vfw_pipe->outport_id[dest_if];
+              if (local_dest_mac_present(dest_if)) {
+                     ether_addr_copy(get_local_link_hw_addr(dest_if),
+                                   &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+              } else {
+                     ret_mac = get_dest_mac_addr_port(dest_address,
+                                   &dest_if, &hw_addr);
+                     if (ret_mac) {
+                            link_hw_laddr_valid[dest_if] = 1;
+                            memcpy(&link_hw_laddr[dest_if], &hw_addr,
+                                          sizeof(struct ether_addr));
+
+                            ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                            ether_addr_copy(get_link_hw_addr(dest_if),
+                                          &ehdr->s_addr);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &hw_addr);
+                                   printf("MAC found for ip 0x%"
+                                          PRIx32", dest_if %d: %s, ",
+                                          dest_address,
+                                          dest_if, buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->s_addr);
+                                   printf("new eth hdr src: %s, ", buf);
+                                   ether_format_addr(buf, sizeof(buf),
+                                                 &ehdr->d_addr);
+                                   printf("new eth hdr dst: %s\n", buf);
+                            }
+
+                     } else {
+                            if (unlikely(ret_mac == 0))
+                                   request_arp(meta_data_addr->output_port,
+                                          nhip);
+
+                            if (vfw_debug >= DEBUG_LEVEL_4) {
+                                   char buf[HW_ADDR_SIZE];
+
+                            ether_format_addr(buf, sizeof(buf),
+                                          &hw_addr);
+                            printf("MAC NOT FOUND for ip 0x%"
+                                          PRIx32", dest_if %"
+                                          PRId16": %s, ",
+                                          dest_address,
+                                          dest_if, buf);
+                     }
+                     /* ICMP req sent, drop packet by
+                      * changing the mask */
+                     pkts_mask &= ~pkt_mask;
+                     vfw_pipe->
+                            counters->pkts_drop_without_arp_entry++;
+              }
+}
+
+       }
+
+       return pkts_mask;
+}
+/**
+ * walk every valid mbuf (denoted by pkts_mask) and apply arp to the packet.
+ * To support synproxy, some (altered) packets may need to be sent back where
+ * they came from. The ip header has already been adjusted, but the ethernet
+ * header has not, so this must be performed here.
+ * Return an updated pkts_mask, since arp may drop some packets
+ *
+ * @param pkts
+ *  A pointer to the packet.
+ * @param pkts_mask
+ *  Packet mask
+ * @param synproxy_reply_mask
+ *  Reply Packet mask for Synproxy
+ * @param vfw_pipe
+ *  A pointer to VFW pipeline.
+ */
+
+       static uint64_t
+rte_vfw_arp_ipv6_packets(struct rte_mbuf **pkts,
+              uint64_t pkts_mask,
+              uint64_t synproxy_reply_mask,
+              struct pipeline_vfw *vfw_pipe)
+{
+       uint64_t pkts_to_arp = pkts_mask;
+       uint8_t nh_ipv6[IPV6_ADD_SIZE];
+       uint32_t ret;
+       uint32_t dest_if = INVALID_DESTIF;
+
+       for (; pkts_to_arp;) {
+              struct ether_addr hw_addr;
+              struct mbuf_tcp_meta_data *meta_data_addr;
+              struct ether_hdr *ehdr;
+              struct rte_mbuf *pkt;
+              uint16_t phy_port;
+
+              uint8_t pos = (uint8_t) __builtin_ctzll(pkts_to_arp);
+              /* bitmask representing only this packet */
+              uint64_t pkt_mask = 1LLU << pos;
+              /* remove this packet from remaining list */
+              pkts_to_arp &= ~pkt_mask;
+              pkt = pkts[pos];
+              int must_reverse = ((synproxy_reply_mask & pkt_mask) != 0);
+
+              phy_port = pkt->port;
+              meta_data_addr = (struct mbuf_tcp_meta_data *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+              ehdr = rte_vfw_get_ether_addr(pkt);
+
+              struct ipv6_hdr *ihdr = (struct ipv6_hdr *)
+                     RTE_MBUF_METADATA_UINT32_PTR(pkt, IP_START);
+
+              uint8_t nhip[IPV6_ADD_SIZE];
+              uint8_t dest_address[IPV6_ADD_SIZE];
+
+              memset(nhip, 0, IPV6_ADD_SIZE);
+
+              rte_mov16(dest_address, ihdr->dst_addr);
+              ret = local_get_nh_ipv6(&dest_address[0], &dest_if,
+                            &nhip[0], vfw_pipe);
+              if (must_reverse) {
+                     rte_sp_exchange_mac_addresses(ehdr);
+                     if (is_phy_port_privte(phy_port)) {
+                            if (!ret) {
+                                   dest_if = get_pub_to_prv_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+
+                     } else {
+                            if (!ret) {
+                                   dest_if = get_prv_to_pub_port(
+                                                 (uint32_t *)
+                                                 &dest_address[0],
+                                                 IP_VERSION_6);
+                                   if (dest_if == INVALID_DESTIF) {
+                                          pkts_mask &= ~pkt_mask;
+                                          vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                                   }
+                                   do_local_nh_ipv6_cache(dest_if,
+                                                 vfw_pipe);
+                            }
+                     }
+
+              } else if (is_phy_port_privte(phy_port)) {
+                     if (!ret) {
+                            dest_if = get_prv_to_pub_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              } else {
+                     if (!ret) {
+                            dest_if = get_pub_to_prv_port((uint32_t *)
+                                          &dest_address[0], IP_VERSION_6);
+                            if (dest_if == INVALID_DESTIF) {
+                                   pkts_mask &= ~pkt_mask;
+                                   vfw_pipe->counters->
+                                          pkts_drop_without_arp_entry++;
+
+                            }
+                            do_local_nh_ipv6_cache(dest_if, vfw_pipe);
+                     }
+
+              }
+
+              meta_data_addr->output_port = vfw_pipe->outport_id[dest_if];
+
+              memset(nh_ipv6, 0, IPV6_ADD_SIZE);
+              if (get_dest_mac_address_ipv6_port(
+                                   &dest_address[0],
+                                   &dest_if,
+                                   &hw_addr,
+                                   &nh_ipv6[0])) {
+                     ether_addr_copy(&hw_addr, &ehdr->d_addr);
+                     ether_addr_copy(get_link_hw_addr(dest_if),
+                                   &ehdr->s_addr);
+
+                     if (vfw_debug >= DEBUG_LEVEL_4) {
+                            char buf[HW_ADDR_SIZE];
+
+                            ether_format_addr(buf, sizeof(buf),
+                                          &hw_addr);
+                            printf("MAC found for  dest_if %d: %s, ",
+                                          dest_if, buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->s_addr);
+                            printf("new eth hdr src: %s, ", buf);
+                            ether_format_addr(buf, sizeof(buf),
+                                          &ehdr->d_addr);
+                            printf("new eth hdr dst: %s\n", buf);
+                     }
+
+              } else {
+                     printf("deleting ipv6\n");
+                     pkts_mask &= ~pkt_mask;
+                     /*Next Neighbor is not yet implemented
+                      * for ipv6.*/
+                     vfw_pipe->counters->
+                            pkts_drop_without_arp_entry++;
+              }
+
+
+       }
+
+       return pkts_mask;
+}
+
+#endif
+/**
+ * Packets processing for connection tracking.
+ *
+ * @param vfw_pipe
+ *  A pointer to the pipeline.
+ * @param ct
+ *  A pointer to the connetion tracker .
+ * @param pkts
+ *  A pointer to a burst of packets.
+ * @param packet_mask_in
+ *  Input packets Mask.
+ */
+
+       static  uint64_t
+vfw_process_buffered_pkts(__rte_unused struct pipeline_vfw *vfw_pipe,
+              struct rte_ct_cnxn_tracker *ct,
+                          struct rte_mbuf **pkts, uint64_t packet_mask_in)
+{
+       uint64_t keep_mask = packet_mask_in;
+       struct rte_synproxy_helper sp_helper;       /* for synproxy */
+
+       keep_mask =
+           rte_ct_cnxn_tracker_batch_lookup_with_synproxy(ct, pkts, keep_mask,
+                                                    &sp_helper);
+
+       if (unlikely(sp_helper.hijack_mask))
+              printf("buffered hijack pkts severe error\n");
+
+       if (unlikely(sp_helper.reply_pkt_mask))
+              printf("buffered reply pkts severe error\n");
+
+       return keep_mask;
+}
+
+/**
+ * Free Packets from mbuf.
+ *
+ * @param ct
+ *  A pointer to the connection tracker to increment drop counter.
+ *
+ * @param pkt
+ *  Packet to be free.
+ */
+static inline void
+vfw_pktmbuf_free(struct rte_ct_cnxn_tracker *ct, struct rte_mbuf *pkt)
+{
+       ct->counters->pkts_drop++;
+       rte_pktmbuf_free(pkt);
+}
+
+static void
+vfw_output_or_delete_buffered_packets(struct rte_ct_cnxn_tracker *ct,
+                                    struct rte_pipeline *p,
+                                    struct rte_mbuf **pkts,
+                                    int num_pkts, uint64_t pkts_mask)
+{
+       int i;
+       struct mbuf_tcp_meta_data *meta_data_addr;
+       uint64_t pkt_mask = 1;
+
+       /* any clear bits in low-order num_pkts bit of
+        * pkt_mask must be discarded */
+
+       for (i = 0; i < num_pkts; i++) {
+              struct rte_mbuf *pkt = pkts[i];
+
+              if (pkts_mask & pkt_mask) {
+                     printf("vfw_output_or_delete_buffered_packets\n");
+                     meta_data_addr = (struct mbuf_tcp_meta_data *)
+                         RTE_MBUF_METADATA_UINT32_PTR(pkt, META_DATA_OFFSET);
+                     rte_pipeline_port_out_packet_insert(
+                                   p, meta_data_addr->output_port, pkt);
+
+              } else {
+                     vfw_pktmbuf_free(ct, pkt);
+              }
+
+              pkt_mask = pkt_mask << 1;
+       }
+}
+
+/**
+ *Packet buffered for synproxy.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+ * @param vfw_pipe
+ *  A pointer to the vfw pipeline.
+ * @param ct
+ *  A pointer to the connection tracker.
+ * @param forward_pkts
+ *  Packet forwarded by synproxy.
+ *
+ */
+static void
+vfw_handle_buffered_packets(struct rte_pipeline *p,
+                            struct pipeline_vfw *vfw_pipe,
+                            struct rte_ct_cnxn_tracker *ct, int forward_pkts)
+{
+       struct rte_mbuf *pkt_list = rte_ct_get_buffered_synproxy_packets(ct);
+
+       if (likely(pkt_list == NULL))       /* only during proxy setup is != NULL */
+              return;
+
+       int pkt_count = 0;
+       uint64_t keep_mask = 0;
+       struct rte_mbuf **pkts = vfw_pipe->pkt_buffer;
+       struct rte_mbuf *pkt;
+
+       while (pkt_list != NULL) {
+              struct mbuf_tcp_meta_data *meta_data =
+              (struct mbuf_tcp_meta_data *)
+              RTE_MBUF_METADATA_UINT32_PTR(pkt_list, META_DATA_OFFSET);
+
+              /* detach head of list and advance list */
+              pkt = pkt_list;
+              pkt_list = meta_data->next;
+
+              if (forward_pkts) {
+
+                     pkts[pkt_count++] = pkt;
+
+                     if (pkt_count == PKT_BUFFER_SIZE) {
+                            /* need to send out packets */
+                            /* currently 0, set all bits */
+                            keep_mask = ~keep_mask;
+
+                            keep_mask =
+                                vfw_process_buffered_pkts(vfw_pipe,
+                                                         ct, pkts,
+                                                         keep_mask);
+                            vfw_output_or_delete_buffered_packets(
+                                          ct, p,
+                                          pkts,
+                                          PKT_BUFFER_SIZE,
+                                          keep_mask);
+                            pkt_count = 0;
+                            keep_mask = 0;
+                     }
+
+              } else {
+                     vfw_pktmbuf_free(ct, pkt);
+              }
+       }
+
+       if (pkt_count != 0) {
+              /* need to send out packets */
+              keep_mask = RTE_LEN2MASK(pkt_count, uint64_t);
+
+              keep_mask =
+                     vfw_process_buffered_pkts(vfw_pipe, ct, pkts,
+                                   keep_mask);
+
+              vfw_output_or_delete_buffered_packets(ct, p, pkts, pkt_count,
+                            keep_mask);
+
+              pkt_count = 0;
+              keep_mask = 0;
+       }
+}
+
+/**
+ * The pipeline port-in action is used to do all the firewall and
+ * connection tracking work.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+  * @param pkts
+ *  A pointer to a burst of packets.
+ * @param n_pkts
+ *  Number of packets to process.
+ * @param arg
+ *  A pointer to pipeline specific data.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+
+static int
+vfw_port_in_action(struct rte_pipeline *p,
+              struct rte_mbuf **pkts,
+              __rte_unused uint32_t n_pkts, __rte_unused void *arg)
+{
+       struct vfw_ports_in_args *port_in_args =
+              (struct vfw_ports_in_args *)arg;
+       struct pipeline_vfw *vfw_pipe =
+              (struct pipeline_vfw *)port_in_args->pipe;
+       struct rte_ct_cnxn_tracker *ct = port_in_args->cnxn_tracker;
+
+       start_tsc_measure(vfw_pipe);
+
+       uint64_t packet_mask_in = RTE_LEN2MASK(n_pkts, uint64_t);
+       uint64_t pkts_drop_mask;
+       uint64_t hijack_mask = 0;
+       uint64_t synproxy_reply_mask = 0;       /* for synproxy */
+       uint64_t keep_mask = packet_mask_in;
+       struct rte_CT_helper ct_helper;
+
+       memset(&ct_helper, 0, sizeof(struct rte_CT_helper));
+
+
+       /*
+        * This routine uses a bit mask to represent which packets in the
+        * "pkts" table are considered valid. Any table entry which exists
+        * and is considered valid has the corresponding bit in the mask set.
+        * Otherwise, it is cleared. Note that the mask is 64 bits,
+        * but the number of packets in the table may be considerably less.
+        * Any mask bits which do correspond to actual packets are cleared.
+        * Various routines are called which may determine that an existing
+        * packet is somehow invalid. The routine will return an altered bit
+        * mask, with the bit cleared. At the end of all the checks,
+        * packets are dropped if their mask bit is a zero
+        */
+
+       if (vfw_debug > 1)
+              printf("Enter in-port action with %p packet mask\n",
+                            (void *)packet_mask_in);
+       vfw_pipe->counters->pkts_received =
+              vfw_pipe->counters->pkts_received + n_pkts;
+       if (VFW_DEBUG)
+              printf("vfw_port_in_action pkts_received: %" PRIu64
+                            " n_pkts: %u\n",
+                            vfw_pipe->counters->pkts_received, n_pkts);
+
+       /* first handle handle any previously buffered packets now released */
+       vfw_handle_buffered_packets(p, vfw_pipe, ct,
+                     FORWARD_BUFFERED_PACKETS);
+
+       /* now handle any new packets on input ports */
+       if (likely(firewall_flag)) {
+              keep_mask =
+                     rte_vfw_packet_filter_and_process(pkts, keep_mask,
+                                   vfw_pipe);
+              vfw_pipe->counters->pkts_fw_forwarded +=
+                     __builtin_popcountll(keep_mask);
+       }
+#ifdef ACL_ENABLE
+       uint64_t conntrack_mask = 0, connexist_mask = 0;
+       keep_mask = lib_acl_pkt_work_key(
+                     vfw_pipe->plib_acl, pkts, keep_mask,
+                     &vfw_pipe->counters->pkts_drop_without_rule,
+                     vfw_rule_table_ipv4_active,
+                     vfw_rule_table_ipv6_active,
+                     action_array_active,
+                     action_counter_table,
+                     &conntrack_mask, &connexist_mask,
+                     vfw_ipv4_enabled,
+                     vfw_ipv6_enabled);
+       vfw_pipe->counters->pkts_acl_forwarded +=
+              __builtin_popcountll(keep_mask);
+       if (conntrack_mask > 0) {
+              keep_mask = conntrack_mask;
+              ct_helper.no_new_cnxn_mask = connexist_mask;
+              cnxn_tracking_is_active = 1;
+       } else
+              cnxn_tracking_is_active = 0;
+#endif
+       if (likely(cnxn_tracking_is_active)) {
+              keep_mask = rte_ct_cnxn_tracker_batch_lookup(ct, pkts,
+                            keep_mask, &ct_helper);
+              synproxy_reply_mask = ct_helper.reply_pkt_mask;
+              hijack_mask = ct_helper.hijack_mask;
+
+       }
+
+
+       keep_mask =
+              rte_vfw_arp_packets(pkts, keep_mask, synproxy_reply_mask,
+                            vfw_pipe);
+
+       if (vfw_debug > 1) {
+              printf("  Exit in-port action with %p packet mask\n",
+                            (void *)keep_mask);
+              if (keep_mask != packet_mask_in)
+                     printf("dropped packets, %p in, %p out\n",
+                                   (void *)packet_mask_in,
+                                   (void *)keep_mask);
+       }
+
+       /* Update mask before returning, so that bad packets are dropped */
+
+       pkts_drop_mask = packet_mask_in & ~keep_mask;
+
+       if (unlikely(pkts_drop_mask != 0)) {
+              /* printf("drop %p\n", (void *) pkts_drop_mask); */
+              rte_pipeline_ah_packet_drop(p, pkts_drop_mask);
+       }
+
+       if (unlikely(hijack_mask != 0))
+              rte_pipeline_ah_packet_hijack(p, hijack_mask);
+
+       vfw_pipe->counters->num_batch_pkts_sum += n_pkts;
+       vfw_pipe->counters->num_pkts_measurements++;
+
+       end_tsc_measure(vfw_pipe, n_pkts);
+
+       return 0;
+}
+/**
+ * The pipeline port-in action is used to do all the firewall and
+ * connection tracking work for IPV4 packets.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+  * @param pkts
+ *  A pointer to a burst of packets.
+ * @param n_pkts
+ *  Number of packets to process.
+ * @param arg
+ *  A pointer to pipeline specific data.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+
+static int
+vfw_port_in_action_ipv4(struct rte_pipeline *p,
+              struct rte_mbuf **pkts,
+              __rte_unused uint32_t n_pkts, __rte_unused void *arg)
+{
+       struct vfw_ports_in_args *port_in_args =
+              (struct vfw_ports_in_args *)arg;
+       struct pipeline_vfw *vfw_pipe =
+              (struct pipeline_vfw *)port_in_args->pipe;
+       struct rte_ct_cnxn_tracker *ct = port_in_args->cnxn_tracker;
+
+       start_tsc_measure(vfw_pipe);
+
+       uint64_t packet_mask_in = RTE_LEN2MASK(n_pkts, uint64_t);
+       uint64_t pkts_drop_mask;
+       uint64_t hijack_mask = 0;
+       uint64_t synproxy_reply_mask = 0;       /* for synproxy */
+       uint64_t keep_mask = packet_mask_in;
+
+       uint64_t conntrack_mask = 0, connexist_mask = 0;
+       struct rte_CT_helper ct_helper;
+       uint8_t j;
+
+       /*
+        * This routine uses a bit mask to represent which packets in the
+        * "pkts" table are considered valid. Any table entry which exists
+        * and is considered valid has the corresponding bit in the mask set.
+        * Otherwise, it is cleared. Note that the mask is 64 bits,
+        * but the number of packets in the table may be considerably less.
+        * Any mask bits which do correspond to actual packets are cleared.
+        * Various routines are called which may determine that an existing
+        * packet is somehow invalid. The routine will return an altered bit
+        * mask, with the bit cleared. At the end of all the checks,
+        * packets are dropped if their mask bit is a zero
+        */
+
+       rte_prefetch0(& vfw_pipe->counters);
+
+#ifdef EN_SWP_ACL
+       /* Pre-fetch all rte_mbuf header */
+       for(j = 0; j < n_pkts; j++)
+              rte_prefetch0(pkts[j]);
+#endif
+       memset(&ct_helper, 0, sizeof(struct rte_CT_helper));
+#ifdef EN_SWP_ACL
+       rte_prefetch0(& vfw_pipe->counters->pkts_drop_ttl);
+       rte_prefetch0(& vfw_pipe->counters->sum_latencies);
+#endif
+
+       if (unlikely(vfw_debug > 1))
+              printf("Enter in-port action IPV4 with %p packet mask\n",
+                            (void *)packet_mask_in);
+       vfw_pipe->counters->pkts_received =
+              vfw_pipe->counters->pkts_received + n_pkts;
+
+       if (unlikely(VFW_DEBUG))
+              printf("vfw_port_in_action_ipv4 pkts_received: %" PRIu64
+                            " n_pkts: %u\n",
+                            vfw_pipe->counters->pkts_received, n_pkts);
+
+       /* first handle handle any previously buffered packets now released */
+       vfw_handle_buffered_packets(p, vfw_pipe, ct,
+                     FORWARD_BUFFERED_PACKETS);
+
+       /* now handle any new packets on input ports */
+       if (likely(firewall_flag)) {
+              keep_mask = rte_vfw_ipv4_packet_filter_and_process(pkts,
+                            keep_mask, vfw_pipe);
+              vfw_pipe->counters->pkts_fw_forwarded +=
+                     __builtin_popcountll(keep_mask);
+       }
+#ifdef ACL_ENABLE
+#ifdef EN_SWP_ACL
+       rte_prefetch0((void*)vfw_pipe->plib_acl);
+       rte_prefetch0((void*)vfw_rule_table_ipv4_active);
+#endif /* EN_SWP_ACL */
+       keep_mask = lib_acl_ipv4_pkt_work_key(
+                     vfw_pipe->plib_acl, pkts, keep_mask,
+                     &vfw_pipe->counters->pkts_drop_without_rule,
+                     vfw_rule_table_ipv4_active,
+                     action_array_active,
+                     action_counter_table,
+                     &conntrack_mask, &connexist_mask);
+       vfw_pipe->counters->pkts_acl_forwarded +=
+              __builtin_popcountll(keep_mask);
+       if (conntrack_mask > 0) {
+              keep_mask = conntrack_mask;
+              ct_helper.no_new_cnxn_mask = connexist_mask;
+              cnxn_tracking_is_active = 1;
+       } else
+              cnxn_tracking_is_active = 0;
+#endif /* ACL_ENABLE */
+
+       if (likely(cnxn_tracking_is_active)) {
+              rte_ct_cnxn_tracker_batch_lookup_type(ct, pkts,
+                            &keep_mask, &ct_helper, IPv4_HEADER_SIZE);
+              synproxy_reply_mask = ct_helper.reply_pkt_mask;
+              hijack_mask = ct_helper.hijack_mask;
+
+       }
+
+#ifdef EN_SWP_ARP
+       for(j = 0; j < (n_pkts & 0x3LLU); j++) {
+               rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   META_DATA_OFFSET));
+               rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   ETHERNET_START));
+       }
+       rte_prefetch0((void*)in_port_dir_a);
+       rte_prefetch0((void*)prv_to_pub_map);
+
+       uint8_t i;
+       for (i = 0; i < (n_pkts & (~0x3LLU)); i += 4) {
+              for (j = i+4; ((j < n_pkts) && (j < i+8)); j++) {
+                     rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                          META_DATA_OFFSET));
+                     rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                          ETHERNET_START));
+              }
+              pkt4_work_vfw_arp_ipv4_packets(&pkts[i], i, &keep_mask,
+                            synproxy_reply_mask, vfw_pipe);
+       }
+       for (j = i; j < n_pkts; j++) {
+              rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   META_DATA_OFFSET));
+              rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   ETHERNET_START));
+       }
+       for (; i < n_pkts; i++) {
+              pkt_work_vfw_arp_ipv4_packets(pkts[i], i, &keep_mask,
+                            synproxy_reply_mask, vfw_pipe);
+       }
+#else
+       rte_prefetch0((void*)in_port_dir_a);
+       rte_prefetch0((void*)prv_to_pub_map);
+       rte_prefetch0((void*) & vfw_pipe->local_lib_arp_route_table);
+       keep_mask = rte_vfw_arp_ipv4_packets(pkts, keep_mask,
+                     synproxy_reply_mask, vfw_pipe);
+#endif
+
+       if (vfw_debug > 1) {
+              printf("  Exit in-port action with %p packet mask\n",
+                            (void *)keep_mask);
+              if (keep_mask != packet_mask_in)
+                     printf("dropped packets, %p in, %p out\n",
+                                   (void *)packet_mask_in,
+                                   (void *)keep_mask);
+       }
+
+       /* Update mask before returning, so that bad packets are dropped */
+
+       pkts_drop_mask = packet_mask_in & ~keep_mask;
+
+       if (unlikely(pkts_drop_mask != 0)) {
+              /* printf("drop %p\n", (void *) pkts_drop_mask); */
+              rte_pipeline_ah_packet_drop(p, pkts_drop_mask);
+       }
+
+       if (unlikely(hijack_mask != 0))
+              rte_pipeline_ah_packet_hijack(p, hijack_mask);
+
+       vfw_pipe->counters->num_batch_pkts_sum += n_pkts;
+       vfw_pipe->counters->num_pkts_measurements++;
+
+       end_tsc_measure(vfw_pipe, n_pkts);
+
+       return 0;
+}
+/**
+ * The pipeline port-in action is used to do all the firewall and
+ * connection tracking work for IPV6 packet.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+  * @param pkts
+ *  A pointer to a burst of packets.
+ * @param n_pkts
+ *  Number of packets to process.
+ * @param arg
+ *  A pointer to pipeline specific data.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+
+static int
+vfw_port_in_action_ipv6(struct rte_pipeline *p,
+              struct rte_mbuf **pkts,
+              __rte_unused uint32_t n_pkts, __rte_unused void *arg)
+{
+       struct vfw_ports_in_args *port_in_args =
+              (struct vfw_ports_in_args *)arg;
+       struct pipeline_vfw *vfw_pipe =
+              (struct pipeline_vfw *)port_in_args->pipe;
+       struct rte_ct_cnxn_tracker *ct = port_in_args->cnxn_tracker;
+
+       start_tsc_measure(vfw_pipe);
+
+       uint64_t packet_mask_in = RTE_LEN2MASK(n_pkts, uint64_t);
+       uint64_t pkts_drop_mask;
+       uint64_t hijack_mask = 0;
+       uint64_t synproxy_reply_mask = 0;       /* for synproxy */
+       uint64_t keep_mask = packet_mask_in;
+
+       uint64_t conntrack_mask = 0, connexist_mask = 0;
+       struct rte_CT_helper ct_helper;
+       uint32_t j;
+
+       /*
+        * This routine uses a bit mask to represent which packets in the
+        * "pkts" table are considered valid. Any table entry which exists
+        * and is considered valid has the corresponding bit in the mask set.
+        * Otherwise, it is cleared. Note that the mask is 64 bits,
+        * but the number of packets in the table may be considerably less.
+        * Any mask bits which do correspond to actual packets are cleared.
+        * Various routines are called which may determine that an existing
+        * packet is somehow invalid. The routine will return an altered bit
+        * mask, with the bit cleared. At the end of all the checks,
+        * packets are dropped if their mask bit is a zero
+        */
+
+       rte_prefetch0(& vfw_pipe->counters);
+
+       /* Pre-fetch all rte_mbuf header */
+       for(j = 0; j < n_pkts; j++)
+               rte_prefetch0(pkts[j]);
+
+       memset(&ct_helper, 0, sizeof(struct rte_CT_helper));
+       rte_prefetch0(& vfw_pipe->counters->pkts_drop_ttl);
+       rte_prefetch0(& vfw_pipe->counters->sum_latencies);
+
+       if (vfw_debug > 1)
+              printf("Enter in-port action with %p packet mask\n",
+                            (void *)packet_mask_in);
+       vfw_pipe->counters->pkts_received =
+              vfw_pipe->counters->pkts_received + n_pkts;
+       if (VFW_DEBUG)
+              printf("vfw_port_in_action pkts_received: %" PRIu64
+                            " n_pkts: %u\n",
+                            vfw_pipe->counters->pkts_received, n_pkts);
+
+       /* first handle handle any previously buffered packets now released */
+       vfw_handle_buffered_packets(p, vfw_pipe, ct,
+                     FORWARD_BUFFERED_PACKETS);
+
+       /* now handle any new packets on input ports */
+       if (likely(firewall_flag)) {
+              keep_mask = rte_vfw_ipv6_packet_filter_and_process(pkts,
+                            keep_mask, vfw_pipe);
+              vfw_pipe->counters->pkts_fw_forwarded +=
+                     __builtin_popcountll(keep_mask);
+       }
+#ifdef ACL_ENABLE
+
+#ifdef EN_SWP_ACL
+       rte_prefetch0((void*)vfw_pipe->plib_acl);
+       rte_prefetch0((void*)vfw_rule_table_ipv6_active);
+#endif /* EN_SWP_ACL */
+       keep_mask = lib_acl_ipv6_pkt_work_key(
+                     vfw_pipe->plib_acl, pkts, keep_mask,
+                     &vfw_pipe->counters->pkts_drop_without_rule,
+                     vfw_rule_table_ipv6_active,
+                     action_array_active,
+                     action_counter_table,
+                     &conntrack_mask, &connexist_mask);
+       vfw_pipe->counters->pkts_acl_forwarded +=
+              __builtin_popcountll(keep_mask);
+       if (conntrack_mask > 0) {
+              keep_mask = conntrack_mask;
+              ct_helper.no_new_cnxn_mask = connexist_mask;
+              cnxn_tracking_is_active = 1;
+       } else
+              cnxn_tracking_is_active = 0;
+#endif /* ACL_ENABLE */
+       if (likely(cnxn_tracking_is_active)) {
+              rte_ct_cnxn_tracker_batch_lookup_type(ct, pkts,
+                            &keep_mask, &ct_helper, IPv6_HEADER_SIZE);
+              synproxy_reply_mask = ct_helper.reply_pkt_mask;
+              hijack_mask = ct_helper.hijack_mask;
+
+       }
+
+#ifdef EN_SWP_ARP
+       for(j = 0; j < (n_pkts & 0x3LLU); j++) {
+               rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   META_DATA_OFFSET));
+               rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   ETHERNET_START));
+       }
+       rte_prefetch0((void*)in_port_dir_a);
+       rte_prefetch0(vfw_pipe->local_lib_nd_route_table);
+       uint32_t i;
+
+       for (i = 0; i < (n_pkts & (~0x3LLU)); i += 4) {
+              for (j = i+4; ((j < n_pkts) && (j < i+8)); j++) {
+                     rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                          META_DATA_OFFSET));
+                     rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                          ETHERNET_START));
+              }
+              pkt4_work_vfw_arp_ipv6_packets(&pkts[i], i, &keep_mask,
+                            synproxy_reply_mask, vfw_pipe);
+       }
+       for (j = i; j < n_pkts; j++) {
+              rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   META_DATA_OFFSET));
+              rte_prefetch0(RTE_MBUF_METADATA_UINT32_PTR(pkts[j],
+                                   ETHERNET_START));
+       }
+       for (; i < n_pkts; i++) {
+              pkt_work_vfw_arp_ipv6_packets(pkts[i], i, &keep_mask,
+                            synproxy_reply_mask, vfw_pipe);
+       }
+#else
+       rte_prefetch0((void*)in_port_dir_a);
+       rte_prefetch0((void*) & vfw_pipe->local_lib_arp_route_table);
+       keep_mask = rte_vfw_arp_ipv6_packets(pkts, keep_mask,
+                     synproxy_reply_mask, vfw_pipe);
+#endif
+
+       if (vfw_debug > 1) {
+              printf("  Exit in-port action with %p packet mask\n",
+                            (void *)keep_mask);
+              if (keep_mask != packet_mask_in)
+                     printf("dropped packets, %p in, %p out\n",
+                                   (void *)packet_mask_in,
+                                   (void *)keep_mask);
+       }
+
+       /* Update mask before returning, so that bad packets are dropped */
+
+       pkts_drop_mask = packet_mask_in & ~keep_mask;
+
+       if (unlikely(pkts_drop_mask != 0)) {
+              /* printf("drop %p\n", (void *) pkts_drop_mask); */
+              rte_pipeline_ah_packet_drop(p, pkts_drop_mask);
+       }
+
+       if (unlikely(hijack_mask != 0))
+              rte_pipeline_ah_packet_hijack(p, hijack_mask);
+
+       vfw_pipe->counters->num_batch_pkts_sum += n_pkts;
+       vfw_pipe->counters->num_pkts_measurements++;
+
+       end_tsc_measure(vfw_pipe, n_pkts);
+
+       return 0;
+}
+
+
+/**
+ * Parse arguments in config file.
+ *
+ * @param vfw_pipe
+ *  A pointer to the pipeline.
+ * @param params
+ *  A pointer to pipeline specific parameters.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int
+pipeline_vfw_parse_args(struct pipeline_vfw *vfw_pipe,
+              struct pipeline_params *params)
+{
+       uint32_t i;
+       int status;
+
+       if (vfw_debug)
+              printf("VFW pipeline_vfw_parse_args params->n_args: %d\n",
+                            params->n_args);
+
+       for (i = 0; i < params->n_args; i++) {
+              char *arg_name = params->args_name[i];
+              char *arg_value = params->args_value[i];
+
+              printf("VFW args[%d]: %s %d, %s\n", i, arg_name,
+                            atoi(arg_value), arg_value);
+#ifdef ACL_ENABLE
+              status = lib_acl_parse_config(vfw_pipe->plib_acl,
+                                   arg_name, arg_value, &vfw_n_rules);
+              if (status < 0) {
+                     printf("rte_ct_set_configuration_options =%s,%s",
+                                   arg_name, arg_value);
+                     return -1;
+              } else if (status == 0)
+                     continue;
+
+#endif              /* traffic_type */
+              if (strcmp(arg_name, "traffic_type") == 0) {
+                     int traffic_type = atoi(arg_value);
+
+                     if (traffic_type == 0 ||
+                                   !(traffic_type == IP_VERSION_4 ||
+                                          traffic_type == IP_VERSION_6)) {
+                            printf("not IPV4/IPV6");
+                            return -1;
+                     }
+
+                     vfw_pipe->traffic_type = traffic_type;
+                     continue;
+              }
+
+
+              /* n_flows */
+              if (strcmp(arg_name, "n_flows") == 0) {
+                     int n_flows = atoi(arg_value);
+
+                     if (n_flows == 0)
+                            return -1;
+
+                     /* must be power of 2, round up if not */
+                     if (!rte_is_power_of_2(n_flows))
+                            n_flows = rte_align32pow2(n_flows);
+
+                     vfw_pipe->n_flows = n_flows;
+                     continue;
+              }
+
+              /* not firewall option, process as cnxn tracking option */
+              status = rte_ct_set_configuration_options(
+                            vfw_pipe->cnxn_tracker,
+                            arg_name, arg_value);
+              if (status < 0) {
+                     printf("rte_ct_set_configuration_options =%s,%s",
+                                   arg_name, arg_value);
+                     return -1;
+              } else if (status == 0)
+                     continue;
+
+       }
+
+       return 0;
+}
+
+static void *pipeline_vfw_msg_req_custom_handler(struct pipeline *p,
+                                              void *msg);
+
+static pipeline_msg_req_handler handlers[] = {
+       [PIPELINE_MSG_REQ_PING] = pipeline_msg_req_ping_handler,
+       [PIPELINE_MSG_REQ_STATS_PORT_IN] =
+           pipeline_msg_req_stats_port_in_handler,
+       [PIPELINE_MSG_REQ_STATS_PORT_OUT] =
+           pipeline_msg_req_stats_port_out_handler,
+       [PIPELINE_MSG_REQ_STATS_TABLE] = pipeline_msg_req_stats_table_handler,
+       [PIPELINE_MSG_REQ_PORT_IN_ENABLE] =
+           pipeline_msg_req_port_in_enable_handler,
+       [PIPELINE_MSG_REQ_PORT_IN_DISABLE] =
+           pipeline_msg_req_port_in_disable_handler,
+       [PIPELINE_MSG_REQ_CUSTOM] = pipeline_vfw_msg_req_custom_handler,
+};
+
+static void *pipeline_vfw_msg_req_synproxy_flag_handler(struct pipeline *p,
+                                                    void *msg);
+static pipeline_msg_req_handler custom_handlers[] = {
+
+       [PIPELINE_VFW_MSG_REQ_SYNPROXY_FLAGS] =
+           pipeline_vfw_msg_req_synproxy_flag_handler
+};
+
+/**
+ * Create and initialize Pipeline Back End (BE).
+ *
+ * @param params
+ *  A pointer to the pipeline specific parameters..
+ * @param arg
+ *  A pointer to pipeline specific data.
+ *
+ * @return
+ *  A pointer to the pipeline create, NULL on error.
+ */
+static void
+*pipeline_vfw_init(struct pipeline_params *params, __rte_unused void *arg)
+{
+       uint32_t size, i;
+
+       /* Check input arguments */
+       if ((params == NULL) ||
+                     (params->n_ports_in == 0) || (params->n_ports_out == 0))
+              return NULL;
+
+       if (vfw_debug)
+              printf("num ports in %d / num ports out %d\n",
+                            params->n_ports_in, params->n_ports_out);
+
+       /* Create a single pipeline instance and initialize. */
+       struct pipeline_vfw *pipe_vfw;
+
+       size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct pipeline_vfw));
+       pipe_vfw = rte_zmalloc(NULL, size, RTE_CACHE_LINE_SIZE);
+
+       if (pipe_vfw == NULL)
+              return NULL;
+
+       struct pipeline *pipe;
+
+       pipe = &pipe_vfw->pipe;
+
+       strncpy(pipe->name, params->name, sizeof(pipe->name));
+       pipe->log_level = params->log_level;
+       pipe_vfw->n_flows = 4096;       /* small default value */
+       pipe_vfw->traffic_type = MIX;
+       pipe_vfw->pipeline_num = 0xff;
+       for (i = 0; i < PIPELINE_MAX_PORT_IN; i++) {
+              pipe_vfw->links_map[i] = 0xff;
+              pipe_vfw->outport_id[i] = 0xff;
+       }
+       PLOG(pipe, HIGH, "VFW");
+
+       /* Create a firewall instance and initialize. */
+       pipe_vfw->cnxn_tracker =
+              rte_zmalloc(NULL, rte_ct_get_cnxn_tracker_size(),
+                            RTE_CACHE_LINE_SIZE);
+
+       if (pipe_vfw->cnxn_tracker == NULL)
+              return NULL;
+#ifdef ACL_ENABLE
+       /* Create a acl instance and initialize. */
+       pipe_vfw->plib_acl =
+              rte_zmalloc(NULL, sizeof(struct lib_acl),
+                            RTE_CACHE_LINE_SIZE);
+
+       if (pipe_vfw->plib_acl == NULL)
+              return NULL;
+#endif
+       timer_lcore = rte_lcore_id();
+       /*
+        * Now allocate a counter block entry. It appears that the
+        * initialization of all instances is serialized on core 0,
+        * so no lock is necessary.
+        */
+       struct rte_VFW_counter_block *counter_ptr;
+
+       if (rte_VFW_hi_counter_block_in_use == MAX_VFW_INSTANCES)
+              /* error, exceeded table bounds */
+              return NULL;
+
+       rte_VFW_hi_counter_block_in_use++;
+       counter_ptr =
+              &rte_vfw_counter_table[rte_VFW_hi_counter_block_in_use];
+       strncpy(counter_ptr->name, params->name, sizeof(counter_ptr->name));
+
+       pipe_vfw->counters = counter_ptr;
+
+       rte_ct_initialize_default_timeouts(pipe_vfw->cnxn_tracker);
+       /* Parse arguments */
+       if (pipeline_vfw_parse_args(pipe_vfw, params))
+              return NULL;
+
+       uint16_t pointers_offset =
+              META_DATA_OFFSET + offsetof(struct mbuf_tcp_meta_data, next);
+
+       if (pipe_vfw->n_flows > 0)
+              rte_ct_initialize_cnxn_tracker_with_synproxy(
+                            pipe_vfw->cnxn_tracker,
+                            pipe_vfw->n_flows,
+                            params->name,
+                            pointers_offset);
+
+       pipe_vfw->counters->ct_counters =
+              rte_ct_get_counter_address(pipe_vfw->cnxn_tracker);
+
+       /* Pipeline */
+       {
+              struct rte_pipeline_params pipeline_params = {
+                     .name = params->name,
+                     .socket_id = params->socket_id,
+                     .offset_port_id = META_DATA_OFFSET +
+                            offsetof(struct mbuf_tcp_meta_data, output_port)
+              };
+
+              pipe->p = rte_pipeline_create(&pipeline_params);
+              if (pipe->p == NULL) {
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+       /* Input ports */
+
+       /*
+        * create a different "arg_ah" for each input port.
+        * They differ only in the recorded port number. Unfortunately,
+        * IP_PIPELINE does not pass port number in to input port handler
+        */
+
+       uint32_t in_ports_arg_size =
+              RTE_CACHE_LINE_ROUNDUP((sizeof(struct vfw_ports_in_args)) *
+                            (params->n_ports_in));
+       struct vfw_ports_in_args *port_in_args =
+              (struct vfw_ports_in_args *)
+              rte_zmalloc(NULL, in_ports_arg_size, RTE_CACHE_LINE_SIZE);
+
+       if (port_in_args == NULL)
+              return NULL;
+
+       pipe->n_ports_in = params->n_ports_in;
+       for (i = 0; i < pipe->n_ports_in; i++) {
+
+              /* initialize this instance of port_in_args as necessary */
+              port_in_args[i].pipe = pipe;
+              port_in_args[i].cnxn_tracker = pipe_vfw->cnxn_tracker;
+
+              struct rte_pipeline_port_in_params port_params = {
+                     .ops =
+                            pipeline_port_in_params_get_ops(&params->port_in
+                                          [i]),
+                     .arg_create =
+                            pipeline_port_in_params_convert(&params->port_in
+                                          [i]),
+                     .f_action = vfw_port_in_action,
+                     .arg_ah = &(port_in_args[i]),
+                     .burst_size = params->port_in[i].burst_size,
+              };
+              if (pipe_vfw->traffic_type == IP_VERSION_4)
+                     port_params.f_action = vfw_port_in_action_ipv4;
+
+               if (pipe_vfw->traffic_type == IP_VERSION_6)
+                     port_params.f_action = vfw_port_in_action_ipv6;
+              int status = rte_pipeline_port_in_create(pipe->p, &port_params,
+                            &pipe->port_in_id[i]);
+
+              if (status) {
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+       /* Output ports */
+       pipe->n_ports_out = params->n_ports_out;
+       for (i = 0; i < pipe->n_ports_out; i++) {
+              struct rte_pipeline_port_out_params port_params = {
+                     .ops = pipeline_port_out_params_get_ops(
+                                   &params->port_out[i]),
+                     .arg_create = pipeline_port_out_params_convert(
+                                   &params->port_out[i]),
+                     .f_action = NULL,
+                     .arg_ah = NULL,
+              };
+
+              int status = rte_pipeline_port_out_create(pipe->p, &port_params,
+                            &pipe->port_out_id[i]);
+
+              if (status) {
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+       int pipeline_num = 0;
+       int dont_care = sscanf(params->name, "PIPELINE%d", &pipeline_num);
+
+       if (dont_care < 0)
+              printf("sscanf unble to read pipeline id\n");
+       pipe_vfw->pipeline_num = (uint8_t) pipeline_num;
+       register_pipeline_Qs(pipe_vfw->pipeline_num, pipe);
+       set_link_map(pipe_vfw->pipeline_num, pipe, pipe_vfw->links_map);
+       set_outport_id(pipe_vfw->pipeline_num, pipe,
+                     pipe_vfw->outport_id);
+       printf("pipeline_num=%d\n", pipeline_num);
+#ifdef ACL_ENABLE
+       /*If this is the first VFW thread, create common VFW Rule tables*/
+       if (rte_VFW_hi_counter_block_in_use == 0) {
+              vfw_rule_table_ipv4_active =
+                     lib_acl_create_active_standby_table_ipv4(1,
+                                   &vfw_n_rules);
+              if (vfw_rule_table_ipv4_active == NULL) {
+                     printf("Failed to create active table for IPV4\n");
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw->cnxn_tracker);
+                     rte_free(pipe_vfw->plib_acl);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+              vfw_rule_table_ipv4_standby =
+                     lib_acl_create_active_standby_table_ipv4(2,
+                                   &vfw_n_rules);
+              if (vfw_rule_table_ipv4_standby == NULL) {
+                     printf("Failed to create standby table for IPV4\n");
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw->cnxn_tracker);
+                     rte_free(pipe_vfw->plib_acl);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+
+              vfw_rule_table_ipv6_active =
+                     lib_acl_create_active_standby_table_ipv6(1,
+                                   &vfw_n_rules);
+
+              if (vfw_rule_table_ipv6_active == NULL) {
+                     printf("Failed to create active table for IPV6\n");
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw->cnxn_tracker);
+                     rte_free(pipe_vfw->plib_acl);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+              vfw_rule_table_ipv6_standby =
+                     lib_acl_create_active_standby_table_ipv6(2,
+                                   &vfw_n_rules);
+              if (vfw_rule_table_ipv6_standby == NULL) {
+                     printf("Failed to create standby table for IPV6\n");
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw->cnxn_tracker);
+                     rte_free(pipe_vfw->plib_acl);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+#endif
+
+       /* Tables */
+
+       pipe->n_tables = 1;
+
+       struct rte_pipeline_table_params table_params = {
+              .ops = &rte_table_stub_ops,
+              .arg_create = NULL,
+              .f_action_hit = NULL,
+              .f_action_miss = NULL,
+              .arg_ah = NULL,
+              .action_data_size = 0,
+       };
+
+       int status = rte_pipeline_table_create(pipe->p,
+                     &table_params,
+                     &pipe->table_id[0]);
+
+       if (status) {
+              rte_pipeline_free(pipe->p);
+              rte_free(pipe);
+              return NULL;
+       }
+
+       struct rte_pipeline_table_entry default_entry = {
+              .action = RTE_PIPELINE_ACTION_PORT_META
+       };
+
+       struct rte_pipeline_table_entry *default_entry_ptr;
+
+       status = rte_pipeline_table_default_entry_add(pipe->p,
+                                                pipe->table_id[0],
+                                                &default_entry,
+                                                &default_entry_ptr);
+
+       if (status) {
+              rte_pipeline_free(pipe->p);
+              rte_free(pipe);
+              return NULL;
+       }
+       for (i = 0; i < pipe->n_ports_in; i++) {
+              int status = rte_pipeline_port_in_connect_to_table(
+                            pipe->p,
+                            pipe->port_in_id[i],
+                            pipe->table_id[0]);
+
+              if (status) {
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+       /* Enable input ports */
+       for (i = 0; i < pipe->n_ports_in; i++) {
+              int status =
+                  rte_pipeline_port_in_enable(pipe->p, pipe->port_in_id[i]);
+
+              if (status) {
+                     rte_pipeline_free(pipe->p);
+                     rte_free(pipe_vfw);
+                     return NULL;
+              }
+       }
+
+       /* Check pipeline consistency */
+       if (rte_pipeline_check(pipe->p) < 0) {
+              rte_pipeline_free(pipe->p);
+              rte_free(pipe_vfw);
+              return NULL;
+       }
+
+       /* Message queues */
+       pipe->n_msgq = params->n_msgq;
+       for (i = 0; i < pipe->n_msgq; i++)
+              pipe->msgq_in[i] = params->msgq_in[i];
+
+       for (i = 0; i < pipe->n_msgq; i++)
+              pipe->msgq_out[i] = params->msgq_out[i];
+
+       /* Message handlers */
+       memcpy(pipe->handlers, handlers, sizeof(pipe->handlers));
+       memcpy(pipe_vfw->custom_handlers, custom_handlers,
+              sizeof(pipe_vfw->custom_handlers));
+
+       return pipe_vfw;
+}
+
+/**
+ * Free resources and delete pipeline.
+ *
+ * @param pipeline
+ *  A pointer to the pipeline.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int pipeline_vfw_free(void *pipeline)
+{
+       struct pipeline *p = (struct pipeline *)pipeline;
+
+       /* Check input arguments */
+       if (p == NULL)
+              return -1;
+
+       /* Free resources */
+       rte_pipeline_free(p->p);
+       rte_free(p);
+       return 0;
+}
+
+/**
+ * Callback function to map input/output ports.
+ *
+ * @param pipeline
+ *  A pointer to the pipeline.
+ * @param port_in
+ *  Input port ID
+ * @param port_out
+ *  A pointer to the Output port.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int
+pipeline_vfw_track(void *pipeline, __rte_unused uint32_t port_in,
+                    uint32_t *port_out)
+{
+       struct pipeline *p = (struct pipeline *)pipeline;
+
+       /* Check input arguments */
+       if ((p == NULL) || (port_in >= p->n_ports_in) || (port_out == NULL))
+              return -1;
+
+       if (p->n_ports_in == 1) {
+              *port_out = 0;
+              return 0;
+       }
+
+       return -1;
+}
+
+/**
+ * Callback function to process timers.
+ *
+ * @param pipeline
+ *  A pointer to the pipeline.
+ *
+ * @return
+ *  0 on success, negative on error.
+ */
+static int pipeline_vfw_timer(void *pipeline)
+{
+       struct pipeline_vfw *p = (struct pipeline_vfw *)pipeline;
+
+       /*
+        * handle any good buffered packets released by synproxy before checking
+        * for packets relased by synproxy due to timeout.
+        * Don't want packets missed
+        */
+
+       vfw_handle_buffered_packets(p->pipe.p, p, p->cnxn_tracker,
+                                   FORWARD_BUFFERED_PACKETS);
+
+       pipeline_msg_req_handle(&p->pipe);
+       rte_pipeline_flush(p->pipe.p);
+
+       rte_ct_handle_expired_timers(p->cnxn_tracker);
+
+       /* now handle packets released by synproxy due to timeout. */
+       vfw_handle_buffered_packets(p->pipe.p, p, p->cnxn_tracker,
+                                   DELETE_BUFFERED_PACKETS);
+
+       return 0;
+}
+
+/**
+ * Callback function to process CLI commands from FE.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+ * @param msg
+ *  A pointer to command specific data.
+ *
+ * @return
+ *  A pointer to message handler on success,
+ *  pipeline_msg_req_invalid_hander on error.
+ */
+void *pipeline_vfw_msg_req_custom_handler(struct pipeline *p, void *msg)
+{
+       struct pipeline_vfw *pipe_vfw = (struct pipeline_vfw *)p;
+       struct pipeline_custom_msg_req *req = msg;
+       pipeline_msg_req_handler f_handle;
+
+       f_handle = (req->subtype < PIPELINE_VFW_MSG_REQS) ?
+           pipe_vfw->custom_handlers[req->subtype] :
+           pipeline_msg_req_invalid_handler;
+
+       if (f_handle == NULL)
+              f_handle = pipeline_msg_req_invalid_handler;
+
+       return f_handle(p, req);
+}
+
+/**
+ * Handler for synproxy ON/OFF CLI command.
+ *
+ * @param p
+ *  A pointer to the pipeline.
+ * @param msg
+ *  A pointer to command specific data.
+ *
+ * @return
+ *  Response message contains status.
+ */
+
+void *pipeline_vfw_msg_req_synproxy_flag_handler(struct pipeline *p,
+                                              void *msg)
+{
+       struct pipeline_vfw *pipe_vfw = (struct pipeline_vfw *)p;
+       struct pipeline_vfw_synproxy_flag_msg_req *req = msg;
+       struct pipeline_vfw_synproxy_flag_msg_rsp *rsp = msg;
+
+       if (req->synproxy_flag == 0) {
+              rte_ct_disable_synproxy(pipe_vfw->cnxn_tracker);
+              rsp->status = 0;
+              printf("synproxy turned OFF for %s\n", p->name);
+       } else if (req->synproxy_flag == 1) {
+              rte_ct_enable_synproxy(pipe_vfw->cnxn_tracker);
+              rsp->status = 0;
+              printf("synproxy turned ON for %s\n", p->name);
+       } else {
+              printf("Invalid synproxy setting\n");
+              rsp->status = -1;
+       }
+
+       return rsp;
+}
+
+struct pipeline_be_ops pipeline_vfw_be_ops = {
+       .f_init = pipeline_vfw_init,
+       .f_free = pipeline_vfw_free,
+       .f_run = NULL,
+       .f_timer = pipeline_vfw_timer,
+       .f_track = pipeline_vfw_track,
+};
diff --git a/VNFs/vFW/pipeline/pipeline_vfw_be.h b/VNFs/vFW/pipeline/pipeline_vfw_be.h
new file mode 100644 (file)
index 0000000..7e90c7c
--- /dev/null
@@ -0,0 +1,218 @@
+/*
+// Copyright (c) 2017 Intel Corporation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+*/
+
+#ifndef __INCLUDE_PIPELINE_VFW_BE_H__
+#define __INCLUDE_PIPELINE_VFW_BE_H__
+
+/**
+ * @file
+ * Pipeline VFW BE.
+ *
+ * Pipeline VFW Back End (BE).
+ * Responsible for packet processing.
+ *
+ */
+#include <stdint.h>
+#include <rte_ether.h>
+
+#include "pipeline_common_be.h"
+#include "rte_cnxn_tracking.h"
+#include "rte_ct_tcp.h"
+#include "lib_acl.h"
+
+/*#define VFW_DEBUG 0*/
+uint8_t vfw_debug;
+extern uint8_t VFW_DEBUG;
+extern uint8_t firewall_flag;
+extern uint8_t cnxn_tracking_is_active;
+#define KEY_SIZE 10              /*IPV4 src_ip + dst_ip + src_port + dst_port */
+#define IP_32BIT_SIZE 4
+#define MAX_VFW_INSTANCES 12       /* max number fw threads, actual usually less */
+#define IPv4_HDR_VERSION 4
+#define IPv6_HDR_VERSION 6
+#define IP_VERSION_CHECK 4
+extern int rte_VFW_hi_counter_block_in_use;
+
+enum pipeline_vfw_key_type {
+       PIPELINE_VFW_IPV4_5TUPLE,
+       PIPELINE_VFW_IPV6_5TUPLE
+};
+ /**
+ * A structure defining the VFW counter block.
+ * One counter block per VFW Thread
+ */
+struct rte_VFW_counter_block {
+       char name[PIPELINE_NAME_SIZE];
+
+       /* as long as a counter doesn't cross cache line, writes are atomic */
+       uint64_t pkts_received;
+       uint64_t bytes_processed; /**< includes all L3 and higher headers. */
+       uint64_t num_batch_pkts_sum;
+       uint32_t num_pkts_measurements;
+       uint32_t unused_counter;
+
+       uint64_t entry_timestamp;
+       uint64_t exit_timestamp;
+       uint64_t internal_time_sum;
+       uint64_t external_time_sum;
+       uint32_t time_measurements;
+       uint32_t count_latencies;
+       /**< Sum latencies */
+       uint64_t sum_latencies;
+       uint64_t pkts_drop_without_rule;
+       uint64_t pkts_acl_forwarded;
+
+       /**< Total packets drop for ttl value by firewall.*/
+       uint64_t pkts_drop_ttl;
+       /**< Total packets drop for bad size by firewall. */
+       uint64_t pkts_drop_bad_size;
+       /**< Total packets drop for fragmented by firewall. */
+       uint64_t pkts_drop_fragmented;
+       /**< Total packets drop for without arp entry by firewall.*/
+       uint64_t pkts_drop_without_arp_entry;
+       /**< Total packets drop for ipv6 not tcp/udp by firewall. */
+       uint64_t pkts_drop_unsupported_type;
+       /**< A pointer to connection tracker counters.*/
+       struct rte_CT_counter_block *ct_counters;
+       /* average latency = sum_latencies / count_latencies */
+       uint64_t pkts_fw_forwarded;
+       uint64_t arpicmpPktCount;
+} __rte_cache_aligned;
+
+/** The counter table for VFW pipeline per thread data.*/
+extern struct rte_VFW_counter_block
+rte_vfw_counter_table[MAX_VFW_INSTANCES] __rte_cache_aligned;
+
+/**
+ * A structure defining the IPv4 5-Tuple for VFW rules.
+ */
+struct pipeline_vfw_key_ipv4_5tuple {
+       uint32_t src_ip;
+       uint32_t src_ip_mask;
+       uint32_t dst_ip;
+       uint32_t dst_ip_mask;
+       uint16_t src_port_from;
+       uint16_t src_port_to;
+       uint16_t dst_port_from;
+       uint16_t dst_port_to;
+       uint8_t proto;
+       uint8_t proto_mask;
+};
+
+/**
+ * A structure defining the IPv6 5-Tuple for VFW rules.
+ */
+struct pipeline_vfw_key_ipv6_5tuple {
+       uint8_t src_ip[16];
+       uint32_t src_ip_mask;
+       uint8_t dst_ip[16];
+       uint32_t dst_ip_mask;
+       uint16_t src_port_from;
+       uint16_t src_port_to;
+       uint16_t dst_port_from;
+       uint16_t dst_port_to;
+       uint8_t proto;
+       uint8_t proto_mask;
+};
+
+/* Messages from CLI for processing by packet processing */
+
+enum pipeline_tcpfw_msg_req_type {
+
+       PIPELINE_TCPFW_MSG_REQ_ENTRY_STATUS,
+       PIPELINE_TCPFW_MSG_REQ_DBG,
+       PIPELINE_TCPFW_MSG_REQ_SYNPROXY_FLAGS,
+       PIPELINE_TCPFW_MSG_REQS
+};
+/**
+ * A structure defining the key to store VFW rule.
+ * For both IPv4 and IPv6.
+ */
+struct pipeline_vfw_key {
+       enum pipeline_vfw_key_type type;
+       union {
+              struct pipeline_vfw_key_ipv4_5tuple ipv4_5tuple;
+              struct pipeline_vfw_key_ipv6_5tuple ipv6_5tuple;
+       } key;
+};
+
+
+
+extern struct pipeline_action_key *action_array_a;
+extern struct pipeline_action_key *action_array_b;
+extern struct pipeline_action_key *action_array_active;
+extern struct pipeline_action_key *action_array_standby;
+extern uint32_t action_array_size;
+
+extern struct action_counter_block
+action_counter_table[MAX_VFW_INSTANCES][action_array_max]
+__rte_cache_aligned;
+
+/**
+ * A structure defining the add VFW rule command response message.
+ */
+struct pipeline_vfw_add_msg_rsp {
+       int status;
+       int key_found;
+       void *entry_ptr;
+};
+
+struct app_pipeline_vfw_entry_params {
+       uint32_t s_addr;
+       uint16_t s_port;
+       uint32_t d_addr;
+       uint16_t d_port;
+
+};
+
+struct pipeline_vfw_entry_key {
+       uint32_t ip1[IP_32BIT_SIZE];
+       uint32_t ip2[IP_32BIT_SIZE];
+       uint16_t port1;
+       uint16_t port2;
+};
+
+/* Messages from CLI for processing by packet processing */
+
+enum pipeline_vfw_msg_req_type {
+       PIPELINE_VFW_MSG_REQ_SYNPROXY_FLAGS,
+       PIPELINE_VFW_MSG_REQS
+};
+
+/*
+ * A structure defining the synproxy ON/OFF command request message.
+ */
+struct pipeline_vfw_synproxy_flag_msg_req {
+       enum pipeline_msg_req_type type;
+       enum pipeline_vfw_msg_req_type subtype;
+
+       /* data */
+       uint8_t synproxy_flag;
+};
+
+/**
+ * A structure defining the synproxy ON/OFF command response message.
+ */
+struct pipeline_vfw_synproxy_flag_msg_rsp {
+       int status;
+       void *entry_ptr;
+};
+extern struct pipeline_be_ops pipeline_vfw_be_ops;
+
+extern int rte_ct_initialize_default_timeouts(struct rte_ct_cnxn_tracker
+                                         *new_cnxn_tracker);
+
+#endif
diff --git a/docs/vFW/INSTALL.rst b/docs/vFW/INSTALL.rst
new file mode 100644 (file)
index 0000000..21b2d36
--- /dev/null
@@ -0,0 +1,177 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, National Center of Scientific Research "Demokritos" and others.
+
+============================
+vFW - Installation Guide
+============================
+
+
+vFW Compilation
+===================
+
+After downloading (or doing a git clone) in a directory (samplevnf)
+
+###### Dependencies
+* DPDK 16.04: Downloaded and installed via vnf_build.sh or manually from [here](http://fast.dpdk.org/rel/dpdk-16.04.tar.xz)
+Both the options are available as part of vnf_build.sh below.
+* libpcap-dev
+* libzmq
+* libcurl
+
+###### Environment variables
+
+Apply all the additional patches in 'patches/dpdk_custom_patch/' and build dpdk
+
+::
+  export RTE_SDK=<dpdk 16.04 directory>
+  export RTE_TARGET=x86_64-native-linuxapp-gcc
+
+This is done by vnf_build.sh script.
+
+Auto Build:
+==========
+$ ./tools/vnf_build.sh in samplevnf root folder
+
+Follow the steps in the screen from option [1] --> [8] and select option [7]
+to build the vnfs.
+It will automatically download DPDK 16.04 and any required patches and will setup
+everything and build vFW VNFs.
+
+Following are the options for setup:
+
+::
+
+  ----------------------------------------------------------
+   Step 1: Environment setup.
+  ----------------------------------------------------------
+  [1] Check OS and network connection
+
+  ----------------------------------------------------------
+   Step 2: Download and Install
+  ----------------------------------------------------------
+  [2] Agree to download
+  [3] Download packages
+  [4] Download DPDK zip (optional, use it when option 4 fails)
+  [5] Install DPDK
+  [6] Setup hugepages
+
+  ----------------------------------------------------------
+   Step 3: Build VNF
+  ----------------------------------------------------------
+  [7] Build VNF
+
+  [8] Exit Script
+
+An vFW executable will be created at the following location
+samplevnf/VNFs/vFW/build/vFW
+
+
+Manual Build:
+============
+1. Download DPDK 16.04 from dpdk.org
+   - http://dpdk.org/browse/dpdk/snapshot/dpdk-16.04.zip
+2. unzip  dpdk-16.04 and apply dpdk patch
+   - cd dpdk-16.04
+        - patch -p0 < VNF_CORE/patches/dpdk_custom_patch/rte_pipeline.patch
+        - patch -p1 < VNF_CORE/patches/dpdk_custom_patch/i40e-fix-link-management.patch
+        - patch -p1 < VNF_CORE/patches/dpdk_custom_patch/i40e-fix-Rx-hang-when-disable-LLDP.patch
+        - patch -p1 < VNF_CORE/patches/dpdk_custom_patch/i40e-fix-link-status-change-interrupt.patch
+        - patch -p1 < VNF_CORE/patches/dpdk_custom_patch/i40e-fix-VF-bonded-device-link-down.patch
+   - build dpdk
+       - make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
+       - cd x86_64-native-linuxapp-gcc
+       - make
+   - Setup huge pages
+       - For 1G/2M hugepage sizes, for example 1G pages, the size must be specified
+          explicitly and can also be optionally set as the default hugepage size for
+          the system. For example, to reserve 8G of hugepage memory in the form of
+          eight 1G pages, the following options should be passed to the kernel:
+               * default_hugepagesz=1G hugepagesz=1G hugepages=8  hugepagesz=2M hugepages=2048
+       - Add this to Go to /etc/default/grub configuration file.
+         - Append "default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048"
+           to the GRUB_CMDLINE_LINUX entry.
+3. Setup Environment Variable
+   - export RTE_SDK=<samplevnf>/dpdk-16.04
+   - export RTE_TARGET=x86_64-native-linuxapp-gcc
+   - export VNF_CORE=<samplevnf>
+     or using ./toot/setenv.sh
+4. Build vFW VNFs
+   - cd <samplevnf>/VNFs/vFW
+   - make clean
+   - make
+5. The vFW executable will be created at the following location
+   - <samplevnf>/VNFs/vFW/build/vFW
+
+Run
+====
+
+Setup Port to run VNF:
+----------------------
+::
+  1. cd <samplevnf>/dpdk-16.04
+  3. ./tool/dpdk_nic_bind.py --status <--- List the network device
+  2. ./tool/dpdk_nic_bind.py -b igb_uio <PCI Port 0> <PCI Port 1>
+  .. _More details: http://dpdk.org/doc/guides-16.04/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
+
+  Make the necessary changes to the config files to run the vFW VNF
+  eg: ports_mac_list = 00:00:00:30:21:01 00:00:00:30:21:00
+
+Firewall
+--------------
+Update the configuration according to system configuration.
+
+::
+  ./vFW -p <port mask> -f <config> -s <script> - SW_LoadB
+  ./vFW -p <port mask> -f <config> -s <script> -hwlb <num_WT> - HW_LoadB
+
+
+Run IPv4
+----------
+::
+  Software LoadB
+  --------------
+  cd <samplevnf>/VNFs/vFW/
+  ./build/vFW -p 0x3 -f ./config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg  -s ./config/VFW_SWLB_IPV4_SinglePortPair_script.tc
+
+
+  Hardware LoadB
+  --------------
+  cd <samplevnf>/VNFs/vFW/
+  ./build/vFW -p 0x3 -f ./config/VFW_HWLB_IPV4_SinglePortPair_4Thread.cfg  -s ./config/VFW_HWLB_IPV4_SinglePortPair_script.cfg --hwlb 4
+
+Run IPv6
+---------
+::
+  Software LoadB
+  --------------
+  cd <samplevnf>/VNFs/vFW
+  ./build/vFW -p 0x3 -f ./config/VFW_SWLB_IPV6_SinglePortPair_4Thread.cfg  -s ./config/VFW_SWLB_IPV6_SinglePortPair_script.tc
+
+
+  Hardware LoadB
+  --------------
+  cd <samplevnf>/VNFs/vFW/
+  ./build/vFW -p 0x3 -f ./config/VFW_HWLB_IPV6_SinglePortPair_4Thread.cfg  -s ./config/VFW_HWLB_IPV6_SinglePortPair_script.tc --hwlb 4
+
+vFW execution on BM & SRIOV:
+--------------------------------
+::
+  To run the VNF, execute the following:
+  samplevnf/VNFs/vFW# ./build/vFW -p 0x3 -f ./config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg  -s ./config/VFW_SWLB_IPV4_SinglePortPair_script.tc
+  Command Line Params:
+  -p PORTMASK: Hexadecimal bitmask of ports to configure
+  -f CONFIG FILE: vFW configuration file
+  -s SCRIPT FILE: vFW script file
+
+vFW execution on OVS:
+-------------------------
+::
+  To run the VNF, execute the following:
+  samplevnf/VNFs/vFW# ./build/vFW -p 0x3 -f ./config/VFW_SWLB_IPV4_SinglePortPair_4Thread.cfg  -s ./config/VFW_SWLB_IPV4_SinglePortPair_script.tc --disable-hw-csum
+  Command Line Params:
+  -p PORTMASK: Hexadecimal bitmask of ports to configure
+  -f CONFIG FILE: vFW configuration file
+  -s SCRIPT FILE: vFW script file
+--disable-hw-csum :Disable TCP/UDP hw checksum
diff --git a/docs/vFW/README.rst b/docs/vFW/README.rst
new file mode 100644 (file)
index 0000000..45e8a17
--- /dev/null
@@ -0,0 +1,166 @@
+.. This work is licensed under a creative commons attribution 4.0 international
+.. license.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) opnfv, national center of scientific research "demokritos" and others.
+
+========================================================
+Virtual Firewall - vFW
+========================================================
+
+1. Introduction
+==============
+The virtual firewall (vFW) is an application implements Firewall. vFW is used
+as a barrier between secure internal and an un-secure external network. The
+firewall performs Dynamic Packet Filtering. This involves keeping track of the
+state of Layer 4 (Transport)traffic,by examining both incoming and outgoing
+packets over time. Packets which don't fall within expected parameters given
+the state of the connection are discarded. The Dynamic Packet Filtering will
+be performed by Connection Tracking component, similar to that supported in
+linux. The firewall also supports Access Controlled List(ACL) for rule based
+policy enforcement. Firewall is built on top of DPDK and uses the packet library.
+
+About DPDK
+----------
+The DPDK IP Pipeline Framework provides a set of libraries to build a pipeline
+application. In this document, vFW will be explained in detail with its own
+building blocks.
+
+This document assumes the reader possesses the knowledge of DPDK concepts and
+packet framework. For more details, read DPDK Getting Started Guide, DPDK
+Programmers Guide, DPDK Sample Applications Guide.
+
+2.  Scope
+==========
+This application provides a standalone DPDK based high performance vFW Virtual
+Network Function implementation.
+
+3. Features
+===========
+The vFW VNF currently supports the following functionality:
+  • Basic packet filtering (malformed packets, IP fragments)
+  • Connection tracking for TCP and UDP
+  • Access Control List for rule based policy enforcement
+  • SYN-flood protection via Synproxy* for TCP
+  • UDP, TCP and ICMP protocol pass-through
+  • CLI based enable/disable connection tracking, synproxy, basic packet
+    filtering
+  • Multithread support
+  • Multiple physical port support
+  • Hardware and Software Load Balancing
+  • L2L3 stack support for ARP/ICMP handling
+  • ARP (request, response, gratuitous)
+  • ICMP (terminal echo, echo response, passthrough)
+  • ICMPv6 and ND (Neighbor Discovery)
+
+4. High Level Design
+====================
+The Firewall performs basic filtering for malformed packets and dynamic packet
+filtering incoming packets using the connection tracker library.
+The connection data will be stored using a DPDK hash table. There will be one
+entry in the hash table for each connection. The hash key will be based on source
+address/port,destination address/port, and protocol of a packet. The hash key
+will be processed to allow a single entry to be used, regardless of which
+direction the packet is flowing (thus changing the source and destination).
+The ACL is implemented as libray stattically linked to vFW, which is used for
+used for rule based packet filtering.
+
+TCP connections and UDP pseudo connections will be tracked separately even if
+theaddresses and ports are identical. Including the protocol in the hash key
+will ensure this.
+
+The Input FIFO contains all the incoming packets for vFW filtering.  The vFW
+Filter has no dependency on which component has written to the Input FIFO.
+Packets will be dequeued from the FIFO in bulk for processing by the vFW.
+Packets will be enqueued to the output FIFO.
+The software or hardware loadbalancing can be used for traffic distribution
+across multiple worker threads. The hardware loadbalancing require ethernet
+flow director support from hardware (eg. Fortville x710 NIC card).
+The Input and Output FIFOs will be implemented using DPDK Ring Buffers.
+
+===================
+5. Components of vFW
+===================
+In vFW, each component is constructed using packet framework pipelines.
+It includes Rx and Tx Driver, Master pipeline, load balancer pipeline and
+vfw worker pipeline components. A Pipeline framework is a collection of input
+ports, table(s),output ports and actions (functions).
+
+Receive and Transmit Driver
+******************************
+Packets will be received in bulk and provided to LoadBalancer(LB) thread.
+Transimit takes packets from worker threads in a dedicated ring and sent to
+hardware queue.
+
+Master Pipeline
+******************************
+The Master component is part of all the IP Pipeline applications. This component
+does not process any packets and should configure with Core 0, to allow
+other cores for processing of the traffic. This component is responsible for
+ 1. Initializing each component of the Pipeline application in different threads
+ 2. Providing CLI shell for the user control/debug
+ 3. Propagating the commands from user to the corresponding components
+
+ARPICMP Pipeline
+******************************
+This pipeline processes the APRICMP packets.
+
+TXRX Pipelines
+******************************
+The TXTX and RXRX pipelines are pass through pipelines to forward both ingress
+and egress traffic to Loadbalancer. This is required when the Software
+Loadbalancer is used.
+
+Load Balancer Pipeline
+******************************
+The vFW support both hardware and software balancing for load balancing of
+traffic across multiple VNF threads. The Hardware load balancing require support
+from hardware like Flow Director for steering of packets to application through
+hardware queues.
+
+The Software Load balancer is also supported if hardware load balancing can't be
+used for any reason. The TXRX along with LOADB pipeline provides support for
+software load balancing by distributing the flows to Multiple vFW worker
+threads.
+Loadbalancer (HW or SW) distributes traffic based on the 5 tuple (src addr, src
+port, dest addr, dest port and protocol) applying an XOR logic distributing to
+active worker threads, thereby maintaining an affinity of flows to worker
+threads.
+
+vFW Pipeline
+******************************
+The vFW performs the basic packet filtering and will drop the invalid and
+malformed packets.The Dynamic packet filtering done using the connection tracker
+library. The packets are processed in bulk and Hash table is used to maintain
+the connection details.
+Every TCP/UDP packets are passed through connection tracker library for valid
+connection. The ACL library integrated to firewall provide rule based filtering.
+
+vFW Topology:
+------------------------
+::
+  IXIA(Port 0)-->(Port 0)VNF(Port 1)-->(Port 1) IXIA
+  operation:
+    Egress --> The packets sent out from ixia(port 0) will be Firewalled to ixia(port 1).
+    Igress --> The packets sent out from ixia(port 1) will be Firewalled to ixia(port 0).
+
+vFW Topology (L4REPLAY):
+------------------------------------
+::
+  IXIA(Port 0)-->(Port 0)VNF(Port 1)-->(Port 0)L4REPLAY
+  operation:
+    Egress --> The packets sent out from ixia will pass through vFW to L3FWD/L4REPLAY.
+    Ingress --> The L4REPLAY upon reception of packets (Private to Public Network),
+                will immediately replay back the traffic to IXIA interface. (Pub -->Priv).
+
+How to run L4Replay:
+--------------------
+::
+  1. After the installation of samplevnf:
+     go to <samplevnf/VNFs/L4Replay>
+  2. ./buid/L4replay -c  core_mask -n no_of_channels(let it be as 2) -- -p PORT_MASK --config="(port,queue,lcore)"
+     eg: ./L4replay -c 0xf -n 4 -- -p 0x3 --config="(0,0,1)"
+
+6. Installation, Compile and Execution
+-----------------------------------------------------------------
+Plase refer to <samplevnf>/docs/vFW/INSTALL.rst for installation, configuration,
+compilation and execution.
diff --git a/docs/vFW/RELEASE_NOTES.rst b/docs/vFW/RELEASE_NOTES.rst
new file mode 100644 (file)
index 0000000..0c88004
--- /dev/null
@@ -0,0 +1,78 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, National Center of Scientific Research "Demokritos" and others.
+
+=========================================================
+Virtual Firewall - vFW
+=========================================================
+
+1.     Introduction
+================
+
+This is a beta release for Sample Virtual Firewall VNF.
+This vFW can application can be run independently (refer INSTALL.rst).
+
+2.     User Guide
+===============
+Refer to README.rst for further details on vFW, HLD, features supported, test
+plan. For build configurations and execution requisites please refer to
+INSTALL.rst.
+
+3. Feature for this release
+===========================
+This release supports following features as part of vFW:
+  - Basic packet filtering (malformed packets, IP fragments)
+  - Connection tracking for TCP and UDP
+  - Access Control List for rule based policy enforcement
+  - SYN-flood protection via Synproxy* for TCP
+  - UDP, TCP and ICMP protocol pass-through
+  - CLI based enable/disable connection tracking, synproxy, basic packet
+    filtering
+  - L2L3 stack support for ARP/ICMP handling
+  - ARP (request, response, gratuitous)
+  - ICMP (terminal echo, echo response, passthrough)
+  - ICMPv6 and ND (Neighbor Discovery)
+  - Hardware and Software Load Balancing
+  - Multithread support
+  - Multiple physical port support
+
+4. System requirements - OS and kernel version
+==============================================
+This is supported on Ubuntu 14.04 and Ubuntu 16.04 and kernel version less than 4.5
+
+   VNFs on BareMetal support:
+               OS: Ubuntu 14.04 or 16.04 LTS
+               kernel: < 4.5
+               http://releases.ubuntu.com/16.04/
+               Download/Install the image: ubuntu-16.04.1-server-amd64.iso
+
+   VNFs on Standalone Hypervisor
+               HOST OS: Ubuntu 14.04 or 16.04 LTS
+               http://releases.ubuntu.com/16.04/
+               Download/Install the image: ubuntu-16.04.1-server-amd64.iso
+            -   OVS (DPDK) - 2.5
+                  -   kernel: < 4.5
+            -   Hypervisor - KVM
+            -   VM OS - Ubuntu 16.04/Ubuntu 14.04
+
+5. Known Bugs and limitations
+=============================
+ - Hadware Load Balancer feature is supported on fortville nic FW version 4.53 and below.
+ - Hardware Checksum offload is not supported for IPv6 traffic.
+ - vFW on sriov is tested upto 4 threads
+ - Http Multiple clients/server with HWLB is not working
+
+6. Future Work
+==============
+Following would be possible enhancement functionalities
+ - Automatic enable/disable of synproxy
+ - Support TCP timestamps with synproxy
+ - FTP ALG integration
+ - Performance optimization on different platforms
+
+7. References
+=============
+Following links provides additional information
+       .. _QUICKSTART: http://dpdk.org/doc/guides-16.04/linux_gsg/quick_start.html
+       .. _DPDKGUIDE: http://dpdk.org/doc/guides-16.04/prog_guide/index.html