clover.git
6 years agoMerge "Updated from global requirements"
Stephen Wong [Sun, 21 Oct 2018 05:53:39 +0000 (05:53 +0000)]
Merge "Updated from global requirements"

6 years agoMerge "Add spinnaker configguide"
Stephen Wong [Sun, 21 Oct 2018 05:50:57 +0000 (05:50 +0000)]
Merge "Add spinnaker configguide"

6 years agoAdd spinnaker configguide 13/63413/3
wutianwei [Thu, 11 Oct 2018 11:25:12 +0000 (19:25 +0800)]
Add spinnaker configguide

- Supply manifest for creation PV
- Add the spinnaker as a service documentation
- set the ci.jenkins.enabled to false

Change-Id: I66d91aa133085c6c5910913da4303c05fcba9800
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years agoUpdated from global requirements 77/63577/1
Cédric Ollivier [Tue, 16 Oct 2018 09:38:22 +0000 (11:38 +0200)]
Updated from global requirements

Change-Id: I81e92ff3db9ca401cc700ff4a9c28c740f7e9060
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
6 years agoUpdated from global requirements 33/63233/1
Cédric Ollivier [Sat, 6 Oct 2018 08:44:15 +0000 (10:44 +0200)]
Updated from global requirements

Change-Id: I92fc0937130b0c70dc5888b64b1082e8816969d4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
6 years agoMerge "Local Documentation Builds"
Stephen Wong [Wed, 19 Sep 2018 20:53:34 +0000 (20:53 +0000)]
Merge "Local Documentation Builds"

6 years agoMerge "Add ModSecurity config guide"
Stephen Wong [Wed, 19 Sep 2018 20:52:46 +0000 (20:52 +0000)]
Merge "Add ModSecurity config guide"

6 years agoMerge "Spinnaker as a Service"
Stephen Wong [Wed, 19 Sep 2018 20:51:18 +0000 (20:51 +0000)]
Merge "Spinnaker as a Service"

6 years agoLocal Documentation Builds 67/62467/1
Aric Gardner [Tue, 18 Sep 2018 15:02:09 +0000 (11:02 -0400)]
Local Documentation Builds

    This adds configuration for performing local documentation builds
    with the following simple command:

    tox -e docs

Change-Id: Ifb8fce66e59432863d042a96308e6c3a89fb11a8
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
6 years agoFix missing psutil dependency in nginx services 01/62001/1
Eddie Arrage [Sat, 8 Sep 2018 00:15:18 +0000 (00:15 +0000)]
Fix missing psutil dependency in nginx services

- A psutil dependency was added in the latest Clover server version
that was not added to the docker builds of the lb/proxy

Change-Id: Id48c3446a93d6b9cef65d453266d0bf15eddf6cc
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoAdd ModSecurity config guide 69/61969/2
JingLu5 [Fri, 7 Sep 2018 08:18:15 +0000 (16:18 +0800)]
Add ModSecurity config guide

This patch adds ModSecurity config guide.
This patch also deploy the modsecurity and ext_authz filter to clover-gateway namespace.

Change-Id: I5ab21e6337b8f8b839ddd028370df378686bd017
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "Add file upload/download and configure URL paths"
Stephen Wong [Wed, 5 Sep 2018 05:05:15 +0000 (05:05 +0000)]
Merge "Add file upload/download and configure URL paths"

6 years agoMerge "Update entry_point to conform with Xtesting"
Stephen Wong [Wed, 5 Sep 2018 05:03:59 +0000 (05:03 +0000)]
Merge "Update entry_point to conform with Xtesting"

6 years agoMerge "Modsecurity as a service"
Stephen Wong [Wed, 5 Sep 2018 05:03:24 +0000 (05:03 +0000)]
Merge "Modsecurity as a service"

6 years agoSpinnaker as a Service 25/61225/3
wutianwei [Wed, 22 Aug 2018 02:49:56 +0000 (10:49 +0800)]
Spinnaker as a Service

JIRA: CLOVER-52

1. Add mainfest to install the spinnaker in kubernetes cluster
2. after using mainfest to install spinnaker,
   we can interacte with the halyard daemon with its REST API
   and we can add/delete/list the dockerRegistry/kubernetes accounts.
3. Add the cloverctl to interate with the halyard daemon

Change-Id: I71bc5977f2d65aab88fa55f7d7a53ab75eb6a46b
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years agoMerge "Add docker and deployment scripts for exemplar video streaming microservice"
Wenjing Chu [Tue, 4 Sep 2018 03:30:48 +0000 (03:30 +0000)]
Merge "Add docker and deployment scripts for exemplar video streaming microservice"

6 years agoUpdate entry_point to conform with Xtesting 23/61723/1
Cédric Ollivier [Sun, 2 Sep 2018 12:52:49 +0000 (14:52 +0200)]
Update entry_point to conform with Xtesting

Functest is being updated to Xtesting 0.60 which leverages on
stevedore [1].

[1] https://docs.openstack.org/stevedore/latest/

Change-Id: I2c162c8e08a580f80b547b3a1cd6cc700d12c858
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
6 years agoMerge "Add envoy.ext_authz filter"
Stephen Wong [Fri, 31 Aug 2018 07:21:35 +0000 (07:21 +0000)]
Merge "Add envoy.ext_authz filter"

6 years agoAdd docker and deployment scripts for exemplar video streaming microservice 03/61603/1
adarsh1001 [Thu, 30 Aug 2018 15:56:08 +0000 (21:26 +0530)]
Add docker and deployment scripts for exemplar video streaming microservice

JIRA: CLOVER-72

This issue falls under the "Edge Cloud-Native Cluster" intern project and is a part of both Clover and Edge cloud projects. The scripts added are of an exemplar video streaming microservice designed to run on the raspberry pi edge cluster. This commit includes the docker scripts to containerize the UV4L stream and the kubernetes deployment script to correctly run the microservice as a pod on the cluster. The detailed description of the project and the instructions for using the scripts are documented in the README.

Change-Id: Ice4e5e7e63c2eb617556eaef1255801ffac851e7
Signed-off-by: adarsh1001 <adarshpalsingh1996@gmail.com>
6 years agoBug fix in SDC after istio to 1.0.0 21/61321/2
Ace Lee [Fri, 24 Aug 2018 07:56:39 +0000 (07:56 +0000)]
Bug fix in SDC after istio to 1.0.0

JIRA: CLOVER-84

There will be some Bug in SDC after we upgrade the Istio to 1.0.0
Istio 1.0 have some concept, for example : virtualservice gateway.
So we change the yaml file using the 1.0.0 concepts.
Add mirror function

Change-Id: Id138cfec2c7d94b44eb508a056c91e193ac1b08b
Signed-off-by: Ace Lee <liyin11@huawei.com>
6 years agoAdd file upload/download and configure URL paths 37/58937/3
Eddie Arrage [Fri, 22 Jun 2018 21:25:54 +0000 (21:25 +0000)]
Add file upload/download and configure URL paths

- Compile nginx from source in order to employ additional modules
- Add nginx-upload-module for high performance file upload that
avoids the need for file copies with a web application.
- File upload allows for placement of files for file download
for performance benchmarking.
- File upload can also be used directly for bi-directional throughput
testing having emulated clients upload files while file downloads
simultaneously occur.
- Nginx file upload stores files with hash to avoid conflicting
file names. Upload block in nginx config is configured to send
REST message to clover-controller with file metadata (original
filename, size, etc.) clover-controller will be responsible for
modifying the hashed filename and placing in a target directory
within an nginx server.
- Build also adds nginx-rtmp module to act as streaming media server
L7 loader will be extended to fetch streaming files from RTMP
servers.

- Add ability to create directories in server site root
and create the location directive(s) in nginx configuration
- Separated upload for configuration (download
files in various paths) from upload for testing (upload
to create bi-directional session throughput)
- Upload for testing does not sent upload metadata to
clover-controller
- Added ability to move upload files to file folders in the nginx site
root to use for download
- Delete files in upload folder
- Fixed issue with 426 Upgrade Required error message
when upload module sends upload metadata to clover-controller
- Added server name to metadata sent to clover-controller

Change-Id: Ib4cf6240f92360b82f378c062675f4fdaa19ca93
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoAdd envoy.ext_authz filter 55/61455/1
JingLu5 [Tue, 28 Aug 2018 08:34:07 +0000 (16:34 +0800)]
Add envoy.ext_authz filter

JIRA: CLOVER-86

This external authorization HTTP filter calls an external HTTP service (ModSecuruty service) to check if the incoming HTTP request is authorized or not. If the request is deemed unauthorized then the request will be denied normally with 403 (Forbidden) response.

Change-Id: I0fe14c73defec027c54f42713cbdf69b0b83e102
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoModsecurity as a service 13/61313/2
JingLu5 [Fri, 24 Aug 2018 02:55:14 +0000 (10:55 +0800)]
Modsecurity as a service

JIRA: CLOVER-68

1. Add Dockerfile and related files to build clover's modsecurity Docekr container
2. Add mainfest to install the Modsecurity in kubernetes cluster

Change-Id: Ia92926e730c04720f931999d7ec30565ce9e54be
Signed-off-by: JingLu5 <lvjing5@huawei.com>
6 years agoMerge "upgrade istio to 1.0.0"
Stephen Wong [Tue, 21 Aug 2018 07:20:24 +0000 (07:20 +0000)]
Merge "upgrade istio to 1.0.0"

6 years agoMerge "Implement initial clover-controller service"
Stephen Wong [Sun, 19 Aug 2018 07:18:24 +0000 (07:18 +0000)]
Merge "Implement initial clover-controller service"

6 years agoMerge "Implement initial cloverctl CLI tool"
Stephen Wong [Sun, 19 Aug 2018 07:18:00 +0000 (07:18 +0000)]
Merge "Implement initial cloverctl CLI tool"

6 years agoMerge "Implement initial Jmeter master/slave containers"
Stephen Wong [Sun, 19 Aug 2018 07:17:28 +0000 (07:17 +0000)]
Merge "Implement initial Jmeter master/slave containers"

6 years agoMerge "Add ansible scripts for raspberry pi based kubernetes edge cluster"
Wenjing Chu [Tue, 14 Aug 2018 23:23:37 +0000 (23:23 +0000)]
Merge "Add ansible scripts for raspberry pi based kubernetes edge cluster"

6 years agoupgrade istio to 1.0.0 27/60827/1
Ace Lee [Fri, 10 Aug 2018 08:10:08 +0000 (08:10 +0000)]
upgrade istio to 1.0.0

JIRA: CLOVER-84

we change the env in clover and change some deploy script.
will upgrade the doc later

Change-Id: I73a78afb91676efc3278b623c5d263a4a215ccd9
Signed-off-by: Ace Lee <liyin11@huawei.com>
6 years agoImplement initial cloverctl CLI tool 39/60539/2
Eddie Arrage [Thu, 2 Aug 2018 23:15:39 +0000 (23:15 +0000)]
Implement initial cloverctl CLI tool

- Uses client-go package to interface to k8s API and implement
functions as cloverkube package.
- Identifies GKE LB IP for clover-controller for user
- Identifies NodePort port number for clover-controller for user
if environment is local k8s (assumes flannel CNI currently)
- Deploys and deletes clover-collector and clover-controller with
native client-go constructs (currently images are defined with
local registry). Future work will implement other clover services
and Istio components. Uses the clover-system namespace.
- Uses Cobra go package to implement CLI (used in kubectl and
istioctl) using cloverctl <verb> <noun> convention.
- Interfaces to clover-controller to configure clover services
(visibility, IDS ...) within the cluster via REST messaging
- Start visibility (collector) engine using input yaml file or
defaults
- Init, stop and clear (truncate Cassandra tables) visibility
engine or get basic stats.
- Add custom rules to IDS from input yaml file and start/stop
IDS
- Generate jmeter testplan on jmeter-master using input yaml
file. Start tests and output log/results from CLI.
- Specify number of jmeter slaves to initiate tests on from
CLI. Automatically find IP addresses of jmeter slaves within
the k8s cluster.
- Sample yaml files for adding IDS rules, starting visibility
engine and generating jmeter test plans.
- Build script to install go and get dependent packages.
- Implement a custom Istio inject package for manual sidecar
injection (cloverinject). Currently, unused as it is built from
Istio 0.8.0/1.0.0 code base.

Change-Id: Ibb8d08cb98267bdffb8905c221473f177d51bbb3
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoMerge "Adding cassandra service with a separate yaml file."
Stephen Wong [Wed, 1 Aug 2018 05:16:34 +0000 (05:16 +0000)]
Merge "Adding cassandra service with a separate yaml file."

6 years agoMerge "Adding clearwater IMS yaml and clearwater live test dockerfile to test istio...
Stephen Wong [Wed, 1 Aug 2018 05:16:00 +0000 (05:16 +0000)]
Merge "Adding clearwater IMS yaml and clearwater live test dockerfile to test istio service-mesh"

6 years agoMerge "[Clover] using elasticsearch to store log"
Stephen Wong [Wed, 1 Aug 2018 05:14:38 +0000 (05:14 +0000)]
Merge "[Clover] using elasticsearch to store log"

6 years agoImplement initial clover-controller service 27/59227/4
Eddie Arrage [Thu, 28 Jun 2018 17:42:28 +0000 (17:42 +0000)]
Implement initial clover-controller service

- First pass of clover-controller which resides within the k8s
cluster and provides interfaces to all Clover services
- Only service that should need to be exposed outside of
cluster
- Docker build of container that uses stack of nginx, gunicorn
and flask to provide REST interface
- REST interface is intended to serve cloverctl CLI and
dashboard browser UI
- Implements GRPC messaging to clover-collector and snort
- GRPC interfaces files for snort/nginx are added to
container from repo. Collector GRPC files will be removed
from controller/control/api once patch below is merged
https://gerrit.opnfv.org/gerrit/#/c/57245/ and added
similarly
- Provides first pass callback for file upload from
clover-server.
- Some REST messages implement JSON for passing params
to internal services
- Redis interface added to obtain data from services.
Currently, a simple interface to retrieve snort event
information
- YAML manifest renderer to add to k8s. Uses NodePort
service currently, defaulting to port 32044.

- Removed collector gRPC interface files with merge of collector
- Expose tracing and monitoring host/port parameters, as these vary
depending on Istio version and Jaeger version
- Add logging to flask blueprints
- Added jmeter blueprint interface with REST for
testplan generation, start test and result retrieval
- Added flask Response to REST reply messages
- Retrieve some basic stats from collector in json
response

Change-Id: I59eaeb860445ade4b45bba22747a61fb0cf0bbd4
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoImplement initial Jmeter master/slave containers 97/60297/1
Eddie Arrage [Sat, 28 Jul 2018 01:02:35 +0000 (01:02 +0000)]
Implement initial Jmeter master/slave containers

- Jmeter can be used for L4-7 functional and performance testing
- Jmeter master has gRPC server for management
- Generates Jmeter test plans from minimal yaml params file
(sample to be added with cloverctl) using template
- Optionally span tests across slave containers to allow greater
loads to be generated
- Specify loop/thread/slave count and URL list, which
dictates target and number of connections that will be attempted
- clover-controller will interface to gRPC interface on Jmeter
master
- Start tests on master and retrieve log/result files
- Render master and slave k8s manifests files

Change-Id: Id144c8f551b7d375ff252c8de0611f895b50387c
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoAdding cassandra service with a separate yaml file. 61/57661/3
iharijono [Wed, 9 May 2018 01:27:59 +0000 (18:27 -0700)]
Adding cassandra service with a separate yaml file.

- Left the file samples/scenarios/service_delivery_controller_opnfv.yaml unchanged.
- Added a yaml definition of Cassandra StatefulSet and its service into a separate file under tools directory
- Cassandra Service run with 1 replica
- Deleted 'data-plane-ns' and use 'default' instead for cassandra containers.
- Revoked changes for samples/scenarios/service_delivery_controller_opnfv.yaml.
- Added new line (Wutien suggested it)

JIRA: CLOVER-000

Change-Id: I2bb4249cf2523f5011d6fefc69dc469a90e20eaf
Signed-off-by: iharijono <indra.harijono@huawei.com>
6 years agoAdd ansible scripts for raspberry pi based kubernetes edge cluster 73/60073/1
adarsh1001 [Sat, 21 Jul 2018 20:41:47 +0000 (02:11 +0530)]
Add ansible scripts for raspberry pi based kubernetes edge cluster

JIRA: CLOVER-71

This issue falls under the "Edge Cloud-Native Cluster" intern project and is a part of both Clover and Edge cloud projects. The detailed description of the project and the instructions for using the scripts are documented in the README.

Change-Id: I4fdb98f17ae0c53f918376ad6fb90be8ff0b0a71
Signed-off-by: adarsh1001 <adarshpalsingh1996@gmail.com>
6 years agoMerge "Initial commit for Clover Collector"
Stephen Wong [Sat, 7 Jul 2018 06:09:37 +0000 (06:09 +0000)]
Merge "Initial commit for Clover Collector"

6 years agoMerge "Sleep 1 sec after set test."
Stephen Wong [Sat, 7 Jul 2018 05:46:00 +0000 (05:46 +0000)]
Merge "Sleep 1 sec after set test."

6 years agoMerge "[clover] Add script to setup k8s"
Stephen Wong [Sat, 7 Jul 2018 05:45:04 +0000 (05:45 +0000)]
Merge "[clover] Add script to setup k8s"

6 years ago[Clover] using elasticsearch to store log 85/59285/1
wutianwei [Fri, 29 Jun 2018 03:29:38 +0000 (11:29 +0800)]
[Clover] using elasticsearch to store log

Change-Id: I0335fa912a3ca2dff5c989fa06183065216f10e4
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years agoFix project LEAD in INFO.yaml 23/59223/1
Aric Gardner [Thu, 28 Jun 2018 17:09:17 +0000 (13:09 -0400)]
Fix project LEAD in INFO.yaml

Change-Id: Iff9c7bfbae8f5ca46ef88634d4864cd27512aa8a
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
6 years agoSleep 1 sec after set test. 67/58567/2
wutianwei [Fri, 15 Jun 2018 06:22:39 +0000 (14:22 +0800)]
Sleep 1 sec after set test.

if we set testid and start test immediately,
the first test's result can't be got from jaeger

Change-Id: Ia2ab8a91d8c5f9956ea4d3d7c2436fb05490acee
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years ago[clover] Add script to setup k8s 55/58355/2
wutianwei [Fri, 8 Jun 2018 01:49:36 +0000 (09:49 +0800)]
[clover] Add script to setup k8s

what the script do
1. git clone the XCI repo
2. set variable to deploy k8s

how to use
run the below command in you teminal
./xci-k8s-setup.sh

requirement
1. don't under the root user
2. run sudo without password

Change-Id: I5015e12d8d3b0db31285d5e817b3c40d6739ba22
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years agoMerge "Streaming logs from application container"
Stephen Wong [Fri, 15 Jun 2018 00:24:57 +0000 (00:24 +0000)]
Merge "Streaming logs from application container"

6 years agoMerge "Add support for node level logging"
Stephen Wong [Fri, 15 Jun 2018 00:24:19 +0000 (00:24 +0000)]
Merge "Add support for node level logging"

6 years agoMerge "fix the sdc sample deploy issue"
Stephen Wong [Fri, 15 Jun 2018 00:23:46 +0000 (00:23 +0000)]
Merge "fix the sdc sample deploy issue"

6 years agoMerge "Fix typo of Jaeger port"
Stephen Wong [Fri, 15 Jun 2018 00:22:11 +0000 (00:22 +0000)]
Merge "Fix typo of Jaeger port"

6 years agoMerge "Fix error in logging installation"
Stephen Wong [Fri, 15 Jun 2018 00:21:42 +0000 (00:21 +0000)]
Merge "Fix error in logging installation"

6 years agoInitial commit for Clover Collector 45/57245/3
Eddie Arrage [Wed, 9 May 2018 18:33:55 +0000 (18:33 +0000)]
Initial commit for Clover Collector

- Added a container named clover-collector using clover
container as a base with build script
- GRPC server to manage collector process
- Cassandra DB client interface to initialize visibility keyspace
- Init messaging adds table schemas for tracing - traces & spans
- Adds table for monitoring - metrics
- Does not implement Cassandra server but developed using
public Cassandra docker container
- Collector process in simple loop that periodically fetches
traces and monitoring data and inserts to Cassandra - not optimized
for batch retrieval yet for monitoring
- CLI interface added to collector process and used
by GRPC server for configuration
- Simple GRPC client script to test GRPC server and start/stop
of collector process
- Collector process can be configured with access for tracing,
monitoring and Cassandra
- Added a return value in monitoring query method

- Added ability to truncate tracing, metrics and spans tables
in cql
- Added cql prepared statements and batch insert for metrics
and spans
- Align cql connection to cql deployment within k8s
- Fix issue with cql host list using ast and collect process
args with background argument
- Added redis interface to accept service/metric list
externally for monitoring (will work in conjunction
with clover-controller)
- Use k8s DNS names and default ports for monitoring, tracing
and cassandra
- Added yaml manifest renderer/template for collector

Change-Id: I3e4353e28844c4ce9c185ff4638012b66c7fff67
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
6 years agoAdd support for node level logging 97/57197/2
Yujun Zhang [Wed, 9 May 2018 07:01:53 +0000 (15:01 +0800)]
Add support for node level logging

Change-Id: Ib5b2240de3276164fe9e272bf36f0d1f89f409c0
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
6 years agoAdding clearwater IMS yaml and clearwater live test dockerfile to test istio service... 01/58101/1
Muhammad Shaikh (Salman) [Wed, 30 May 2018 00:41:19 +0000 (00:41 +0000)]
Adding clearwater IMS yaml and clearwater live test dockerfile to test istio service-mesh

Checking into CLEARWATER_ISTIO branch

This part of the project is intended to validate the clearwater IMS with istio service-mesh.

Change-Id: Ia5ba86301a363fcf9cfe0bac525606b0d897713e
Signed-off-by: Muhammad Shaikh (Salman) <muhammad.shaikh@huawei.com>
6 years agofix the sdc sample deploy issue 81/57981/2
wutianwei [Mon, 28 May 2018 01:02:14 +0000 (09:02 +0800)]
fix the sdc sample deploy issue

there is a issue "No module named google.protobuf",
when trying to run the services docker.
Add the protobuf in services Dockerfile.

Change-Id: I280dc1d5908bcec784e9e1e7c4d07e145f092cdb
Signed-off-by: wutianwei <wutianwei1@huawei.com>
6 years agoStreaming logs from application container 49/57249/2
Yujun Zhang [Thu, 10 May 2018 07:26:08 +0000 (15:26 +0800)]
Streaming logs from application container

Change-Id: I6a1e526bec4160bcdac32d4124acb110b9cf6959
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
6 years agoRemove INFO file, we only need to maintain INFO.yaml 91/57291/1
Aric Gardner [Thu, 10 May 2018 18:56:16 +0000 (14:56 -0400)]
Remove INFO file, we only need to maintain INFO.yaml

Change-Id: I51fbdaf6991cb43a7cb1b2ce01099e0ed385df13
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
6 years agoFix error in logging installation 95/57195/2
Yujun Zhang [Wed, 9 May 2018 03:18:38 +0000 (11:18 +0800)]
Fix error in logging installation

It must be done in two steps otherwise the mixer adapter may not
be initialized correctly.

Change-Id: Ie59e811fc287fbd52a007be45f0f9c74983149b3
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
6 years agoFix typo of Jaeger port 91/57191/1
Yujun Zhang [Tue, 8 May 2018 08:34:23 +0000 (16:34 +0800)]
Fix typo of Jaeger port

Change-Id: I70b766fe26e750fef6a622344d69ad4f6e2b8962
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoUpdate docs with edits and proper release tag 13/56813/3
Eddie Arrage [Fri, 27 Apr 2018 18:52:41 +0000 (18:52 +0000)]
Update docs with edits and proper release tag

- Incorporated feedback from doc reviews
- Fix some rendering issues
- Add redis inspect section
- Update SDC deploy instructions using Fraser release tag

Change-Id: I573dcd04066ad83b9c659fae645c65ab4aaa2007
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoMerge "Write A-B configuration guide and align with SDC guide"
Stephen Wong [Thu, 26 Apr 2018 07:39:48 +0000 (07:39 +0000)]
Merge "Write A-B configuration guide and align with SDC guide"

7 years agoWrite A-B configuration guide and align with SDC guide 57/56557/3
Eddie Arrage [Thu, 26 Apr 2018 01:20:49 +0000 (01:20 +0000)]
Write A-B configuration guide and align with SDC guide

- Document A-B sample validation script
- Remove redundant TOC in docs
- Provide reference links in SDC guide
- Additional edits to SDC guide

Change-Id: Id4135c99df688f7de1af18017c847a6546082bfc
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoUpdated user guide with edits and reference links 93/56593/1
Eddie Arrage [Thu, 26 Apr 2018 04:26:06 +0000 (04:26 +0000)]
Updated user guide with edits and reference links

- Provided some overall edits to user guide
- Fixed titles of index files for release notes and user guide
- Added links to SDC, A-B configuration guides and logging, tracing
and monitoring install/validation docs

Change-Id: I9a0e1e0a2c12b20400eec5a5642f7c5de2dbd7bf
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoMerge "Experimental commit for A-B testing with Clover Fraser release and on the...
Stephen Wong [Wed, 25 Apr 2018 06:27:52 +0000 (06:27 +0000)]
Merge "Experimental commit for A-B testing with Clover Fraser release and on the SDC application"

7 years agoMerge "Created config guide for SDC sample app"
Stephen Wong [Wed, 25 Apr 2018 06:26:36 +0000 (06:26 +0000)]
Merge "Created config guide for SDC sample app"

7 years agoCreated config guide for SDC sample app 47/55847/7
Eddie Arrage [Tue, 17 Apr 2018 02:31:44 +0000 (02:31 +0000)]
Created config guide for SDC sample app

- Overview with micro-service diagram
- Source diagram file in GIMP with layers for editing by
others
- Deploying the sample
- Using the sample
- Exposing Jaeger Tracing and Prometheus monitoring browser
interfaces
- Modifying run-time micro-service configuration including
modifying load balancer server list and adding a custom snort rule
- Uninstalling the sample

- Updated overview with service description, table and traffic
flow description, general edits
- Link to A/B configuration guide and doc index file
- Additional edits
- Diagram for Jaeger UI with SDC

Change-Id: I5d851316c05a9e1bd48c8aab5511a98116e6893d
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoSimplification of Clover Fraser official docs 31/56431/2
Stephen Wong [Tue, 24 Apr 2018 22:26:47 +0000 (15:26 -0700)]
Simplification of Clover Fraser official docs

(Double commit from master:I89adbef74aa74071a055dcdf62aa0925e263ffe3,
gerrit 56167)

Change-Id: I86f12d5ba67f09177eca758c184c614ee9d6dd9d
Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
7 years agoMerge "Fix snort rule with blank content & WR packet in alert"
Stephen Wong [Tue, 24 Apr 2018 22:19:25 +0000 (22:19 +0000)]
Merge "Fix snort rule with blank content & WR packet in alert"

7 years agoFraser official docs simplification 67/56167/3
Stephen Wong [Fri, 20 Apr 2018 22:37:07 +0000 (15:37 -0700)]
Fraser official docs simplification
[TBD]: link to configguide for sample VNF, link to A-B testing configguide

Change-Id: I89adbef74aa74071a055dcdf62aa0925e263ffe3
Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
7 years agoExperimental commit for A-B testing with Clover Fraser release 21/56021/3
Stephen Wong [Wed, 18 Apr 2018 23:27:59 +0000 (16:27 -0700)]
Experimental commit for A-B testing with Clover Fraser release
and on the SDC application

Change-Id: I6e1bd84a6d674a2c4c4484722b20415f5402a59c
Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
7 years agoFix snort rule with blank content & WR packet in alert 05/56305/1
Eddie Arrage [Tue, 24 Apr 2018 00:22:07 +0000 (00:22 +0000)]
Fix snort rule with blank content & WR packet in alert

- Fix bug with addition of content field in rule definition
that causes rules with a blank content fields to inhibit
snort from starting successfully.
- Write more of the packet data for snort alert into Redis
- Above includes X-Real-IP, X-Forwarded-For header fields
for http traffic from proxy that shows source IP

Some packet data is missing in alerts from snort.

Change-Id: I2c5c29e514d1ca9e8e5b9b3f7990afa87c6311b9
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoMerge "Add section to help user understand how logging works"
Stephen Wong [Thu, 19 Apr 2018 23:47:59 +0000 (23:47 +0000)]
Merge "Add section to help user understand how logging works"

7 years agoMerge "Cover validation of elasticsearch cluster"
Stephen Wong [Thu, 19 Apr 2018 23:47:05 +0000 (23:47 +0000)]
Merge "Cover validation of elasticsearch cluster"

7 years agoMinor edits to release notes and user guides 07/56007/1
rpaik [Wed, 18 Apr 2018 17:56:27 +0000 (10:56 -0700)]
Minor edits to release notes and user guides

Change-Id: I093fb995f108194367334c4215780b5884d0207a
Signed-off-by: rpaik <rpaik@linuxfoundation.org>
7 years agoCover validation of elasticsearch cluster 75/55875/2
Yujun Zhang [Tue, 17 Apr 2018 08:28:38 +0000 (16:28 +0800)]
Cover validation of elasticsearch cluster

- cluster health is not red
- indics found
- log entry created by istio found
- requests in and out http load balance matches

pytest is used as the test runner and wrapped in `validate.py`

Change-Id: Iad540b69d05118fadc97df679cf3424513c15e38
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoMerge "Remove unused argument"
Stephen Wong [Tue, 17 Apr 2018 05:38:13 +0000 (05:38 +0000)]
Merge "Remove unused argument"

7 years agoMerge "Added dependent python packages to Clover container"
Stephen Wong [Tue, 17 Apr 2018 05:37:16 +0000 (05:37 +0000)]
Merge "Added dependent python packages to Clover container"

7 years agoMerge "Extended snort rule add to allow content field"
Stephen Wong [Tue, 17 Apr 2018 05:36:04 +0000 (05:36 +0000)]
Merge "Extended snort rule add to allow content field"

7 years agoAdd section to help user understand how logging works 51/55851/2
Yujun Zhang [Tue, 17 Apr 2018 03:49:10 +0000 (11:49 +0800)]
Add section to help user understand how logging works

Change-Id: Iebfb747450cc08e930eabd36a87670236b23ffff
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoExtended snort rule add to allow content field 35/55835/1
Eddie Arrage [Mon, 16 Apr 2018 19:00:34 +0000 (19:00 +0000)]
Extended snort rule add to allow content field

- Exposed the 'content' field in the GRPC server AddRules method
- Allows the 'MALWARE-CNC User-Agent ASafaWeb Scan' signature
in the community rules to be copied to local rules
- Above ensures more deterministic alerts by snort each time
the signature is hit
- Added here to support the SDC configuration guide, which details
how to add this scan rule via GRPC client script

Change-Id: I6945c1e500075444134543bb9eb6003a03f1d5cc
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoAdded dependent python packages to Clover container 31/55831/1
Eddie Arrage [Mon, 16 Apr 2018 17:07:54 +0000 (17:07 +0000)]
Added dependent python packages to Clover container

- Added pip grpcio and argparse packages to docker build
- Allows service (nginx/snort) client sample scripts to be
executed using the Clover container without having to clone
the repo

Change-Id: Ifeda6d58a9a381cb80372255f41ad703a089ea4b
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoRemove unused argument 81/55781/1
Yujun Zhang [Mon, 16 Apr 2018 01:23:00 +0000 (09:23 +0800)]
Remove unused argument

`BRANCH` is no longer required since we copy source code from working directory
instead of remote git repository.

Change-Id: I44776538a9efbca72e8d165e7790603cdafbe395
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoMerge "Add SDC deploy/clean scripts"
Stephen Wong [Sun, 15 Apr 2018 06:58:59 +0000 (06:58 +0000)]
Merge "Add SDC deploy/clean scripts"

7 years agoMerge "Add support for install istio of specified version"
Stephen Wong [Sat, 14 Apr 2018 00:05:02 +0000 (00:05 +0000)]
Merge "Add support for install istio of specified version"

7 years agoAdd SDC deploy/clean scripts 03/55703/2
Eddie Arrage [Fri, 13 Apr 2018 19:15:29 +0000 (19:15 +0000)]
Add SDC deploy/clean scripts

- Added deploy/clean scripts for use in Clover container
- Deployment of entire SDC scenario
- Deployment includes istio install for manual sidecar injection
without TLS authentication (deploy.sh)
- Added Jaeger tracing and Prometheus monitoring install (view.sh)
- Exposes NodePort for monitoring/tracing to access UIs outside
of cluster
- Clean.sh attempts to remove all of the above

Change-Id: Id9548a77d71465a814a6e0cb1cbdf02d37235590
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoAdd support for install istio of specified version 57/55557/3
Yujun Zhang [Thu, 12 Apr 2018 08:59:54 +0000 (16:59 +0800)]
Add support for install istio of specified version

Change-Id: Ibfe0002daff58d30e7fffbb8828d8853a7e963a6
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoUpdated from global requirements 23/55523/1
Cédric Ollivier [Thu, 12 Apr 2018 07:18:56 +0000 (09:18 +0200)]
Updated from global requirements

Change-Id: Ieeaf87ab920f1862e3a1b9ac3316d387ff64954f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoMerge "Update from requirements"
Stephen Wong [Thu, 12 Apr 2018 06:06:51 +0000 (06:06 +0000)]
Merge "Update from requirements"

7 years agoMerge "Create index to be included in opnfvdocs site"
Stephen Wong [Thu, 12 Apr 2018 06:04:46 +0000 (06:04 +0000)]
Merge "Create index to be included in opnfvdocs site"

7 years agoMerge "Modified validation script for tracing to support CI"
Stephen Wong [Thu, 12 Apr 2018 06:03:07 +0000 (06:03 +0000)]
Merge "Modified validation script for tracing to support CI"

7 years agoUpdate from requirements 09/55509/1
Yujun Zhang [Thu, 12 Apr 2018 04:05:57 +0000 (12:05 +0800)]
Update from requirements

Pipfile.lock ensures a consistent environment

Change-Id: Id2e544c77a67ce8fa010fba9c357735496f62a26
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoCreate index to be included in opnfvdocs site 87/55487/1
Yujun Zhang [Thu, 12 Apr 2018 01:16:06 +0000 (09:16 +0800)]
Create index to be included in opnfvdocs site

Change-Id: Icbfe547697a8d879f4af8d9f9fbde2211b63129c
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoModified validation script for tracing to support CI 87/55387/1
Eddie Arrage [Wed, 11 Apr 2018 02:44:51 +0000 (02:44 +0000)]
Modified validation script for tracing to support CI

- Changed default Jaeger ports to 16686 for use with basic
kubernetes port-forward and CI scripts
- Added CLI to validate script to disable istio service check
by default. This requires at least a single http request
to istio-ingress after Jaeger deployment. It can be enabled
with 'python validate.py -s'. Port and IP address for Jaeger
can optionally be specified with '-ip' and '-port' options
- Modified tracing doc to add k8s port-forward example in addition
to k8s expose

Change-Id: I10fb4d3cccfa50370d44ed7446f67a49c538bba9
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoUpdated from global requirements 65/55365/1
Cédric Ollivier [Tue, 10 Apr 2018 19:26:55 +0000 (21:26 +0200)]
Updated from global requirements

Change-Id: I9e6c925744ed928f7a5c6fbe54942e8b3895b1b9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
7 years agoAdd existing public redis pod/service 49/54849/3
Eddie Arrage [Wed, 4 Apr 2018 00:28:12 +0000 (00:28 +0000)]
Add existing public redis pod/service

- Use a community yaml for redis in k8s as simple data store
- Redis can be used for tracing and also by the snort-ids
to store alerts that can be processed by other services
- If flannel is used, the redis CLI can be accessed on the
host OS with redis-cli -h <flannel ip>
- Within the k8s cluster, the redis service can be accessed with
DNS using name 'redis'
- The same yaml for redis is also included in toplevel manifest for SDC
scenario. Included here if intention is to use separately (tracing
only)

Change-Id: Ibad283a4cc8938fe01f5de6b7743bdb5511be3af
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
7 years agoMerge "Fix Nginx lb in k8s/istio"
Stephen Wong [Thu, 5 Apr 2018 23:20:26 +0000 (23:20 +0000)]
Merge "Fix Nginx lb in k8s/istio"

7 years agoMerge "Added toplevel manifests for SDC sample scenario"
Stephen Wong [Thu, 5 Apr 2018 07:07:03 +0000 (07:07 +0000)]
Merge "Added toplevel manifests for SDC sample scenario"

7 years agoMerge "Aligned snort-ids default config for SDC scenario"
Stephen Wong [Thu, 5 Apr 2018 05:30:06 +0000 (05:30 +0000)]
Merge "Aligned snort-ids default config for SDC scenario"

7 years agoMerge "Move design document to dedicated folder"
Stephen Wong [Thu, 5 Apr 2018 05:24:20 +0000 (05:24 +0000)]
Merge "Move design document to dedicated folder"

7 years agoAdd clover initial docker image build script 85/54685/5
QiLiang [Sat, 31 Mar 2018 09:21:30 +0000 (17:21 +0800)]
Add clover initial docker image build script

- install dependent deb/pip packages
- install basic tools istioctl, kubectl
- install clover source code
- build/upload docker image script
- update requirements.txt
- update module import path

- To use this image use need setup kube-config file.
    e.g. `docker run -v /root/config:/root/.kube/config -it clover bash`

Change-Id: I91044bb99ce8e2b785ef03212d961a97b3d42233
Signed-off-by: QiLiang <liangqi1@huawei.com>
7 years agoMove design document to dedicated folder 81/54781/2
Yujun Zhang [Tue, 3 Apr 2018 08:48:24 +0000 (16:48 +0800)]
Move design document to dedicated folder

Change-Id: I20c85b7116cd2b29d0efcaae5ee0b0381a685bbb
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
7 years agoAligned snort-ids default config for SDC scenario 45/54845/2
Eddie Arrage [Tue, 3 Apr 2018 23:57:58 +0000 (23:57 +0000)]
Aligned snort-ids default config for SDC scenario

- Modified snort-ids alert process to use k8s DNS name
'proxy-access-control' to align with SDC scenario naming
- Added default port 50054 to the manifest yaml template and
rendering script for communication with proxy-access-control

Change-Id: Ib04ee75e5d8ea9921b16b3b4469bed87b1cd2018
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>