fuel.git
8 years agoImproved caching functionality and tracability 57/2557/5
Stefan K. Berg [Thu, 15 Oct 2015 10:51:26 +0000 (12:51 +0200)]
Improved caching functionality and tracability

In order to enable the caching of more build artifacts than the Fuel
iso during CI builds, the caching functionality and the CI build
frontend "build.sh" has been rewritten.

The build.sh script will now rely on the "make cache" functionality of
the top Makefile to make sure that the build is using cache handling.

The underlying cache logic is implemented in "cache.mk" for the top
makefile and those recursive levels that do not produce cachable
artifacts themselves in that they are only calling the SUBDIRS of
their lower level (like f_isoroot).

All "leaf" Makefiles will however need to implement three cache
targets in their top Makefile (for visibility):

clean-cache:
        Clean all files relating to the handling of caches.
get-cache:
        Attempt to fetch a cached artifact using a SHA1 key.
put-cache:
        Store a built artifact into the cache.

They can just implement a simple "no-op" functionality for thes
targets if they do not have any use of the caching functionality.

If they are to use the caching functionality, they must make sure to
implement a logic that produces a SHA1 hash based on the source of the
data they are to cache - for upstream repos one suggestion is to use
the commit ID of the used upstream HEAD. For examples, see the top
Makefile that implements this logic for the Fuel ISO build, taking
into consideration the commit IDs of all the upstream repositories
used.

To improve tracability, the root directory of the ISO will contain the
file gitinfo.txt, which is meant to detail the upstream repo and
commit ID used for all upstream dependencies of a build. If you are
adding additional upstream dependencies, make sure to use the
repo_info.sh tool to add this data.

The cache tool cache.sh has no notion of the cache data it is storing
- from the tools perspective, cache data is just a binary blob piped
in or out of the tool.

This blob is stored by the cache tool at the cache location as
<SHA1>.blob, together with an associated meta file <SHA1>.meta.

The cache meta file currently holds just one line:

    Expires: <epoch time>

This file is expected to be used to iterate through the cache objects
and retire those who's expiry date has passed. Currently objects will
always have an expiry date of two weeks into the future, but down the
road the "cache put" functionality may be amended with an optional
"age" argument that can set a different expiration time.

New tools in this commit:

  cache.sh - the cache logic
  repo_info.sh - the (optionally recursive) repo information logger

Change-Id: I8a40546c21febeecc9de6d82c0ceb6bc60b04205
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
8 years agoUpdated logic for selecting Ubuntu repo for mirroring 65/2365/5
Stefan K. Berg [Thu, 8 Oct 2015 16:06:00 +0000 (18:06 +0200)]
Updated logic for selecting Ubuntu repo for mirroring

A slight improvement to Michal Skalski's original concept just
to filter out repos where updates are in progress (which would
leave us hanging).

Change-Id: Id641b3aa82b991c23b5742d1f64ff79cfbbd708c
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
8 years agoTemporary fix for forcing cache rebuild 17/2317/2
Stefan K. Berg [Wed, 7 Oct 2015 18:54:05 +0000 (20:54 +0200)]
Temporary fix for forcing cache rebuild

The caching functionality will be greatly improved for the next
release - this is a quick fix for having the possiblility to
invalidate the cache forcefully when needed.

Change-Id: I4d83907255105fb454af6fb426193acc744fdba9
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
8 years agoMerge "Rebased patch due to upstream changes"
Stefan K. Berg [Fri, 9 Oct 2015 05:09:08 +0000 (05:09 +0000)]
Merge "Rebased patch due to upstream changes"

8 years agoRebased patch due to upstream changes 69/2369/1
Stefan K. Berg [Thu, 8 Oct 2015 18:55:06 +0000 (20:55 +0200)]
Rebased patch due to upstream changes

Change-Id: I63e39ff3d08bcfb80b83bb0c42082deb680ad727
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
8 years agoCAN MERGE 95/2295/4
Jonas Bjurel [Wed, 7 Oct 2015 14:10:16 +0000 (16:10 +0200)]
CAN MERGE
VERIFIED
Description: Improved handling of docker clean-up
JIRA: FUEL-71

Change-Id: I565f77e227da2dcbaecd86582ab6890508726607
Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
8 years agoModifications due to upstream changes 93/2293/1
Stefan K. Berg [Wed, 7 Oct 2015 14:11:07 +0000 (16:11 +0200)]
Modifications due to upstream changes

Removed a patch which is now incorporated upstream.
Added debmirror as a requirement for the build Docker
container.

Change-Id: I92fa8571019ece9bac41bec2baa3629d286a3a51
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
8 years agoAdd AMT adapter 51/2251/1
Liyi Meng [Mon, 5 Oct 2015 07:55:20 +0000 (09:55 +0200)]
Add AMT adapter

This adapter allow fuel deploy over Intel AMT/vPro system.
Please note that the adapter use amttool to interact with
target system, which imply that it only support up to
vPro v7.

Change-Id: I75f0882ea914b57d7d62338ed803a45104bc2d4e
Signed-off-by: Liyi Meng <liyi.meng@ericsson.com>
8 years agoMove of genesis/fuel master branch: commit 563547b4a9f44090f32c0e17d040114854563760 11/2211/1
Jonas Bjurel [Fri, 2 Oct 2015 11:29:56 +0000 (13:29 +0200)]
Move of genesis/fuel master branch: commit 563547b4a9f44090f32c0e17d040114854563760
Note: other installers and /common are removed

Change-Id: Ie5a2b0b7f3e7fa2eda191710c98456eeec17ed61
Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
8 years agoModified INFO 64/1164/1
Jonas Bjurel [Wed, 12 Aug 2015 14:30:48 +0000 (16:30 +0200)]
Modified INFO
Added Licence.rst

Change-Id: I610e75eccd44adff177122dd8e3a93a44e83d28e
Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
8 years agoAdding .gitreview INFO and License 94/994/1
Aric Gardner [Fri, 10 Jul 2015 17:21:00 +0000 (13:21 -0400)]
Adding .gitreview INFO and License

Change-Id: I31333946c4095c3e96f541f3fcc62cdeabd17a77
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
8 years agoInitial empty repository
Aric Gardner [Fri, 10 Jul 2015 15:07:46 +0000 (15:07 +0000)]
Initial empty repository