X-Git-Url: https://gerrit.opnfv.org/gerrit/gitweb?a=blobdiff_plain;f=src%2Fceph%2Fdoc%2Fdev%2Fcorpus.rst;fp=src%2Fceph%2Fdoc%2Fdev%2Fcorpus.rst;h=76fa43db0a771b3fb6dacdb7ea296706f6a1bf78;hb=812ff6ca9fcd3e629e49d4328905f33eee8ca3f5;hp=0000000000000000000000000000000000000000;hpb=15280273faafb77777eab341909a3f495cf248d9;p=stor4nfv.git

diff --git a/src/ceph/doc/dev/corpus.rst b/src/ceph/doc/dev/corpus.rst
new file mode 100644
index 0000000..76fa43d
--- /dev/null
+++ b/src/ceph/doc/dev/corpus.rst
@@ -0,0 +1,95 @@
+
+Corpus structure
+================
+
+ceph.git/ceph-object-corpus is a submodule.::
+
+ bin/   # misc scripts
+ archive/$version/objects/$type/$hash  # a sample of encoded objects from a specific version
+
+You can also mark known or deliberate incompatibilities between versions with::
+
+ archive/$version/forward_incompat/$type
+
+The presence of a file indicates that new versions of code cannot
+decode old objects across that $version (this is normally the case).
+
+
+How to generate an object corpus
+--------------------------------
+
+We can generate an object corpus for a particular version of ceph like so.
+
+#. Checkout a clean repo (best not to do this where you normally work)::
+
+	git clone ceph.git
+	cd ceph
+	git submodule update --init --recursive
+
+#. Build with flag to dump objects to /tmp/foo::
+
+	rm -rf /tmp/foo ; mkdir /tmp/foo
+	./do_autogen.sh -e /tmp/foo
+	make
+
+#. Start via vstart::
+
+	cd src
+	MON=3 OSD=3 MDS=3 RGW=1 ./vstart.sh -n -x
+
+#. Use as much functionality of the cluster as you can, to exercise as many object encoder methods as possible::
+
+	./rados -p rbd bench 10 write -b 123
+	./ceph osd out 0
+	./init-ceph stop osd.1
+	for f in ../qa/workunits/cls/*.sh ; do PATH=".:$PATH" $f ; done
+	../qa/workunits/rados/test.sh
+	./ceph_test_librbd
+	./ceph_test_libcephfs
+	./init-ceph restart mds.a
+
+Do some more stuff with rgw if you know how.
+
+#. Stop::
+
+	./stop.sh
+
+#. Import the corpus (this will take a few minutes)::
+
+	test/encoding/import.sh /tmp/foo `./ceph-dencoder version` ../ceph-object-corpus/archive
+	test/encoding/import-generated.sh ../ceph-object-corpus/archive
+
+#. Prune it!  There will be a bazillion copies of various objects, and we only want a representative sample.::
+
+	pushd ../ceph-object-corpus
+	bin/prune-archive.sh
+	popd
+
+#. Verify the tests pass::
+
+	make check-local
+
+#. Commit it to the corpus repo and push::
+
+	pushd ../ceph-object-corpus
+	git checkout -b wip-new
+	git add archive/`../src/ceph-dencoder version`
+	git commit -m `../src/ceph-dencoder version`
+	git remote add cc ceph.com:/git/ceph-object-corpus.git
+	git push cc wip-new
+	popd
+
+#. Go test it out::
+
+	cd my/regular/tree
+	cd ceph-object-corpus
+	git fetch origin
+	git checkout wip-new
+	cd ../src
+	make check-local
+
+#. If everything looks good, update the submodule master branch, and commit the submodule in ceph.git.
+
+
+
+