5 The layout of a file controls how its contents are mapped to Ceph RADOS objects. You can
6 read and write a file's layout using *virtual extended attributes* or xattrs.
8 The name of the layout xattrs depends on whether a file is a regular file or a directory. Regular
9 files' layout xattrs are called ``ceph.file.layout``, whereas directories' layout xattrs are called
10 ``ceph.dir.layout``. Where subsequent examples refer to ``ceph.file.layout``, substitute ``dir`` as appropriate
11 when dealing with directories.
15 Your linux distribution may not ship with commands for manipulating xattrs by default,
16 the required package is usually called ``attr``.
22 String, giving ID or name. Which RADOS pool a file's data objects will be stored in.
25 String. Within the data pool, which RADOS namespace the objects will
26 be written to. Empty by default (i.e. default namespace).
29 Integer in bytes. The size (in bytes) of a block of data used in the RAID 0 distribution of a file. All stripe units for a file have equal size. The last stripe unit is typically incomplete–i.e. it represents the data at the end of the file as well as unused “space” beyond it up to the end of the fixed stripe unit size.
32 Integer. The number of consecutive stripe units that constitute a RAID 0 “stripe” of file data.
35 Integer in bytes. File data is chunked into RADOS objects of this size.
39 RADOS enforces a configurable limit on object sizes: if you increase CephFS
40 object sizes beyond that limit then writes may not succeed. The OSD
41 setting is ``rados_max_object_size``, which is 128MB by default.
42 Very large RADOS objects may prevent smooth operation of the cluster,
43 so increasing the object size limit past the default is not recommended.
45 Reading layouts with ``getfattr``
46 ---------------------------------
48 Read the layout information as a single string:
53 $ getfattr -n ceph.file.layout file
55 ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data"
57 Read individual layout fields:
61 $ getfattr -n ceph.file.layout.pool file
63 ceph.file.layout.pool="cephfs_data"
64 $ getfattr -n ceph.file.layout.stripe_unit file
66 ceph.file.layout.stripe_unit="4194304"
67 $ getfattr -n ceph.file.layout.stripe_count file
69 ceph.file.layout.stripe_count="1"
70 $ getfattr -n ceph.file.layout.object_size file
72 ceph.file.layout.object_size="4194304"
76 When reading layouts, the pool will usually be indicated by name. However, in
77 rare cases when pools have only just been created, the ID may be output instead.
79 Directories do not have an explicit layout until it is customized. Attempts to read
80 the layout will fail if it has never been modified: this indicates that layout of the
81 next ancestor directory with an explicit layout will be used.
86 $ getfattr -n ceph.dir.layout dir
87 dir: ceph.dir.layout: No such attribute
88 $ setfattr -n ceph.dir.layout.stripe_count -v 2 dir
89 $ getfattr -n ceph.dir.layout dir
91 ceph.dir.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
94 Writing layouts with ``setfattr``
95 ---------------------------------
97 Layout fields are modified using ``setfattr``:
102 0 rbd,1 cephfs_data,2 cephfs_metadata,
104 $ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2
105 $ setfattr -n ceph.file.layout.stripe_count -v 8 file2
106 $ setfattr -n ceph.file.layout.object_size -v 10485760 file2
107 $ setfattr -n ceph.file.layout.pool -v 1 file2 # Setting pool by ID
108 $ setfattr -n ceph.file.layout.pool -v cephfs_data file2 # Setting pool by name
112 When the layout fields of a file are modified using ``setfattr``, this file must be empty, otherwise an error will occur.
116 # touch an empty file
118 # modify layout field successfully
119 $ setfattr -n ceph.file.layout.stripe_count -v 3 file1
121 # write something to file1
122 $ echo "hello world" > file1
123 $ setfattr -n ceph.file.layout.stripe_count -v 4 file1
124 setfattr: file1: Directory not empty
129 If you wish to remove an explicit layout from a directory, to revert to
130 inherting the layout of its ancestor, you can do so:
134 setfattr -x ceph.dir.layout mydir
136 Similarly, if you have set the ``pool_namespace`` attribute and wish
137 to modify the layout to use the default namespace instead:
141 # Create a dir and set a namespace on it
143 setfattr -n ceph.dir.layout.pool_namespace -v foons mydir
144 getfattr -n ceph.dir.layout mydir
145 ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_a pool_namespace=foons"
147 # Clear the namespace from the directory's layout
148 setfattr -x ceph.dir.layout.pool_namespace mydir
149 getfattr -n ceph.dir.layout mydir
150 ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_a"
153 Inheritance of layouts
154 ----------------------
156 Files inherit the layout of their parent directory at creation time. However, subsequent
157 changes to the parent directory's layout do not affect children.
161 $ getfattr -n ceph.dir.layout dir
163 ceph.dir.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
165 # Demonstrate file1 inheriting its parent's layout
167 $ getfattr -n ceph.file.layout dir/file1
169 ceph.file.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
171 # Now update the layout of the directory before creating a second file
172 $ setfattr -n ceph.dir.layout.stripe_count -v 4 dir
175 # Demonstrate that file1's layout is unchanged
176 $ getfattr -n ceph.file.layout dir/file1
178 ceph.file.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
180 # ...while file2 has the parent directory's new layout
181 $ getfattr -n ceph.file.layout dir/file2
183 ceph.file.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
186 Files created as descendents of the directory also inherit the layout, if the intermediate
187 directories do not have layouts set:
191 $ getfattr -n ceph.dir.layout dir
193 ceph.dir.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
195 $ getfattr -n ceph.dir.layout dir/childdir
196 dir/childdir: ceph.dir.layout: No such attribute
197 $ touch dir/childdir/grandchild
198 $ getfattr -n ceph.file.layout dir/childdir/grandchild
199 # file: dir/childdir/grandchild
200 ceph.file.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
203 Adding a data pool to the MDS
204 ---------------------------------
206 Before you can use a pool with CephFS you have to add it to the Metadata Servers.
210 $ ceph fs add_data_pool cephfs cephfs_data_ssd
211 # Pool should now show up
213 .... data pools: [cephfs_data cephfs_data_ssd ]
215 Make sure that your cephx keys allows the client to access this new pool.