3 ===============================================
4 rbd -- manage rados block device (RBD) images
5 ===============================================
12 | **rbd** [ -c *ceph.conf* ] [ -m *monaddr* ] [--cluster *cluster-name*]
13 [ -p | --pool *pool* ] [ *command* ... ]
19 **rbd** is a utility for manipulating rados block device (RBD) images,
20 used by the Linux rbd driver and the rbd storage driver for QEMU/KVM.
21 RBD images are simple block devices that are striped over objects and
22 stored in a RADOS object store. The size of the objects the image is
23 striped over must be a power of two.
29 .. option:: -c ceph.conf, --conf ceph.conf
31 Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to
32 determine monitor addresses during startup.
34 .. option:: -m monaddress[:port]
36 Connect to specified monitor (instead of looking through ceph.conf).
38 .. option:: --cluster cluster-name
40 Use different cluster name as compared to default cluster name *ceph*.
42 .. option:: -p pool-name, --pool pool-name
44 Interact with the given pool. Required by most commands.
46 .. option:: --no-progress
48 Do not output progress information (goes to standard error by
49 default for some commands).
55 .. option:: --image-format format-id
57 Specifies which object layout to use. The default is 2.
59 * format 1 - (deprecated) Use the original format for a new rbd image. This
60 format is understood by all versions of librbd and the kernel rbd module,
61 but does not support newer features like cloning.
63 * format 2 - Use the second rbd format, which is supported by
64 librbd and kernel since version 3.11 (except for striping). This adds
65 support for cloning and is more easily extensible to allow more
66 features in the future.
68 .. option:: -s size-in-M/G/T, --size size-in-M/G/T
70 Specifies the size of the new rbd image or the new size of the existing rbd
71 image in M/G/T. If no suffix is given, unit M is assumed.
73 .. option:: --object-size size-in-B/K/M
75 Specifies the object size in B/K/M. Object size will be rounded up the
76 nearest power of two; if no suffix is given, unit B is assumed. The default
77 object size is 4M, smallest is 4K and maximum is 32M.
79 .. option:: --stripe-unit size-in-B/K/M
81 Specifies the stripe unit size in B/K/M. If no suffix is given, unit B is
82 assumed. See striping section (below) for more details.
84 .. option:: --stripe-count num
86 Specifies the number of objects to stripe over before looping back
87 to the first object. See striping section (below) for more details.
89 .. option:: --snap snap
91 Specifies the snapshot name for the specific operation.
93 .. option:: --id username
95 Specifies the username (without the ``client.`` prefix) to use with the map command.
97 .. option:: --keyring filename
99 Specifies a keyring file containing a secret for the specified user
100 to use with the map command. If not specified, the default keyring
101 locations will be searched.
103 .. option:: --keyfile filename
105 Specifies a file containing the secret key of ``--id user`` to use with the map command.
106 This option is overridden by ``--keyring`` if the latter is also specified.
108 .. option:: --shared lock-tag
110 Option for `lock add` that allows multiple clients to lock the
111 same image if they use the same tag. The tag is an arbitrary
112 string. This is useful for situations where an image must
113 be open from more than one client at once, like during
114 live migration of a virtual machine, or for use underneath
115 a clustered filesystem.
117 .. option:: --format format
119 Specifies output formatting (default: plain, json, xml)
121 .. option:: --pretty-format
123 Make json or xml formatted output more human-readable.
125 .. option:: -o krbd-options, --options krbd-options
127 Specifies which options to use when mapping or unmapping an image via the
128 rbd kernel driver. krbd-options is a comma-separated list of options
129 (similar to mount(8) mount options). See kernel rbd (krbd) options section
130 below for more details.
132 .. option:: --read-only
134 Map the image read-only. Equivalent to -o ro.
136 .. option:: --image-feature feature-name
138 Specifies which RBD format 2 feature should be enabled when creating
139 an image. Multiple features can be enabled by repeating this option
140 multiple times. The following features are supported:
142 * layering: layering support
143 * striping: striping v2 support
144 * exclusive-lock: exclusive locking support
145 * object-map: object map support (requires exclusive-lock)
146 * fast-diff: fast diff calculations (requires object-map)
147 * deep-flatten: snapshot flatten support
148 * journaling: journaled IO support (requires exclusive-lock)
150 .. option:: --image-shared
152 Specifies that the image will be used concurrently by multiple clients.
153 This will disable features that are dependent upon exclusive ownership
156 .. option:: --whole-object
158 Specifies that the diff should be limited to the extents of a full object
159 instead of showing intra-object deltas. When the object map feature is
160 enabled on an image, limiting the diff to the object extents will
161 dramatically improve performance since the differences can be computed
162 by examining the in-memory object map instead of querying RADOS for each
163 object within the image.
167 Specifies the limit for the number of snapshots permitted.
172 .. TODO rst "option" directive seems to require --foo style options, parsing breaks on subcommands.. the args show up as bold too
174 :command:`ls` [-l | --long] [*pool-name*]
175 Will list all rbd images listed in the rbd_directory object. With
176 -l, also show snapshots, and use longer-format output including
177 size, parent (if clone), format, etc.
179 :command:`du` [-p | --pool *pool-name*] [*image-spec* | *snap-spec*]
180 Will calculate the provisioned and actual disk usage of all images and
181 associated snapshots within the specified pool. It can also be used against
182 individual images and snapshots.
184 If the RBD fast-diff feature is not enabled on images, this operation will
185 require querying the OSDs for every potential object within the image.
187 :command:`info` *image-spec* | *snap-spec*
188 Will dump information (such as size and object size) about a specific rbd image.
189 If image is a clone, information about its parent is also displayed.
190 If a snapshot is specified, whether it is protected is shown as well.
192 :command:`create` (-s | --size *size-in-M/G/T*) [--image-format *format-id*] [--object-size *size-in-B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *image-spec*
193 Will create a new rbd image. You must also specify the size via --size. The
194 --stripe-unit and --stripe-count arguments are optional, but must be used together.
196 :command:`clone` [--object-size *size-in-B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*] [--image-shared] *parent-snap-spec* *child-image-spec*
197 Will create a clone (copy-on-write child) of the parent snapshot.
198 Object size will be identical to that of the parent image unless
199 specified. Size will be the same as the parent snapshot. The --stripe-unit
200 and --stripe-count arguments are optional, but must be used together.
202 The parent snapshot must be protected (see `rbd snap protect`).
203 This requires image format 2.
205 :command:`flatten` *image-spec*
206 If image is a clone, copy all shared blocks from the parent snapshot and
207 make the child independent of the parent, severing the link between
208 parent snap and child. The parent snapshot can be unprotected and
209 deleted if it has no further dependent clones.
211 This requires image format 2.
213 :command:`children` *snap-spec*
214 List the clones of the image at the given snapshot. This checks
215 every pool, and outputs the resulting poolname/imagename.
217 This requires image format 2.
219 :command:`resize` (-s | --size *size-in-M/G/T*) [--allow-shrink] *image-spec*
220 Resize rbd image. The size parameter also needs to be specified.
221 The --allow-shrink option lets the size be reduced.
223 :command:`rm` *image-spec*
224 Delete an rbd image (including all data blocks). If the image has
225 snapshots, this fails and nothing is deleted.
227 :command:`export` [--export-format *format (1 or 2)*] (*image-spec* | *snap-spec*) [*dest-path*]
228 Export image to dest path (use - for stdout).
229 The --export-format accepts '1' or '2' currently. Format 2 allow us to export not only the content
230 of image, but also the snapshots and other properties, such as image_order, features.
232 :command:`import` [--export-format *format (1 or 2)*] [--image-format *format-id*] [--object-size *size-in-B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *src-path* [*image-spec*]
233 Create a new image and imports its data from path (use - for
234 stdin). The import operation will try to create sparse rbd images
235 if possible. For import from stdin, the sparsification unit is
236 the data block size of the destination image (object size).
238 The --stripe-unit and --stripe-count arguments are optional, but must be
241 The --export-format accepts '1' or '2' currently. Format 2 allow us to import not only the content
242 of image, but also the snapshots and other properties, such as image_order, features.
244 :command:`export-diff` [--from-snap *snap-name*] [--whole-object] (*image-spec* | *snap-spec*) *dest-path*
245 Export an incremental diff for an image to dest path (use - for stdout). If
246 an initial snapshot is specified, only changes since that snapshot are included; otherwise,
247 any regions of the image that contain data are included. The end snapshot is specified
248 using the standard --snap option or @snap syntax (see below). The image diff format includes
249 metadata about image size changes, and the start and end snapshots. It efficiently represents
250 discarded or 'zero' regions of the image.
252 :command:`merge-diff` *first-diff-path* *second-diff-path* *merged-diff-path*
253 Merge two continuous incremental diffs of an image into one single diff. The
254 first diff's end snapshot must be equal with the second diff's start snapshot.
255 The first diff could be - for stdin, and merged diff could be - for stdout, which
256 enables multiple diff files to be merged using something like
257 'rbd merge-diff first second - | rbd merge-diff - third result'. Note this command
258 currently only support the source incremental diff with stripe_count == 1
260 :command:`import-diff` *src-path* *image-spec*
261 Import an incremental diff of an image and applies it to the current image. If the diff
262 was generated relative to a start snapshot, we verify that snapshot already exists before
263 continuing. If there was an end snapshot we verify it does not already exist before
264 applying the changes, and create the snapshot when we are done.
266 :command:`diff` [--from-snap *snap-name*] [--whole-object] *image-spec* | *snap-spec*
267 Dump a list of byte extents in the image that have changed since the specified start
268 snapshot, or since the image was created. Each output line includes the starting offset
269 (in bytes), the length of the region (in bytes), and either 'zero' or 'data' to indicate
270 whether the region is known to be zeros or may contain other data.
272 :command:`cp` (*src-image-spec* | *src-snap-spec*) *dest-image-spec*
273 Copy the content of a src-image into the newly created dest-image.
274 dest-image will have the same size, object size, and image format as src-image.
276 :command:`mv` *src-image-spec* *dest-image-spec*
277 Rename an image. Note: rename across pools is not supported.
279 :command:`image-meta list` *image-spec*
280 Show metadata held on the image. The first column is the key
281 and the second column is the value.
283 :command:`image-meta get` *image-spec* *key*
284 Get metadata value with the key.
286 :command:`image-meta set` *image-spec* *key* *value*
287 Set metadata key with the value. They will displayed in `image-meta list`.
289 :command:`image-meta remove` *image-spec* *key*
290 Remove metadata key with the value.
292 :command:`object-map rebuild` *image-spec* | *snap-spec*
293 Rebuild an invalid object map for the specified image. An image snapshot can be
294 specified to rebuild an invalid object map for a snapshot.
296 :command:`snap ls` *image-spec*
297 Dump the list of snapshots inside a specific image.
299 :command:`snap create` *snap-spec*
300 Create a new snapshot. Requires the snapshot name parameter specified.
302 :command:`snap rollback` *snap-spec*
303 Rollback image content to snapshot. This will iterate through the entire blocks
304 array and update the data head content to the snapshotted version.
306 :command:`snap rm` [--force] *snap-spec*
307 Remove the specified snapshot.
309 :command:`snap purge` *image-spec*
310 Remove all snapshots from an image.
312 :command:`snap protect` *snap-spec*
313 Protect a snapshot from deletion, so that clones can be made of it
314 (see `rbd clone`). Snapshots must be protected before clones are made;
315 protection implies that there exist dependent cloned children that
316 refer to this snapshot. `rbd clone` will fail on a nonprotected
319 This requires image format 2.
321 :command:`snap unprotect` *snap-spec*
322 Unprotect a snapshot from deletion (undo `snap protect`). If cloned
323 children remain, `snap unprotect` fails. (Note that clones may exist
324 in different pools than the parent snapshot.)
326 This requires image format 2.
328 :command:`snap limit set` [--limit] *limit* *image-spec*
329 Set a limit for the number of snapshots allowed on an image.
331 :command:`snap limit clear` *image-spec*
332 Remove any previously set limit on the number of snapshots allowed on
335 :command:`map` [-o | --options *krbd-options* ] [--read-only] *image-spec* | *snap-spec*
336 Map the specified image to a block device via the rbd kernel module.
338 :command:`unmap` [-o | --options *krbd-options* ] *image-spec* | *snap-spec* | *device-path*
339 Unmap the block device that was mapped via the rbd kernel module.
341 :command:`showmapped`
342 Show the rbd images that are mapped via the rbd kernel module.
344 :command:`nbd map` [--device *device-path*] [--read-only] *image-spec* | *snap-spec*
345 Map the specified image to a block device via the rbd-nbd tool.
347 :command:`nbd unmap` *device-path*
348 Unmap the block device that was mapped via the rbd-nbd tool.
351 Show the list of used nbd devices via the rbd-nbd tool.
353 :command:`status` *image-spec*
354 Show the status of the image, including which clients have it open.
356 :command:`feature disable` *image-spec* *feature-name*...
357 Disable the specified feature on the specified image. Multiple features can
360 :command:`feature enable` *image-spec* *feature-name*...
361 Enable the specified feature on the specified image. Multiple features can
364 :command:`lock list` *image-spec*
365 Show locks held on the image. The first column is the locker
366 to use with the `lock remove` command.
368 :command:`lock add` [--shared *lock-tag*] *image-spec* *lock-id*
369 Lock an image. The lock-id is an arbitrary name for the user's
370 convenience. By default, this is an exclusive lock, meaning it
371 will fail if the image is already locked. The --shared option
372 changes this behavior. Note that locking does not affect
373 any operation other than adding a lock. It does not
374 protect an image from being deleted.
376 :command:`lock remove` *image-spec* *lock-id* *locker*
377 Release a lock on an image. The lock id and locker are
378 as output by lock ls.
380 :command:`bench` --io-type <read | write> [--io-size *size-in-B/K/M/G/T*] [--io-threads *num-ios-in-flight*] [--io-total *size-in-B/K/M/G/T*] [--io-pattern seq | rand] *image-spec*
381 Generate a series of IOs to the image and measure the IO throughput and
382 latency. If no suffix is given, unit B is assumed for both --io-size and
383 --io-total. Defaults are: --io-size 4096, --io-threads 16, --io-total 1G,
386 :command:`trash ls` [*pool-name*]
387 List all entries from trash.
389 :command:`trash mv` *image-spec*
390 Move an image to the trash. Images, even ones actively in-use by
391 clones, can be moved to the trash and deleted at a later time.
393 :command:`trash rm` *image-id*
394 Delete an image from trash. If image deferment time has not expired
395 you can not removed it unless use force. But an actively in-use by clones
396 or has snapshots can not be removed.
398 :command:`trash restore` *image-id*
399 Restore an image from trash.
404 | *image-spec* is [*pool-name*]/*image-name*
405 | *snap-spec* is [*pool-name*]/*image-name*\ @\ *snap-name*
407 The default for *pool-name* is "rbd". If an image name contains a slash
408 character ('/'), *pool-name* is required.
410 You may specify each name individually, using --pool, --image and --snap
411 options, but this is discouraged in favor of the above spec syntax.
416 RBD images are striped over many objects, which are then stored by the
417 Ceph distributed object store (RADOS). As a result, read and write
418 requests for the image are distributed across many nodes in the
419 cluster, generally preventing any single node from becoming a
420 bottleneck when individual images get large or busy.
422 The striping is controlled by three parameters:
424 .. option:: object-size
426 The size of objects we stripe over is a power of two. It will be rounded up the nearest power of two.
427 The default object size is 4 MB, smallest is 4K and maximum is 32M.
429 .. option:: stripe_unit
431 Each [*stripe_unit*] contiguous bytes are stored adjacently in the same object, before we move on
434 .. option:: stripe_count
436 After we write [*stripe_unit*] bytes to [*stripe_count*] objects, we loop back to the initial object
437 and write another stripe, until the object reaches its maximum size. At that point,
438 we move on to the next [*stripe_count*] objects.
440 By default, [*stripe_unit*] is the same as the object size and [*stripe_count*] is 1. Specifying a different
441 [*stripe_unit*] requires that the STRIPINGV2 feature be supported (added in Ceph v0.53) and format 2 images be
445 Kernel rbd (krbd) options
446 =========================
448 Most of these options are useful mainly for debugging and benchmarking. The
449 default values are set in the kernel and may therefore depend on the version of
452 Per client instance `rbd map` options:
454 * fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be assumed by
457 * ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
459 * share - Enable sharing of client instances with other mappings (default).
461 * noshare - Disable sharing of client instances with other mappings.
463 * crc - Enable CRC32C checksumming for data writes (default).
465 * nocrc - Disable CRC32C checksumming for data writes.
467 * cephx_require_signatures - Require cephx message signing (since 3.19,
470 * nocephx_require_signatures - Don't require cephx message signing (since
473 * tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
476 * notcp_nodelay - Enable Nagle's algorithm on client sockets (since 4.0).
478 * cephx_sign_messages - Enable message signing (since 4.4, default).
480 * nocephx_sign_messages - Disable message signing (since 4.4).
482 * mount_timeout=x - A timeout on various steps in `rbd map` and `rbd unmap`
483 sequences (default is 60 seconds). In particular, since 4.2 this can be used
484 to ensure that `rbd unmap` eventually times out when there is no network
485 connection to a cluster.
487 * osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
489 * osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
491 Per mapping (block device) `rbd map` options:
493 * rw - Map the image read-write (default).
495 * ro - Map the image read-only. Equivalent to --read-only.
497 * queue_depth=x - queue depth (since 4.2, default is 128 requests).
499 * lock_on_read - Acquire exclusive lock on reads, in addition to writes and
500 discards (since 4.9).
502 * exclusive - Disable automatic exclusive lock transitions (since 4.12).
506 * force - Force the unmapping of a block device that is open (since 4.9). The
507 driver will wait for running requests to complete and then unmap; requests
508 sent to the driver after initiating the unmap will be failed.
514 To create a new rbd image that is 100 GB::
516 rbd create mypool/myimage --size 102400
518 To use a non-default object size (8 MB)::
520 rbd create mypool/myimage --size 102400 --object-size 8M
522 To delete an rbd image (be careful!)::
524 rbd rm mypool/myimage
526 To create a new snapshot::
528 rbd snap create mypool/myimage@mysnap
530 To create a copy-on-write clone of a protected snapshot::
532 rbd clone mypool/myimage@mysnap otherpool/cloneimage
534 To see which clones of a snapshot exist::
536 rbd children mypool/myimage@mysnap
538 To delete a snapshot::
540 rbd snap rm mypool/myimage@mysnap
542 To map an image via the kernel with cephx enabled::
544 rbd map mypool/myimage --id admin --keyfile secretfile
546 To map an image via the kernel with different cluster name other than default *ceph*::
548 rbd map mypool/myimage --cluster cluster-name
554 To create an image and a clone from it::
556 rbd import --image-format 2 image mypool/parent
557 rbd snap create mypool/parent@snap
558 rbd snap protect mypool/parent@snap
559 rbd clone mypool/parent@snap otherpool/child
561 To create an image with a smaller stripe_unit (to better distribute small writes in some workloads)::
563 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
565 To change an image from one image format to another, export it and then
566 import it as the desired image format::
568 rbd export mypool/myimage@snap /tmp/img
569 rbd import --image-format 2 /tmp/img mypool/myimage2
571 To lock an image for exclusive use::
573 rbd lock add mypool/myimage mylockid
577 rbd lock remove mypool/myimage mylockid client.2485
579 To list images from trash::
583 To defer delete an image (use *--delay* to set delay-time, default is 0)::
585 rbd trash mv mypool/myimage
587 To delete an image from trash (be careful!)::
589 rbd trash rm mypool/myimage-id
591 To force delete an image from trash (be careful!)::
593 rbd trash rm mypool/myimage-id --force
595 To restore an image from trash::
597 rbd trash restore mypool/myimage-id
599 To restore an image from trash and rename it::
601 rbd trash restore mypool/myimage-id --image mynewimage
607 **rbd** is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to
608 the Ceph documentation at http://ceph.com/docs for more information.
614 :doc:`ceph <ceph>`\(8),
615 :doc:`rados <rados>`\(8)