1 ========================
3 ========================
5 .. index:: Ceph Block Device; QEMU KVM
7 The most frequent Ceph Block Device use case involves providing block device
8 images to virtual machines. For example, a user may create a "golden" image
9 with an OS and any relevant software in an ideal configuration. Then, the user
10 takes a snapshot of the image. Finally, the user clones the snapshot (usually
11 many times). See `Snapshots`_ for details. The ability to make copy-on-write
12 clones of a snapshot means that Ceph can provision block device images to
13 virtual machines quickly, because the client doesn't have to download an entire
14 image each time it spins up a new virtual machine.
17 .. ditaa:: +---------------------------------------------------+
19 +---------------------------------------------------+
21 +---------------------------------------------------+
23 +------------------------+-+------------------------+
25 +------------------------+ +------------------------+
28 Ceph Block Devices can integrate with the QEMU virtual machine. For details on
29 QEMU, see `QEMU Open Source Processor Emulator`_. For QEMU documentation, see
30 `QEMU Manual`_. For installation details, see `Installation`_.
32 .. important:: To use Ceph Block Devices with QEMU, you must have access to a
39 The QEMU command line expects you to specify the pool name and image name. You
40 may also specify a snapshot name.
42 QEMU will assume that the Ceph configuration file resides in the default
43 location (e.g., ``/etc/ceph/$cluster.conf``) and that you are executing
44 commands as the default ``client.admin`` user unless you expressly specify
45 another Ceph configuration file path or another user. When specifying a user,
46 QEMU uses the ``ID`` rather than the full ``TYPE:ID``. See `User Management -
47 User`_ for details. Do not prepend the client type (i.e., ``client.``) to the
48 beginning of the user ``ID``, or you will receive an authentication error. You
49 should have the key for the ``admin`` user or the key of another user you
50 specify with the ``:id={user}`` option in a keyring file stored in default path
51 (i.e., ``/etc/ceph`` or the local directory with appropriate file ownership and
52 permissions. Usage takes the following form::
54 qemu-img {command} [options] rbd:{pool-name}/{image-name}[@snapshot-name][:option1=value1][:option2=value2...]
56 For example, specifying the ``id`` and ``conf`` options might look like the following::
58 qemu-img {command} [options] rbd:glance-pool/maipo:id=glance:conf=/etc/ceph/ceph.conf
60 .. tip:: Configuration values containing ``:``, ``@``, or ``=`` can be escaped with a
61 leading ``\`` character.
64 Creating Images with QEMU
65 =========================
67 You can create a block device image from QEMU. You must specify ``rbd``, the
68 pool name, and the name of the image you wish to create. You must also specify
69 the size of the image. ::
71 qemu-img create -f raw rbd:{pool-name}/{image-name} {size}
75 qemu-img create -f raw rbd:data/foo 10G
77 .. important:: The ``raw`` data format is really the only sensible
78 ``format`` option to use with RBD. Technically, you could use other
79 QEMU-supported formats (such as ``qcow2`` or ``vmdk``), but doing
80 so would add additional overhead, and would also render the volume
81 unsafe for virtual machine live migration when caching (see below)
85 Resizing Images with QEMU
86 =========================
88 You can resize a block device image from QEMU. You must specify ``rbd``,
89 the pool name, and the name of the image you wish to resize. You must also
90 specify the size of the image. ::
92 qemu-img resize rbd:{pool-name}/{image-name} {size}
96 qemu-img resize rbd:data/foo 10G
99 Retrieving Image Info with QEMU
100 ===============================
102 You can retrieve block device image information from QEMU. You must
103 specify ``rbd``, the pool name, and the name of the image. ::
105 qemu-img info rbd:{pool-name}/{image-name}
109 qemu-img info rbd:data/foo
112 Running QEMU with RBD
113 =====================
115 QEMU can pass a block device from the host on to a guest, but since
116 QEMU 0.15, there's no need to map an image as a block device on
117 the host. Instead, QEMU can access an image as a virtual block
118 device directly via ``librbd``. This performs better because it avoids
119 an additional context switch, and can take advantage of `RBD caching`_.
121 You can use ``qemu-img`` to convert existing virtual machine images to Ceph
122 block device images. For example, if you have a qcow2 image, you could run::
124 qemu-img convert -f qcow2 -O raw debian_squeeze.qcow2 rbd:data/squeeze
126 To run a virtual machine booting from that image, you could run::
128 qemu -m 1024 -drive format=raw,file=rbd:data/squeeze
130 `RBD caching`_ can significantly improve performance.
131 Since QEMU 1.2, QEMU's cache options control ``librbd`` caching::
133 qemu -m 1024 -drive format=rbd,file=rbd:data/squeeze,cache=writeback
135 If you have an older version of QEMU, you can set the ``librbd`` cache
136 configuration (like any Ceph configuration option) as part of the
139 qemu -m 1024 -drive format=raw,file=rbd:data/squeeze:rbd_cache=true,cache=writeback
141 .. important:: If you set rbd_cache=true, you must set cache=writeback
142 or risk data loss. Without cache=writeback, QEMU will not send
143 flush requests to librbd. If QEMU exits uncleanly in this
144 configuration, filesystems on top of rbd can be corrupted.
146 .. _RBD caching: ../rbd-config-ref/#rbd-cache-config-settings
149 .. index:: Ceph Block Device; discard trim and libvirt
151 Enabling Discard/TRIM
152 =====================
154 Since Ceph version 0.46 and QEMU version 1.1, Ceph Block Devices support the
155 discard operation. This means that a guest can send TRIM requests to let a Ceph
156 block device reclaim unused space. This can be enabled in the guest by mounting
157 ``ext4`` or ``XFS`` with the ``discard`` option.
159 For this to be available to the guest, it must be explicitly enabled
160 for the block device. To do this, you must specify a
161 ``discard_granularity`` associated with the drive::
163 qemu -m 1024 -drive format=raw,file=rbd:data/squeeze,id=drive1,if=none \
164 -device driver=ide-hd,drive=drive1,discard_granularity=512
166 Note that this uses the IDE driver. The virtio driver does not
169 If using libvirt, edit your libvirt domain's configuration file using ``virsh
170 edit`` to include the ``xmlns:qemu`` value. Then, add a ``qemu:commandline``
171 block as a child of that domain. The following example shows how to set two
172 devices with ``qemu id=`` to different ``discard_granularity`` values.
174 .. code-block:: guess
176 <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
178 <qemu:arg value='-set'/>
179 <qemu:arg value='block.scsi0-0-0.discard_granularity=4096'/>
180 <qemu:arg value='-set'/>
181 <qemu:arg value='block.scsi0-0-1.discard_granularity=65536'/>
186 .. index:: Ceph Block Device; cache options
191 QEMU's cache options correspond to the following Ceph `RBD Cache`_ settings.
200 rbd_cache_max_dirty = 0
206 QEMU's cache settings override Ceph's cache settings (including settings that
207 are explicitly set in the Ceph configuration file).
209 .. note:: Prior to QEMU v2.4.0, if you explicitly set `RBD Cache`_ settings
210 in the Ceph configuration file, your Ceph settings override the QEMU cache
213 .. _QEMU Open Source Processor Emulator: http://wiki.qemu.org/Main_Page
214 .. _QEMU Manual: http://wiki.qemu.org/Manual
215 .. _RBD Cache: ../rbd-config-ref/
216 .. _Snapshots: ../rbd-snapshot/
217 .. _Installation: ../../install
218 .. _User Management - User: ../../rados/operations/user-management#user