5 The ``rados`` module is a thin Python wrapper for ``librados``.
10 To install Python libraries for Ceph, see `Getting librados for Python`_.
16 You can create your own Ceph client using Python. The following tutorial will
17 show you how to import the Ceph Python module, connect to a Ceph cluster, and
18 perform object operations as a ``client.admin`` user.
20 .. note:: To use the Ceph Python bindings, you must have access to a
21 running Ceph cluster. To set one up quickly, see `Getting Started`_.
23 First, create a Python source file for your Ceph client. ::
32 To use the ``rados`` module, import it into your source file.
34 .. code-block:: python
40 Configure a Cluster Handle
41 --------------------------
43 Before connecting to the Ceph Storage Cluster, create a cluster handle. By
44 default, the cluster handle assumes a cluster named ``ceph`` (i.e., the default
45 for deployment tools, and our Getting Started guides too), and a
46 ``client.admin`` user name. You may change these defaults to suit your needs.
48 To connect to the Ceph Storage Cluster, your application needs to know where to
49 find the Ceph Monitor. Provide this information to your application by
50 specifying the path to your Ceph configuration file, which contains the location
51 of the initial Ceph monitors.
53 .. code-block:: python
58 #Create Handle Examples.
59 cluster = rados.Rados(conffile='ceph.conf')
60 cluster = rados.Rados(conffile=sys.argv[1])
61 cluster = rados.Rados(conffile = 'ceph.conf', conf = dict (keyring = '/path/to/keyring'))
63 Ensure that the ``conffile`` argument provides the path and file name of your
64 Ceph configuration file. You may use the ``sys`` module to avoid hard-coding the
65 Ceph configuration path and file name.
67 Your Python client also requires a client keyring. For this example, we use the
68 ``client.admin`` key by default. If you would like to specify the keyring when
69 creating the cluster handle, you may use the ``conf`` argument. Alternatively,
70 you may specify the keyring path in your Ceph configuration file. For example,
71 you may add something like the following line to you Ceph configuration file::
73 keyring = /path/to/ceph.client.admin.keyring
75 For additional details on modifying your configuration via Python, see `Configuration`_.
78 Connect to the Cluster
79 ----------------------
81 Once you have a cluster handle configured, you may connect to the cluster.
82 With a connection to the cluster, you may execute methods that return
83 information about the cluster.
85 .. code-block:: python
91 cluster = rados.Rados(conffile='ceph.conf')
92 print "\nlibrados version: " + str(cluster.version())
93 print "Will attempt to connect to: " + str(cluster.conf_get('mon initial members'))
96 print "\nCluster ID: " + cluster.get_fsid()
98 print "\n\nCluster Statistics"
99 print "=================="
100 cluster_stats = cluster.get_cluster_stats()
102 for key, value in cluster_stats.iteritems():
106 By default, Ceph authentication is ``on``. Your application will need to know
107 the location of the keyring. The ``python-ceph`` module doesn't have the default
108 location, so you need to specify the keyring path. The easiest way to specify
109 the keyring is to add it to the Ceph configuration file. The following Ceph
110 configuration file example uses the ``client.admin`` keyring you generated with
118 keyring=/path/to/keyring/ceph.client.admin.keyring
124 When connected to the cluster, the ``Rados`` API allows you to manage pools. You
125 can list pools, check for the existence of a pool, create a pool and delete a
128 .. code-block:: python
130 :emphasize-lines: 6, 13, 18, 25
132 print "\n\nPool Operations"
133 print "==============="
135 print "\nAvailable Pools"
136 print "----------------"
137 pools = cluster.list_pools()
142 print "\nCreate 'test' Pool"
143 print "------------------"
144 cluster.create_pool('test')
146 print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))
147 print "\nVerify 'test' Pool Exists"
148 print "-------------------------"
149 pools = cluster.list_pools()
154 print "\nDelete 'test' Pool"
155 print "------------------"
156 cluster.delete_pool('test')
157 print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))
164 Reading from and writing to the Ceph Storage Cluster requires an input/output
165 context (ioctx). You can create an ioctx with the ``open_ioctx()`` method of the
166 ``Rados`` class. The ``ioctx_name`` parameter is the name of the pool you wish
169 .. code-block:: python
172 ioctx = cluster.open_ioctx('data')
175 Once you have an I/O context, you can read/write objects, extended attributes,
176 and perform a number of other operations. After you complete operations, ensure
177 that you close the connection. For example:
179 .. code-block:: python
182 print "\nClosing the connection."
186 Writing, Reading and Removing Objects
187 -------------------------------------
189 Once you create an I/O context, you can write objects to the cluster. If you
190 write to an object that doesn't exist, Ceph creates it. If you write to an
191 object that exists, Ceph overwrites it (except when you specify a range, and
192 then it only overwrites the range). You may read objects (and object ranges)
193 from the cluster. You may also remove objects from the cluster. For example:
195 .. code-block:: python
197 :emphasize-lines: 2, 5, 8
199 print "\nWriting object 'hw' with contents 'Hello World!' to pool 'data'."
200 ioctx.write_full("hw", "Hello World!")
202 print "\n\nContents of object 'hw'\n------------------------\n"
203 print ioctx.read("hw")
205 print "\nRemoving object 'hw'"
206 ioctx.remove_object("hw")
209 Writing and Reading XATTRS
210 --------------------------
212 Once you create an object, you can write extended attributes (XATTRs) to
213 the object and read XATTRs from the object. For example:
215 .. code-block:: python
217 :emphasize-lines: 2, 5
219 print "\n\nWriting XATTR 'lang' with value 'en_US' to object 'hw'"
220 ioctx.set_xattr("hw", "lang", "en_US")
222 print "\n\nGetting XATTR 'lang' from object 'hw'\n"
223 print ioctx.get_xattr("hw", "lang")
229 If you want to examine the list of objects in a pool, you may
230 retrieve the list of objects and iterate over them with the object iterator.
233 .. code-block:: python
235 :emphasize-lines: 1, 6, 7
237 object_iterator = ioctx.list_objects()
242 rados_object = object_iterator.next()
243 print "Object contents = " + rados_object.read()
245 except StopIteration :
248 The ``Object`` class provides a file-like interface to an object, allowing
249 you to read and write content and extended attributes. Object operations using
250 the I/O context provide additional functionality and asynchronous capabilities.
256 The ``Rados`` class provides an interface into the Ceph Storage Daemon.
262 The ``Rados`` class provides methods for getting and setting configuration
263 values, reading the Ceph configuration file, and parsing arguments. You
264 do not need to be connected to the Ceph Storage Cluster to invoke the following
265 methods. See `Storage Cluster Configuration`_ for details on settings.
267 .. currentmodule:: rados
268 .. automethod:: Rados.conf_get(option)
269 .. automethod:: Rados.conf_set(option, val)
270 .. automethod:: Rados.conf_read_file(path=None)
271 .. automethod:: Rados.conf_parse_argv(args)
272 .. automethod:: Rados.version()
275 Connection Management
276 ---------------------
278 Once you configure your cluster handle, you may connect to the cluster, check
279 the cluster ``fsid``, retrieve cluster statistics, and disconnect (shutdown)
280 from the cluster. You may also assert that the cluster handle is in a particular
281 state (e.g., "configuring", "connecting", etc.).
284 .. automethod:: Rados.connect(timeout=0)
285 .. automethod:: Rados.shutdown()
286 .. automethod:: Rados.get_fsid()
287 .. automethod:: Rados.get_cluster_stats()
288 .. automethod:: Rados.require_state(*args)
294 To use pool operation methods, you must connect to the Ceph Storage Cluster
295 first. You may list the available pools, create a pool, check to see if a pool
296 exists, and delete a pool.
298 .. automethod:: Rados.list_pools()
299 .. automethod:: Rados.create_pool(pool_name, auid=None, crush_rule=None)
300 .. automethod:: Rados.pool_exists()
301 .. automethod:: Rados.delete_pool(pool_name)
305 Input/Output Context API
306 ========================
308 To write data to and read data from the Ceph Object Store, you must create
309 an Input/Output context (ioctx). The `Rados` class provides a `open_ioctx()`
310 method. The remaining ``ioctx`` operations involve invoking methods of the
311 `Ioctx` and other classes.
313 .. automethod:: Rados.open_ioctx(ioctx_name)
314 .. automethod:: Ioctx.require_ioctx_open()
315 .. automethod:: Ioctx.get_stats()
316 .. automethod:: Ioctx.change_auid(auid)
317 .. automethod:: Ioctx.get_last_version()
318 .. automethod:: Ioctx.close()
324 .. The Ceph Storage Cluster allows you to make a snapshot of a pool's state.
325 .. Whereas, basic pool operations only require a connection to the cluster,
326 .. snapshots require an I/O context.
328 .. Ioctx.create_snap(self, snap_name)
329 .. Ioctx.list_snaps(self)
330 .. SnapIterator.next(self)
331 .. Snap.get_timestamp(self)
332 .. Ioctx.lookup_snap(self, snap_name)
333 .. Ioctx.remove_snap(self, snap_name)
335 .. not published. This doesn't seem ready yet.
340 The Ceph Storage Cluster stores data as objects. You can read and write objects
341 synchronously or asynchronously. You can read and write from offsets. An object
342 has a name (or key) and data.
345 .. automethod:: Ioctx.aio_write(object_name, to_write, offset=0, oncomplete=None, onsafe=None)
346 .. automethod:: Ioctx.aio_write_full(object_name, to_write, oncomplete=None, onsafe=None)
347 .. automethod:: Ioctx.aio_append(object_name, to_append, oncomplete=None, onsafe=None)
348 .. automethod:: Ioctx.write(key, data, offset=0)
349 .. automethod:: Ioctx.write_full(key, data)
350 .. automethod:: Ioctx.aio_flush()
351 .. automethod:: Ioctx.set_locator_key(loc_key)
352 .. automethod:: Ioctx.aio_read(object_name, length, offset, oncomplete)
353 .. automethod:: Ioctx.read(key, length=8192, offset=0)
354 .. automethod:: Ioctx.stat(key)
355 .. automethod:: Ioctx.trunc(key, size)
356 .. automethod:: Ioctx.remove_object(key)
359 Object Extended Attributes
360 --------------------------
362 You may set extended attributes (XATTRs) on an object. You can retrieve a list
363 of objects or XATTRs and iterate over them.
365 .. automethod:: Ioctx.set_xattr(key, xattr_name, xattr_value)
366 .. automethod:: Ioctx.get_xattrs(oid)
367 .. automethod:: XattrIterator.next()
368 .. automethod:: Ioctx.get_xattr(key, xattr_name)
369 .. automethod:: Ioctx.rm_xattr(key, xattr_name)
376 From an I/O context, you can retrieve a list of objects from a pool and iterate
377 over them. The object interface provide makes each object look like a file, and
378 you may perform synchronous operations on the objects. For asynchronous
379 operations, you should use the I/O context methods.
381 .. automethod:: Ioctx.list_objects()
382 .. automethod:: ObjectIterator.next()
383 .. automethod:: Object.read(length = 1024*1024)
384 .. automethod:: Object.write(string_to_write)
385 .. automethod:: Object.get_xattrs()
386 .. automethod:: Object.get_xattr(xattr_name)
387 .. automethod:: Object.set_xattr(xattr_name, xattr_value)
388 .. automethod:: Object.rm_xattr(xattr_name)
389 .. automethod:: Object.stat()
390 .. automethod:: Object.remove()
395 .. _Getting Started: ../../../start
396 .. _Storage Cluster Configuration: ../../configuration
397 .. _Getting librados for Python: ../librados-intro#getting-librados-for-python