1 .. index:: control, commands
11 Monitor commands are issued using the ceph utility::
13 ceph [-m monhost] {command}
15 The command is usually (though not always) of the form::
17 ceph {subsystem} {command}
23 Execute the following to display the current status of the cluster. ::
28 Execute the following to display a running summary of the status of the cluster,
33 Execute the following to show the monitor quorum, including which monitors are
34 participating and which one is the leader. ::
38 Execute the following to query the status of a single monitor, including whether
39 or not it is in the quorum. ::
41 ceph [-m monhost] mon_status
44 Authentication Subsystem
45 ========================
47 To add a keyring for an OSD, execute the following::
49 ceph auth add {osd} {--in-file|-i} {path-to-osd-keyring}
51 To list the cluster's keys and their capabilities, execute the following::
56 Placement Group Subsystem
57 =========================
59 To display the statistics for all placement groups, execute the following::
61 ceph pg dump [--format {format}]
63 The valid formats are ``plain`` (default) and ``json``.
65 To display the statistics for all placement groups stuck in a specified state,
66 execute the following::
68 ceph pg dump_stuck inactive|unclean|stale|undersized|degraded [--format {format}] [-t|--threshold {seconds}]
71 ``--format`` may be ``plain`` (default) or ``json``
73 ``--threshold`` defines how many seconds "stuck" is (default: 300)
75 **Inactive** Placement groups cannot process reads or writes because they are waiting for an OSD
76 with the most up-to-date data to come back.
78 **Unclean** Placement groups contain objects that are not replicated the desired number
79 of times. They should be recovering.
81 **Stale** Placement groups are in an unknown state - the OSDs that host them have not
82 reported to the monitor cluster in a while (configured by
83 ``mon_osd_report_timeout``).
85 Delete "lost" objects or revert them to their prior state, either a previous version
86 or delete them if they were just created. ::
88 ceph pg {pgid} mark_unfound_lost revert|delete
94 Query OSD subsystem status. ::
98 Write a copy of the most recent OSD map to a file. See
101 ceph osd getmap -o file
103 .. _osdmaptool: ../../man/8/osdmaptool
105 Write a copy of the crush map from the most recent OSD map to
108 ceph osd getcrushmap -o file
110 The foregoing functionally equivalent to ::
112 ceph osd getmap -o /tmp/osdmap
113 osdmaptool /tmp/osdmap --export-crush file
115 Dump the OSD map. Valid formats for ``-f`` are ``plain`` and ``json``. If no
116 ``--format`` option is given, the OSD map is dumped as plain text. ::
118 ceph osd dump [--format {format}]
120 Dump the OSD map as a tree with one line per OSD containing weight
123 ceph osd tree [--format {format}]
125 Find out where a specific object is or would be stored in the system::
127 ceph osd map <pool-name> <object-name>
129 Add or move a new item (OSD) with the given id/name/weight at the specified
132 ceph osd crush set {id} {weight} [{loc1} [{loc2} ...]]
134 Remove an existing item (OSD) from the CRUSH map. ::
136 ceph osd crush remove {name}
138 Remove an existing bucket from the CRUSH map. ::
140 ceph osd crush remove {bucket-name}
142 Move an existing bucket from one position in the hierarchy to another. ::
144 ceph osd crush move {id} {loc1} [{loc2} ...]
146 Set the weight of the item given by ``{name}`` to ``{weight}``. ::
148 ceph osd crush reweight {name} {weight}
150 Mark an OSD as lost. This may result in permanent data loss. Use with caution. ::
152 ceph osd lost {id} [--yes-i-really-mean-it]
154 Create a new OSD. If no UUID is given, it will be set automatically when the OSD
157 ceph osd create [{uuid}]
159 Remove the given OSD(s). ::
161 ceph osd rm [{id}...]
163 Query the current max_osd parameter in the OSD map. ::
167 Import the given crush map. ::
169 ceph osd setcrushmap -i file
171 Set the ``max_osd`` parameter in the OSD map. This is necessary when
172 expanding the storage cluster. ::
176 Mark OSD ``{osd-num}`` down. ::
178 ceph osd down {osd-num}
180 Mark OSD ``{osd-num}`` out of the distribution (i.e. allocated no data). ::
182 ceph osd out {osd-num}
184 Mark ``{osd-num}`` in the distribution (i.e. allocated data). ::
186 ceph osd in {osd-num}
188 Set or clear the pause flags in the OSD map. If set, no IO requests
189 will be sent to any OSD. Clearing the flags via unpause results in
190 resending pending requests. ::
195 Set the weight of ``{osd-num}`` to ``{weight}``. Two OSDs with the
196 same weight will receive roughly the same number of I/O requests and
197 store approximately the same amount of data. ``ceph osd reweight``
198 sets an override weight on the OSD. This value is in the range 0 to 1,
199 and forces CRUSH to re-place (1-weight) of the data that would
200 otherwise live on this drive. It does not change the weights assigned
201 to the buckets above the OSD in the crush map, and is a corrective
202 measure in case the normal CRUSH distribution is not working out quite
203 right. For instance, if one of your OSDs is at 90% and the others are
204 at 50%, you could reduce this weight to try and compensate for it. ::
206 ceph osd reweight {osd-num} {weight}
208 Reweights all the OSDs by reducing the weight of OSDs which are
209 heavily overused. By default it will adjust the weights downward on
210 OSDs which have 120% of the average utilization, but if you include
211 threshold it will use that percentage instead. ::
213 ceph osd reweight-by-utilization [threshold]
215 Describes what reweight-by-utilization would do. ::
217 ceph osd test-reweight-by-utilization
219 Adds/removes the address to/from the blacklist. When adding an address,
220 you can specify how long it should be blacklisted in seconds; otherwise,
221 it will default to 1 hour. A blacklisted address is prevented from
222 connecting to any OSD. Blacklisting is most often used to prevent a
223 lagging metadata server from making bad changes to data on the OSDs.
225 These commands are mostly only useful for failure testing, as
226 blacklists are normally maintained automatically and shouldn't need
227 manual intervention. ::
229 ceph osd blacklist add ADDRESS[:source_port] [TIME]
230 ceph osd blacklist rm ADDRESS[:source_port]
232 Creates/deletes a snapshot of a pool. ::
234 ceph osd pool mksnap {pool-name} {snap-name}
235 ceph osd pool rmsnap {pool-name} {snap-name}
237 Creates/deletes/renames a storage pool. ::
239 ceph osd pool create {pool-name} pg_num [pgp_num]
240 ceph osd pool delete {pool-name} [{pool-name} --yes-i-really-really-mean-it]
241 ceph osd pool rename {old-name} {new-name}
243 Changes a pool setting. ::
245 ceph osd pool set {pool-name} {field} {value}
249 * ``size``: Sets the number of copies of data in the pool.
250 * ``pg_num``: The placement group number.
251 * ``pgp_num``: Effective number when calculating pg placement.
252 * ``crush_ruleset``: rule number for mapping placement.
254 Get the value of a pool setting. ::
256 ceph osd pool get {pool-name} {field}
260 * ``pg_num``: The placement group number.
261 * ``pgp_num``: Effective number of placement groups when calculating placement.
262 * ``lpg_num``: The number of local placement groups.
263 * ``lpgp_num``: The number used for placing the local placement groups.
266 Sends a scrub command to OSD ``{osd-num}``. To send the command to all OSDs, use ``*``. ::
268 ceph osd scrub {osd-num}
270 Sends a repair command to OSD.N. To send the command to all OSDs, use ``*``. ::
274 Runs a simple throughput benchmark against OSD.N, writing ``TOTAL_DATA_BYTES``
275 in write requests of ``BYTES_PER_WRITE`` each. By default, the test
276 writes 1 GB in total in 4-MB increments.
277 The benchmark is non-destructive and will not overwrite existing live
278 OSD data, but might temporarily affect the performance of clients
279 concurrently accessing the OSD. ::
281 ceph tell osd.N bench [TOTAL_DATA_BYTES] [BYTES_PER_WRITE]
287 Change configuration parameters on a running mds. ::
289 ceph tell mds.{mds-id} injectargs --{switch} {value} [--{switch} {value}]
293 ceph tell mds.0 injectargs --debug_ms 1 --debug_mds 10
295 Enables debug messages. ::
299 Displays the status of all metadata servers. ::
303 Marks the active MDS as failed, triggering failover to a standby if present.
305 .. todo:: ``ceph mds`` subcommands missing docs: set, dump, getmap, stop, setmap
315 e2: 3 mons at {a=127.0.0.1:40000/0,b=127.0.0.1:40001/0,c=127.0.0.1:40002/0}, election epoch 6, quorum 0,1,2 a,b,c
318 The ``quorum`` list at the end lists monitor nodes that are part of the current quorum.
320 This is also available more directly::
322 ceph quorum_status -f json-pretty
324 .. code-block:: javascript
338 "quorum_leader_name": "a",
341 "fsid": "ba807e74-b64f-4b72-b43f-597dfe60ddbc",
342 "modified": "2016-12-26 14:42:09.288066",
343 "created": "2016-12-26 14:42:03.573585",
354 "addr": "127.0.0.1:40000\/0",
355 "public_addr": "127.0.0.1:40000\/0"
360 "addr": "127.0.0.1:40001\/0",
361 "public_addr": "127.0.0.1:40001\/0"
366 "addr": "127.0.0.1:40002\/0",
367 "public_addr": "127.0.0.1:40002\/0"
374 The above will block until a quorum is reached.
376 For a status of just the monitor you connect to (use ``-m HOST:PORT``
379 ceph mon_status -f json-pretty
382 .. code-block:: javascript
395 "required_con": "9025616074522624",
399 "quorum_con": "1152921504336314367",
404 "outside_quorum": [],
405 "extra_probe_peers": [],
409 "fsid": "ba807e74-b64f-4b72-b43f-597dfe60ddbc",
410 "modified": "2016-12-26 14:42:09.288066",
411 "created": "2016-12-26 14:42:03.573585",
422 "addr": "127.0.0.1:40000\/0",
423 "public_addr": "127.0.0.1:40000\/0"
428 "addr": "127.0.0.1:40001\/0",
429 "public_addr": "127.0.0.1:40001\/0"
434 "addr": "127.0.0.1:40002\/0",
435 "public_addr": "127.0.0.1:40002\/0"
441 A dump of the monitor state::
445 dumped monmap epoch 2
447 fsid ba807e74-b64f-4b72-b43f-597dfe60ddbc
448 last_changed 2016-12-26 14:42:09.288066
449 created 2016-12-26 14:42:03.573585
450 0: 127.0.0.1:40000/0 mon.a
451 1: 127.0.0.1:40001/0 mon.b
452 2: 127.0.0.1:40002/0 mon.c