Ceph RADOS Block Device (RBD)

Bertha 。 2024-04-19 09:03 49阅读 0赞

### Ceph RADOS Block Device (RBD) ###

UPDATED: 2019-09-12 04:28

If you use KVM or QEMU as your hypervisor, you can configure the Compute service to use [Ceph RADOS block devices (RBD)][Ceph RADOS block devices _RBD] for volumes.

Ceph is a massively scalable, open source, distributed storage system. It is comprised of an object store, block store, and a POSIX-compliant distributed file system. The platform can auto-scale to the exabyte level and beyond. It runs on commodity hardware, is self-healing and self-managing, and has no single point of failure. Due to its open-source nature, you can install and use this portable storage platform in public or private clouds.

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FsbHdheTI_size_16_color_FFFFFF_t_70][]

Ceph architecture

### RADOS[¶][Link 1] ###

Ceph is based on Reliable Autonomic Distributed Object Store (RADOS). RADOS distributes objects across the storage cluster and replicates objects for fault tolerance. RADOS contains the following major components:

*Object Storage Device (OSD) Daemon*

The storage daemon for the RADOS service, which interacts with the OSD (physical or logical storage unit for your data). You must run this daemon on each server in your cluster. For each OSD, you can have an associated hard drive disk. For performance purposes, pool your hard drive disk with raid arrays, or logical volume management (LVM). By default, the following pools are created: data, metadata, and RBD.

*Meta-Data Server (MDS)*

Stores metadata. MDSs build a POSIX file system on top of objects for Ceph clients. However, if you do not use the Ceph file system, you do not need a metadata server.

*Monitor (MON)*

A lightweight daemon that handles all communications with external applications and clients. It also provides a consensus for distributed decision making in a Ceph/RADOS cluster. For instance, when you mount a Ceph shared on a client, you point to the address of a MON server. It checks the state and the consistency of the data. In an ideal setup, you must run at least three `ceph-mon` daemons on separate servers.

### Ways to store, use, and expose data[¶][Link 2] ###

To store and access your data, you can use the following storage systems:

*RADOS*

Use as an object, default storage mechanism.

*RBD*

Use as a block device. The Linux kernel RBD (RADOS block device) driver allows striping a Linux block device over multiple distributed object store data objects. It is compatible with the KVM RBD image.

*CephFS*

Use as a file, POSIX-compliant file system.

Ceph exposes RADOS; you can access it through the following interfaces:

*RADOS Gateway*

OpenStack Object Storage and Amazon-S3 compatible RESTful interface (see [RADOS\_Gateway][RADOS_Gateway]).

*librados*

and its related C/C++ bindings

*RBD and QEMU-RBD*

Linux kernel and QEMU block devices that stripe data across multiple objects.

### Driver options[¶][Link 3] ###

The following table contains the configuration options supported by the Ceph RADOS Block Device driver.

<table>  Description of Ceph storage configuration options 
 <a href="https://docs.openstack.org/cinder/queens/configuration/block-storage/drivers/ceph-rbd-volume-driver.html#id2" rel="nofollow">¶</a>  
 <thead> 
  <tr> 
   <th>Configuration option = Default value</th> 
   <th>Description</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td><strong>[DEFAULT]</strong></td> 
   <td>&nbsp;</td> 
  </tr> 
  <tr> 
   <td><code>rados_connect_timeout</code>&nbsp;=&nbsp;<code>-1</code></td> 
   <td>(Integer) Timeout value (in seconds) used when connecting to ceph cluster. If value &lt; 0, no timeout is set and default librados value is used.</td> 
  </tr> 
  <tr> 
   <td><code>rados_connection_interval</code>&nbsp;=&nbsp;<code>5</code></td> 
   <td>(Integer) Interval value (in seconds) between connection retries to ceph cluster.</td> 
  </tr> 
  <tr> 
   <td><code>rados_connection_retries</code>&nbsp;=&nbsp;<code>3</code></td> 
   <td>(Integer) Number of retries if connection to ceph cluster failed.</td> 
  </tr> 
  <tr> 
   <td><code>rbd_ceph_conf</code>&nbsp;=</td> 
   <td>(String) Path to the ceph configuration file</td> 
  </tr> 
  <tr> 
   <td><code>rbd_cluster_name</code>&nbsp;=&nbsp;<code>ceph</code></td> 
   <td>(String) The name of ceph cluster</td> 
  </tr> 
  <tr> 
   <td><code>rbd_flatten_volume_from_snapshot</code>&nbsp;=&nbsp;<code>False</code></td> 
   <td>(Boolean) Flatten volumes created from snapshots to remove dependency from volume to snapshot</td> 
  </tr> 
  <tr> 
   <td><code>rbd_max_clone_depth</code>&nbsp;=&nbsp;<code>5</code></td> 
   <td>(Integer) Maximum number of nested volume clones that are taken before a flatten occurs. Set to 0 to disable cloning.</td> 
  </tr> 
  <tr> 
   <td><code>rbd_pool</code>&nbsp;=&nbsp;<code>rbd</code></td> 
   <td>(String) The RADOS pool where rbd volumes are stored</td> 
  </tr> 
  <tr> 
   <td><code>rbd_secret_uuid</code>&nbsp;=&nbsp;<code>None</code></td> 
   <td>(String) The libvirt uuid of the secret for the rbd_user volumes</td> 
  </tr> 
  <tr> 
   <td><code>rbd_store_chunk_size</code>&nbsp;=&nbsp;<code>4</code></td> 
   <td>(Integer) Volumes will be chunked into objects of this size (in megabytes).</td> 
  </tr> 
  <tr> 
   <td><code>rbd_user</code>&nbsp;=&nbsp;<code>None</code></td> 
   <td>(String) The RADOS client name for accessing rbd volumes - only set when using cephx authentication</td> 
  </tr> 
  <tr> 
   <td><code>replication_connect_timeout</code>&nbsp;=&nbsp;<code>5</code></td> 
   <td>(Integer) Timeout value (in seconds) used when connecting to ceph cluster to do a demotion/promotion of volumes. If value &lt; 0, no timeout is set and default librados value is used.</td> 
  </tr> 
 </tbody> 
</table>

[Ceph RADOS block devices _RBD]: https://ceph.com/ceph-storage/block-storage/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FsbHdheTI_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/5183ca44a82a4fc9b455e14237eea477.png
[Link 1]: https://docs.openstack.org/cinder/queens/configuration/block-storage/drivers/ceph-rbd-volume-driver.html#rados
[Link 2]: https://docs.openstack.org/cinder/queens/configuration/block-storage/drivers/ceph-rbd-volume-driver.html#ways-to-store-use-and-expose-data
[RADOS_Gateway]: https://ceph.com/wiki/RADOS_Gateway
[Link 3]: https://docs.openstack.org/cinder/queens/configuration/block-storage/drivers/ceph-rbd-volume-driver.html#driver-options