Skip to content

Instantly share code, notes, and snippets.

@fmount
Last active October 30, 2025 21:41
Show Gist options
  • Select an option

  • Save fmount/70e74066de6885c07db6423b42ea2816 to your computer and use it in GitHub Desktop.

Select an option

Save fmount/70e74066de6885c07db6423b42ea2816 to your computer and use it in GitHub Desktop.

Step 1: Create Pool and configure allow_ec_overwrites

$ ceph osd pool create images_data erasure
$ ceph osd pool create images_meta replicated
$ ceph osd pool set images_data allow_ec_overwrites true

Note: allow_ec_overwrites is required and must be set in Ceph.

Verification steps:

$ ceph osd pool get images_data erasure_code_profile
erasure_code_profile: default
$ ceph osd erasure-code-profile get default
k=2
m=2
plugin=isa
technique=reed_sol_van

Step 2: Create an image

import rados
import rbd

CONF_FILE = '/etc/ceph/ceph.conf'

METADATA_POOL_NAME = 'images_meta' # The pool opened by ioctx (Metadata)
EC_DATA_POOL_NAME = 'images_data'       # The pool where image data will go (Erasure-Coded)
IMAGE_NAME = 'my_ec_image'
IMAGE_SIZE = 4 * 1024**3  # 4 GiB (4 GiB)

# RBD_FEATURE_DATAPOOL is required to specify a separate data_pool
RBD_FEATURE_DATAPOOL = (1 << 8)

try:
    # Connect to RADOS cluster
    with rados.Rados(
        conffile=CONF_FILE,
    ) as cluster:

        # I/O context (Metadata pool)
        with cluster.open_ioctx(METADATA_POOL_NAME) as ioctx:

            rbd_inst = rbd.RBD()

            print(f"Attempting to create image '{IMAGE_NAME}' in pool '{EC_DATA_POOL_NAME}'...")

            # Create the RBD image
            rbd_inst.create(
                ioctx,
                IMAGE_NAME,
                IMAGE_SIZE,
                features=RBD_FEATURE_DATAPOOL,
                data_pool=EC_DATA_POOL_NAME # <-- Pass the data pool here
            )

            print(f"Successfully created RBD image '{IMAGE_NAME}'.")
            print(f"Metadata is in: {METADATA_POOL_NAME}")
            print(f"Data is set to be stored in: {EC_DATA_POOL_NAME}")

            # I/O part
            with rbd.Image(ioctx, IMAGE_NAME) as image:

                # Write and read data
                data_to_write = b'Test data for EC pool verification!' * 10
                image.write(data_to_write, 0)
                print(f"Wrote {len(data_to_write)} bytes for EC verification.")

                read_data = image.read(len(data_to_write), 0)
                print(f"Verified read: {read_data[:10]}...")

except rados.Error as e:
    print(f"Ceph RADOS Error: {e}")
except rbd.Error as e:
    print(f"Ceph RBD Error: {e}")
$ python test_rbd_ec.py

Attempting to create image 'my_ec_image' in metadata pool 'images_meta'...
Successfully created RBD image 'my_ec_image'.
Metadata is in: images_meta
Data is set to be stored in: images_data
Wrote 350 bytes for EC verification.
Verified read: b''...

Step 3: Verification

$ rbd ls images_meta
my_ec_image
$ rbd info images_meta/my_ec_image
rbd image 'my_ec_image':
        size 4 GiB in 1024 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 3aaac533865d8
        data_pool: images_data   <<<<<============ THIS IS THE CRITICAL LINE TO CHECK
        block_name_prefix: rbd_data.21.3aaac533865d8
        format: 2
        features: data-pool
        op_features:
        flags:
        create_timestamp: Thu Oct 30 17:29:43 2025
        access_timestamp: Thu Oct 30 17:29:43 2025
        modify_timestamp: Thu Oct 30 17:29:43 2025

if data_pool shows the name of your erasure-coded pool, the image data is correctly configured to use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment