Skip to content

Instantly share code, notes, and snippets.

@markus-hentsch

markus-hentsch/openstack_devstack_standardized_image_encryption_testing.md

Last active May 23, 2025 15:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save markus-hentsch/051e34252f4f9d0a4cbb7472665d872b to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save markus-hentsch/051e34252f4f9d0a4cbb7472665d872b to your computer and use it in GitHub Desktop.
Download ZIP
OpenStack Standardized Image Encryption - Manual DevStack Testing
Raw
openstack_devstack_standardized_image_encryption_testing.md

OpenStack Standardized Image Encryption - Manual DevStack Testing

The below instructions are manual steps to validate the implementation of the Standardized Image Encryption on a DevStack as per upstream patchsets uploaded at https://review.opendev.org/q/topic:%22LUKS-image-encryption%22

For further information see the corresponding Glance Spec and Cinder Spec.

Note: the instructions were created during the implementation phase of the patchsets and details about the process of using image encryption might change in the future and may not reflect the behavior of stable OpenStack releases.

Preparation

source /opt/stack/devstack/openrc admin admin

# security group to allow SSH access
openstack security group create --description "SSH" admin-access-group
openstack security group rule create --proto tcp --dst-port 22 admin-access-group

# prepare networks and router
PUBLIC_NETWORK=... # name of public network with Floating IPs
openstack network create admin-private
openstack subnet create --network admin-private --subnet-range 10.1.2.0/24 admin-private-subnet
openstack router create admin-public-router
openstack router set --external-gateway $PUBLIC_NETWORK admin-public-router
openstack router add subnet admin-public-router admin-private-subnet

# encrypted Volume Type
# (this assumes that you are using LVM-based Cinder backend in DevStack)
BASIC_TYPE=$(openstack volume type list -f value -c Name | grep lvm)
openstack volume type create \
    --property volume_backend_name="$BASIC_TYPE" \
    --encryption-provider luks \
    --encryption-cipher aes-xts-plain64 \
    --encryption-key-size 256 \
    --encryption-control-location front-end \
    "LVM-LUKS"

Image encryption

Example image file for testing:

wget https://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img
qemu-img convert -O raw cirros-0.6.2-x86_64-disk.img cirros.raw

LUKS with plain passphrase

  • image format will be uncompressed LUKS, which can later be directly transferred to a volume by Cinder
  • passphrase will be uploaded as "passphrase" secret type in Barbican; this will make Cinder skip binascii.hexlify() and use it as LUKS passphrase directly
echo -n "muchsecretsuchwow" > secret_file.key

# upload passphrase to Barbican in plain text
SECRET_TYPE="passphrase"
openstack secret store --file secret_file.key \
--name luks-image-passphrase --secret-type $SECRET_TYPE

SECRET_ID=$(openstack secret list -f value --name luks-image-passphrase \
    | head -n1 | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)

echo $SECRET_ID

# encrypt the image with LUKS using qemu-img
qemu-img convert -f raw -O luks \
--object secret,id=sec,file=secret_file.key \
-o key-secret=sec \
-o cipher-alg=aes-256 \
-o cipher-mode=xts \
-o hash-alg=sha256 \
-o ivgen-alg=plain64 \
-o ivgen-hash-alg=sha256 \
cirros.raw cirros.luks

# upload the encrypted image and reference Barbican secret
openstack image create --file cirros.luks \
    --disk-format luks \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=on_image_deletion \
    --property os_encrypt_format=LUKSv1 \
    cirros-luks

LUKS with hexlified passphrase

  • image format will be uncompressed LUKS, which can later be directly transferred to a volume by Cinder
  • passphrase will be uploaded as "symmetric" secret type in Barbican; this will make Cinder convert it to passphrase using binascii.hexlify() (like it does for its own images of LUKS volumes)
echo -n "muchsecretsuchwow" > secret_file.key
# it is important that the file does not end with a newline!
python3 -c "import binascii;
print(binascii.hexlify('$(cat secret_file.key)'\
.encode('utf-8')).decode('utf-8'), end='')" \
    > secret_file.hex

# upload passphrase to Barbican in plain text (Cinder will hexlify later)
SECRET_TYPE="symmetric"
openstack secret store --file secret_file.key \
--name luks-image-key --secret-type $SECRET_TYPE

SECRET_ID=$(openstack secret list -f value --name luks-image-key \
    | head -n1 | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)

echo $SECRET_ID

# encrypt the image with LUKS using qemu-img, use hexlified passphrase
qemu-img convert -f raw -O luks \
--object secret,id=sec,file=secret_file.hex \
-o key-secret=sec \
-o cipher-alg=aes-256 \
-o cipher-mode=xts \
-o hash-alg=sha256 \
-o ivgen-alg=plain64 \
-o ivgen-hash-alg=sha256 \
cirros.raw cirros.luks.hexlified

# upload the encrypted image and reference Barbican secret
openstack image create --file cirros.luks.hexlified \
    --disk-format luks \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=on_image_deletion \
    --property os_encrypt_format=LUKSv1 \
    cirros-luks-hexlified

qcow2+LUKS with plain passphrase

  • image format will be qcow2-wrapped LUKS, which will later be converted to raw LUKS in Cinder
  • passphrase will be uploaded as "passphrase" secret type in Barbican; this will make Cinder skip binascii.hexlify() and use it as LUKS passphrase directly
echo -n "muchsecretsuchwow" > secret_file.key

# upload passphrase to Barbican in plain text
SECRET_TYPE="passphrase"
openstack secret store --file secret_file.key \
--name qcow-image-passphrase --secret-type $SECRET_TYPE

SECRET_ID=$(openstack secret list -f value --name qcow-image-passphrase \
    | head -n1 | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)

echo $SECRET_ID

# encrypt the image with qcow2+LUKS using qemu-img
qemu-img convert -f raw -O qcow2 \
--object secret,id=sec,file=secret_file.key \
-o encrypt.format=luks \
-o encrypt.key-secret=sec \
-o encrypt.cipher-alg=aes-256 \
-o encrypt.cipher-mode=xts \
-o encrypt.hash-alg=sha256 \
-o encrypt.ivgen-alg=plain64 \
-o encrypt.ivgen-hash-alg=sha256 \
cirros.raw cirros.luks.qcow2

# upload the encrypted image and reference Barbican secret
openstack image create --file cirros.luks.qcow2 \
    --disk-format qcow2 \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=on_image_deletion \
    --property os_encrypt_format=LUKSv1 \
    cirros-qcow2luks

qcow2+LUKS with hexlified passphrase

  • image format will be qcow2-wrapped LUKS, which will later be converted to raw LUKS in Cinder
  • passphrase will be uploaded as "symmetric" secret type in Barbican; this will make Cinder convert it to passphrase using binascii.hexlify() (like it does for its own images of LUKS volumes)
echo -n "muchsecretsuchwow" > secret_file.key
# it is important that the file does not end with a newline!
python3 -c "import binascii;
print(binascii.hexlify('$(cat secret_file.key)'\
.encode('utf-8')).decode('utf-8'), end='')" \
    > secret_file.hex

# upload passphrase to Barbican in plain text (Cinder will hexlify later)
SECRET_TYPE="symmetric"
openstack secret store --file secret_file.key \
--name qcow-image-key --secret-type $SECRET_TYPE

SECRET_ID=$(openstack secret list -f value --name qcow-image-key \
    | head -n1 | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)

echo $SECRET_ID

# encrypt the image with qcow2+LUKS using qemu-img, use hexlified passphrase
qemu-img convert -f raw -O qcow2 \
--object secret,id=sec,file=secret_file.hex \
-o encrypt.format=luks \
-o encrypt.key-secret=sec \
-o encrypt.cipher-alg=aes-256 \
-o encrypt.cipher-mode=xts \
-o encrypt.hash-alg=sha256 \
-o encrypt.ivgen-alg=plain64 \
-o encrypt.ivgen-hash-alg=sha256 \
cirros.raw cirros.luks.qcow2.hexlified

# upload the encrypted image and reference Barbican secret
openstack image create --file cirros.luks.qcow2.hexlified \
    --disk-format qcow2 \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=on_image_deletion \
    --property os_encrypt_format=LUKSv1 \
    cirros-qcow2luks-hexlified

Testing

Volumes

openstack volume create --type LVM-LUKS --size 1 \
    --image cirros-luks vol-from-luks-img
openstack volume create --type LVM-LUKS --size 1 \
    --image cirros-luks-hexlified vol-from-luks-hexlified-img
openstack volume create --type LVM-LUKS --size 1 \
    --image cirros-qcow2luks vol-from-qcow2luks-img
openstack volume create --type LVM-LUKS --size 1 \
    --image cirros-qcow2luks-hexlified vol-from-qcow2luks-hexlified-img

Floating IPs

PUBLIC_NETWORK=... # name of public network with Floating IPs

# the commands below only create Floating IPs that do not exist with the same
# description already
if [ $(openstack floating ip list --long -f value -c Description \
    | grep "serverIP1" | wc -l) == 0 ]; \
then openstack floating ip create --description "serverIP1" $PUBLIC_NETWORK; fi
if [ $(openstack floating ip list --long -f value -c Description \
    | grep "serverIP2" | wc -l) == 0 ]; \
then openstack floating ip create --description "serverIP2" $PUBLIC_NETWORK; fi
if [ $(openstack floating ip list --long -f value -c Description \
    | grep "serverIP3" | wc -l) == 0 ]; \
then openstack floating ip create --description "serverIP3" $PUBLIC_NETWORK; fi
if [ $(openstack floating ip list --long -f value -c Description \
    | grep "serverIP4" | wc -l) == 0 ]; \
then openstack floating ip create --description "serverIP4" $PUBLIC_NETWORK; fi

Servers (VMs)

# get Floating IP addresses
FIP1=$(openstack floating ip list --long -f value -c Description -c "Floating IP Address" \
    | grep "serverIP1" | cut -d ' ' -f1)
FIP2=$(openstack floating ip list --long -f value -c Description -c "Floating IP Address" \
    | grep "serverIP2" | cut -d ' ' -f1)
FIP3=$(openstack floating ip list --long -f value -c Description -c "Floating IP Address" \
    | grep "serverIP3" | cut -d ' ' -f1)
FIP4=$(openstack floating ip list --long -f value -c Description -c "Floating IP Address" \
    | grep "serverIP4" | cut -d ' ' -f1)

FLAVOR=m1.tiny
openstack server create --flavor $FLAVOR --network admin-private \
    --volume vol-from-luks-img --security-group admin-access-group \
    vm-from-luks-img-vol
openstack server create --flavor $FLAVOR --network admin-private \
    --volume vol-from-luks-hexlified-img --security-group admin-access-group \
    vm-from-luks-img-vol-hex
openstack server create --flavor $FLAVOR --network admin-private \
    --volume vol-from-qcow2luks-img --security-group admin-access-group \
    vm-from-qcow2luks-img-vol
openstack server create --flavor $FLAVOR --network admin-private \
    --volume vol-from-qcow2luks-hexlified-img \
    --security-group admin-access-group \
    vm-from-qcow2luks-img-vol-hex

openstack server add floating ip vm-from-luks-img-vol $FIP1
openstack server add floating ip vm-from-luks-img-vol-hex $FIP2
openstack server add floating ip vm-from-qcow2luks-img-vol $FIP3
openstack server add floating ip vm-from-qcow2luks-img-vol-hex $FIP4

# OPTION A) console logs
# (wait a bit after server startup before executing these)
openstack console log show vm-from-luks-img-vol | tail -n10
openstack console log show vm-from-luks-img-vol-hex | tail -n10
openstack console log show vm-from-qcow2luks-img-vol | tail -n10
openstack console log show vm-from-qcow2luks-img-vol-hex | tail -n10

# OPTION B) SSH tests (requires provider network connectivity)
# default password for cirros image is 'gocubsgo'
ssh cirros@$FIP1
ssh cirros@$FIP2
ssh cirros@$FIP3
ssh cirros@$FIP4

Image import

Required Glance configuration for image import testing

The image import testing requires appropriate multi-store backends being configured. See the Glance documentation on Multi Store Support for more details.

The following are small snippets for configuring two file-based backends that can be used on a DevStack to configure Glance for testing purposes.

sudo mkdir /opt/stack/data/glance/images/local1
sudo mkdir /opt/stack/data/glance/images/local2
sudo chown stack:stack /opt/stack/data/glance/images/local*
sudo mv /opt/stack/data/glance/images/*-* /opt/stack/data/glance/images/local1

Adjust the /etc/glance/glance-api.conf file to configure the backends:

[DEFAULT]
...
enabled_backends = local1:file, local2:file

# note: remove any other previous contents of the glance_store section!
[glance_store]
default_backend = local1

[local1]
filesystem_store_datadir = /opt/stack/data/glance/images/local1

[local2]
filesystem_store_datadir = /opt/stack/data/glance/images/local2

# if using multi-store (enabled_backends), then os_glance_tasks_store
# and os_glance_staging_store become mandatory sections if their path
# is to be changed!
[os_glance_tasks_store]
filesystem_store_datadir = /opt/stack/data/glance/images/tasks

[os_glance_staging_store]
filesystem_store_datadir = /opt/stack/data/glance/images/staging

Also adjust /etc/glance/glance-image-import.conf for the web download test as necessary:

[import_filtering_opts]
allowed_ports = [80,443,33445]

(port 33445 added for local hosting purposes)

Finally, restart the Glance API service:

sudo systemctl restart devstack@g-api

Image Import Test Cases

The import test cases will use cirros.luks and the corresponding Barbican secret as prepared by LUKS with plain passphrase above.

glance-direct
SECRET_ID=$(openstack secret list -f value --name luks-image-passphrase \
    | head -n1 | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)
openstack image create \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=none \
    --property os_encrypt_format=LUKSv1 \
    --disk-format luks \
    imported-cirros-luks-img

# image status should be 'queued' now:
openstack image show imported-cirros-luks-img -f value -c status

# upload encrypted image data to the staging area
openstack image stage --file cirros.luks --progress imported-cirros-luks-img

# image status should be 'uploading' now:
openstack image show imported-cirros-luks-img -f value -c status

openstack image import --method glance-direct --wait imported-cirros-luks-img

# image status should be 'active' now:
openstack image show imported-cirros-luks-img -f value -c status

# create volume and server based on imported image
openstack volume create --type lvmdriver-1-LUKS --size 1 \
    --image imported-cirros-luks-img test-luks-img-imported
openstack server create --flavor m1.tiny --network admin-private \
    --volume test-luks-img-imported --security-group admin-access-group \
    vm-luksimg-imported-vol
openstack console log show vm-luksimg-imported-vol | tail -n10
copy-image
openstack image import --method copy-image --store local2 \
    --wait imported-cirros-luks-img

IMAGE_ID=$(openstack image show -f value -c id imported-cirros-luks-img)

# this should show two entries with the same hash:
md5sum /opt/stack/data/glance/images/*/$IMAGE_ID

# this should show both files identified as LUKS-encrypted:
file /opt/stack/data/glance/images/*/$IMAGE_ID
web-download

First, start a local HTTP server serving the encrypted image in a dedicated shell:

mkdir serve
cp cirros.luks serve/
cd serve
python3 -m http.server 33445

In a separate shell, execute the test case:

URL="http://localhost:33445/cirros.luks"

openstack image create \
    --property os_encrypt_key_id=$SECRET_ID \
    --property os_encrypt_key_deletion_policy=none \
    --property os_encrypt_format=LUKSv1 \
    --disk-format luks \
    downloaded-cirros-luks-img

openstack image import --method web-download --uri $URL downloaded-cirros-luks-img

# wait for 'active' status
openstack image show downloaded-cirros-luks-img -f value -c status

# create volume and server based on downloaded image
openstack volume create --type lvmdriver-1-LUKS --size 1 \
    --image downloaded-cirros-luks-img test-luks-img-downloaded
openstack server create --flavor m1.tiny --network admin-private \
    --volume test-luks-img-downloaded --security-group admin-access-group \
    vm-luksimg-downloaded-vol
openstack console log show vm-luksimg-downloaded-vol | tail -n10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment