Skip to content

Instantly share code, notes, and snippets.

@bcdurden
Last active October 28, 2025 22:10
Show Gist options
  • Select an option

  • Save bcdurden/832c7f18b5ad0299849801b1bbe07cd4 to your computer and use it in GitHub Desktop.

Select an option

Save bcdurden/832c7f18b5ad0299849801b1bbe07cd4 to your computer and use it in GitHub Desktop.
Create Harvester RKE2 Airgap-friendly Node Image via Packer

RKE2 and Packer

These files are configured in a way so that an Ubuntu cloud-image is modified by downloading the RKE2 install script from upstream as well as installing the qemu-guest-agent. This is done so the Ubuntu image can now function in an airgapped environment as an RKE2 node. Previous methods I've done involved using libguestfs tools and it was a bit clunky due to how it needed to be managed. Packer's QEMU provider fixes that for me.

Unfortunately, Packer's QEMU provider must run locally as there is no Harvester provider that would run these commands on a remote Harvester cluster to save us the dependency problem. Perhaps in the future we can explore that level of sophistication, but for now this works great.

There is a post-install provisioner that uploads the resulting image to Harvester using a VirtualMachineImage CR template. If you do not wish to upload to Harvester, feel free to comment out that section in spec.pkr.hcl, it is located towards the bottom and starts with the lines post-processor "shell-local"

Notes

Keep in mind that Packer has VM specs such as core and memory counts purely to build a temporary VM instance in order to load and modify the upstream Canonical cloud image for Ubuntu. The image output format is qcow2 and as such, it does not contain any specifications around core/memory/storage. qcow2 is not the equivalent of ova or ovf, it is closer to vmdk.

Packer automatically handles rebuilding and compressing the finalized image and there are some cleanup steps that reset the cloud-init status so the image can be run as a new image where cloud-init runs again.

Tips for Prod

In a production environment, the upload to Harvester would likely be done in a separate step unless you're building in an airgap. Further, you might have mandates to start with a server iso as opposed to a cloud image. Given that, there are a lot more steps involved to install one of those that involves Packer's UI interaction capabilities. There is plenty of information online around Ubuntu as an example with packer for that, it should be easy enough to transpose.

Also, in production, you'll want to add checksum compute capabilities and likely add steps that generate a manifest for your image to be added to Hauler so that it may be brought into the airgap.

HowTo

Ensure qemu and yq are installed and that you are running on a Linux OS of some sort. Edit the build.sh file and ensure your harvester password is correct.

Run the build script ./build.sh, see example below:

$ ./build.sh 
2024/04/08 08:27:23 [INFO] Packer version: 1.10.2 [go1.20.12 linux amd64]
2024/04/08 08:27:23 [TRACE] discovering plugins in /usr/bin
2024/04/08 08:27:23 [TRACE] discovering plugins in .
2024/04/08 08:27:23 [TRACE] discovering plugins in /home/deathstar/.config/packer/plugins
2024/04/08 08:27:23 [INFO] Discovered potential plugin: qemu = /home/deathstar/.config/packer/plugins/github.com/hashicorp/qemu/packer-plugin-qemu_v1.1.0_x5.0_linux_amd64
2024/04/08 08:27:23 [INFO] found external [-packer-default-plugin-name-] builders from qemu plugin
2024/04/08 08:27:23 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2024/04/08 08:27:23 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/deathstar/.packerconfig
2024/04/08 08:27:23 [WARN] Config file doesn't exist: /home/deathstar/.packerconfig
2024/04/08 08:27:23 [INFO] Setting cache directory: /home/deathstar/.cache/packer
2024/04/08 08:27:23 [TRACE] listing potential installations for "github.com/hashicorp/qemu" that match ">= 1.0.9". plugingetter.ListInstallationsOptions{FromFolders:[]string{"/usr/bin", ".", "/home/deathstar/.config/packer/plugins"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc0003e2180)}}}}
2024/04/08 08:27:23 [TRACE] Found the following "github.com/hashicorp/qemu" installations: [{/home/deathstar/.config/packer/plugins/github.com/hashicorp/qemu/packer-plugin-qemu_v1.1.0_x5.0_linux_amd64 v1.1.0}]
2024/04/08 08:27:23 [INFO] found external [-packer-default-plugin-name-] builders from qemu plugin
...
==> image_build.qemu.ubuntu_cloud: Running post-processor:  (type shell-local)
==> image_build.qemu.ubuntu_cloud (shell-local): Running local shell script: upload.sh
2024/04/08 08:30:44 packer-post-processor-shell-local plugin: [INFO] (shell-local): starting local command: bash -c HARVESTER_PASSWORD='<sensitive>' HARVESTER_VIP='10.10.0.10' IMAGE_FILE='output//ubuntu-jammy-rke2-amd64.img' IMAGE_NAME='ubuntu-jammy-rke2' NAMESPACE='default' PACKER_BUILDER_TYPE='qemu' PACKER_BUILD_NAME='ubuntu_cloud' PACKER_HTTP_ADDR='10.0.2.2:0' PACKER_HTTP_IP='10.0.2.2' PACKER_HTTP_PORT='0'  /home/deathstar/airgapping_harvester_with_hauler/services/ubuntu_image/upload.sh
2024/04/08 08:30:44 packer-post-processor-shell-local plugin: [INFO] (shell-local communicator): Executing local shell command [bash -c HARVESTER_PASSWORD='<sensitive>' HARVESTER_VIP='10.10.0.10' IMAGE_FILE='output//ubuntu-jammy-rke2-amd64.img' IMAGE_NAME='ubuntu-jammy-rke2' NAMESPACE='default' PACKER_BUILDER_TYPE='qemu' PACKER_BUILD_NAME='ubuntu_cloud' PACKER_HTTP_ADDR='10.0.2.2:0' PACKER_HTTP_IP='10.0.2.2' PACKER_HTTP_PORT='0'  /home/deathstar/airgapping_harvester_with_hauler/services/ubuntu_image/upload.sh]
...
Build 'image_build.qemu.ubuntu_cloud' finished after 3 minutes 37 seconds.
==> Wait completed after 3 minutes 37 seconds
#!/bin/bash
rm -rf output/
HARVESTER_PASSWORD=mypassword
packer init .
PACKER_LOG=1 packer build -var="harvester_vip=10.10.0.10" -var "harvester_password=${HARVESTER_PASSWORD}" .
source "file" "user_data" {
content = <<EOF
#cloud-config
ssh_pwauth: True
package_update: true
packages:
- qemu-guest-agent
password: superpassword
chpasswd: { expire: False }
ssh_pwauth: True
runcmd:
- - systemctl
- enable
- '--now'
- qemu-guest-agent.service
- mkdir -p /var/lib/rancher/rke2-artifacts && wget https://get.rke2.io -O /var/lib/rancher/install.sh && chmod +x /var/lib/rancher/install.sh
EOF
target = "user-data"
}
source "file" "meta_data" {
content = <<EOF
{"instance-id":"packer-worker.tenant-local","local-hostname":"packer-worker"}
EOF
target = "meta-data"
}
build {
sources = ["source.file.user_data", "source.file.meta_data"]
provisioner "shell-local" {
inline = ["genisoimage -output cidata.iso -input-charset utf-8 -volid cidata -joliet -r user-data meta-data"]
}
}
packer {
required_plugins {
qemu = {
version = ">= 1.0.9"
source = "github.com/hashicorp/qemu"
}
}
}
source "qemu" "ubuntu_cloud" {
vm_name = "${var.image_name}-amd64.img"
iso_url = var.ubuntu_url
iso_checksum = var.ubuntu_checksum
disk_image = true
boot_command = []
boot_wait = "10s"
# QEMU specific configuration
cpus = 2
memory = 4096
accelerator = "kvm" # use none here if not using KVM
disk_size = "10G"
disk_compression = true
efi_boot = true
output_directory = var.output_location
# SSH configuration so that Packer can log into the Image
ssh_password = "superpassword"
ssh_username = "ubuntu"
ssh_timeout = "5m"
shutdown_command = "sudo cloud-init clean --logs --machine-id && sudo shutdown -P now"
headless = true
net_device = "virtio-net"
qemuargs = [
["-cdrom", "cidata.iso"]
]
}
build {
name = "image_build"
sources = [ "source.qemu.ubuntu_cloud" ]
# Wait till Cloud-Init has finished setting up the image on first-boot
provisioner "shell" {
inline = [
"while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for Cloud-Init...'; tail -n10 /var/log/cloud-init-output.log; sleep 5; done"
]
}
post-processor "shell-local" {
execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"]
script = "upload.sh"
environment_vars = [
"HARVESTER_VIP=${var.harvester_vip}",
"HARVESTER_PASSWORD=${var.harvester_password}",
"IMAGE_NAME=${var.image_name}",
"IMAGE_FILE=${var.output_location}/${var.image_name}-amd64.img",
"NAMESPACE=${var.namespace}"
]
}
}
#!/bin/bash
# params:
# HARVESTER_VIP
# HARVESTER_PASSWORD
# IMAGE_NAME
# IMAGE_FILE
# NAMESPACE
export TOKEN=$(curl -sk -X POST https://${HARVESTER_VIP}/v3-public/localProviders/local?action=login -H 'content-type: application/json' -d '{"username":"admin","password":"'${HARVESTER_PASSWORD}'"}' | jq -r '.token')
export IMAGE_SIZE=$(stat -c%s "$IMAGE_FILE")
yq -j '.metadata.name = "'${IMAGE_NAME}'" | .spec.displayName = "'${IMAGE_NAME}'"' vmi_template.yaml | \
curl -sk -X POST -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" --data-binary @- "https://${HARVESTER_VIP}/v1/harvester/harvesterhci.io.virtualmachineimages/${NAMESPACE}" &> /dev/null
curl -sk -X POST -H "Authorization: Bearer ${TOKEN}" -F "chunk=@${IMAGE_FILE}" "https://${HARVESTER_VIP}/v1/harvester/harvesterhci.io.virtualmachineimages/${NAMESPACE}/${IMAGE_NAME}?action=upload&size=${IMAGE_SIZE}"
variable "harvester_vip" {
type = string
}
variable "harvester_password" {
type = string
sensitive = true
}
variable "ubuntu_url" {
type = string
default = "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"
}
variable "ubuntu_checksum" {
type = string
default = "file:https://cloud-images.ubuntu.com/jammy/current/SHA256SUMS"
}
variable "image_name" {
type = string
default = "ubuntu-jammy-rke2"
}
variable "namespace" {
type = string
default = "default"
}
variable "output_location" {
type = string
default = "output/"
}
---
apiVersion: harvesterhci.io/v1beta1
kind: VirtualMachineImage
metadata:
name:
annotations:
harvesterhci.io/storageClassName: harvester-longhorn
labels:
harvesterhci.io/image-type: raw_qcow2
harvesterhci.io/os-type: linux
namespace: default
spec:
displayName:
retry: 3
sourceType: upload
storageClassParameters:
migratable: 'true'
numberOfReplicas: '3'
staleReplicaTimeout: '30'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment