Skip to content

Instantly share code, notes, and snippets.

@raveenb

raveenb/setup-ci-runner.sh

Last active January 12, 2026 13:05
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save raveenb/687611354264f5f12741e9a2be9b443a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save raveenb/687611354264f5f12741e9a2be9b443a to your computer and use it in GitHub Desktop.
Download ZIP
Self-hosted CI runner setup for GitHub Actions + GitLab CI. Cut your CI/CD costs by 90%. MIT License.
Raw
setup-ci-runner.sh
#!/bin/bash
#
# =============================================================================
# Self-Hosted CI Runner Setup Script
# For GitHub Actions + GitLab CI on a Single VM
# =============================================================================
#
# Author: Luminary Lane (https://luminarylane.app)
# License: MIT
# Version: 1.2.0
#
# This script sets up a fresh Ubuntu 24.04 VM as a dual CI runner for both
# GitHub Actions and GitLab CI, with built-in maintenance and hygiene.
#
# -----------------------------------------------------------------------------
# WHY SELF-HOST?
# -----------------------------------------------------------------------------
#
# THE PROBLEM: GitHub's CI Pricing Changes (January 2026)
#
# GitHub has announced significant pricing changes for private repositories:
# - Increased per-minute costs for hosted runners
# - New charges for self-hosted runner management access
# - Variable costs that scale unpredictably with your build frequency
#
# For startups with active development, this means CI costs can spike
# unexpectedly during crunch time—exactly when you can least afford it.
#
# -----------------------------------------------------------------------------
# THE SOLUTION: Variable Cost → Fixed Cost
# -----------------------------------------------------------------------------
#
# Self-hosting transforms your CI expenses from unpredictable variable costs
# to a predictable fixed monthly expense:
#
# BEFORE (Variable - GitHub Hosted):
# - Linux: $0.008/min - you pay only for minutes used
# - Example: 50 builds/day × 10 min × 30 days = 15,000 min = $120/month
# - Costs spike during active development sprints
# - Hard to budget when build frequency varies
#
# AFTER (Fixed - Self-Hosted):
# - 4-core/32GB VM: ~$30-50/month (varies by provider)
# - Same cost whether you run 100 or 1000 builds
# - Serves BOTH GitHub Actions AND GitLab CI
# - Add more runners on same VM for parallelism (both GH + GL support this)
# - Full control, no queue times, predictable expenses
#
# BREAK-EVEN: ~60-100 build hours/month. Above that, self-hosted wins.
# RESULT: For active teams, 70-90% cost reduction + budget predictability
#
# -----------------------------------------------------------------------------
# USAGE
# -----------------------------------------------------------------------------
#
# Option 1 - Direct run:
# curl -fsSL https://gist.githubusercontent.com/raveenb/687611354264f5f12741e9a2be9b443a/raw/setup-ci-runner.sh | sudo bash
#
# Option 2 - Download and customize:
# wget https://gist.githubusercontent.com/raveenb/687611354264f5f12741e9a2be9b443a/raw/setup-ci-runner.sh
# # Edit the CONFIGURATION section below
# chmod +x setup-ci-runner.sh
# sudo ./setup-ci-runner.sh
#
# After running, you still need to register the runners:
# 1. GitHub: cd /opt/actions-runner && ./config.sh --url ... --token ...
# 2. GitLab: sudo gitlab-runner register
#
# -----------------------------------------------------------------------------
# CHANGELOG
# -----------------------------------------------------------------------------
#
# v1.2.0 (2026-01-12):
# - Added Docker Compose V2 plugin installation (required for modern workflows)
# - Ubuntu's docker.io package doesn't include Compose plugin by default
#
# v1.1.0:
# - Added file descriptor limits configuration
# - Added Playwright browser dependencies option
#
# v1.0.0:
# - Initial release
#
# -----------------------------------------------------------------------------
# MIT LICENSE
# -----------------------------------------------------------------------------
#
# Copyright (c) 2026 Luminary Lane
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
#
# =============================================================================
# =============================================================================
# CONFIGURATION - Customize these values for your setup
# =============================================================================
# GitHub Actions Runner version (check: https://github.com/actions/runner/releases)
ACTIONS_RUNNER_VERSION="2.321.0"
# Swap file size (recommended: equal to RAM for <32GB, half of RAM for >32GB)
SWAP_SIZE="4G"
# Swap behavior (10 = only swap under memory pressure, default is 60)
SWAPPINESS=10
# Journal log limits
JOURNAL_MAX_SIZE="500M"
JOURNAL_MAX_AGE="7day"
# Cleanup schedule (cron format: minute hour day month weekday)
CLEANUP_SCHEDULE="0 3 * * *" # Daily at 3 AM
# Disk space alert threshold (percentage)
DISK_ALERT_THRESHOLD=80
# Docker image retention (hours) - images older than this are pruned
DOCKER_IMAGE_RETENTION_HOURS=168 # 7 days
# Temp file retention (days)
TEMP_FILE_RETENTION_DAYS=7
# Install Playwright browser dependencies? (true/false)
# Set to false if you don't run browser/E2E tests
INSTALL_PLAYWRIGHT_DEPS=true
# Install GitLab CLI (glab)? (true/false)
INSTALL_GITLAB_CLI=true
# Install GitHub CLI (gh)? (true/false)
INSTALL_GITHUB_CLI=true
# File descriptor limits (important for database connections, browser testing)
# Default Linux limit of 1024 is too low for CI workloads
FILE_DESCRIPTOR_LIMIT=65536
# =============================================================================
# END CONFIGURATION - No need to modify below this line
# =============================================================================
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
log_step() { echo -e "${BLUE}[STEP]${NC} $1"; }
# Check if running as root or with sudo
if [[ $EUID -ne 0 ]]; then
log_error "This script must be run as root or with sudo"
exit 1
fi
echo ""
echo "=============================================="
echo " Self-Hosted CI Runner Setup"
echo " GitHub Actions + GitLab CI"
echo "=============================================="
echo ""
log_info "Starting CI Runner setup..."
log_info "This will take 5-10 minutes depending on your connection speed."
echo ""
# =============================================================================
# 1. System Update
# =============================================================================
log_step "1/14 - Updating system packages..."
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get upgrade -y -qq
# =============================================================================
# 2. Install Base Build Tools
# =============================================================================
log_step "2/14 - Installing base build tools..."
apt-get install -y -qq \
build-essential \
curl \
wget \
git \
jq \
unzip \
ca-certificates \
gnupg \
lsb-release \
software-properties-common
# =============================================================================
# 3. Install Docker
# =============================================================================
log_step "3/14 - Installing Docker..."
apt-get install -y -qq docker.io
systemctl enable docker
systemctl start docker
# Add current user to docker group (if not root)
if [ -n "${SUDO_USER:-}" ]; then
usermod -aG docker "$SUDO_USER"
log_info "Added $SUDO_USER to docker group"
fi
# =============================================================================
# 4. Install Docker Compose V2 Plugin
# =============================================================================
log_step "4/14 - Installing Docker Compose V2 plugin..."
# Ubuntu's docker.io package doesn't include the Compose plugin.
# Many CI workflows use 'docker compose' (V2 syntax) which requires the plugin.
# Without it, you'll get: "unknown shorthand flag: 'f' in -f"
DOCKER_COMPOSE_URL="https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64"
DOCKER_PLUGINS_DIR="/usr/local/lib/docker/cli-plugins"
mkdir -p "$DOCKER_PLUGINS_DIR"
curl -SL "$DOCKER_COMPOSE_URL" -o "$DOCKER_PLUGINS_DIR/docker-compose"
chmod +x "$DOCKER_PLUGINS_DIR/docker-compose"
# Verify installation
if docker compose version &>/dev/null; then
log_info "Docker Compose V2 installed: $(docker compose version --short)"
else
log_error "Docker Compose V2 installation failed"
exit 1
fi
# =============================================================================
# 5. Install GitHub CLI (gh)
# =============================================================================
if [ "$INSTALL_GITHUB_CLI" = true ]; then
log_step "5/14 - Installing GitHub CLI..."
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg 2>/dev/null
chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null
apt-get update -qq
apt-get install -y -qq gh
else
log_step "5/14 - Skipping GitHub CLI (disabled in config)"
fi
# =============================================================================
# 6. Install GitLab CLI (glab)
# =============================================================================
if [ "$INSTALL_GITLAB_CLI" = true ]; then
log_step "6/14 - Installing GitLab CLI..."
snap install glab 2>/dev/null || log_warn "Could not install glab via snap"
else
log_step "6/14 - Skipping GitLab CLI (disabled in config)"
fi
# =============================================================================
# 7. Install Playwright/Browser Testing Dependencies
# =============================================================================
if [ "$INSTALL_PLAYWRIGHT_DEPS" = true ]; then
log_step "7/14 - Installing Playwright browser dependencies..."
apt-get install -y -qq --no-install-recommends \
libasound2t64 \
libatk-bridge2.0-0t64 \
libatk1.0-0t64 \
libatspi2.0-0t64 \
libcairo2 \
libcups2t64 \
libdbus-1-3 \
libdrm2 \
libgbm1 \
libgbm-dev \
libglib2.0-0t64 \
libgtk-3-0t64 \
libgtk2.0-0t64 \
libnotify-dev \
libnspr4 \
libnss3 \
libpango-1.0-0 \
libx11-6 \
libxcb1 \
libxcomposite1 \
libxdamage1 \
libxext6 \
libxfixes3 \
libxkbcommon0 \
libxrandr2 \
libxss1 \
libxtst6 \
xauth \
xvfb \
fonts-freefont-ttf \
fonts-ipafont-gothic \
fonts-liberation \
fonts-noto-color-emoji \
fonts-tlwg-loma-otf \
fonts-unifont \
fonts-wqy-zenhei \
libfontconfig1 \
libfreetype6 \
xfonts-cyrillic \
xfonts-scalable
else
log_step "7/14 - Skipping Playwright dependencies (disabled in config)"
fi
# =============================================================================
# 8. Install GitHub Actions Runner
# =============================================================================
log_step "8/14 - Installing GitHub Actions Runner v${ACTIONS_RUNNER_VERSION}..."
mkdir -p /opt/actions-runner
cd /opt/actions-runner
curl -sL -o actions-runner-linux-x64-${ACTIONS_RUNNER_VERSION}.tar.gz \
https://github.com/actions/runner/releases/download/v${ACTIONS_RUNNER_VERSION}/actions-runner-linux-x64-${ACTIONS_RUNNER_VERSION}.tar.gz
tar xzf ./actions-runner-linux-x64-${ACTIONS_RUNNER_VERSION}.tar.gz
rm -f ./actions-runner-linux-x64-${ACTIONS_RUNNER_VERSION}.tar.gz
# Set ownership if running via sudo
if [ -n "${SUDO_USER:-}" ]; then
chown -R "$SUDO_USER":"$SUDO_USER" /opt/actions-runner
fi
# =============================================================================
# 9. Install GitLab Runner
# =============================================================================
log_step "9/14 - Installing GitLab Runner..."
curl -sL https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | bash
apt-get install -y -qq gitlab-runner
# =============================================================================
# 10. Configure Journal Log Limits
# =============================================================================
log_step "10/14 - Configuring systemd journal limits..."
mkdir -p /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/size-limit.conf << EOF
[Journal]
SystemMaxUse=${JOURNAL_MAX_SIZE}
SystemKeepFree=1G
MaxRetentionSec=${JOURNAL_MAX_AGE}
EOF
systemctl restart systemd-journald
# =============================================================================
# 11. Enable Unattended Security Updates
# =============================================================================
log_step "11/14 - Enabling unattended security updates..."
apt-get install -y -qq unattended-upgrades apt-listchanges
cat > /etc/apt/apt.conf.d/20auto-upgrades << 'EOF'
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::AutocleanInterval "7";
EOF
cat > /etc/apt/apt.conf.d/50unattended-upgrades << 'EOF'
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
"${distro_id}ESMApps:${distro_codename}-apps-security";
"${distro_id}ESM:${distro_codename}-infra-security";
};
Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "false";
EOF
systemctl enable unattended-upgrades
# =============================================================================
# 12. Setup Swap File
# =============================================================================
log_step "12/14 - Setting up swap file..."
SWAP_FILE="/swapfile"
if [ ! -f "$SWAP_FILE" ]; then
fallocate -l $SWAP_SIZE $SWAP_FILE
chmod 600 $SWAP_FILE
mkswap $SWAP_FILE
swapon $SWAP_FILE
echo "$SWAP_FILE none swap sw 0 0" >> /etc/fstab
log_info "Created ${SWAP_SIZE} swap file"
else
log_info "Swap file already exists, skipping"
fi
# Set swappiness
sysctl vm.swappiness=${SWAPPINESS}
grep -q "vm.swappiness" /etc/sysctl.conf || echo "vm.swappiness=${SWAPPINESS}" >> /etc/sysctl.conf
# =============================================================================
# 13. Configure File Descriptor Limits
# =============================================================================
log_step "13/14 - Configuring file descriptor limits..."
# Default Linux limit of 1024 is too low for CI workloads with:
# - Database connections (MongoDB, PostgreSQL, Redis)
# - Browser testing (Cypress, Playwright)
# - Node.js servers (Next.js, Express)
# - Multiple parallel processes
# When limit is hit, servers hang without error messages
# Add to /etc/security/limits.conf
if ! grep -q "# CI Runner file descriptor limits" /etc/security/limits.conf 2>/dev/null; then
cat >> /etc/security/limits.conf << EOF
# CI Runner file descriptor limits - added by setup-ci-runner.sh
* soft nofile ${FILE_DESCRIPTOR_LIMIT}
* hard nofile ${FILE_DESCRIPTOR_LIMIT}
* soft nproc ${FILE_DESCRIPTOR_LIMIT}
* hard nproc ${FILE_DESCRIPTOR_LIMIT}
runner soft nofile ${FILE_DESCRIPTOR_LIMIT}
runner hard nofile ${FILE_DESCRIPTOR_LIMIT}
runner soft nproc ${FILE_DESCRIPTOR_LIMIT}
runner hard nproc ${FILE_DESCRIPTOR_LIMIT}
EOF
log_info "Added file descriptor limits to /etc/security/limits.conf"
else
log_info "File descriptor limits already configured"
fi
# =============================================================================
# 14. Setup Automated Cleanup
# =============================================================================
log_step "14/14 - Setting up automated cleanup..."
# Create the comprehensive cleanup script
cat > /opt/ci-runner-cleanup.sh << CLEANUP_SCRIPT
#!/bin/bash
#
# CI Runner Cleanup Script
# Auto-generated by setup-ci-runner.sh
# Runs daily to prevent disk space issues
#
LOG_FILE="/var/log/ci-runner-cleanup.log"
exec >> "\$LOG_FILE" 2>&1
echo "=========================================="
echo "CI Runner cleanup started at \$(date)"
echo "=========================================="
echo ""
echo ">>> Disk usage BEFORE cleanup:"
df -h / | tail -1
# Docker Cleanup
echo ""
echo ">>> Docker cleanup..."
docker container prune -f --filter "until=24h" 2>/dev/null || true
docker image prune -f 2>/dev/null || true
docker image prune -af --filter "until=${DOCKER_IMAGE_RETENTION_HOURS}h" 2>/dev/null || true
docker volume prune -f 2>/dev/null || true
docker network prune -f 2>/dev/null || true
docker builder prune -f --filter "until=${DOCKER_IMAGE_RETENTION_HOURS}h" 2>/dev/null || true
# Temp Files Cleanup
echo ""
echo ">>> Temp files cleanup..."
find /tmp -type f -atime +${TEMP_FILE_RETENTION_DAYS} -delete 2>/dev/null || true
find /var/tmp -type f -atime +${TEMP_FILE_RETENTION_DAYS} -delete 2>/dev/null || true
rm -rf /root/.npm/_cacache/* 2>/dev/null || true
find /root/.cache -type f -atime +14 -delete 2>/dev/null || true
# GitHub Actions workspace
echo ""
echo ">>> GitHub Actions workspace cleanup..."
if [ -d "/opt/actions-runner/_work" ]; then
find "/opt/actions-runner/_work" -type d -name "_temp" -mtime +3 -exec rm -rf {} + 2>/dev/null || true
find "/opt/actions-runner/_work/_tool" -mindepth 2 -maxdepth 2 -type d -mtime +14 -exec rm -rf {} + 2>/dev/null || true
echo "Actions workspace cleaned"
fi
# GitLab Runner builds
echo ""
echo ">>> GitLab Runner builds cleanup..."
if [ -d "/home/gitlab-runner/builds" ]; then
find "/home/gitlab-runner/builds" -mindepth 2 -maxdepth 2 -type d -mtime +7 -exec rm -rf {} + 2>/dev/null || true
echo "GitLab builds cleaned"
fi
# Old logs
echo ""
echo ">>> Old logs cleanup..."
find /var/log -name "*.log" -size +100M -exec sh -c 'tail -1000 "\$1" > "\$1.tmp" && mv "\$1.tmp" "\$1"' _ {} \; 2>/dev/null || true
find /var/log -name "*.gz" -mtime +30 -delete 2>/dev/null || true
find /var/log -name "*.old" -mtime +7 -delete 2>/dev/null || true
# Disk alert
echo ""
echo ">>> Checking disk space..."
DISK_USAGE=\$(df / | tail -1 | awk '{print \$5}' | tr -d '%')
if [ "\$DISK_USAGE" -gt ${DISK_ALERT_THRESHOLD} ]; then
echo "!!! WARNING: Disk usage is \${DISK_USAGE}% (threshold: ${DISK_ALERT_THRESHOLD}%) !!!"
logger -t ci-runner-cleanup -p user.warning "CI Runner disk usage: \${DISK_USAGE}%"
else
echo "Disk usage OK: \${DISK_USAGE}%"
fi
echo ""
echo ">>> Disk usage AFTER cleanup:"
df -h / | tail -1
echo ""
echo "CI Runner cleanup completed at \$(date)"
echo "=========================================="
CLEANUP_SCRIPT
chmod +x /opt/ci-runner-cleanup.sh
# Create cron job
cat > /etc/cron.d/ci-runner-cleanup << EOF
# CI Runner maintenance - auto-generated
${CLEANUP_SCHEDULE} root /opt/ci-runner-cleanup.sh
EOF
chmod 644 /etc/cron.d/ci-runner-cleanup
# =============================================================================
# Cleanup APT
# =============================================================================
log_info "Cleaning up..."
apt-get autoremove -y -qq
apt-get clean -qq
# =============================================================================
# Summary
# =============================================================================
echo ""
echo "=============================================="
echo -e "${GREEN} CI Runner Setup Complete!${NC}"
echo "=============================================="
echo ""
echo "Installed components:"
echo " - Docker: $(docker --version 2>/dev/null | cut -d' ' -f3 | tr -d ',')"
echo " - Docker Compose: $(docker compose version --short 2>/dev/null || echo 'Not installed')"
if [ "$INSTALL_GITHUB_CLI" = true ]; then
echo " - GitHub CLI (gh): $(gh --version 2>/dev/null | head -1 | cut -d' ' -f3)"
fi
if [ "$INSTALL_GITLAB_CLI" = true ]; then
echo " - GitLab CLI (glab): $(glab --version 2>/dev/null | head -1 | awk '{print $2}')"
fi
echo " - GitLab Runner: $(gitlab-runner --version 2>&1 | grep Version | awk '{print $2}')"
echo " - Actions Runner: ${ACTIONS_RUNNER_VERSION}"
if [ "$INSTALL_PLAYWRIGHT_DEPS" = true ]; then
echo " - Playwright deps: Installed"
fi
echo ""
echo "System hygiene:"
echo " - Journal logs: Max ${JOURNAL_MAX_SIZE}, ${JOURNAL_MAX_AGE} retention"
echo " - Security updates: Unattended-upgrades enabled"
echo " - Swap: ${SWAP_SIZE}, swappiness=${SWAPPINESS}"
echo " - File descriptors: ${FILE_DESCRIPTOR_LIMIT} (nofile + nproc)"
echo " - Cleanup cron: ${CLEANUP_SCHEDULE}"
echo " - Disk alert: >${DISK_ALERT_THRESHOLD}% triggers warning"
echo ""
echo "----------------------------------------------"
echo "NEXT STEPS"
echo "----------------------------------------------"
echo ""
echo "1. Register GitHub Actions Runner:"
echo " cd /opt/actions-runner"
echo " ./config.sh --url https://github.com/YOUR_ORG --token YOUR_TOKEN"
echo " sudo ./svc.sh install"
echo " sudo ./svc.sh start"
echo ""
echo "2. Register GitLab Runner:"
echo " sudo gitlab-runner register"
echo " # URL: https://gitlab.com (or your instance)"
echo " # Token: from GitLab Settings > CI/CD > Runners"
echo " # Executor: docker"
echo " # Default image: node:20"
echo ""
echo "3. (Optional) Reboot to apply all changes:"
echo " sudo reboot"
echo ""
echo "----------------------------------------------"
echo "Logs: /var/log/ci-runner-cleanup.log"
echo "Script: /opt/ci-runner-cleanup.sh (run manually to test)"
echo "----------------------------------------------"
echo ""
echo "Happy shipping! - Luminary Lane"
echo "https://luminarylane.app"
echo ""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment