General

Info

!ATTENTION! This script is intentionally designed to completely wipe all data on at least two drives! !ATTENTION!

I used standard Arch Linux on two NVME drives. The goal was as much encryption as possible with the smallest performance impact possible.

Why did I make a script? Because I tire of typing these commands manually through this experiment.

Should you try it? That is up to you. The installer.sh is designed to wipe all data on two drives. This script is designed specifically for my personal laptop. Most likely one or more scripts in this gist are broken even on my personal laptop, because this is an experiment.

Why then is this gist public? GitHub's CDN started returning 404 errors on the raw source links for my gist's scripts. Making the gist public solved the issue.

Notes

At the time of this writing GRUB does not boot on NVME & EFI.

The Plan

• NVMe drives should be aligned to 4KiB, which all partitioners seem to do now.
• EFI partition on primary drive.
• EXT4 boot partition on primary drive.
• BTFS, RAID 0 (stripe).
• Encrypted Swap on both drives, without hibernate because I'm unsure how to avoid dbl password entry at boot/resume (NYI).
• dm_crypt for encryption, using AES-XTS, 256bit key, because benchmarks:
  • aes-xts 256b 3449.7 MiB/s 3435.7 MiB/s
  • aes-xts 512b 2732.7 MiB/s 2757.1 MiB/s
• Only enter password once on boot, so use a modified mkinitcpio.

Sources

• Manuals.
  1. BTRFS, e.g. btrfs on top of dmcrypt
  2. Arch Linux, e.g.:
  • dm-crypt/Drive preparation
  • dm-crypt/Swap encryption
  • Solid State Drives/NVMe
  • Beginner's Guide
  • Installation Guide
  3. dm_crypt, etc.
• Internet in general.
• Btrfs on dm-crypt: An In-Depth Guide
• dude42's guide on decrypting multiple devices for /.

How To Use

Installer

1. Boot your arch linux install media.
2. Connect to the internet.

# View your connections.
ip link

# The dhcpcd daemon is enabled on boot for wired devices, and will attempt to start a connection.
# https://wiki.archlinux.org/index.php/beginners'_guide#Connect_to_the_Internet

# If not using wired, connect to wifi.
wifi-menu

3. Download & run the script.

# Download and run the script.
wget https://gist.githubusercontent.com/redeemed2011/3387d18d5e8f0274b0e3/raw/ad6cad46e418e6eb2593612ba90a8028e9ce60bc/installer.sh -O setup.sh && sh setup.sh

Add WiFi

There is no gui for how we use wifi in this configuration. Currently you must do the following:

wpa_passphrase WIFI_SSID PASSWORD | sudo tee -a /etc/wpa_supplicant/wpa_supplicant.conf

Snapshot/Maintenance Script

Whenever you need to boot to a live arch disk or similar so that you may, say, rollback the root btrfs subvol to an earlier snapshot, download and run this script.

1. Boot your arch linux install media.
2. Connect to the internet.

# View your connections.
ip link

# The dhcpcd daemon is enabled on boot for wired devices, and will attempt to start a connection.
# https://wiki.archlinux.org/index.php/beginners'_guide#Connect_to_the_Internet

# If not using wired, connect to wifi.
wifi-menu

3. Download & run the script.

# Download and run the script.
# NOTE: Short URL resolves to https://gist.github.com/redeemed2011/3387d18d5e8f0274b0e3/raw/open-btrfs-pool.sh
wget https://goo.gl/o7m4A9 -O open.sh && sh open.sh

4. Delete the root subvolume (or rename it).

cd /mnt/btrfs-pool
btrfs subvol delete root

5. Pick your snapshot. Snapshot info is available via the info.xml in each snapshot's root folder.

cat snapshots/root/1/info.xml

5. Snapshots are read only, so make a new snapshot pointing to the now-non-existant root subvol.

btrfs subvol snapshot snapshots/root/1 root

KVM & VFIO

The follow options will need to be added the QEMU virtual machine script.

-enable-kvm \
-device vfio-pci,host=01:00.0,x-vga=on,multifunction=on \
-device vfio-pci,host=01:00.1 \

The host= will need to be set to the bus addresses of the devices being passed through. The video device will need x-vga=on in order to display video.

Because the video of the VM is on separate hardware, seemless keyboard/mouse integration will not work.

There are ways around losing the keyboard and mouse integration, like Synergy. Another option is to setup a dumb video device on the QEMU VM.

-device qxl \
-vga none \

This will create a qxl video device but not connected to any video driver. When kicking off the QEMU VM, QEMU will start up a blank video screen window. Clicking on the window will pass the keyboard and mouse to the VM. The mouse cursor will seem choppy in the VM but input is registered as normal.

KVM Example GPU VM

General KVM Script

#!/bin/bash

# QEMU name and PID
OPTS="-name windows-10-pro"
OPTS="$OPTS -pidfile /tmp/windows-10-pro.pid"

# Processor core2duo qemu64 host
OPTS="$OPTS -cpu qemu64,kvm=off"
OPTS="$OPTS -smp 8,sockets=1,cores=4,threads=2"
OPTS="$OPTS -enable-kvm"

# Machine
OPTS="$OPTS -machine type=pc-i440fx-2.1,accel=kvm"
#OPTS="$OPTS -machine type=q35,accel=kvm"

# The following setting enables S3 (suspend to RAM). OVMF supports S3
# suspend/resume. Disable when using Q35
OPTS="$OPTS -global PIIX4_PM.disable_s3=0"

# Memory
OPTS="$OPTS -m 8G"
OPTS="$OPTS -mem-path /dev/hugepages"
OPTS="$OPTS -mem-prealloc"
OPTS="$OPTS -balloon none"

# Hardware clock
OPTS="$OPTS -rtc clock=host,base=utc"

# Sound hardware
QEMU_PA_SAMPLES=128
export QEMU_AUDIO_DRV=pa
OPTS="$OPTS -soundhw hda"

# Graphic card passthrough (Gigabyte GeForce GTX 980 G1 Gaming)
OPTS="$OPTS -device vfio-pci,host=02:00.0,multifunction=on"
OPTS="$OPTS -device vfio-pci,host=02:00.1"

# USB 3.0 passthrough (NEC/Renesas)
#OPTS="$OPTS -usb -usbdevice host:0x03f0:0xd407"
#OPTS="$OPTS -usb -usbdevice host:0x1d6b:0x0003"
#OPTS="$OPTS -usb -usbdevice host:0x1a40:0x0101"

#OPTS="$OPTS -device vfio-pci,host=03:00.0"

# Keyboard layout
OPTS="$OPTS -k en-us"

# Boot priority
OPTS="$OPTS -boot order=c"

# OVMF
cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /home/tanasaki/qemu/VMs/windows10/my_vars.fd
OPTS="$OPTS -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd"
OPTS="$OPTS -drive if=pflash,format=raw,file=/home/tanasaki/qemu/VMs/windows10/my_vars.fd"

# System drive
OPTS="$OPTS -drive id=disk0,if=none,cache=unsafe,format=raw,file=/home/tanasaki/qemu/VMs/windows10/windows10.img"
OPTS="$OPTS -device driver=virtio-scsi-pci,id=scsi0"
OPTS="$OPTS -device scsi-hd,drive=disk0"

# Other drive
#OPTS="$OPTS -drive id=disk1,if=none,cache=none,aio=native,format=raw,file=/dev/disk/by-id/ata-Hitachi_HDS721050CLA660_JP1570FR1ZWP7K"
#OPTS="$OPTS -device driver=virtio-scsi-pci,id=scsi1"
#OPTS="$OPTS -device scsi-hd,drive=disk1"

# Other drive 2
#OPTS="$OPTS -drive id=disk2,if=none,cache=none,aio=native,format=raw,file=/dev/disk/by-id/ata-Hitachi_HDS5C3020ALA632_ML0220F30NX2DD"
#OPTS="$OPTS -device driver=virtio-scsi-pci,id=scsi2"
#OPTS="$OPTS -device scsi-hd,drive=disk2"

# Windows 10 Pro installer
OPTS="$OPTS -drive id=cd0,if=none,format=raw,readonly,file=/home/tanasaki/qemu/OS_ISOs/Windows10/Windows10.iso"
OPTS="$OPTS -device driver=ide-cd,bus=ide.0,drive=cd0"

# Virtio driver
OPTS="$OPTS -drive id=virtiocd,if=none,format=raw,file=/home/tanasaki/qemu/OS_ISOs/virtio-win.iso"
OPTS="$OPTS -device driver=ide-cd,bus=ide.1,drive=virtiocd"

# OVMF emits a number of info / debug messages to the QEMU debug console, at
# ioport 0x402. We configure qemu so that the debug console is indeed
# available at that ioport. We redirect the host side of the debug console to
# a file.
#OPTS="$OPTS -global isa-debugcon.iobase=0x402 -debugcon file:/tmp/windows_10_pro.ovmf.log"

# QEMU accepts various commands and queries from the user on the monitor
# interface. Connect the monitor with the qemu process's standard input and
# output.
#OPTS="$OPTS -monitor stdio"

# A USB tablet device in the guest allows for accurate pointer tracking
# between the host and the guest.
#OPTS="$OPTS -device piix3-usb-uhci -device usb-tablet"

# Network
#OPTS="$OPTS -netdev tap,vhost=on,ifname=$VM,script=/usr/local/bin/vm_ifup_brlan,id=brlan"
#OPTS="$OPTS -device virtio-net-pci,mac=52:54:00:xx:xx:xx,netdev=brlan"

# Disable display
#OPTS="$OPTS -vga qxl"
#OPTS="$OPTS -vga none"
#OPTS="$OPTS -serial null"
#OPTS="$OPTS -parallel null"
#OPTS="$OPTS -monitor none"
#OPTS="$OPTS -display none"
#OPTS="$OPTS -daemonize"

# QEMU Guest Agent
#OPTS="$OPTS -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0"
#OPTS="$OPTS -device virtio-serial"
#OPTS="$OPTS -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0"

sudo taskset -c 0-7 qemu-system-x86_64 $OPTS

Define the VM

mkdir -p ~/Documents/
tee ~/Documents/gamingvm.xml <<'EOF' > /dev/null
<?xml version="1.0" encoding="UTF-8"?>
<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
  <name>gamingvm</name>
  <uuid>01bd2ed1-b465-4eba-b6e4-47c6ac8171c6</uuid>
  <memory unit="GB">16</memory>
  <currentMemory unit="GB">16</currentMemory>
  <vcpu placement="static">6</vcpu>
  <cpu mode="host-passthrough">
  <topology sockets="1" cores="3" threads="2" />
  </cpu>
  <os>
  <type arch="x86_64" machine="q35">hvm</type>
  <loader>/usr/share/qemu/bios.bin</loader>
  <bootmenu enable="yes" />
  </os>
  <features>
  <hyperv>
  <relaxed state="on" />
  <vapic state="on" />
  <spinlocks state="on" retries="8191" />
  </hyperv>
  <acpi />
  </features>
  <clock offset="localtime">
  <timer name="hypervclock" present="yes" />
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
  <emulator>/usr/bin/qemu-system-x86_64</emulator>
  <interface type='bridge'>
  <mac address='52:54:00:a0:41:92'/>
  <source bridge='br0'/>
  <model type='rtl8139'/>
  <rom bar='off'/>
  </interface>
  <sound model='ich6'/>
  <controller type="usb" index="0" />
  <controller type="usb" index="1" />
  <controller type="usb" index="2" />
  <controller type="usb" index="3" />
  <controller type="sata" index="0" />
  <controller type="pci" index="0" model="pcie-root" />
  <controller type="pci" index="1" model="dmi-to-pci-bridge" />
  <controller type="pci" index="2" model="pci-bridge" />
  <memballoon model="none" />
  <sound model="ich6" />
  <disk type="file" device="cdrom">
  <driver name="qemu" type="raw" />
  <source file="[OS disc image]" />
  <target dev="hdc" bus="ide" />
  <readonly />
  <address type="drive" controller="0" bus="1" unit="0" />
  </disk>
  <disk type="file" device="disk">
  <source file="[Primary HDD image]" />
  <target dev="vda" bus="ide" />
  <address type="drive" bus="0" />
  </disk>
  <!--
  <disk type="file" device="disk">
  <source file="[Secondary HDD image]" />
  <target dev="vdb" bus="ide" />
  <address type="drive" bus="1" />
  </disk>
  -->
  </devices>
  <qemu:commandline>
  <qemu:env name="QEMU_PA_SAMPLES" value="4096" />
  <qemu:env name="QEMU_AUDIO_DRV" value="pa" />
  <qemu:arg value="-device" />
  <qemu:arg value="ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1" />
  <qemu:arg value="-device" />
  <qemu:arg value="vfio-pci,host=[GPU],bus=root.1,addr=00.0,multifunction=on,x-vga=on" />
  <!-- DISABLING AUDIO FOR THE TIME BEING
  <qemu:arg value="-device" />
  <qemu:arg value="vfio-pci,host=[Audio],bus=root.1,addr=00.1" />
  -->
  <qemu:arg value="-cpu" />
  <qemu:arg value="host" />
  </qemu:commandline>
</domain>
EOF

sed -i "s/\[GPU\]/${NVIDIA_SLOT}/" ~/Documents/gamingvm.xml
sed -i 's@\[Primary HDD image\]@\~/Documents/disk.img@' ~/Documents/gamingvm.xml

qemu-img create -f raw -o size=50G ~/Documents/disk.img

5b. Replace [OS disc image] with the path to an .iso for an install disc. I highly recommend using Windows 10 considering it's future support for dx12 and it has given me far less problems than Windows 8. This tutorial is also written for Win10, but Windows 8 is practically the same.

5c. Replace [Memory] with the amount of memory you want the VM to have. Note the value is in Gigabytes. Be sure not to give more than your system ram amount and leave some space for the host.

5d. Replace [Primary HDD Image] with the path to the image you created in step four. You can also add the path for a second hdd where [Secondary HDD Image]. You can also add more HDDs by copying the elements and increasing the number of bus and increasing the letter on vdx (so third disk would be bus=2 vdc, fourth would be bus=3, vdd, etc.).

5e. Choose the amount of cores and threads you want your VM to have. If you're on a processor with 8 threads, I recommend 3 cores and 2 threads (2 threads per core). In this instance, I would change [Cores*Threads] to 6, [Cores] to 3, and [Threads] to 2.

5f. Change [GPU] and [Audio] to the slot/function numbers from lspci. For example, I would use "01:00.0" and "01:00.1"

5f. Terminal: "virsh define [Path to XML file]". The terminal should return this if everything is correct. Code:

Domain gamingvm defined from [Path to XML file]

Setting up input devices

6a. Create a new XML file like the following

<hostdev mode='subsystem' type='usb' managed='no'>
<source>
<vendor id='0x[Before Colon]'/>
<product id='0x[After Colon]'/>
</source>
</hostdev>

6b. Terminal: "lsusb". Identify which device is your keyboard. Then look at the numbers after "ID" from the output. In the XML file, replace [Before Colon] with the set of numbers/letters before the colon, then replace [After Colon] with the set of numbers/letters after the colon. Be sure to keep the 0x. It sounds a bit confusing, but here is an example:

My keyboard from lsusb

Bus 002 Device 005: ID 413c:2011 Dell Computer Corp. Multimedia Pro Keyboard

My XML file would to look like this

<hostdev mode='subsystem' type='usb' managed='no'>
<source>
<vendor id='0x413c'/>
<product id='0x2011'/>
</source>
</hostdev>

Note: You could just add the hostdev element in the devices element in the XML file you created in step four and define the vm again. However, putting the USB devices in separate files allows you to attach/detach much easier. This is especially useful if the VM fails to boot and you have no way of interacting with your host OS.

6c. Repeat that for your mouse. If your mouse/keyboard has multiple entries, be sure to do it all of them (for instance, a wireless mouse would show up twice. Once for the USB cord, and once for the wireless adapter).

6d. Create a file with the following content

#!/bin/bash
if [[ $(virsh list | grep gamingvm) != "" ]]
then
vm=gamingvm
fi

virsh attach-device $vm [Mouse]
virsh attach-device $vm [Keyboard]

6e. Replace [Mouse] and [Keyboard] with paths to the files you created in the previous steps. If you had to make more than two files, just add lines with the path to those at the end

6f. Make this file executable by either right clicking it and going into the properties or using "chmod +x".

6g. Make a copy of the file from 6d, except replace the word attach with detach so it says virsh detach-device $vm .... This script can be used for detaching your input devices. Make sure it is executable.

Preparing virtual machine

7a. Create the following file

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
  echo "This script must be run as root" 1>&2
  su
  exit 0
fi

set -x

echo 1 > /sys/module/kvm/parameters/ignore_msrs
virsh start gamingvm

if [[ $(virsh list | grep gamingvm) == "" ]]
then
  exit 0
fi

set +x

exit 0

7b. Terminal:

sudo gedit /etc/libvirt/qemu.conf

7c. Add the following lines (or find them and uncomment them)

user = "root"
group = "root"
cgroup_device_acl = [
  "/dev/null", "/dev/full", "/dev/zero",
  "/dev/random", "/dev/urandom",
  "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
  "/dev/rtc","/dev/hpet", "/dev/vfio/vfio",
  "/dev/vfio/[GROUP]"
]
clear_emulator_capabilities = 0

7d. Terminal:

find /sys/kernel/iommu_groups/ -type l

Now look for what directory the GPU you are passing through is in. Replace [GROUP] with whatever number group your device is in.

Preparing PulseAudio

Note: If you are going to use HDMI audio, you do not need to follow these steps.

8a. Terminal:

cp /etc/pulse/default.pa ~/.pulse/

8b. Terminal:

gedit ~/.pulse/

8c. Add the following to the end of the file load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1

8d. Terminal:

su

8d. Terminal:

mkdir ~/.pulse/

8e. Terminal:

echo "default-server = 127.0.0.1" > ~/.pulse/client.conf

Networking

Note: This uses netctl which should already be installed. If for some reason it's not, install it via "pacman -S netctl" and configure it by referring to the [Click here to view the link]

9a. Terminal:

cd /etc/netctl

9b. Terminal:

sudo gedit [profile name]

[profile name] is the profile for the device that you use to get your interenet connection. It is likely called wired. If you are unsure run the command ls to view the available profiles.

9c. Change the IP=... line to IP=no. If the Address, DNS, or Gateway lines are present, delete them. Take note of the Interface=... line for future reference.

9d. Terminal:

sudo gedit bridge

This will create a new bridge profile.

9e. Add the following to file

Description="Bridge connection"
Interface=br0
Connection=bridge
BindsToInterfaces=([Interface])
IP=static
Address='192.168.1.202/24'
Gateway='192.168.1.1'
## Ignore (R)STP and immediately activate the bridge
SkipForwardingDelay=yes

9e. Change [Interface] to the name of the interface you took note of in step 10c. Also feel free to change the IP address to what you want it to be.

9f. Terminal:

netctl enable bridge

9g. Reboot to ensure networking on the host is still working.​

Install The OS

10a. Look at the "Kernel Patches" section of this tutorial below. Check to see if you need to enable those patches. If you do, enable the patches before you procede.​

10b. Make the file from step seven executable and run it. If everything goes smoothly, switching to the input of your GPU should display the installation screen. If not, scroll down to the troubleshooting portion of the thread.

10c. Run the file you made in step six to enable keyboard and mouse.

10d. Install the OS like usual.

10e. You now (hopefully) have a VM capable of playing Windows games!