archlinux-ami-setup-with-xfce4-for-photogrammetry.md

Arch Linux AMI Setup with XFCE4 for Photogrammetry

Purpose

The purpose for this Gist is to document for myself the steps to setup Arch Linux AMI in AWS with XFCE4 as the desktop environment. As well as, setting up a RDP server. The goal is to have an instance in AWS with GPU resources for the purposes of photogrammetry.

NOTE: Pay attention to $ and # when commands are shown. # indicates that the command needs to be run as root - - OR - - with elevated priveleges. $ indicates that the command is invoked as a regular user.

Steps Overview

Install Arch Linux AMI
Update system
Install Helpful Software
User setup
Install NVIDIA Driver
Install XFCE4
Install display manager
XFCE Configuration
Install Remote Desktop Server
Connect with RDP Client
Photogrammetry Software Install (COLMAP)

Step 1 - Install Arch Linux AMI

I like to stick with the std kernel: Arch Linux AMIs for Amazon Web Services
You will need to select an EC2 Instance Type that supports GPUs. I use the p2.xlarge
Ensure that you open ports 22 - SSH - and 3389 - RDP

Step 2 - Update System

SSH with the default user arch: ssh -i </path/to/.pem file> arch@<EC2 IP>
# pacman-key --init
# pacman-key --populate
# pacman -Syu

Step 3 - Install Helpful Software

# pacman -S base-devel git nano xdg-user-dirs

Step 4 - User Setup

arch user already exists and it has sudo priveleges but things work better if a new user is created and HAS a password.

# useradd -m pg - PG is an acrovym for Photogrammetry
# passwd pg
# usermod -aG wheel,video,audio,storage pg
# EDITOR=nano visudo
Remove # from %wheel ALL=(ALL) ALL
Save and Exit
Create the usual suspects for user pg:
1. In pg home directory run: $ xdg-user-dirs-update
Setup AUR:
1. $ mkdir /home/pg/.aur
Setup SSH for pg:
1. # cp -R /home/arch/.ssh /home/pg/
2. # chown "pg:pg" /home/pg/.ssh
3. # chown "pg:pg" /home/pg/.ssh/authorized_keys

Step 5 - Install NVIDIA Driver

NOTE: Identify the driver you need with $ lspci | grep -E 'VGA|3D' or $ lspci -k | grep -A 2 -E "(VGA|3D)" and NVIDIA website.

As user pg:

For p2 Instance Types (as of the time of this writing):
1. $ cd .aur
2. $ git clone https://aur.archlinux.org/nvidia-470xx-utils.git
3. $ cd nvidia-470xx-utils/
4. $ makepkg -si
For all others (p3, g3, etc.):
1. # pacman -S nvidia-dkms
# nano /etc/mkinitcpio.conf
Remove kms from the HOOKS array
Save and Exit
Regenerate the initramfs: # mkinitcpio -P
# reboot
Verify driver is working: $ nvidia-smi:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.223.02   Driver Version: 470.223.02   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:1E.0 Off |                    0 |
| N/A   26C    P0    73W / 149W |      0MiB / 11441MiB |     98%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Step 6 - Install XFCE4

# pacman -S xfce4 xfce4-goodies xorg-server xorg-apps

Step 7 - Install Display Manager

# pacman -S sddm
$ cd /home/pg/.aur
$ git clone https://aur.archlinux.org/archlinux-themes-sddm.git - theme for SDDM
$ cd archlinux-themes-sddm/
$ makepkg -si
# systemctl enable sddm
# systemctl start sddm

Step 8 - XFCE Configuration

Enable Audio

# pacman -S pipewire pipewire-alsa pipewire-pulse
# reboot

Install Archive Utility

# pacman -S file-roller

Step 9 - Install Remote Desktop Server

$ cd /home/pg/.aur
$ git clone https://aur.archlinux.org/xorgxrdp.git
$ cd xorgxrdp/
$ makepkg -si
If you see: ==> ERROR: 'pacman' failed to install missing dependencies. then just install them manually:
1. # pacman -S nasm xorg-server-devel
2. $ cd ..
3. $ git clone https://aur.archlinux.org/xrdp.git
4. $ cd xrdp/
5. $ makepkg -si
6. $ cd ../xorgxrdp/
7. $ makepkg -si
If you see: ==> ERROR: One or more PGP signatures could not be verified!:
1. $ gpg --recv-key <The key that shows in the error message, most likely: 03993B4065E7193B>
2. $ makepkg -si
# systemctl enable xrdp
# systemctl start xrdp
# nano /etc/xrdp/sesman.ini
Change param=Xorg to param=/usr/lib/Xorg, otherwise X server will error: [ERROR] There is no X server active on display
Save and Exit
Add xinitrc file: $ cp /etc/X11/xinit/xinitrc ~/.xinitrc
Modify xinitrc:
1. $ nano .xinitrc
2. Add exec startxfce4 under the LAST if block in file
3. Save and Exit
# reboot

Step 10 - Connect with RDP Client

Use your preferred RDP client to connect to the RDP server you just installed above. I use Microsoft Remote Desktop to connect from a Mac to the EC2 instance.

Step 11 - Photogrammetry Software Install (COLMAP)

For this step you can use any photogrammetry software - within reason. I have better luck with COLMAP and that is what I'll be showing here. Although there's an AUR for COLMAP I'll build from source for simplicity.

$ git clone https://github.com/colmap/colmap
Install dependencies:
1. For p2 Instance Types (as of the time of this writing):
  1. # pacman -S cgal ceres-solver glew freeimage flann cmake ninja python-sphinx
  2. $ cd ~/.aur
  3. $ git clone https://aur.archlinux.org/cuda-11.0.git
  4. $ cd cuda-11.0/
  5. $ makepkg -si
2. For all others (p3, g3, etc.):
  1. # pacman -S cgal ceres-solver glew freeimage flann cmake ninja python-sphinx cuda
3. Verify NVIDIA compiler exists and is functional $ /opt/cuda/bin/nvcc -V:
```
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Sep__8_19:17:24_PDT_2023
Cuda compilation tools, release 12.3, V12.3.52
Build cuda_12.3.r12.3/compiler.33281558_0
```
4. $ cd /home/pg/.aur
5. $ git clone https://aur.archlinux.org/metis.git
6. $ git clone https://aur.archlinux.org/gklib.git
7. $ cd gklib/
8. $ makepkg -si
9. $ cd ../metis/
10. $ makepkg -si
$ cd /<path to>/colmap/
$ mkdir build
$ cd build/
We need to ensure cmake finds nvcc so that the build is configured for CUDA: $ export PATH=/opt/cuda/bin:$PATH
$ cmake .. -DCMAKE_CUDA_ARCHITECTURES=all
$ make
colmap executable is located at build/src/colmap/exe/
In the exe directory run ./colmap gui to launch the GUI and have fun :-)

NOTE: To verify that colmap supports CUDA run ./colmap -h and look for:

COLMAP 3.9 -- Structure-from-Motion and Multi-View Stereo
              (Commit f6b94fc2 on 2023-11-21 with CUDA)

Use the directions to get going. I keep the defaults in Automatic reconstruction modal box and check the Dense model checkbox. To ensure colmap is using the GPU you can run nvidia-smi -a | grep Utilization -A2. If Gpu > 0% then you're good.

jhillix/archlinux-ami-setup-with-xfce4-for-photogrammetry.md

Select an option

No results found

Select an option

No results found