Jimmy Yen cawa0505

Running llama.cpp on RX 480

Even though the RX 480 is not a very recent GPU, the 8 GB VRAM are quite decent and make it possible to run LLMs much faster than with the CPU.

To make it running, three steps are necessary: Installing vulkan, building shaderc and finally building lama.cpp including the vulkan backend (assuming git, a c compiler, cmake, ninja, etc. are installed on a sufficiently recent Ubuntu).

Step 1: Installing Vulkan

Run sudo apt install libvulkan-dev vulkan-tools glslang-tools.

Date of the guide : September, 2025

Introduction

In this post, I will provide the solution that worked on my system on how to install Radeon Open Compute (ROCm) on Arch (linux-6.6.7.arch1-1) for RX 6900 XT (Should work on other 6000 series). ROCm is an open-source software platform that allows GPU-accelerated computation. This tool is a prerequist to use GPU Acceleration on TensorFlow or PyTorch. In this guide I will use Paru as my AUR package helper, feel free to use any other (https://wiki.archlinux.org/title/AUR_helpers). I will assume you have a working operating system and know what you do with it (Otherwise Arch will be painfull for you).

Building on Ubuntu 23.04

You have 2 options, use the script descrived bellow or follow the instrutions

script in this gist if you want the source code

wget https://gist.githubusercontent.com/Vertecedoc4545/6e54487f07a1888b656b656c0cdd9764/raw/2c5e8ccb428fc331307e2f653cab88174c051310/build-ubuntu-23.sh
chmod +x build-ubuntu-23.sh
./build-ubuntu-23.sh

Setup ZSH

This gist will guide you through to setup starship along with zsh suggestions and zsh syntax highlight for your codespaces.

	"""QA Chatbot streaming using FastAPI, LangChain Expression Language , OpenAI, and Chroma.

	Features
	--------
	- Persistent Chat Memory:
	Stores chat history in a local file.
	- Persistent Vector Store:
	Stores document embeddings in a local vector store.
	- Standalone Question Generation:
	Rephrases follow-up questions to standalone questions in their original language.

	{ config, pkgs, ... }:

	{
	imports =
	[ # Include the results of the hardware scan.
	./hardware-configuration.nix
	];

	# Use the systemd-boot EFI boot loader.
	boot.loader.systemd-boot.enable = true;

	#!/bin/bash
	set -e -o errexit -o pipefail -o nounset

	###################################
	# This script can be used by itself, but it's recommended that you read
	# a tutorial on Proxmox forum first: https://forum.proxmox.com/threads/hey-proxmox-community-lets-talk-about-resources-isolation.124256/
	###################################

	# Do not modify these variables (set by Proxmox when calling the script)
	vmId="$1"

	##/etc/pve/qemu-server/100.conf
	##Network and disks not included in this example

	##Set halt_poll_ns
	#set_halt_poll 0
	##CPU pinning
	#cpu_taskset 1-7
	#assign_interrupts --sleep=10s 1-7 --all

	agent: 1

	set $cursor_size 54

	```
	# GTK
	# This is the only place where you must set GTK scaling
	set $gnome-schema org.gnome.desktop.interface
	exec_always {
	gsettings set $gnome-schema gtk-theme 'Matcha-dark-sea'
	gsettings set $gnome-schema icon-theme 'Numix-Square'
	gsettings set org.gnome.desktop.interface text-scaling-factor 2.73