vpeopleonatank

Ollama + Open-WebUI + NVIDIA GPU

A Docker Compose stack for running local LLMs on your own hardware with NVIDIA GPU acceleration, a web UI, and one-command model setup via profiles.

What's Included

Steps to manage multiple CUDA environments

Latest Update: May 19th, 2024

This gist contains all the steps required to:

Install multiple CUDA versions (e.g., CUDA 11.8 and CUDA 12.1
Manage multiple CUDA environments on Ubuntu using the utility called environment modules.
Use this approach to avoid CUDA environment conflicts.

Environment Modules is a package that provides for the dynamic modification of a user's environment via modulefiles. You can find more on it at https://modules.readthedocs.io/en/latest/

Why

You may want to use AdGuard's DNS over HTTPS[^2] service if you

Want to make it harder for your ISP to know what websites you are requesting
Want to block most traditional ads from your web browsing experience across your entire PC

Setup instructions

Open PowerShell

Install TrueNAS SCALE on a partition instead of the full disk

The TrueNAS installer doesn't have a way to use anything less than the full device. This is usually a waste of resources when installing to a modern NVMe which is usually several hundred of GB. TrueNAS SCALE will use only a few GB for its system files so installing to a 16GB partition would be helpful.

The easiest way to solve this is to modify the installer script before starting the installation process.

	from moviepy.editor import *


	def getCroppedFrames(movie, crop_size, resize_size):
	cropped_movie = movie.crop(x1=0, y1=0, width=crop_size["w"], height=crop_size["h"])
	resized_movie = cropped_movie.resize(height=resize_size["h"])
	return resized_movie


	def makeGIF(path, movie, fps, program):

	Download File URL - https://download.splunk.com/products/splunk/releases/9.0.1/linux/splunk-9.0.1-82c987350fde-Linux-x86_64.tgz

	Splunk Linux Tar file - wget -O splunk-9.0.1-82c987350fde-Linux-x86_64.tgz "https://download.splunk.com/products/splunk/releases/9.0.1/linux/splunk-9.0.1-82c987350fde-Linux-x86_64.tgz"

	Splunk Linux rpm file - wget -O splunk-9.0.1-82c987350fde-linux-2.6-x86_64.rpm "https://download.splunk.com/products/splunk/releases/9.0.1/linux/splunk-9.0.1-82c987350fde-linux-2.6-x86_64.rpm"

	Splunk Linux Debian file - wget -O splunk-9.0.1-82c987350fde-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/9.0.1/linux/splunk-9.0.1-82c987350fde-linux-2.6-amd64.deb"

	Splunk Linux Windows file - wget -O splunk-9.0.1-82c987350fde-x64-release.msi "https://download.splunk.com/products/splunk/releases/9.0.1/windows/splunk-9.0.1-82c987350fde-x64-release.msi"

	# TFLite quantized inference example
	#
	# Based on:
	# https://www.tensorflow.org/lite/performance/post_training_integer_quant
	# https://www.tensorflow.org/lite/api_docs/java/org/tensorflow/lite/Tensor.QuantizationParams

	import numpy as np
	import tensorflow as tf

	# Location of tflite model file (float32 or int8 quantized)

	version: '3'

	services:

	homeassistant:
	container_name: home-assistant
	image: homeassistant/home-assistant:stable
	volumes:
	- ./homeassistant_data:/config
	environment:

	Note : Proxmox 6.1

	VI : /etc/apt/sources.list

	# security updates
	deb http://security.debian.org jessie/updates main contrib

	# PVE pve-no-subscription repository provided by proxmox.com,
	# NOT recommended for production use
	deb http://download.proxmox.com/debian jessie pve-no-subscription

	#!/bin/bash
	# Automatically compile and install FFMPEG with NVIDIA hardware acceleration on Debian
	# Based on https://www.tal.org/tutorials/ffmpeg_nvidia_encode
	# Verified working on Debian 10 and 11

	# Abort on error
	set -e

	suite=stable