- Request access to gated repositories you might use in advance: Various Gemma & Llama model generations etc.
- RunPod:
- Create a >= 100 GB network volume because you can't expand it later (copying to a new one is a hassle)
- Be aware which storage is persistant (the network volume) and which isn't (everything else, incl. home directory)
- Push (almost) everything to git repositories to avoid data loss
- Remember to turn off your devboxes
- I use the
runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04image
- Consider using my runpod_cli package to automate the setup + automatically shut down pods
- A couple of TransformerLens reminders (as discussed in the call):
- https://transformerlensorg.github.io/TransformerLens/generated/model_properties_table.html
os.environ["HF_HOME"] = "/workspace/hf_home"model = HookedTransformer.from_pretrained_no_processing("Qwen/Qwen2.5-32B-Instruct")from transformer_lens.utils import test_prompttest_prompt("The longest river in the world is the", " Amazon", model)- Use context manager for hooks
Last active
September 30, 2025 14:31
-
-
Save Stefan-Heimersheim/0b4daac9de94777492cbed243a910ce9 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Make sure to add your SSH key to your RunPod account | |
| echo "export HF_HOME=/workspace/hf_home/" >> ~/.bashrc | |
| apt-get update | |
| apt-get upgrade -y | |
| apt-get install -y sudo git vim ssh net-tools htop curl zip unzip tmux rsync libopenmpi-dev iputils-ping make fzf restic ripgrep wget pandoc poppler-utils pigz bzip2 nano locales | |
| pip install uv | |
| # Alternative: curl -LsSf https://astral.sh/uv/install.sh | sh | |
| uv pip install --system --compile-bytecode ipykernel kaleido nbformat numpy scipy scikit-learn scikit-image transformers datasets torchvision pandas matplotlib seaborn plotly jaxtyping einops tqdm ruff basedpyright umap-learn ipywidgets virtualenv pytest git+https://github.com/callummcdougall/eindex.git transformer_lens nnsight | |
| apt-get install -y libnss3 libatk-bridge2.0-0 libcups2 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libgbm1 libxkbcommon0 libpango-1.0-0 libcairo2 libasound2 | |
| # Optional: | |
| python_version=$(python --version | cut -d' ' -f2 | cut -d'.' -f1-2) | |
| uv venv ~/.venv --python $python_version --system-site-packages |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "remote.SSH.defaultExtensions": [ | |
| "ms-python.python", // Python extension | |
| "ms-toolsai.jupyter", // Jupyter notebook support | |
| "ms-python.debugpy", // Debugger for Python | |
| "anysphere.cursorpyright", // Cursor Pyright extension | |
| "charliermarsh.ruff", // Ruff linter for Python | |
| "redhat.vscode-yaml", // YAML support | |
| ], | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment