Peiran Yao xavieryao

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

Upgrading to Lion or Yosemite and WebStorm 9, I noticed key repeat was
turned off for the IdeaVim plugin h j k l keys.

System-wide key repeat

defaults write -g ApplePressAndHoldEnabled -bool false in a terminal will enable
key repeat for every app. This can alternatively be found in the accessibility settings in OS X' preferences.

	# Open "Automator.app", add a new "Quick Action"

	# Workflow receives current "files or folders" in "Finder"
	#+ Run Shell Script
	#+ Shell: /bin/bash
	#+ Pass input: as arguments

	# Deployment: Keyboard / Shortcuts, bind 'Cmd-Shift-V'
	# Usage: copy something to clipboard, click on a folder, press 'Cmd-Shift-V'.
	# Contents will be pasted into 'pbpaste.txt'.

	import torch

	def jacobian(y, x, create_graph=False):
	jac = []
	flat_y = y.reshape(-1)
	grad_y = torch.zeros_like(flat_y)
	for i in range(len(flat_y)):
	grad_y[i] = 1.
	grad_x, = torch.autograd.grad(flat_y, x, grad_y, retain_graph=True, create_graph=create_graph)
	jac.append(grad_x.reshape(x.shape))

	from __future__ import print_function
	import torch
	import torch.nn as nn
	import torch.nn.functional as F
	from torch.autograd import Variable

	def sample_gumbel(shape, eps=1e-20):
	U = torch.rand(shape).cuda()
	return -Variable(torch.log(-torch.log(U + eps) + eps))

Peiran Yao xavieryao

Anti-hype LLM reading list

Foundational Concepts

Pre-Transformer Models

System-wide key repeat

App specific key repeat