Meet Patel MeetThePatel

MeetThePatel / REFERENCE+AMSGrad+Power=0_9.txt

Created June 25, 2025 23:09

Reference vs. Reference+AMSGrad+GradPower(p=0.9)

	import os
	import sys

	with open(sys.argv[0]) as f:
	code = f.read() # read the code of this file ASAP, for logging
	import uuid
	import time
	import copy
	import glob
	from dataclasses import dataclass

MeetThePatel / d16d3609-21cb-4f29-852f-04b3994ea7e8.txt

Created June 20, 2025 20:29

Testing GALA (adapted to Muon and Adam) for NanoGPT speedrun

MeetThePatel / benchmark.py

Last active May 20, 2025 16:15

FMA in Adam(fused=True) benchmarks

	import torch
	import torch.nn as nn
	from torch.utils.benchmark import Timer
	from torch.optim import Adam
	import subprocess

	DEVICE = "cuda"


	def print_git_info():

MeetThePatel / test_correctness.py

Last active May 13, 2025 17:30

Adagrad(fused=True) benchmark script

	import torch
	import torch.nn as nn
	import torch.nn.functional as F


	class TinyModel(nn.Module):
	def __init__(self, D=128):
	super().__init__()
	self.net = nn.Sequential(nn.Linear(D, D), nn.ReLU(), nn.Linear(D, D))