(I’ve been working on …)
Machine Learning
- Modded NanoGPT - an LLM speedrunning competition where I have set several records 1. I’ve written a corresponding blogpost on muon. My work here has been extensively tested and used in Andrej Karpathy’s Nanochat (>42k github stars).
- Prime Environments - creating Reinforcement Learning envs to test and train agents, including SWE Bench
- Spotify MCP Server - get your LLM to control spotify (>500 github stars!)
- Semantic Finder - vector search in your browser (>300 github stars!)
Graphics
- Theory of Human Tetrachromacy - Best Paper Honorable Mention at Siggraph ‘24
- Various experiments/simulations in C++: Raytracing I, II, 3D Mesh Engine
Footnotes
-
At the time of writing this, 8 records including: incorporating Flash Attention 3, adding a better matrix sign function, speeding up Muon, improving weight decay, adding batch size scheduling, and inventing an efficient form of multitoken prediction. ^