!til &reveng-ann As a sign-off, for the teaching portion decided to majorly revise the first part of my writings (on solving Caesar ciphers using a transformer) to ensure overall quality. Gave a project recap and had a fun time!

!til &reveng-ann I was coding along with a popular article on ChatGPT by Stephen Wolfram. It suggests that tiny neural nets can't reproduce a certain function (using automated training). By reverse-engineering how some larger nets solved it, I discovered that actually the tiny nets can reproduce the function, and arbitrarily well. This discovery will be the basis of my jam entry.

I'm starting a new low-level YT series called "Handcrafted Transformers." It introduces how Large Language Models like ChatGPT are built by coding them from scratch. What makes the series unique is that we find the neural net weights by hand (using math), rather than training them using GPUs/Calculus. https://youtu.be/PvutWdJnwJ8