CIRCLE WITH A DOT

@ambrosia_engine @totoroot @mitchellh And btw this is the whole (still minimized) GPT algorithm on one screen of python code by Karpathy: https://karpathy.ai/microgpt.html - stunning there is nearly everything that basic LLM needs (just without stolen data and optimizations, which actually made the crucial leap.)
Crazy that the world is upside
down because of something like this.

@ambrosia_engine @totoroot @mitchellh + I consider very useful to go through algorithms like Word2Vec which is highly comprehensible and de-mystifying in some way. Even it is not the same LLMs do, the intuition of word embeddings is there very clear. It can reveal a lot of “the basic math only” that is actually going on in it. This is a great article with informative visuals: https://jalammar.github.io/illustrated-word2vec/

@ambrosia_engine @totoroot @mitchellh Good point with the literacy. Even I am not in the industry I signed-up for two ML courses on my alma-mater at Masaryk University to better understand how it works. During courses I slowly lost any residual feeling of “magic” and “allmightyness” and stopped using LLM with exception for limited couching in coding which is my hobby (with a deliberately introduced friction and docs-checking). It helps when I can explain deeper insights to others.