From simple neurons to memory - the evolution of language models
From a single neuron in 1958, through MLP and RNN with the forgetting problem, to LSTM with memory gates. The evolution of architectures that led to Transformers. read more →
How a computer reads text - from counting words to vectors
From tokenization through TF-IDF and Markov chains, to Word2Vec. How a computer turns text into numbers so that language models can process it. read more →
Semiotics - why an LLM doesn't "think", yet still means something
Saussure, Peirce and Derrida as the key to understanding LLMs. Why a model is not a mind, but a machine of signs - and why that is enough to generate meaningful text. read more →
Linguistic features - what you need to know before you understand how an LLM thinks
Five layers of language - phonetics, morphology, syntax, semantics, pragmatics - and how an LLM handles each of them. read more →