In the previous post we built our linguistic onion - five layers, from phonetics to pragmatics, and we saw how an LLM handles each of them. But after writing that post, one big "but..." lingered in my head.
\nBecause - does an LLM even "understand" what it generates? Does it have some internal model of the world? Does it think?
\nAnd then I stumbled into semiotics. And it turns out that semiotics gives us a brilliant frame for thinking about LLMs - not as an artificial mind, but as a machine of signs. And suddenly everything starts to make sense. Or at least it makes more sense than before ;-)
\nThis is the second post in the "Understanding LLM" series. Today we shift perspective: instead of looking at the layers of language, we look at the very nature of what signs are and how meaning comes into being at all. And why that is key to understanding what an LLM is - and what it is not.
\nBefore we get to LLMs, we need to nail down the basics. Because "semiotics" is one of those words that sounds smart, but what does it actually mean?
\nSemiotics is the study of signs and of how signs create meaning. That's it. Doesn't sound so scary anymore, right? ;-)
\nAnd a "sign" in semiotics is anything that means something. Something that stands for something else. Simple examples:
\nEach of these signs represents something else. And that "representing" is exactly what semiotics studies.
\nTip
\nExperiment: Look around you - how many signs do you see right now? At this moment I see: a WiFi icon (I have internet), a notification on my phone (someone texted), a logo on my coffee mug (a brand). Three signs and I didn't even get up from my chair.
\nIn the previous post we had semantics - the study of the meaning of words and sentences. So how does semiotics differ?
\nIn short:
\n| \n | What it asks | \nExample | \n
|---|---|---|
| Semantics | \n"What does this mean?" | \nWhat does the word "zamek" mean? | \n
| Semiotics | \n"How does this sign even work?" | \nHow is it that a red light MEANS "stop"? | \n
Semantics asks about specific meanings. Semiotics asks about the mechanism of meaning - how it is that anything means anything at all.
\nAnd that is exactly why semiotics is so important for understanding LLMs. Because the question is not "what does an LLM mean", but "how an LLM operates on signs".
\nIn semiotics there are two main traditions you need to know. Two approaches, two ways of thinking about signs. And note - both are important for understanding LLMs, but each from a different angle.
\nSaussure (a Swiss linguist, who lived at the turn of the 19th and 20th centuries) said: a sign consists of two parts.
\n\nAnd the key thing: the relationship between signifier and signified is arbitrary. There is no logical reason why the sequence of letters "c-a-t" means that particular animal. It just... caught on. In Polish it's "kot", in German "Katze", in Japanese "猫" (neko) - every language has a different sequence of sounds/letters for the same concept.
\nBut Saussure says something even more important: the meaning of a word arises from its relationship to other words in the system. "Cat" means what it means because it is NOT "dog", it is NOT "house", it is NOT "car". Meaning is differential - it arises from difference.
\n\n\nThis sounds abstract, but in a moment you'll see that this is exactly what embeddings do in an LLM. Really ;-)
\n
Peirce (an American philosopher, a bit earlier than Saussure, but roughly around the same time) had a different approach. For him a sign is not a static pair, but a dynamic process.
\n\nPeirce's triad:
\nAnd that is the key difference: for Saussure a sign is static (a pair), for Peirce it is dynamic, alive, a process. Meaning is not "contained" in the sign - it arises in the process of interpretation.
\nPeirce divided signs into three categories that are super intuitive:
\n| Type | \nDescription | \nExamples | \n
|---|---|---|
| Icon | \nA resemblance between the sign and the object | \nA portrait, a map, the emoji 😺 (looks a bit like a cat), a folder icon on a computer | \n
| Index | \nA causal or physical connection | \nFootprints in the sand (someone walked here), smoke (fire), a cough (illness), a thermometer (temperature) | \n
| Symbol | \nA convention, a social agreement | \nThe word "cat", a national flag, a red light = stop, the mathematical "=" | \n
Note
\nThe word "symbol" in everyday language means something different than in Peirce's semiotics! In semiotics a symbol is a sign based purely on convention - it doesn't resemble the object (like an icon) and isn't physically tied to it (like an index). The Polish flag doesn't "resemble" Poland and isn't physically connected to it - we simply agreed that these colors stand for that country.
\nTest yourself - what type of sign is this?1
\nDavid Manheim, an AI researcher, used a beautiful metaphor coming from Peirce's semiotics. He called it the "Hall of Mirrors Problem".
\nImagine: you're in a room full of mirrors. You see reflections of reflections of reflections... and there's no window to the outside anywhere. You don't see the real world - you see only... more mirrors.
\nThat is exactly what an LLM does.
\n\nLet's look at this through the lens of Peirce's triad:
\nAn LLM has never seen the world. It has never tasted a strawberry, never touched ice, never heard laughter. It draws all its "knowledge" about the world from text - from what other people wrote about the world.
\nSo when an LLM writes "strawberries are sweet" - it doesn't know they are sweet. It knows that in the texts it was trained on, the word "strawberries" often appears near the word "sweet". That is the difference. And that is precisely the semiotic difference.
\nWarning
\nParadox: An LLM can write a beautiful description of a sunset, even though it has never seen the sun. But it can also write a beautiful description of a sunset on Mars - even though nobody has ever seen a sunset there yet. How does it "know"? From science fiction text. So: signs refer to signs, which refer to signs... a hall of mirrors ;-)
\nOK, now what I promised - let's see how Saussure's theory plays out in code. Because this is genuinely fascinating.
\nRemember what Saussure said? The meaning of a word arises from its relationship to other words. "Cat" means what it means because it isn't "dog", it isn't "house", etc. Meaning is relational.
\nNow think about word embeddings - which we met in the previous post. Each word is represented as a vector in a high-dimensional space. And words with similar meanings are close to each other in that space.
\nThis is exactly Saussure's relational theory of the sign, just implemented in math!
\nfrom gensim .downloader import load \n\nmodel = load ("glove-wiki-gigaword-50" )\n\nking = model ["king" ]\nqueen = model ["queen" ]\nman = model ["man" ]\nwoman = model ["woman" ]\n\nresult = king - man + woman \n\nfrom gensim .models import KeyedVectors \nsimilarities = model .cosine_similarities (result , [queen ])\nprint (f"Similarity to 'queen': { similarities [ 0 ]:.3f } " )It will print something like: Similarity to 'queen': 0.850
But look at this from Saussure's perspective:
\n\nBoth Saussure and Word2Vec say the same thing: meaning is not in the sign itself - it's in its relation to other signs. Saussure came up with this as a theory of language. Engineers at Google came up with Word2Vec as an algorithm. And they reached the same conclusion.
\nWell... almost. Because there's one catch. Saussure assumed that behind the signifier (form) stands the signified (concept). In embeddings we only have a position in space - we have relations, but do we have a "concept"? Is the vector [0.50, 0.68, ...] the concept of "king"?
And that is exactly the question that leads us to the next semiotician...
\nJacques Derrida (a French philosopher, 1960s and 70s) did something bold. He looked at all of Saussure and said: "Wait. And why do you assume speech is more important than writing?"
\nSaussure (and the entire Western philosophical tradition) treated speech as "primary" - closer to thought, closer to meaning. Writing was "derivative" - merely a record of speech, a "sign of a sign". Derrida called this logocentrism - the belief that at the end of the chain of signs there is some "presence", some "thought", some "intention" that bestows meaning.
\nAnd Derrida turned this upside down: writing is not subordinate to speech. Writing is a system in its own right.
\nWhy does this matter for LLMs? Because Elad Vromen, in his paper "Language Models as Semiotic Machines", noticed something brilliant:
\n\n\n\nAn LLM trains on writing (text). It builds a model of writing. It generates new writing. Nowhere in this process is there "speech", "mind", or "intention". Saussure's entire hierarchy - speech > writing - gets inverted. Writing is the only reality an LLM knows.
\n
So when we ask "does an LLM understand language?" - we're asking a bad question. It's like asking "does a book understand what is written in it?" A book doesn't "understand" - but it contains signs that we - the readers - interpret. An LLM is something in between: it isn't a book (because it generates new text), but it isn't a mind either (because it has no access to meanings outside of text).
\nImportant
\nDerrida in a nutshell for LLMs: An LLM doesn't model a "mind" or a "world". An LLM models writing - a system of signs that has its own logic, its own rules, its own coherence. And that writing is enough to generate text that means something to us. But it is us who give it meaning - not the model.
\nThis also explains a phenomenon you've probably noticed: an LLM sometimes says things that are statistically correct, but nonsensical. Because in the system of writing the model operates in, those words fit together well. But we - as humans with access to the world (to objects in Peirce's sense) - see that it makes no sense. The model lacks that "grounding" in reality.
\nDerrida, in his famous essay "Signature Event Context", spoke of the iterability of signs - the fact that a sign can be repeated in a new context and take on new meaning. The word "good" can be a compliment, irony, or part of "good evening" - context changes everything.
\nAnd that is exactly what we see in LLMs: the same prompt in a different context yields a different answer. The model doesn't "understand" context - but statistically it picks up contextual patterns from the training data. So: iterability of signs in its purest, mathematical form.
\nNow we move onto very practical ground. Because if an LLM is a machine of signs, then a prompt - what you type into it - is a semiotic act. Not simply "a command". But an act that creates the frame for meaning.
\nLet's look at the interaction with an LLM through the lens of Peirce's triad:
\n\nSo:
\nNotice: meaning arises only in step 3. The LLM generates a sequence of tokens, but it's you who gives it sense. This is exactly what Umberto Eco meant with his concept of the "open work" - a text doesn't have one fixed meaning. A text is a "frame" that the reader fills with interpretation.
\nTry it yourself. Fire up ChatGPT (or Claude, Gemini - whatever you have) and send these three prompts, each in a new conversation:
\n"Explain what gravity is.""Explain gravity to a five-year-old.""Explain gravity in the style of a Shakespearean sonnet."Each prompt is about the same topic (gravity). But each one changes the semiotic frame - it shifts the tone, the register, the genre, the expected form of the answer.
\n| Prompt | \nWhat changes... | \nSemiotic frame | \n
|---|---|---|
| "Explain gravity" | \n- | \nNeutral, encyclopedic | \n
| "...to a five-year-old" | \nThe audience, the simplicity of language | \nEducational, age-appropriate | \n
| "...in the style of Shakespeare" | \nThe genre, the form, the style | \nLiterary, artistic | \n
This is exactly what Picca calls the "semiotic contract". When you write a prompt, you don't "ask for information" - you set the conditions under which meaning will be constructed. You ask for gravity in Shakespeare mode? You get a hybrid of physics and poetry. That's neither "real" gravity nor "real" Shakespeare - it's a semiotic collage, a new configuration of signs.
\nTip
\nBonus experiment: Try this: "Explain the concept of entropy using metaphors from fairy tales." You'll see how an LLM connects two completely different semiotic zones - physics and fairy tales. That is exactly what semiotics calls translation between cultural codes.
\nOne more concept worth knowing. Juri Lotman, a Russian semiotician, came up with the concept of the semiosphere.
\nThe semiosphere is the space in which signs live. It's an ecology of meanings - a network of cultural codes, genres, discourses, ideologies that interact with one another. Just as the biosphere is the space in which organisms live, the semiosphere is the space in which signs live.
\n\nAn LLM's training data is nothing else but a giant cross-section of the semiosphere. When the model reads Wikipedia, Twitter, books, scientific papers, source code - it absorbs (statistically!) all that diversity of cultural codes.
\nAnd that's why it can write in the style of Shakespeare, translate from German, joke like a comedian, and quote legal regulations - because all of that lives in the semiosphere, and the model has "seen" samples from every zone.
\nBut there's a flip side: the model also absorbs the prejudices, stereotypes, and dominant narratives contained in the semiosphere. Because the semiosphere isn't neutral - it's a cultural space with a history, with power, with ideology. And the LLM, operating on signs from that space, reproduces them in its outputs.
\nWarning
\nWhy does an LLM sometimes talk nonsense? Because the semiosphere is full of contradictions. On one page of the internet "the earth is round", on another "the earth is flat". The model sees both signs and has no access to the "object" (the actual earth) to decide which sign is true. It operates in a hall of mirrors - signs refer to signs, not to reality.
\nTest which semiotician better explains these LLM phenomena. A reminder:
\nWho better explains the fact that...?2
\nHere's our semiotic map in a nutshell:
\n| Semiotician | \nKey concept | \nWhat it tells us about LLMs | \n
|---|---|---|
| Saussure | \nMeaning = relation between signs | \nEmbeddings realize the relational concept of the sign | \n
| Peirce | \nTriad sign-object-interpretant | \nAn LLM has no access to the object, only to signs (hall of mirrors) | \n
| Derrida | \nWriting as a system in itself | \nAn LLM models writing, not mind; training data = the only source | \n
| Lotman | \nSemiosphere - the ecology of signs | \nTraining data is a cross-section of the semiosphere; an LLM navigates its zones | \n
| Eco | \nThe open work | \nAn LLM's output has no single meaning - it requires your interpretation | \n
So: an LLM has no mind. But that doesn't mean it does nothing interesting. An LLM operates on signs - it reconfigures them, connects zones of the semiosphere, creates new configurations of signs. And those new configurations mean something to us - as interpreters.
\nThat is the semiotic answer to "does an LLM understand?". An LLM doesn't "understand" in the human sense. But it generates signs that enter our semiosphere and become part of our process of interpretation. And that process is real, important, and powerful.
\nI know semiotics sounds at first like something from another planet, but I hope you now see how it connects to LLMs.
\nWhich semiotic concept do you find most useful for understanding LLMs? Saussure and his relationality? Peirce and his hall of mirrors? Or maybe Derrida and his "writing"?
\nBecause frankly - I'm still learning this perspective myself. But I feel this is the moment where linguistics, philosophy, and programming meet in one place and create something genuinely fascinating.
\n\n\nWhat's in the next post? We move from theory to practice: How a computer reads text - from counting words to vectors. We'll see how tokenization, TF-IDF, Markov chains, and Word2Vec realize in code what Saussure (meaning is relational!) and Derrida (writing is a system in its own right!) were talking about.
\n
See you next time!
\nSources and interesting links:
\nIf you want to go deeper, here are the materials I used:
\nAnswers: 1) index (causal - temperature causes the liquid to expand), 2) icon (a resemblance to the dog), 3) symbol (pure convention), 4) index (a physical trace of someone who walked here), 5) icon (a resemblance to a person in a wheelchair). ↩
\nMy answers: 1) Peirce - the model has no access to the object (the experience of poetry), but it generates representamens (text) that we interpret. 2) Saussure - this is pure relationality! A king is defined by what it is not. 3) Peirce - context changes interpretation, and the interpretant creates a new sign. 4) Peirce - the absence of the object in the triad. 5) Peirce + Lotman - the model reproduces signs from the medical zone of the semiosphere without access to the real object (the patient's body). The diagnosis is a sign without grounding. ↩
\n