Semiotics - why an LLM doesn't "think", yet still means something

In the previous post we built our linguistic onion - five layers, from phonetics to pragmatics, and we saw how an LLM handles each of them. But after writing that post, one big "but..." lingered in my head.

Because - does an LLM even "understand" what it generates? Does it have some internal model of the world? Does it think?

And then I stumbled into semiotics. And it turns out that semiotics gives us a brilliant frame for thinking about LLMs - not as an artificial mind, but as a machine of signs. And suddenly everything starts to make sense. Or at least it makes more sense than before ;-)

This is the second post in the "Understanding LLM" series. Today we shift perspective: instead of looking at the layers of language, we look at the very nature of what signs are and how meaning comes into being at all. And why that is key to understanding what an LLM is - and what it is not.

---

What is semiotics?

Before we get to LLMs, we need to nail down the basics. Because "semiotics" is one of those words that sounds smart, but what does it actually mean?

Semiotics is the study of signs and of how signs create meaning. That's it. Doesn't sound so scary anymore, right? ;-)

And a "sign" in semiotics is anything that means something. Something that stands for something else. Simple examples:

• 🔴 A red light at an intersection = STOP
• 😂 A tears-of-joy emoji = I'm laughing (or: I'm dying of laughter)
• 💨 The smell of smoke = a fire is somewhere nearby
• 🐾 Paw prints in the snow = a dog passed by here (or a wolf, or... better not think about it :D)

Each of these signs represents something else. And that "representing" is exactly what semiotics studies.

Semiotics vs semantics

In the previous post we had semantics - the study of the meaning of words and sentences. So how does semiotics differ?

In short:

	What it asks	Example
Semantics	"What does this mean?"	What does the word "zamek" mean?
Semiotics	"How does this sign even work?"	How is it that a red light MEANS "stop"?

Semantics asks about specific meanings. Semiotics asks about the mechanism of meaning - how it is that anything means anything at all.

And that is exactly why semiotics is so important for understanding LLMs. Because the question is not "what does an LLM mean", but "how an LLM operates on signs".

---

Two giants: Saussure vs Peirce

In semiotics there are two main traditions you need to know. Two approaches, two ways of thinking about signs. And note - both are important for understanding LLMs, but each from a different angle.

Ferdinand de Saussure: the sign as a pair

Saussure (a Swiss linguist, who lived at the turn of the 19th and 20th centuries) said: a sign consists of two parts.

• Signifier (signifiant) - the form of the sign. What you see, hear, touch. E.g. the sequence of letters "c-a-t" or the sound /kat/.
• Signified (signifie) - the concept, the idea that this form triggers in your head. E.g. a furry animal that meows and ignores you for most of the day :D

And the key thing: the relationship between signifier and signified is arbitrary. There is no logical reason why the sequence of letters "c-a-t" means that particular animal. It just... caught on. In Polish it's "kot", in German "Katze", in Japanese "猫" (neko) - every language has a different sequence of sounds/letters for the same concept.

But Saussure says something even more important: the meaning of a word arises from its relationship to other words in the system. "Cat" means what it means because it is NOT "dog", it is NOT "house", it is NOT "car". Meaning is differential - it arises from difference.

This sounds abstract, but in a moment you'll see that this is exactly what embeddings do in an LLM. Really ;-)

Charles Sanders Peirce: the sign as a process

Peirce (an American philosopher, a bit earlier than Saussure, but roughly around the same time) had a different approach. For him a sign is not a static pair, but a dynamic process.

Peirce's triad:

• Representamen - the sign itself, the form (equivalent of Saussure's "signifier")
• Object - what the sign refers to (a thing in the world, a concept)
• Interpretant - the effect the sign produces in the receiver's mind. And note: this interpretant ITSELF becomes a new sign, which again has its own interpretant, and so on... an infinite chain of interpretation.

And that is the key difference: for Saussure a sign is static (a pair), for Peirce it is dynamic, alive, a process. Meaning is not "contained" in the sign - it arises in the process of interpretation.

Three kinds of signs according to Peirce

Peirce divided signs into three categories that are super intuitive:

Type	Description	Examples
Icon	A resemblance between the sign and the object	A portrait, a map, the emoji 😺 (looks a bit like a cat), a folder icon on a computer
Index	A causal or physical connection	Footprints in the sand (someone walked here), smoke (fire), a cough (illness), a thermometer (temperature)
Symbol	A convention, a social agreement	The word "cat", a national flag, a red light = stop, the mathematical "="

Test yourself - what type of sign is this?¹

1. 🌡️ A thermometer showing 37°C
2. 📸 A photo of your dog
3. 🟢 A green light = "go"
4. 🐾 Boot prints in the snow
5. ♿ An accessibility icon

---

Hall of Mirrors - or, the LLM is stuck in mirrors

David Manheim, an AI researcher, used a beautiful metaphor coming from Peirce's semiotics. He called it the "Hall of Mirrors Problem".

Imagine: you're in a room full of mirrors. You see reflections of reflections of reflections... and there's no window to the outside anywhere. You don't see the real world - you see only... more mirrors.

That is exactly what an LLM does.

Let's look at this through the lens of Peirce's triad:

• Representamen (sign) - text, tokens, words - ✅ the LLM has access
• Object (reality) - the world, experience, physics - ❌ the LLM has no access
• Interpretant (interpretation) - understanding - ❓ the LLM generates something that looks like interpretation

An LLM has never seen the world. It has never tasted a strawberry, never touched ice, never heard laughter. It draws all its "knowledge" about the world from text - from what other people wrote about the world.

So when an LLM writes "strawberries are sweet" - it doesn't know they are sweet. It knows that in the texts it was trained on, the word "strawberries" often appears near the word "sweet". That is the difference. And that is precisely the semiotic difference.

---

Saussure in code: embeddings as a system of signs

OK, now what I promised - let's see how Saussure's theory plays out in code. Because this is genuinely fascinating.

Remember what Saussure said? The meaning of a word arises from its relationship to other words. "Cat" means what it means because it isn't "dog", it isn't "house", etc. Meaning is relational.

Now think about word embeddings - which we met in the previous post. Each word is represented as a vector in a high-dimensional space. And words with similar meanings are close to each other in that space.

This is exactly Saussure's relational theory of the sign, just implemented in math!

from gensim.downloader import load

model = load("glove-wiki-gigaword-50")

king = model["king"]
queen = model["queen"]
man = model["man"]
woman = model["woman"]

result = king - man + woman

from gensim.models import KeyedVectors
similarities = model.cosine_similarities(result, [queen])
print(f"Similarity to 'queen': {similarities[0]:.3f}")

It will print something like: Similarity to 'queen': 0.850

But look at this from Saussure's perspective:

Both Saussure and Word2Vec say the same thing: meaning is not in the sign itself - it's in its relation to other signs. Saussure came up with this as a theory of language. Engineers at Google came up with Word2Vec as an algorithm. And they reached the same conclusion.

Well... almost. Because there's one catch. Saussure assumed that behind the signifier (form) stands the signified (concept). In embeddings we only have a position in space - we have relations, but do we have a "concept"? Is the vector [0.50, 0.68, ...] the concept of "king"?

And that is exactly the question that leads us to the next semiotician...

---

Derrida: writing as the foundation

Jacques Derrida (a French philosopher, 1960s and 70s) did something bold. He looked at all of Saussure and said: "Wait. And why do you assume speech is more important than writing?"

Saussure (and the entire Western philosophical tradition) treated speech as "primary" - closer to thought, closer to meaning. Writing was "derivative" - merely a record of speech, a "sign of a sign". Derrida called this logocentrism - the belief that at the end of the chain of signs there is some "presence", some "thought", some "intention" that bestows meaning.

And Derrida turned this upside down: writing is not subordinate to speech. Writing is a system in its own right.

Why does this matter for LLMs? Because Elad Vromen, in his paper "Language Models as Semiotic Machines", noticed something brilliant:

An LLM trains on writing (text). It builds a model of writing. It generates new writing. Nowhere in this process is there "speech", "mind", or "intention". Saussure's entire hierarchy - speech > writing - gets inverted. Writing is the only reality an LLM knows.

So when we ask "does an LLM understand language?" - we're asking a bad question. It's like asking "does a book understand what is written in it?" A book doesn't "understand" - but it contains signs that we - the readers - interpret. An LLM is something in between: it isn't a book (because it generates new text), but it isn't a mind either (because it has no access to meanings outside of text).

This also explains a phenomenon you've probably noticed: an LLM sometimes says things that are statistically correct, but nonsensical. Because in the system of writing the model operates in, those words fit together well. But we - as humans with access to the world (to objects in Peirce's sense) - see that it makes no sense. The model lacks that "grounding" in reality.

For the curious: Derrida and iterability

Derrida, in his famous essay "Signature Event Context", spoke of the iterability of signs - the fact that a sign can be repeated in a new context and take on new meaning. The word "good" can be a compliment, irony, or part of "good evening" - context changes everything.

And that is exactly what we see in LLMs: the same prompt in a different context yields a different answer. The model doesn't "understand" context - but statistically it picks up contextual patterns from the training data. So: iterability of signs in its purest, mathematical form.

---

The prompt as a semiotic act

Now we move onto very practical ground. Because if an LLM is a machine of signs, then a prompt - what you type into it - is a semiotic act. Not simply "a command". But an act that creates the frame for meaning.

Peirce's triad in practice

Let's look at the interaction with an LLM through the lens of Peirce's triad:

So:

1. You create the prompt (representamen)
2. The LLM processes signs and generates an answer (a new representamen)
3. You interpret the answer (you become the interpretant)
4. Your interpretation leads to a new prompt... and the cycle repeats

Notice: meaning arises only in step 3. The LLM generates a sequence of tokens, but it's you who gives it sense. This is exactly what Umberto Eco meant with his concept of the "open work" - a text doesn't have one fixed meaning. A text is a "frame" that the reader fills with interpretation.

Experiment: how a prompt changes the semiotic frame

Try it yourself. Fire up ChatGPT (or Claude, Gemini - whatever you have) and send these three prompts, each in a new conversation:

1. "Explain what gravity is."
2. "Explain gravity to a five-year-old."
3. "Explain gravity in the style of a Shakespearean sonnet."

Each prompt is about the same topic (gravity). But each one changes the semiotic frame - it shifts the tone, the register, the genre, the expected form of the answer.

Prompt	What changes...	Semiotic frame
"Explain gravity"	-	Neutral, encyclopedic
"...to a five-year-old"	The audience, the simplicity of language	Educational, age-appropriate
"...in the style of Shakespeare"	The genre, the form, the style	Literary, artistic

This is exactly what Picca calls the "semiotic contract". When you write a prompt, you don't "ask for information" - you set the conditions under which meaning will be constructed. You ask for gravity in Shakespeare mode? You get a hybrid of physics and poetry. That's neither "real" gravity nor "real" Shakespeare - it's a semiotic collage, a new configuration of signs.

---

The semiosphere - the ecology of signs

One more concept worth knowing. Juri Lotman, a Russian semiotician, came up with the concept of the semiosphere.

The semiosphere is the space in which signs live. It's an ecology of meanings - a network of cultural codes, genres, discourses, ideologies that interact with one another. Just as the biosphere is the space in which organisms live, the semiosphere is the space in which signs live.

An LLM's training data is nothing else but a giant cross-section of the semiosphere. When the model reads Wikipedia, Twitter, books, scientific papers, source code - it absorbs (statistically!) all that diversity of cultural codes.

And that's why it can write in the style of Shakespeare, translate from German, joke like a comedian, and quote legal regulations - because all of that lives in the semiosphere, and the model has "seen" samples from every zone.

But there's a flip side: the model also absorbs the prejudices, stereotypes, and dominant narratives contained in the semiosphere. Because the semiosphere isn't neutral - it's a cultural space with a history, with power, with ideology. And the LLM, operating on signs from that space, reproduces them in its outputs.

---

Quiz: Saussure or Peirce?

Test which semiotician better explains these LLM phenomena. A reminder:

• Saussure: sign = a pair (signifier + signified), relational meaning, a static system
• Peirce: sign = a triad (representamen + object + interpretant), a process of interpretation, dynamics

Who better explains the fact that...?²

1. An LLM can write poetry, even though it has never felt poetry - Saussure or Peirce?
2. The embedding "king" - "man" + "woman" = "queen" - Saussure or Peirce?
3. The same prompt gives different answers depending on the context of the conversation - Saussure or Peirce?
4. An LLM has no access to the world, only to text - Saussure or Peirce?
5. We all know someone who asked ChatGPT for a medical diagnosis and got one - Saussure or Peirce?

---

Summary - a semiotic map of the LLM

Here's our semiotic map in a nutshell:

Semiotician	Key concept	What it tells us about LLMs
Saussure	Meaning = relation between signs	Embeddings realize the relational concept of the sign
Peirce	Triad sign-object-interpretant	An LLM has no access to the object, only to signs (hall of mirrors)
Derrida	Writing as a system in itself	An LLM models writing, not mind; training data = the only source
Lotman	Semiosphere - the ecology of signs	Training data is a cross-section of the semiosphere; an LLM navigates its zones
Eco	The open work	An LLM's output has no single meaning - it requires your interpretation

So: an LLM has no mind. But that doesn't mean it does nothing interesting. An LLM operates on signs - it reconfigures them, connects zones of the semiosphere, creates new configurations of signs. And those new configurations mean something to us - as interpreters.

That is the semiotic answer to "does an LLM understand?". An LLM doesn't "understand" in the human sense. But it generates signs that enter our semiosphere and become part of our process of interpretation. And that process is real, important, and powerful.

---

I know semiotics sounds at first like something from another planet, but I hope you now see how it connects to LLMs.

Which semiotic concept do you find most useful for understanding LLMs? Saussure and his relationality? Peirce and his hall of mirrors? Or maybe Derrida and his "writing"?

Because frankly - I'm still learning this perspective myself. But I feel this is the moment where linguistics, philosophy, and programming meet in one place and create something genuinely fascinating.

What's in the next post? We move from theory to practice: How a computer reads text - from counting words to vectors. We'll see how tokenization, TF-IDF, Markov chains, and Word2Vec realize in code what Saussure (meaning is relational!) and Derrida (writing is a system in its own right!) were talking about.

See you next time!

---

Sources and interesting links:

If you want to go deeper, here are the materials I used:

• Davide Picca, "Not Minds, but Signs: Reframing LLMs through Semiotics" - arXiv - the main inspiration for this post; a proposal to look at LLMs as semiotic machines
• Elad Vromen, "Language Models as Semiotic Machines" - arXiv - a brilliant connection of Saussure, Derrida, and embeddings
• David Manheim, "Language Models' Hall of Mirrors Problem" - PhilPapers - the source of the "hall of mirrors" metaphor
• "Semiotic reflections and modelling" - arXiv - prompts as semiotic phenomena
• "The meaning of prompts: a semiotic approach to human-LLM interaction" - Emerald - prompts as meaning-making acts
• "A Semiotic Perspective on Generative AI and LLMs" - TurtlesAI - an accessible introduction to the semiotics of generative AI
• "The Semiotic Perspectives of Peirce and Saussure: A Brief Comparative Study" - ScienceDirect - a comparison of the two main semiotic traditions
• Carlos E. Perez, "Semantics vs Semiotics: How LLMs Should Be Understood" - LinkedIn - a short post on why semiotics > semantics for LLMs

1. Answers: 1) index (causal - temperature causes the liquid to expand), 2) icon (a resemblance to the dog), 3) symbol (pure convention), 4) index (a physical trace of someone who walked here), 5) icon (a resemblance to a person in a wheelchair). ↩

2. My answers: 1) Peirce - the model has no access to the object (the experience of poetry), but it generates representamens (text) that we interpret. 2) Saussure - this is pure relationality! A king is defined by what it is not. 3) Peirce - context changes interpretation, and the interpretant creates a new sign. 4) Peirce - the absence of the object in the triad. 5) Peirce + Lotman - the model reproduces signs from the medical zone of the semiosphere without access to the real object (the patient's body). The diagnosis is a sign without grounding. ↩