What is a Large Concept Model (LCM)?
An LCM is an experimental AI model that works with whole sentences, or ideas, instead of individual words. Meta introduced the first one.
What is an LCM?
Today's large language models (LLMs) generate text one token at a time, predicting the next piece of a word at every step. People, when they speak, usually form the idea first and then choose the words. A Large Concept Model tries something closer to that second path: the unit it predicts is a "concept" that represents the meaning of a whole sentence.
Meta's research team introduced the approach in a paper published in December 2024.
How does it work?
- Text is split into sentences.
- Each sentence is turned into an embedding vector that carries its meaning. Meta used a system called SONAR, which represents many languages in the same meaning space.
- The model looks at the vectors of earlier sentences and predicts the vector of the next one.
- The predicted vector is turned back into a sentence in the desired language.
Because meaning is represented independently of language, the same model can work at the level of ideas across many languages.
How is it different from an LLM?
An analogy: an LLM is like someone typing a text one keystroke at a time. An LCM resembles a writer who first lines up the main ideas of a paragraph and then builds the sentences. The theoretical advantage is better coherence over long texts and reasoning that does not depend on a particular language.
LCMs are still at the research stage. Widely used chat assistants continue to run on token based LLMs.
Why follow it?
The LCM is one of the experiments showing that language models may be able to work at a more abstract level than word prediction. It is not something businesses will use directly in the short term, but it is a useful idea to watch for where AI models may be heading.