What is a masked language model (MLM)?
An MLM is a language model trained by guessing hidden words. It understands text better than it writes it; BERT is the best known one.
What is an MLM?
A masked language model learns by hiding some words in sentences during training and asking the model to guess them. In "We had tea and [MASK] for breakfast", the model learns to fill the gap with a fitting word such as "toast" or "eggs".
The best known model of this kind is BERT, introduced by Google in 2018. BERT hides about 15 percent of the words during training.
How is it different from GPT style models?
The GPT style models behind ChatGPT read text left to right and predict the next word, which makes them good at generating text. An MLM looks at the words on both the left and the right of the gap when it guesses. That two way reading helps it grasp a word's meaning in context, but it is not suited to writing long, fluent text.
A GPT style model is like a writer continuing a story. An MLM is like a reader solving fill in the blank questions while seeing the whole sentence.
Where are MLMs used?
Today MLMs work behind the scenes rather than in the chat window:
- Search: measuring how closely a query and a document match in meaning. Google announced in 2019 that it had started using BERT to improve search results.
- Producing embeddings: many embedding models used in RAG systems come from the BERT family.
- Classification: spam detection, sentiment analysis, sorting customer requests by topic.
- Entity recognition: pulling people, companies, dates and amounts out of text.
Do they still matter?
Yes. Large generative models dominate the headlines, but MLMs are small, fast and cheap, so they remain widely used for understanding and classification. RoBERTa, DeBERTa and BERTurk, trained for Turkish, are well known members of the family.