# What is Mixture of Experts (MoE)? — QANATONE > MoE splits a model into many expert sub-networks and runs only the relevant ones at each step, making large models cheaper to run. Kaynak: https://www.qanatone.com/en/nedir/moe-nedir/ --- [Explainers](https://www.qanatone.com/en/nedir/) # What is Mixture of Experts (MoE)? 15 Sep 2026 2 min read Model Type · MoE MoE splits a model into many expert sub-networks and runs only the relevant ones at each step, making large models cheaper to run. ## What is MoE? Mixture of Experts is an architecture that splits a model into many "expert" sub-networks instead of one big block. For each input a router decides which experts should run. All experts live inside the model, but only a few work at each step. Think of a large hospital. Patients do not see every doctor; the front desk sends each one to two or three specialists based on their complaint. The hospital may employ hundreds of doctors, yet the time spent on any single patient stays limited. ## How does it work? In language models routing happens at the [token](https://www.qanatone.com/en/nedir/token-nedir/) level: for every token the router scores the experts and picks the best few. Mistral's open source Mixtral 8x7B, for example, has eight experts per layer and runs two of them for each token. Experts are not divided into human readable topics such as "history" or "code". They learn during training which kinds of patterns they handle well. ## What is the advantage? The model's total capacity (parameter count) can be huge, while the computation for each token covers only the active experts. That makes it possible to run larger, more capable models on the same budget, with faster and cheaper [inference](https://www.qanatone.com/en/nedir/inference-nedir/). Many models from DeepSeek, Mistral and Alibaba's Qwen series use MoE, and Google has said Gemini 1.5 is built on an MoE architecture. ## Drawbacks Inactive experts still have to sit in memory, so MoE models are cheap in compute but heavy in memory. They are also harder to train: the router can overload some experts and leave others idle, and extra techniques are needed to balance the load. Even so, a large share of the biggest models are moving toward this architecture. ## More articles [Architecture · Orchestration ### What is orchestration in AI systems? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/orkestrasyon-nedir/)[Model Type · SAM ### What is the Segment Anything Model (SAM)? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/sam-nedir/)[Model Type · SLM ### What is a small language model (SLM)? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/slm-nedir/)[Marketing · Social Media ### What is social media marketing? A strategy for 2026 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/sosyal-medya-pazarlamasi-nedir/)[Prompting · Basics ### What is a prompt and how do you write a good one? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/prompt-nedir/)[Prompting · System Prompt ### What is a system prompt and how is it different? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/system-prompt-nedir/)[Model Settings · Temperature ### What is temperature in AI models? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/temperature-nedir/)[Core Concept · Token ### What is a token in AI and how is it counted? 15 Sep 2026 · 2 min](https://www.qanatone.com/en/nedir/token-nedir/)