Skip to Content

A new AI model every few days: what really changes?

From parameters to reasoning: how language models work, what changes with each release, and how to choose the right one for your business.

In recent weeks, AI labs have announced a steady stream of new models, often billed as the most capable ever, yet business leaders still struggle to separate genuine breakthroughs from routine updates.

Animation: a timeline from mid-October to mid-November 2026 showing the successive arrivals, on their respective dates, of DeepSeek-V4.1-Flash-1017, Kimi K3.1 Pro, GPT-6.1 Terra, Gemini 4 Carbon, Mistral Large 4, Claude Fable 5.2, and Qwen4-Max.

A selection of the models announced between mid-October and November 16, 2026.

The pace of releases is dizzying. Between mid-October and early November 2026, DeepSeek-V4.1-Flash-1017, Kimi K3.1 Pro, GPT-6.1 Terra, Gemini 4 Carbon, and the open weights of Mistral Large 4 arrived, followed by Claude Fable 5.2 and, on November 16, 2026, Alibaba's Qwen4-Max. To read the headlines, you would think there was a revolution every other day. Yet Stanford University's 2026 AI Index counts 102 notable models in 2025[1], selected by Epoch AI for their historical or technical significance—fewer than the year before.

The paradox emerges when looking at the specifications of the latest systems. Take Qwen4-Max: its overall structure has 2.4 trillion parameters, the same size as its predecessor, Qwen3.8-Max, released on August 3, 2026. Yet according to Alibaba, the new model activates 71 billion parameters per token rather than 95, while on the internal Work360 benchmark, the average cost to complete a complex task dropped from $4.43 to $1.67[2].

How can two models with the same total number of parameters have such different compute requirements and costs? Gross size alone explains almost nothing. To understand what really changes from one version to the next, you have to look under the hood and follow the steps that turn numbers into answers.

Contents

  1. Parameters: billions of numbers to avoid writing rules by hand
  2. Predicting the next word: how probability works
  3. The Transformer and attention: connecting concepts without losing track
  4. How a model learns: from gradient descent to reasoning
  5. What really changes from one model to the next
  6. How to read the next announcement without getting dazzled
  7. How we do it
  8. What matters beyond the pace of releases

Parameters: billions of numbers to avoid writing rules by hand

No programmer could hand-code every rule of a language: the exceptions would overwhelm any software. To get around this obstacle, engineers use neural networks—software structures composed of interconnected elementary units.

How a single unit works is straightforward: it takes incoming numbers, multiplies each by a value that determines its importance—called a weight or parameter, much like a knob adjusting connection strength—sums the results, and decides how much of the total to pass along. These units are arranged in layers, or successive stages of processing, where each performs its calculations and passes the signal to the next.

In today's systems, the number of these knobs is staggering. In Qwen3.8, an open-weight model (one anyone can download), the weights published by Alibaba amount to exactly 2,446,182,725,504 numbers, stored across 213 separate files; the network is structured into 92 layers, through which each individual fragment of text travels as a list of 8,192 numbers[3]. Inside, there are no plain-text sentences or facts: only numerical series capturing statistical correlations.

Animation: a neural network unit receives three signals, each passing through a knob that adjusts its weight; the unit sums them and determines how much to pass forward; beside it, many units arranged in layers form a network. Below, the number of parameters in the published weights of Qwen3.8: 2,446,182,725,504.

Illustrative example of a single unit and a small network; the figure reflects the published weights of Qwen3.8.

If the software had to turn every knob for every word, computing and electricity costs would be unsustainable. To solve this problem, engineers use the Mixture-of-Experts (MoE) architecture, an idea that a Google Brain team brought to scale in 2017. Instead of running the entire network on every pass, parameters are divided into independent groups governed by a router, a fast dispatch algorithm. In Qwen3.8, there are 512 total experts: for each token, the router activates only 10 plus one shared, putting 95 billion parameters to work out of the 2.4 trillion total. But what exactly does this machine calculate, word by word?

Animation: a grid of 512 experts; a token reaches the router, which sends it to 10 experts and to one shared expert, while the others remain idle: 95 billion parameters at work out of 2.4 trillion.

Qwen3.8's Mixture-of-Experts: 512 experts, with 10 plus one shared expert at work for each token. Which ones light up is an example.

Predicting the next word: how probability works

At its core, a language model performs a single task repeatedly: estimating what should follow a given piece of text. Documents are broken down into tokens, or text fragments: for Google's models, a token is about four characters, and 100 tokens correspond to about 60-80 English words. The full inventory of fragments forms the model's vocabulary, which in Qwen3.8 spans 248,320 tokens.

At each step, the network assigns a probability score to every token in its vocabulary, summing to 100%. In Google's documentation, for the English prompt “When I hear rain on my roof, I _______ in my kitchen”, a model assigns 9.4% to “cook soup”, 5.2% to “warm up a kettle”, and 2.5% to “nap”[4].

If the algorithm always picked the single most probable option, responses would be flat and repetitive. To prevent this, generation relies on sampling guided by temperature: at zero, the model always selects the top choice; as temperature increases, selection opens up to less frequent alternatives. The top-k and top-p parameters prune the long tail of unlikely words: for Qwen3.8, its developers recommend a temperature of 1, top-k at 20, and top-p at 0.95. A response is a rapid sequence of hundreds of these draws.

Animation: for the phrase “The technician restarted the...”, the model assigns a probability to each candidate word (server, computer, router, system, phone), shown as a bar; at low temperature, the most likely option almost always wins; at high temperature, the bars even out; “server” is then sampled, filling the slot, and a new slot appears.

Illustrative example: next-token probabilities, how temperature alters them, and sampling.

To make coherent choices, the system must hold earlier text in its context window—its working short-term memory. Up from roughly 8,000 tokens in older systems, the latest models, such as Qwen4-Max, reach 1 million tokens: about 50,000 lines of code or 8 full-length novels. But how does the network know which past words actually matter for the next one?

The Transformer and attention: connecting concepts without losing track

Take an everyday office sentence: “The customer called the technician because his computer wouldn't start.” To a human, it is obvious that “his” refers to the customer, but for a machine, resolving that ambiguity requires measuring the logical relationship between distant words. Before the Transformer, many models processed text sequentially, word by word, making parallel computation difficult.

The answer came in 2017 with the paper “Attention Is All You Need” by Google researchers, which introduced the Transformer architecture built on the attention mechanism. Each token asks a question of all the other elements in the text; each element presents a label and some content: the better the label answers the question, the more of that content is taken in. This comparison happens across multiple parallel channels, known as attention heads, each free to track a different relationship.

Animation: in the sentence “The customer called the technician because his computer wouldn't start”, the word “his” looks back at all preceding words with thin arcs, then draws heavily from “customer” (the thickest arc), slightly from “technician”, and almost nothing from the rest.

Illustrative example: attention links “his” to the word that matters, “customer”.

Because it processes all tokens simultaneously, the Transformer trains rapidly on GPUs (graphics processors specialized for parallel computing): the big model described in the original paper was trained in just 3.5 days on 8 GPUs[5].

One challenge remains: comparing every token with every other causes compute requirements to scale with the square of sequence length. Running full attention across million-token contexts would be extremely expensive. That is why several recent models combine different layer types: in Qwen4, for every full attention layer there are three lighter layers that compress earlier history into fixed-size states, while the main attention layer steps in selectively only on portions of context flagged as relevant. But who determines the values of all these parameters?

How a model learns: from gradient descent to reasoning

At the start of training, all the knobs are set to random values. During pre-training (the initial training phase on massive amounts of raw data), the network reads real-world text; for each token, the system measures the gap between the prediction and the actual word, known as the loss (the error metric). For each parameter, it calculates the slope, known as the gradient, which indicates whether nudging the knob up or down increases or decreases the error. Each knob is then turned a tiny step in the direction that lowers the error, and the cycle is repeated an enormous number of times. It is like descending a mountain in fog, feeling the slope beneath your feet with every step.

Animation: loss represented as a valley varying with the setting of a knob; a ball starts at a random position, the slope beneath it indicates the downward direction, and it rolls down in small, decreasing steps until reaching the valley floor.

Illustrative example: gradient descent on a single knob.

The technique that calculates all these slopes across the entire network, working backward from output to input, is backpropagation, formalized by Rumelhart, Hinton, and Williams in Nature in 1986. This process is repeated on a massive scale: Qwen3 was pre-trained on 36 trillion tokens across 119 languages and dialects. On Qwen4-Max, the development team reports that during training on its 2.4 trillion parameters, the loss curve dropped smoothly and steadily, without spikes.

The scaling laws published in 2020 show that error drops predictably as parameters, data, and compute increase. In 2022, DeepMind showed with Chinchilla that data matters just as much as size: for the same compute budget, Chinchilla with 70 billion parameters on 1.4 trillion tokens outperformed Gopher, at 280 billion parameters on 300 billion tokens[6].

This growth faces two limits: according to Epoch AI, the compute used to train frontier models grows by a factor of about 5 a year, and a study by Cottier and colleagues estimates that if cost growth continues, the largest single training runs will exceed one billion dollars by 2027; furthermore, according to Epoch AI, the stock of high-quality human public text amounts to about 300 trillion tokens and, if trends continue, could be entirely exhausted between 2026 and 2032.

Crucially, a base network can only continue text. Turning it into a useful assistant requires post-training—the subsequent training phase that teaches the model how to converse and respond. One method is reinforcement learning from human feedback (RLHF), using human demonstrations and rankings by reviewers: in 2022, the 1.3-billion-parameter InstructGPT was preferred over the 175-billion-parameter GPT-3, despite having a hundred times fewer parameters. Another approach is reinforcement learning with objective verification: by rewarding the model only when the end result is correct according to rules or code, without the conventional human demonstration phase, DeepSeek-R1-Zero (published in Nature) showed how the network learns to think longer, boosting accuracy on AIME 2024 (a US math competition) from 15.6% to 77.9%[7].

What really changes from one model to the next

In vendor announcements there is almost never any new magic, just work on six specific levers.

The first lever is a new pre-trained base: a ground-up rebuild, which is both rare and expensive. In Qwen4-Max, according to internal benchmarks from the Qwen team, the new base model makes fewer errors: its loss on text is 5–13% lower across all evaluated domains, while activating 25% fewer parameters than the Qwen3.8-Max base.

The second lever is architecture: making experts work more efficiently lowers the marginal cost per token. Ho et al. estimated that between 2012 and 2023, algorithmic advances halved the compute needed to achieve a given performance level roughly every 8 months, even though the bulk of overall gains still came from scaling compute and data.

Bar chart: Qwen3.8-Max and Qwen4-Max both have 2.4 trillion parameters; 95 billion versus 71 billion are at work for each token; the average cost per task on the Work360 benchmark drops from $4.43 to $1.67.

Qwen3.8-Max and Qwen4-Max: same size, different bill, according to Alibaba.

The third lever is post-training, the process that refines an existing base model's behavior: in our view, it is the lever behind many of the releases that follow each other most closely. Updates bearing dates in their names, such as DeepSeek-V4.1-Flash-1017—described by DeepSeek as faster with shorter responses than the previous version—often leave unstated whether the base was retrained or merely fine-tuned. Even when the base changes, post-training carries significant weight: for Qwen3.8, developers attributed its long-horizon task capabilities primarily to scaling reinforcement learning environments.

The fourth lever is reasoning on demand (effort). Snell et al. found that on problems within reach of a smaller model, allocating more thinking time can enable it to outperform a system 14 times larger; however, intermediate reasoning steps come at a price, and Alibaba notes that on Work360 in High effort mode, Qwen4-Max uses about 35% fewer tokens than Qwen3.8-Max.

The fifth lever is distilled models: using the technique outlined by Hinton et al. in 2015, compact models learn from the probability distributions generated by a larger model (which is how Alibaba trained the lightweight versions of Qwen3), delivering fast responses at low cost, typically with a slight trade-off in accuracy.

The sixth lever is packaging: APIs, extended context windows, and pricing. Here the advantages are measurable: according to Artificial Analysis, GPT-6.1 Terra costs half as much or less per token compared to GPT-5.6 Terra; and the model itself, when it does not know the answer, guesses in 38% of cases, compared to 88% for its predecessor[8].

Technical progress is real: research organization METR found that the length of the software tasks frontier models can complete half the time, measured by how long they would take a person, has doubled roughly every seven months since 2019, and perhaps faster since 2024[9]. At the same time, the gaps have narrowed: on Arena, the leaderboard where people vote blind between two anonymous answers, the top four models in March 2026 were separated by just 25 Elo points, down from 97 points a year earlier[10]. When raw intelligence levels out, the choice has to look at costs and at fit with business processes.

How to read the next announcement without getting dazzled

When faced with a new product announcement, a company shouldn't ask which model looks smartest on marketing charts, but which one actually makes business sense to deploy. Here are seven practical steps to evaluate a release:

  1. New base model or touch-up. Check whether the announcement describes a ground-up pre-training run or simply refines existing production models.
  2. Who validated the numbers. Benchmarks saturate quickly: on GPQA Diamond (PhD-level science questions), average accuracy reached 93% in 2025, surpassing the 81.2% human expert baseline, while data contamination (test questions leaking into training datasets) can artificially inflate scores. Furthermore, the study The Leaderboard Illusion showed that some providers can test many private variants (27 in Meta's case before Llama 4) and choose which scores to make public[11].
  3. Total cost per task. Don't look solely at the price per million tokens: when reasoning is enabled, as in Gemini models, thinking tokens are billed alongside output tokens, making total cost per completed task the true metric.
  4. Reasoning effort. Match the reasoning level to the actual workload: as Anthropic's guidelines note, for most everyday tasks, maximum effort adds substantial cost for relatively modest gains in quality.
  5. Testing on your own use cases. Build an internal benchmark with a few dozen real business tasks; keeping prompts and tooling consistent, track quality, latency, token consumption, cost per task, and technical failures.
  6. Data governance and privacy. Clarify where servers are located and confirm contractually that your company data will not be used to train public models.
  7. Architectural flexibility. Keep proprietary business knowledge decoupled from any single AI vendor, allowing you to swap models as prices drop without re-architecting integrations.

How we do it

In the projects we run with companies, we apply this principle of modularity through our platform, Snake.

Snake is not locked into any single AI provider: it connects to the leading models—including Anthropic's Claude, OpenAI's GPT, and Google's Gemini—and we select the right engine for each specific project, balancing answer quality, cost per interaction, and data privacy requirements. The underlying engine can be swapped over time without rebuilding the project from scratch: “change the model, not your knowledge.”

To keep answers verifiable, Snake answers from company documents and always cites the document it draws on; if information is insufficient, it says so and hands the conversation over to a person. Documents remain on European infrastructure and are not used to train public models.

What matters beyond the pace of releases

New AI models will likely continue to drop every few days, but what actually changes is almost always how parameters are trained, selected, and billed. For business leaders, the priority is not chasing every headline, but measuring performance on internal documents and tracking real cost per completed task.

Want to know how we choose AI models for our clients? Let's talk.

Want to know more? Leave us your email below and we'll get back to you as soon as possible.

Send 

Thank you for your message

Our team will get back to you as soon as possible.
In the meantime, we invite you to visit our website.

Sources:

  1. Stanford HAI — “Artificial Intelligence Index Report 2026” – hai.stanford.edu
  2. Qwen Team — “Qwen4-Max: Your New Teammate Has Arrived” (November 16, 2026) – qwen.ai
  3. Hugging Face / Qwen Team — “Qwen3.8-2.4T-A95B” (August 8, 2026) – huggingface.co
  4. Google for Developers — “Introduction to Large Language Models” (January 9, 2026) – developers.google.com
  5. Google Research / arXiv — “Attention Is All You Need” (June 12, 2017) – arxiv.org
  6. DeepMind / arXiv — “Training Compute-Optimal Large Language Models” (March 29, 2022) – arxiv.org
  7. DeepSeek-AI / Nature — “DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning” (September 17, 2025) – nature.com
  8. Artificial Analysis — “OpenAI has released GPT-6.1 Terra, pushing the cost efficiency frontier” (October 26, 2026) – artificialanalysis.ai
  9. METR / arXiv — “Measuring AI Ability to Complete Long Software Tasks” (July 10, 2026) – arxiv.org
  10. Stanford HAI — “AI Index Report 2026, Chapter 2: Technical Performance” – hai.stanford.edu
  11. Cohere Labs / arXiv — “The Leaderboard Illusion” (April 29, 2025) – arxiv.org
Outlook down for hours: is the cloud still a good idea?
What happened to Microsoft 365, how a cloud service for millions of users works, how it is monitored, and why it is still worth it.