You Can Download It. You Just Can't Run It. · Issue #178

Opening

Reader, on July 17, Beijing-based Moonshot AI released Kimi K3. With 2.8 trillion parameters, it’s a model that calls itself the world’s largest open-weight¹ model. Anyone can still download it from Hugging Face. The button works. The problem comes after. The weight files alone exceed 1.5 terabytes, and there’s no way to run this model on a personal computer—or even on most corporate servers.

Let me give you the conclusion up front: the “open” in today’s frontier-grade open models isn’t the openness we used to know. The freedom to download belongs to everyone, but the ability to run it has been left to clouds, corporations, and states. The door is open, but the threshold has become a mountain. Today, let’s measure that gap in numbers.

The Map Redrawn in Two Weeks

Let’s start with the board. By the Intelligence Index from benchmark firm Artificial Analysis, the top open-weight models right now rank Kimi K3 (57), GLM-5.2 (51), and DeepSeek V4 Flash (50). Fourth and fifth place go to a smaller Kimi variant and MiniMax. That means all top five spots are Chinese models.

This leaderboard has been rewritten in just the past two weeks. Moonshot released Kimi K3 on July 17, and two days later Alibaba previewed the 2.4-trillion-parameter Qwen3.8-Max, claiming it trails only Fable 5. In between, OpenAI cut prices on its flagship model by 80%. Then on July 31, DeepSeek shipped the full release of V4 Flash, and on August 3, Alibaba shipped the full release of Qwen3.8-Max. The weights are slated for release next week.

It’s not just performance that’s climbing. The models are getting bigger. GLM-5.2 has 753 billion parameters, DeepSeek V4-Pro has 1.6 trillion, Qwen3.8-Max has 2.4 trillion, and Kimi K3 has 2.8 trillion. The reason for the growth is clear: it’s the cost of chasing the frontier. DeepSeek V4 Flash’s score of 50 trails OpenAI’s latest closed model by just one point, and Kimi K3 outright claims Fable/Sol-level intelligence. Squeezing out those last few points at the top demands this much scale. And the reason models can grow this large is that virtually every big model today uses a Mixture of Experts (MoE)² architecture. It’s a design that wakes only a fraction of the whole network to process each token. Of Qwen3.8-Max’s 2.4 trillion parameters, only 95 billion participate in any given computation; of DeepSeek V4 Flash’s 284 billion, only 13 billion do.

Here’s where the central asymmetry of this piece comes in. Compute cost tracks active parameters, but memory requirements track total parameters. Since you never know which dormant expert will be called next, every single one has to sit resident in memory. That’s why API pricing is dirt cheap (DeepSeek charges $0.28 per million output tokens), while running it at home requires enough graphics memory to hold all 2.8 trillion parameters. The reason the API is cheap and the reason you can’t run it at home come from the exact same design.

To put 1.5 terabytes in perspective: the top consumer graphics card, the RTX 5090, has 32 gigabytes of memory. You’d need to plug in 48 of them just to barely fit the weights alone—and that’s before even counting the KV cache³ for a 1-million-token context. Let’s also look at the pace on the other side. Consumer GPU memory grew from 24 gigabytes in 2020 to 32 gigabytes in 2025—a 33% increase over five years. Over roughly the same period, the largest open model went from Llama 405B in 2024 to 2.8T in 2026—a sevenfold jump in two years. The cake is growing far faster than the kitchen.

Even a GPU Specialist Picked Up a Pickaxe

So who’s actually eating this cake, and how? As it happens, a good real-world benchmark came out last week. Wafer, a startup that sells open-model inference on AMD chips, published its own record of actually serving Kimi K3. You have to read it knowing that AMD serving is literally their business, but the process is documented in such detail that this, itself, is the real payoff.

Let’s start with the numbers. Kimi K3 cannot fit both its weights and KV cache into a standard 8-GPU Nvidia B200 node (192 gigabytes per GPU, 1.5 terabytes total). So you either need the higher-end 288-gigabyte B300 chip, or you have to join two B200 nodes together. But there’s one more chip with 288 gigabytes of memory per GPU: AMD’s MI355X. In Wafer’s measurements, an 8-GPU MI355X node produced 952 tokens per second, and its throughput per dollar of GPU rental—48 tokens—beat both the B300 (33 tokens) and the dual-B200 setup (7 tokens). The moment the model grew large enough to shift the bottleneck from compute to memory capacity, the pecking order flipped.

Comparing this to a record from a month earlier makes the boundary vivid. When the same team served GLM-5.2 (753 billion parameters) in early July, the story was different. GLM-5.2 fits within a single Nvidia node, so even after quantization⁴ and framework surgery, AMD only managed to land as a value alternative: “80% of the performance at half the price.” Change one variable—whether the model crosses the 1.5-terabyte line—and the outcome flips entirely.

But what caught my attention in this benchmark wasn’t the ranking—it was the difficulty. Even a company whose entire business is GPU optimization hit a scheduler crash on the very first request, had to write a missing function in PyTorch by hand, and resorted to workarounds bypassing shape constraints in attention computation—before finally reaching target performance. And the team’s own assessment is that this time was actually the easy one. For prior models, they’d had to write custom kernels from scratch or fix two framework bugs at a time. There are weaknesses too. Loading a 170,000-token document for the first time takes 51 seconds—more than twice as slow as Nvidia B300’s 23 seconds. And even after clearing all of this, the node rental still runs about $20 an hour. Run it for a month and you’re at roughly ₩20,000,000—unmistakably a business-scale number.

Of course, there’s a flip side. DeepSeek V4 Flash is small by design at 284 billion total parameters, and its 3-bit quantized version comes in at 103 gigabytes—small enough to run on a high-end workstation with 128 gigabytes of RAM. Alibaba, too, has previewed a 27-billion-parameter compact version alongside Qwen3.8-Max. So “open that you can actually run” hasn’t disappeared. It’s just that it sits a rung below the frontier, and the frontier itself has left individual hands—that’s the accurate way to put it.

The Math Behind Giving It Away for Free, Anyway

This leaves one strange question. Why does everyone race to give away, for free, something most individual developers can’t even run?

Wall Street’s answer offers a clue. In an early-August Alibaba report, Citi framed it this way: since models get updated on a matter of weeks, companies are moving toward a “model-agnostic” strategy, swapping in whichever model fits each task. The more that happens, the weaker any individual model’s moat becomes, and the competitive focus shifts to the platform and infrastructure that host the model. That’s the logic behind the buy rating: it favors Alibaba, which owns everything from chips to cloud to models to apps. You should factor in that this is a sell-side report with its own incentives, but the direction lines up exactly with Alibaba’s actual behavior. The same day it promised to give away the weights for free, it also launched QwenWork, an enterprise agent product, bolted onto DingTalk. The model is the flyer; the actual product is the cloud and the workplace tools.

This also connects to the calculation from Liang Wenfeng, DeepSeek’s founder, that I quoted in last month’s issue on DeepSeek’s funding round.

“If you want to make 100x profit, open source gets in the way—but if you’re satisfied with a reasonable profit, it makes no difference at all.”

So the real buyers of frontier-grade open weights were never individuals in the first place. They’re clouds that sell it bolted onto their own infrastructure, companies that can’t let data leave their walls, governments that want intelligence kept within their borders, and companies like Wafer that turn serving itself into a business. Corporate demand, in particular, is concrete. For industries like finance, healthcare, and defense—where data can’t be sent to an external API—weights are intelligence you can bring inside your own perimeter, along with the freedom to fine-tune on your own data and the assurance that the model won’t suddenly change under you one day. Open weights are exactly what tops the ingredient list when governments talk about “sovereign AI.” In the end, releasing the weights is closer to outsourcing distribution to these players.

There’s one more party quietly smiling within this distribution network. The moment the model-size race translates into a memory-capacity race, the chip’s vital point stops being the compute engine and becomes the high-bandwidth memory (HBM)⁵ stacked beside it. What fills the MI355X’s 288 gigabytes is 12-layer HBM3E from Samsung Electronics and Micron. Samsung, which had long struggled to pass Nvidia’s qualification, walked through AMD’s door instead—and the bloating of Chinese models is exactly what’s widening that door. Plenty of people can’t get a slice of the cake, but whoever’s selling the steamer pans is making a killing.

There’s Math Running in the Opposite Direction, Too

Tucked inside this two-week whirlwind was one domestic Korean announcement, too. On July 27, Kakao released Kanana-2, its second small-model series. It trained a 3-billion-parameter model from scratch, then pruned and distilled it down to 1.3 billion; the 1.3-billion model is explicitly labeled for on-device deployment. Set next to Kimi K3, it’s 1/2,000th the size. It’s 2.6 gigabytes next to 1.5 terabytes—a smartphone standing beside a server rack.

cant The opposite direction isn’t a coincidence—it’s a calculation. The one card a company that can’t play the frontier capital game holds is a distribution network already sitting in tens of millions of hands—and at that scale, server inference costs are a catastrophe. If the model runs on-device, inference cost converges to zero and data never leaves the phone. It’s the retail version of the “intelligence within the perimeter” I mentioned earlier. The design, too, is entirely focused on saving memory.

kakaocorp/kanana-2-1.3b-base · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

Kakao says it used an attention architecture that cuts the KV cache by up to 72.7%. In its own evaluations, it beats comparable Qwen models by a factor of two on Korean knowledge Q&A, but falls behind on math. It’s clearly in a different league from the frontier, but if the actual goal is everyday tasks like summarization or classification, the math changes. Rather than waiting for trickle-down from the giants, it’s a strategy to win on its own turf.

Oz’s Lens

When I review free-tier strategies in my go-to-market consulting work, there’s one thing I always check: who, ultimately, gets sent the bill for this “free,” and in what form? Through that lens, today’s open weights are a different thing wearing the same word as the open source of the Linux era. With Linux, whoever downloaded it was also the one who ran it—it ran even on a beat-up old 486. Frontier-grade open weights, by contrast, structurally separate the downloader from the runner. The language is accessibility, but the function is wholesale distribution.

So every time I see the phrase “open-source AI democratizes intelligence,” I’d urge you to pause for a beat of doubt. What’s being democratized isn’t the retail market for intelligence—it’s the wholesale market. The beneficiaries aren’t individuals; they’re the layer that takes that intelligence and resells it.

But let me note the fact running in the opposite direction, too, in fairness. The trickle-down is real. The class of model that runs on a personal workstation today is roughly at last year’s frontier level. It does trickle down. Just with a delay, and always one rung below. Acknowledging that delay and that rung is, to my mind, how you use the word “open” honestly.

Closing

To sum up:

First, the entire top of the open-weight leaderboard is now Chinese models, and their scale swelled to 2.8 trillion parameters within two weeks.

Second, because of the MoE architecture, the API can be dirt cheap even as self-hosting requires memory scaled to the full parameter count. That’s why the freedom to download and the ability to run have split apart.

Third, the real buyers of frontier-grade open weights are clouds, corporations, and states rather than individuals, and the trickle-down from the size race flows into memory chips.

When Alibaba actually releases its 2.4-trillion-parameter weights next week, I’ll dedicate a separate issue to unpacking that “bill for free.”

Reader, has your team tried running open models yourselves? Tell me in the comments—how many billions of parameters was realistic, and where did you hit the wall and turn back to the API? I’ll gather the examples and fold them into the next issue.

💬 Share your experience running open models yourself in the comments. I’ll fold it into the next issue. 📨 If you know a colleague weighing self-hosting, forward this along.

📎 References & Further Reading

Primary sources

Ian Ye, “Is memory the moat?”, Wafer Blog, July 31, 2026. ··· The Kimi K3 benchmark that forms the backbone of this issue. It documents the kernel-patching process in full, so if you’re curious about the real difficulty of “running an open model,” start here.
Ian Ye, “Performance per dollar is getting faster and cheaper”, Wafer Blog, July 3, 2026. ··· The GLM-5.2 benchmark from a month earlier. It serves as this issue’s control group, showing the landscape when a model does “fit” on a single node.
Artificial Analysis, “DeepSeek V4 Flash 0731 Analysis”, August 2026. ··· The source of the open-weight intelligence rankings and pricing data.
Qwen Team, “Qwen3.8 Launch”, August 3, 2026. ··· The primary source for the 2.4-trillion-parameter count and the weight-release plan.
Citi Research, “Alibaba: Qwen3.8-Max Official Launch Assessment”, August 2026. ··· The source of the “model-agnostic” framing. Keep in mind this is a buy-rated sell-side report.
Kanana LLM Team, “Kanana-2 SLM”, Kakao Tech Blog, July 27, 2026. ··· The primary source on the compression process that pares a 3B model down to 1.3B. The weights are available on Hugging Face.

Background

Unsloth, “DeepSeek-V4 Local Deployment Guide”, August 2026. ··· How to actually run the 103-gigabyte quantized version. If you’re curious about the current ceiling of “open you can run.”
Seoul Economic Daily, “‘AMD: MI350X Uses Samsung HBM3E’”, June 2025. ··· Background on the Korean memory chips inside the MI355X.

Issues worth reading alongside this one

Why Did Jensen Huang Praise Chinese Models? ··· Today’s benchmark data touches the hidden assumption behind his line that “free AI is good for chips.”
The Four-Hour Video Call That Raised DeepSeek ₩11 Trillion ··· The very meeting where Liang Wenfeng’s “reasonable profit” calculation, quoted today, originated.

📝 Glossary

The author, Kwangseob Ahn, is a professor of business administration at Sejong University and lead consultant at OBF (Oswarld Boutique Consulting Firm). He teaches statistics and data analysis, including business data management and business analytics, while leading GTM and AI strategy consulting in the field, designing the seam between technology and business. He has published academic research on a memory architecture for AI dialogue systems (HEMA) and runs Daily Arxiv, a daily curation of global AI papers. He holds a master's from Korea University's Graduate School of Technology Management and a KMBA. He is the author of Homo Brainless: The People Who Outsource Their Thinking.

Footnotes

Open weights: A method of releasing a trained model’s weight files for anyone to download. It’s a narrower concept than full open source, which also opens training data and code—but these days the two terms are often used interchangeably. ↩
Mixture of Experts (MoE): An architecture that splits a model into multiple expert sub-networks and, for each token, activates only the ones needed. It saves compute, but since you never know in advance which piece will be called, all of them have to stay resident in memory. ↩
KV cache: The working memory a model stores so it doesn’t have to recompute earlier text while reading a long passage. The longer the context and the more concurrent users, the larger this footprint grows. ↩
Quantization: Compression that lowers the precision of weight values to shrink file size and memory use. Done well, it can cut capacity to a fraction with almost no loss in performance. ↩
HBM (High-Bandwidth Memory): Memory stacked vertically right next to the GPU for ultra-fast data exchange. It’s effectively this memory that determines an AI chip’s capacity and speed, and the market is split among three companies: Samsung Electronics, SK Hynix, and Micron. ↩