What It Really Costs To Run AI Locally

You cannot run Claude on your own hardware. Anthropic has never published open weights for any Claude model, so Opus 5 is not a price question, it is simply unavailable. What people actually mean when they say "run it locally" is running the nearest open-weight equivalent, and as of July 2026 that is Moonshot AI's Kimi K3.

This guide prices that machine honestly, top to bottom, with a source on every number. Then it explains the part that makes most of the spending pointless, which is not the money and is not the GPU.

1. 1
   Kimi K3, released 27 July 2026 by Moonshot AI. 2.8 trillion total parameters, of which about 104 billion are active per token. It is a Mixture-of-Experts model: 896 specialist experts, 16 activated per token.
2. 2
   1,048,576 token context window. Native text, image and video input. An OpenAI and Anthropic compatible API.
3. 3
   Modified MIT licence, permissive enough for commercial use. Training data and training code are not included, so this is open WEIGHT, not open SOURCE. The distinction matters if you were planning to fine tune from scratch.

At the time of release it is the largest open-weight model anyone has shipped. That is the whole reason it is the reference point for this question.

Start with the number everyone quotes and then look at what it assumes, because the assumption is where the honesty lives.

1. 1
   Seven NVIDIA RTX PRO 6000 Blackwell cards, 96GB each, is 672GB of aggregate GPU memory.
2. 2
   At NVIDIA's own US Marketplace listing as of August 2026, that card is $16,000. It launched at $8,565 and has been repriced twice inside one generation, driven by the GDDR7 shortage. Seven of them is $112,000.

Now the assumption. Seven cards clears the floor for the 1-bit quantized build only. Unsloth's UD-IQ1_S GGUF needs a little over 594GB of combined RAM and VRAM, so 672GB fits it.

1. 1
   The full precision weights as Moonshot shipped them are 1.56TB in MXFP4, averaging about 4.49 stored bits per parameter. At 96GB a card that is roughly eighteen cards, not seven. Anyone quoting seven is quoting the 1-bit number, whether or not they say so.

And $112,000 is cards alone. It does not include a motherboard that can physically host them, four or more power supplies, a chassis, system RAM, or cooling. On a build of this shape those are not rounding errors.

1. 1
   The RTX PRO 6000 Blackwell has a 600W TDP. Seven cards is 4,200W, or 4.2 kW, before the rest of the system.
2. 2
   Left running 24/7 that is roughly 3,066 kWh a month. At the 2026 US residential average, which the EIA puts between 17.65 and 18.83 cents per kWh depending on the month, you are paying about $565 a month to power the GPUs and nothing else.

Two things worth saying out loud. That figure is GPUs only, so the real bill is higher. And 4.2 kW continuous is more than most domestic ring circuits are rated for, which is a wiring conversation before it is a budget conversation.

Here is the finding that changes the answer, and it comes from someone who actually ran it rather than from a spec sheet.

1. 1
   On a rented instance with four A100-SXM4-40GB cards and 2TB of system RAM, running the UD-IQ1_S build, generation measured 0.10 tokens per second, held steady across the sampled window.
2. 2
   One token every ten seconds. A 500 token answer takes an hour and 23 minutes. The person running it aborted a 128 token benchmark after 17 minutes because it had not finished.
3. 3
   GPU utilization sat between 0 and 1 percent for the entire run, with only 64GB of the available 160GB VRAM in use.

That last line is the one to sit with. The GPUs were idle. Almost none of the memory was in use. Nothing about that run was short of compute or short of capacity.

A Mixture-of-Experts model only activates a fraction of itself per token, which is what makes a 2.8 trillion parameter model tractable at all. But the experts it needs are not the same ones from token to token, and if the full set does not fit in GPU memory, every token has to drag its active experts across the bus out of system RAM.

That read is the wall. It is a bandwidth problem, not a throughput problem, and it does not respond to any of the things people instinctively buy.

• Adding GPUs does not help unless you add enough of them to hold the entire model at once. Partial residency does not give you partial speed, it gives you the slow path on nearly every token.
• Faster GPUs do not help, because the GPUs are already waiting.
• More system RAM does not help. Having the weights is not the same as being able to move them.

This is the sentence worth keeping: you can buy your way to capacity, you cannot buy your way past bandwidth. The money buys the tank. It does not buy the pipe.

1. 1
   Claude Opus 5 is $5 per million input tokens and $25 per million output tokens on Anthropic's published pricing.

An hour of genuinely intense coding, with prompt caching doing its normal job on the repeated context, lands around 70 cents. That is the comparison the hardware has to beat, and against $112,000 of depreciating cards plus $565 a month of electricity, it does not get close for an individual.

Be fair about what that comparison is and is not. Renting is an operating cost that never stops, and it means your code leaves your machine. Those are real objections. They are just not cost objections.