In early 2026, after watching OpenAI roll ads into ChatGPT and Anthropic raise at valuations that implied aggressive future pricing, I decided to run an experiment. For several weeks of real project work, I switched from my usual Claude Code setup to a 100% self-hosted workflow on my 2024 M4 Pro Max MacBook. No cloud APIs, no data leaving my machine. I wanted to see what local-first AI development actually felt like in practice. Not in theory, not in a weekend benchmark, but across real deliverables.
What follows is what I learned: where local models surprised me, where they fell short, and why the question of who controls your AI tooling matters more than most developers realize.
Why This Matters Now
Per-token API prices have dropped over the past year thanks to fierce competition. But zoom out and the picture looks different. OpenAI is projecting $14 billion in losses for 2026 and expects to burn through a cumulative $115 billion before reaching profitability around 2029 or 2030. Anthropic, while leaner, still anticipates burning close to $20 billion before reaching positive cash flow.
The money has to come from somewhere. OpenAI's answer arrived in February 2026: ads in ChatGPT. Free-tier and Go-plan users now see sponsored content matched to the subject of their conversations, with ad personalization enabled by default. Anthropic committed to keeping Claude ad-free, but its latest funding round valued the company at $180 billion. That kind of valuation requires aggressive revenue growth.
The pattern is familiar from other tech cycles. Prices drop to capture market share, VC money subsidizes the difference, and once the market is locked in, the economics shift. Gartner forecasts that by 2026, AI service cost will become a more important competitive factor than raw model performance for most buyers.
If you are building your workflow around a model you do not control, you are building on ground that can shift under you at any time.
Security and Control Considerations
Cost is only one dimension. The security risks of third-party AI tools are becoming harder to hand-wave away. OpenClaw is the clearest example of what can go wrong.
OpenClaw exploded to over 85,000 GitHub stars in weeks. It can browse the web, read and write files, send emails, and control desktop applications. A January 2026 audit found 512 vulnerabilities, eight of them critical, including one-click remote code execution. Researchers found over 135,000 internet-facing instances, nearly a thousand running with no authentication at all.
The supply chain risk compounds this. Researchers identified users contributing malicious "skills" to ClawHub, OpenClaw's plugin marketplace, including one user who uploaded 354 malicious packages. Gartner found that 22% of enterprise customers already have employees running OpenClaw on corporate networks without official approval.
OpenClaw is an extreme case, but the underlying pattern applies to any AI tool where you do not control the execution environment. When models run locally, the attack surface shrinks to your own machine and your own network. You control the permissions, the integrations, and the update cycle.
Data Sovereignty and Compliance
The regulatory pressure is moving in the same direction. The EU AI Act reaches full enforcement in August 2026, with penalties up to 7% of global annual turnover. DORA demands proof that organizations can maintain operational continuity even when third-party providers fail. The EU Data Act explicitly prohibits vendor lock-in for data storage.
For developers working with client code, medical records, financial data, or anything covered by HIPAA, PCI-DSS, or GDPR, self-hosted AI is the shortest path to compliance. When the model runs on your hardware, every prompt and every response stays within your perimeter.
Gartner coined the term "geopatriation" to describe the trend of moving workloads back into local or sovereign environments, predicting that by 2030, over 75% of European and Middle Eastern enterprises will geopatriate their virtual workloads. AI workloads are leading that migration.
Where Local Models Open Up Flexibility
Beyond cost and compliance, local-first AI opens up customization options that cloud APIs simply do not offer.
Fine-tuning on your own data.
With tools like Unsloth and QLoRA, you can fine-tune open-weight models on your codebase, your documentation, and your team's conventions. LoRA adapters are small (often under 100MB) and can be swapped depending on which project you are working on.
Custom tool integrations.
OpenCode's provider system is built around an open configuration format. You define exactly which tools the model has access to, how it calls them, and what guardrails are in place. No third-party plugin marketplace, no supply chain attack vector beyond your own config files.
Model distillation.
You can use a frontier model to generate synthetic training data for a specific workflow, then fine-tune a smaller local model to replicate it at a fraction of the latency and cost.
Deterministic behavior.
Local models produce consistent outputs when you control the temperature and seed. Cloud APIs can change model behavior with silent updates at any time. If reproducibility matters, local is the only guarantee.
Hardware-aware optimization.
Apple's MLX framework is purpose-built for Apple Silicon. On an M4 Pro Max, GLM-4.7 Flash runs at 60-85 tokens per second in 4-bit quantization. That is fast enough for interactive coding, with no network dependency.
The Tools I Used
GLM-4.7 Flash
A 30B-parameter Mixture-of-Experts model with ~3B active parameters per forward pass. Scores 59.2% on SWE-bench Verified. Lightweight enough to use as a planning model with fast responses and a low memory footprint.
Devstral Small 2
Mistral's 24B-parameter dense transformer under Apache 2.0. Scores 68.0% on SWE-bench Verified with a 256K context window. Built specifically for agentic coding: exploring codebases, editing multiple files, orchestrating multi-step changes.
OpenCode
An open-source, provider-agnostic coding agent with 100K+ GitHub stars. Two built-in agents, Plan (read-only) and Build (full access), that you can assign to different models. The key to the whole workflow.
How the Workflow Came Together
The setup I landed on uses both models in a plan-then-build pattern through OpenCode. Ollama serves the models locally, and OpenCode's configuration lets you point each agent at a different model.
I assigned GLM-4.7 Flash to the Plan agent. It is fast, cheap on resources, and good at reading through a codebase, outlining an approach, and suggesting implementation strategies. Because Plan mode is read-only, there is no risk of premature changes. I could iterate on the approach quickly before committing to anything.
For the Build agent, I used Devstral Small 2. Its larger context window and stronger agentic capabilities made it well-suited for multi-file edits. Once a plan was solid, I switched agents and let Devstral handle the implementation.
This split is not arbitrary. Using a lightweight model for planning keeps iteration fast, while reserving the heavier model for execution ensures higher-quality code changes. After the first week, the rhythm felt natural. Plan fast, build carefully.
There was friction early on. The initial setup took a couple of hours of configuration and troubleshooting. Memory management on a 32GB machine required attention. Running both models simultaneously caused pressure, so I learned to let idle models unload between tasks. It was not as seamless as opening a browser tab to Claude. But once the workflow was dialed in, those rough edges faded.
Working Effectively with Smaller Models
Small models are not dumb models, but they are less forgiving of lazy prompting. Frontier cloud models can power through vague instructions. Local models in the 3B-24B active parameter range need you to meet them halfway.
Be explicit about the output format.
Instead of "refactor this function," say "refactor to use early returns, keep the same signature, and add a JSDoc comment." Small models follow specific instructions well but guess poorly when instructions are open-ended.
Front-load the important context.
Put the most relevant code and key constraints at the top of your prompt. Push background information toward the end. Small models have shorter effective attention spans even when the context window is technically large.
Use one-shot examples.
If you want the model to follow a specific pattern, include one concrete example. Small models imitate examples more reliably than they interpret abstract descriptions.
Break complex tasks into steps.
Ask a small model to implement the JWT utility first, review the output, then move to the next step. OpenCode's Plan/Build split naturally supports this incremental approach.
Keep the system prompt lean.
Every token in the system prompt competes with your actual task for the model's attention budget. Strip it down to role, constraints, and output format.
Specify what NOT to do.
Small models are more prone to over-generating. If you do not want the model to rewrite surrounding code or add verbose comments, say so explicitly.
Pin file paths and function names.
Use exact paths and names rather than descriptions like "the auth handler." Small models resolve ambiguity less reliably, and precise references eliminate guesswork.
Trim context between tasks.
Start a fresh conversation rather than accumulating stale context when switching focus. Small models degrade faster when the context fills with irrelevant previous turns.
What I Gave Up and What I Gained
I will not pretend the local setup matches Claude Code or Codex on every task. Frontier models are still better at complex multi-step reasoning, large-scale refactors spanning dozens of files, and tasks that require deep knowledge of obscure libraries. When I hit a wall on something genuinely hard, I occasionally reached for a cloud API. I do not think that is a failure of the experiment. Knowing when to use which tool is the point.
But for 80-90% of my daily coding work, including implementing features from a clear plan, writing tests, refactoring within a well-understood codebase, and exploring unfamiliar code, the local setup was more than capable. The advantages added up quickly: zero API costs, zero network latency, zero data leaving my machine, full control over model behavior, and no risk of a provider changing pricing or silently updating the model underneath me.
Cloud AI is not going away, and it should not. For tasks that genuinely require frontier-scale reasoning, for teams without the hardware, or for workflows where convenience outweighs control, cloud models remain the right choice. The goal is not to reject cloud AI. It is to stop depending on it by default.
The combination of OpenCode, GLM-4.7 Flash, and Devstral Small 2 is not a compromise. It is a different set of trade-offs, and for developers who care about sovereignty, cost predictability, and control, it is increasingly the right set of trade-offs.
If you need help setting this up, you are welcome to reach out.
Talk to Zach
