I Found 40% of My AI Instructions Were Reflexes, Not Decisions.
The Setup
Nate B. Jones laid out an argument recently that I was inspired by. Every AI system in production contains an invisible layer of workarounds for the last model’s weaknesses, and most teams stopped seeing them as workarounds a long time ago. He calls it compensating complexity. When a better model arrives, those workarounds don’t just become unnecessary, they become constraints. I don’t discover these things on my own. I get inspired by smart people who see patterns I haven’t named yet, and then I try to figure out how the pattern plays in my world. My world is an AI Operating System I’ve been building for months (honestly, more of a methodology than an OS at this point… nine months in and what I’m still trying to figure out what this is). It’s a structured context layer that makes AI think like my team instead of a generic model. It works. But after hearing Nate’s argument, I had a suspicion: my carefully tuned system might be full of scar tissue I’d stopped seeing. So I adapted his framework for my own context and ran the audit.
My Approach + AI Role
Nate’s audit is built for engineering teams working on code pipelines and system prompts. I needed something that worked for the stuff practitioners actually maintain: project instructions, knowledge files, the persistent way-we-work-together layer that shapes every AI interaction. Less codebase, more operating philosophy. I adapted his classification into four categories: Outcomes: What the system should accomplish. Goals, success criteria (your strategy decks and messaging frameworks have these). These survive any model upgrade. Constraints: What must always be true regardless of how the model gets there. Values, safety rails, non-negotiables (also in those same strategy decks). These also survive any upgrade. Scaffolding: Step-by-step procedures telling the model how to do something. Likely necessary for earlier models. May now prevent a smarter model from finding a better path. Compensating Complexity: Workarounds for specific failure modes. Instructions added because the model kept getting something wrong. Strongest deletion candidates. I fed my full instruction set into AI with this framework and asked it to classify every line. The AI did the pattern recognition. I made the judgment calls about what to actually cut.
What Actually Happened
42 distinct instructions went in. 28 came out. Biggest cut: a Mode Detection section… seven instructions defining three operational modes with explicit detection logic. That section existed because older models couldn’t infer the right register from context. Current models can. Replaced by a single sentence: “Match the register to the question.” Other cuts: elaborations of metaphors the model already understands from knowledge files. “How to think about the domain” coaching a capable model infers from the outcome statement. Defensive instructions (“these modes are not rigid categories”) that only existed to prevent over-literal application of other instructions that were themselves unnecessary. And then the big dig. I had instructions compensating for the side effects of other compensating instructions. Layers of patches on patches. Each one made sense when it was added. Together they were a tax on every conversation. It reminded me of something deeply human. We do this to ourselves all the time. You have a painful experience in your teens, a bad relationship, a moment where you got burned being too open… and you develop a response. A protective behavior. Maybe you stop trusting easily, or you overexplain yourself before anyone asks. Decades later, those responses are so automatic you don’t even see them as responses anymore. They’re just “how I am.” Then that nagging feeling hits you, perhaps it’s not helpful anymore. It isn’t until you do the real reflection that you trace them back to specific moments and realize: this was a scar I developed, and the response it created stopped serving me a long time ago. Same pattern here, except with AI it’s happening on a timeline of weeks and months, not years and decades. The instruction that was a solution became a constraint. And unless you sit down and trace each one back to why it exists, you’ll keep building on top of scar tissue without knowing it. Yes, I’ve just somehow blended psychology and AI together on this topic. It just makes sense honestly. And I have Leona Lewis’ Bleeding Love song in my head as I write this. Moving on. The sophistication ratio (outcome + constraint lines divided by total) went from 52% to 79%. Not by adding more outcomes - but by removing everything that wasn’t one. One key design choice: nothing got deleted permanently. Every removed instruction went into a “scaffolding annex” with a note on what failure would indicate it needs to come back. Delete and test, not delete and forget.
The Real Insight
I’ve been writing AI instructions for months. With AI, through AI, iterating constantly. And 40% of what I’d built was procedures the model no longer needs. That ratio matters because models are now good enough to figure out procedures on their own. The bottleneck has shifted. It’s no longer “can the model follow my steps?” It’s “do I actually know what good looks like well enough to describe it as an outcome instead of a procedure?” And honestly? I catch myself realizing I don’t always know. I just know the procedures very well (I’ve got a PM background after all). There’s a difference. The instructions you wrote six months ago were the right judgment calls for that moment. But the moment has moved. The question isn’t whether you have compensating complexity in your systems. You do. The question is whether you’re auditing for it or just accumulating it.
Try This If
You maintain any AI system with persistent instructions: Custom GPTs, project workspaces, system prompts, agent setups. If those instructions have been growing for more than three months, you almost certainly have compensating complexity. Classify every instruction into the four categories. Be honest about which ones are outcomes vs. procedures wearing outcome clothing. Cut the scaffolding. Test. Re-add only what the model demonstrably fails without. Before adding any new instruction, ask:
- Is this an outcome or a procedure?
- If it’s a procedure: has the model actually failed without it?
- Am I encoding this because “it worked last time”?
- Will the next model still need this?
Systems Lens
My AI Operating System (methodology?) is designed to be a persistent context layer. The parts that describe what good work looks like get more valuable as models improve. The parts that describe how to produce good work become constraints the moment the model outgrows them. I’m building a practice around this now (in between meetings): every time a model upgrades, audit what I can delete. Every time I’m tempted to add an instruction, test whether the model needs it first. The discipline is restraint, not engineering. And yes, I’ve confessed many times about this discipline before. For a builder who’s been adding layers of sophistication for months, this is its own kind of deep reflection identity work.