Make It Figure Out Less
The Setup
A few months ago I built an AI burn reporting tool for an account. It’s just an AI project folder with instructions. The goal was simple: let people ask questions about burn data in a conversation instead of needing to be an Excel wiz, cleaning data, building pivot tables, and formatting charts by hand every time. The output was an interactive HTML with trends, analysis, insights, and visuals I never thought of. The first version worked. It was also slow: a full build took 15 to 16 minutes between analysis and HTML output. The interactive HTML was impressive but a bit meh. So I did what felt obvious, I tried to make the AI smarter. Better prompts. More context. Clearer instructions about what “good” looked like. That turned out to be the wrong direction. The real gains came from making it figure out less.
My Approach + AI Role
This played out in three phases, forced by the realities of everyday work and finding time to explore with AI, each one removing a different chunk of from-scratch reasoning. Phase 1: Built it, knew it was rough, shipped it anyway. The first prototype was a single workflow that ingested a raw hours export, calculated burn, and produced a formatted report and HTML. It worked end to end, but the AI was re-learning everything on every run. What the columns meant. What burn reporting looks for. How to format the output. Fifteen minutes of thinking for something a person with a template could do in less. But you can chat with it after it! Phase 2: Gave it the answers it kept re-figuring out. Two changes did most of the work. Knowledge files saved to the project folder that explain what the data actually is and what burn reporting always looks for, so it stops re-learning the file structure every run. And an HTML shell file the conversation fills in each time you run it, so it stops reasoning out how to build the output and what on-brand means from scratch each time. That dropped the full build to around 4 to 5 minutes, depending on file size and how messy the export data is. Phase 3: Built a workflow. The newest piece is an ingestor skill that handles the raw hours export exactly as downloaded. No pre-cleaning, no renaming, no deleting report metadata, no saving as CSV. The skill finds the headers, filters to billable by default, checks the currency, and maps the messy columns to a clean, consistent set. When it hits a missing or wrong-currency cost figure, it follows a documented set of rules. And if nothing reliable matches, leave the row unresolved and flag it rather than invent a number. That last rule is the whole point. It behaves like a disciplined contractor following a checklist, not like something improvising.
What Actually Happened
The simple version: a full build went from 15 minutes for 1 data file in Phase 1 down to 2 minutes with 3 data files in Phase 3. Every improvement across the phases was similar: Removing something the AI had been figuring out from scratch on every run. The team has been running on this tool for weeks now. It is not perfect (I will probably be tweaking it for a while), but the shift from “pull the export, build the pivot, format the chart” to “upload the file and ask your question” has stuck. People are using it without being asked to. What surprised me: none of the speed gains came from a better model or a cleverer prompt. They came from knowledge files, a fixed output template, and an ingestor skill that handles the messy parts before the AI starts the “real work”. Every time I made the AI figure out less, the output got faster and more reliable.
The Real Insight
There’s so much more to do in recognizing what is “contractor work” versus “architect work”. The speed at which one can look at an Excel file and understand its structure (and what it’s meant to do) is so innate in someone who works with it every day that I didn’t recognize it until I looked at what AI was doing in its thinking. It was refiguring it out every time. Once I recognized that and provided instructions so it stops refiguring it out every time, that’s when we can get to the useful high-value judgment work right away. If you are building a file, assembling a reference table, or producing something that future conversations need to draw on, that is contractor work. You want it repeatable, checkable, and organized. The ingestor from the latest phase is exactly this: it produces the clean file that everything downstream AI work reads from, and it speeds up future work. If you are asking questions across material you already trust, that is the conversation. You stand on the contractor’s foundation and explore. The point is that once I stopped treating every AI interaction as the same kind of work, the architecture started making sense.
Try This If…
You have an AI workflow that is slow, inconsistent, or producing output that is “close but not quite right” every time. Start here. List every piece of knowledge the AI is re-figuring on each run. Read what it’s doing in the thinking panes and it will be clear. Column definitions, formatting rules, what “good” looks like for your context. Write those down as reference files and load them into the project folder. The meta move: ask AI what’s a task or repeatable thing it keeps doing that it should create a skill for. Then ask. Is the AI building something (a file, a table, a report) or answering something (a question across existing material)? If it is building, give it a fixed output template to fill in rather than letting it reason out the structure from scratch. Watch for this. The temptation to make the AI “smarter” when the real problem is that you are asking it to figure out too much. In my experience so far, a disciplined contractor with a clear checklist has beaten a brilliant improviser every time.
Systems Lens
Here’s a hypothesis I’m working on. I’m trying to use the contractor environment to run weekly intelligence scans across certain internal and external communication channels. The goal is to surface what is underneath the noise. The trends and patterns teams are too heads-down to notice, so leaders can get a read on what is actually happening without yet another meeting (and the prep work that no one has time for). The contractor does the audit and pattern detection because that work is repeatable and checkable. Once it stabilizes, I want to pull the outputs into a shared AI project folder where a we can query across all of it with thoughtful questions for deeper state strategic discussions/meetings. I expect to build a harness made up of new knowledge files and maybe a skill or two so it’s composable, portable, efficient, and powerful. The caveat for me: this is high friction right now. The work lives across two separate working platforms held back by data/IT security concerns, and I would love it if it were all in one place. So I’m going to be porting it manually. I am learning this as I go from mistakes and I can’t wait to see what comes of it. It’s not just about model capability and clever prompts. The gains I am finding are almost entirely about building the seemingly “boring” knowledge/skills for it to amplify our everyday work from.