A friend who is a content strategist sent me a desperate message: her company had given 300 people Claude access and everyone thought “they could write content now”. She has been spending most of her time fixing those articles.
When you give AI context but no playbook, it's like hiring 100 junior writers and handing them a PDF of your brand guide. Then you disappear.
One intern reads "we value clarity" and produces minimalist content. Another reads the same document and writes 5,000-word epics. One prioritises SEO keywords. Another prioritises readability. Some hallucinate facts when they don't know the answer. Some miss nuance because they don't understand the stakes.
No coordination. No consistency. Just 100 different interpretations of what "good" looks like.
Another real example: A prospect called me last week. She is a CMO who said she had already figured out the SEO and AI search strategy for the Crypto brand she was working for, even though she had never done SEO before or hired anyone with that skill set. She only needed help to review what she had done and implement it. How did she do it? She has an "SEO folder" in Claude. She uploaded resources, best practices, and competitor research. The AI read all of it.
But when I audited their actual output, I found the most critical issues, those issues that will simply avoid her website from showing up on Google and LLMs: duplicated SEO titles across different pages, thin content scoring 16 out of 61 against competitors, broken HTML structure with multiple H1s, pages invisible to LLMs. Confident in the setup. Chaos in the results.
When I explained what was happening technically, she was shocked. The folder didn't train the AI. It just gave it reading material.
The Hammer vs. The Engineer
The difference between hiring an expert and using AI is the same between hiring an engineer or buying a hammer.
This is the distinction that changes everything.
A hammer is powerful. It doesn't care what you're building. You can smash walls or construct houses depending on who's holding it. By itself, it's neutral. Without direction, it hits randomly.
An engineer knows what you're building, why you're building it, what could go wrong, and how to handle the things that never go to plan. The engineer makes the hammer powerful.
Most organisations bought the hammer. They're missing the engineer.
You can't turn AI into an engineer by uploading documents. An engineer needs something much more specific: a documented playbook, processes explicit decision-making frameworks, and the ability to handle edge cases the way your expert actually would.
What Training Actually Looks Like (Not What You Think)
A ChatGPT folder or Claude folder is context. It says: "Here's information. Use it however you think is best."
Training is explicit instructions. It says: "Here's how to think. Here's the priority. Here's what matters. Here's what to do when things conflict."
Real training happens in a Custom GPT or Claude Skill, because that's where you can encode your expertise into a system.
Your engineer (the person who actually knows the work) needs to:
- Document the SOP step-by-step. Not vague advice. Explicit. "We research keywords using Semrush. We prioritise search volume first, competition second, brand fit third. We structure content with an intro hook, H2s for main ideas, H3s for details."
- Create decision frameworks. What happens when X? "If the keyword is branded, we focus on differentiation. If the keyword is intent-based, we match the search intent first. If there's conflicting data, we prioritise what our audience actually needs."
- Provide annotated examples. "Here's a piece we created for a similar keyword. Here's why it works. Here's what makes it better than the competition."
- Test and refine edge cases. What happens when the keyword is too niche? When you're competing against massive brands? When the audience is completely new to the topic?
Only the engineer can do this work. Not the intern. Not the generalist. Not someone learning the job as they go. The expert.
This takes 40-60 hours upfront. One-time cost. But it's also non-negotiable.
Why Context (Folders) Fails at Scale
A folder works fine when one person is using it once. They read the documents, they add their judgment, they review the output themselves. That's AI-assisted work.
It fails the moment you need:
- Consistency across 100 pieces of content
- Different people using the same system and producing similar results
- Output you can't review in detail because there's too much of it
- Edge cases the folder didn't explicitly address
When these things happen, a folder-based setup produces exactly what you'd expect: 100 interns all guessing independently.
The CM0’s duplicated titles? That's what happens when 100 interns all make keyword decisions independently and nobody's coordinated the strategy.
The CM0’s thin content and poor structure? That's what happens when 100 interns interpret "good content" differently.
This is what happens when 100 interns try SEO without a shared playbook.
The Setup That Actually Works
You need three layers, not one person hoping AI solves everything.
Layer 1: The Engineer (your expert)
Builds the Custom GPT or Claude Skill. Encodes the SOPs, creates the decision frameworks, writes the examples, trains the team. 40-60 hours upfront. They're overseeing quality, setting standards, ensuring the system behaves like they would.
Layer 2: The Operator (mid-level person)
Runs the system. Feeds inputs. Manages the workflow. Doesn't need to be an expert. Just needs to understand the process and know when something looks wrong.
Layer 3: The Junior Reviewer (intern or junior staff)
Applies checklists after the expert has trained them. Spots edge cases the system might have missed. This is now manageable because the AI isn't making a thousand micro-decisions inconsistently.
Compare this to a folder-based setup where the junior reviewer is drowning because they're trying to catch quality issues in AI output that's all over the place.
Real Example: The Difference Trained vs. Untrained Makes
I worked with the most popular crypto podcast in Latam. They have a content creator who knows content production deeply. He didn't upload her knowledge to a folder. We worked together to encode it into a system. The team went from producing 1 blog post per day to 5-10 per day. Quality stayed consistent. Traffic grew 33x.
Why? Because the AI wasn't 100 interns guessing. It was 100 interns following one expert's playbook.
The CMO’s folder setup:
Access to AI. No direction. Confident leadership. Broken fundamentals. The best case study of what happens when you have the hammer but not the engineer.
Who You Actually Need (It's Not Who You Think)
The engineer needs to:
- Know the work so well they can explain it step-by-step
- Understand edge cases and exceptions (this is where experience matters)
- Think systematically: what's the decision tree, not just the happy path
- Write clearly: instructions must be unambiguous
- Test rigorously: does the system behave like the expert would
An intern can't do this. They don't have pattern recognition.
A generalist can't do this. They don't know the domain deeply enough.
Most organisations think they can save money by putting a junior person in charge of quality control. They're actually creating a new, worse bottleneck. The intern can't judge quality in a domain they're still learning.
Managing AI output requires MORE expertise, not less.
The real investment isn't a bigger headcount. It's putting your expert in a position where they can multiply what they know instead of getting buried doing it all manually.
The Cost of Doing It Right vs. Wrong
A 5-minute task done 10 times a day is 12+ days of work per year on a single workflow. Multiply that across your team, and you're looking at thousands of pounds in operational cost.
But the hidden cost of scaling without standards is brand damage. Content that doesn't match your voice. SEO that cannibalises itself. Pages invisible to LLMs. False confidence in a broken system.
The CMO thinks the "SEO folder" is working. She's ranking. But for what? Misspellings of her own brand. One audit showed 260-word pages competing against 2,000-word resources from established players. She's not winning. She's just busy producing output.
The Question Every Organisation Needs to Answer
Do you have an engineer, or just a hammer?
- Engineer = someone who can encode expertise into SOPs, decision frameworks, and train a Custom GPT or Skill.
- Hammer = access to AI, no direction.
If you have a hammer but no engineer, you're scaling chaos.
If you have an engineer but no hammer, you're bottlenecked.
You need both.
Frequently Asked Questions
What's the difference between a ChatGPT folder and a Custom GPT?
A ChatGPT folder gives the AI context. A Custom GPT lets you encode training. With a folder, the AI reads your documents and makes decisions in the moment. With a Custom GPT, you define the decision rules upfront, so every output is consistent with your playbook.
Can an intern manage the quality of AI output if I've trained the AI well?
Yes, if the system is genuinely trained. With a well-built Custom GPT and clear decision frameworks, an intern can apply a checklist and spot edge cases. But if the AI hasn't been trained with SOPs and decision trees, you're asking them to judge quality in a domain they don't understand yet. That's a recipe for failure.
How long does it take to properly train an AI system?
The engineering work (documenting SOPs, building decision frameworks, creating examples) takes dozens or hundreds upfront. That's the non-negotiable investment. Then you're set. The system scales without you needing to rebuild it for every new task.
Is a Custom GPT better than a Claude Skill?
Custom GPTs work well for ChatGPT. Claude Skills work well within Claude. The framework is the same: you're encoding your expertise into explicit instructions and decision trees. Pick the tool that fits your workflow. The principle is what matters.
What happens if I skip the training and just use a folder?
You get context but not instructions. You get inconsistent quality, pages invisible to search engines, and a false sense of progress. You're producing volume without strategy. Eventually, you notice the output isn't working. By then, you've wasted months on the wrong setup.
Can I train an AI system myself if I don't have an expert on staff?
Not well. The engineer needs to understand the work at a deep level, which usually means years of experience. If you don't have that, you need to either hire someone with that expertise or work with a consultant who can help you encode what you actually do into a system.
More insights on AI search and AI marketing on my blog: https://victoriaolsina.com/blog/
